We run Retool as self-hosted, having installed id via Helm in Google Cloud GKE cluster, and are using cached queries. However, when invalidating the cache of a query, subsequent executions of the query sometimes return a cached old result and sometimes return the uncached, new result.
It seems to be related to us having multiple instances of the backend deployed, i.e. the tryretool/backend:3.283.1-edge image in 2 pods of the helm deployment.
I can see in the backend logs, that the invalidateCache call is received by only one instance of the backend. And I can see, that query executions to the same instance after the invalidation return the new data, the first execution return a non-cached result with the new data, the subsequent queries returning a cached result with the new data. However, query executions to the other instance, that did not receive the invalidateCache call, always return a cached result with the old, stale data, until the cache times out.
Is there something, that needs to be configured to make the invalidateCache work across multiple instances of the backend in a Helm deployed Retool app or is query caching not working at all in such a setup?
