Self hosted Retool going down

Self Hosted Retool
1

We use the 3.52.1 version and face frequent issues when the app crashes.
Here are a few log messages:

"error":{"level":"warn","msg":"Query timed out after 120000ms
{"endpoint":{"path":"/api/pages/uuids/:pageUuid/query","service":"backend","team":"@tryretool/core-data"},"environment":"production","error":{"level":"warn","msg":"Query timed out after 120000ms","stack":"Error: Query timed out after 120000ms\n    at ServerResponse.<anonymous> (/retool_backend/bundle/main.js:4781:4360295)\n    at ServerResponse.emit (node:events:517:28)\n    at ServerResponse.emit (/retool_backend/node_modules/.pnpm/dd-trace@4.23.0/node_modules/dd-trace/packages/datadog-instrumentations/src/http/server.js:36:19)\n    at Socket.socketOnTimeout (node:_http_server:750:33)\n    at Socket.emit (node:events:517:28)\n    at Socket.emit (node:domain:489:12)\n    at Socket._onTimeout (node:net:598:8)\n    at listOnTimeout (node:internal/timers:569:17)\n    at process.processTimers (node:internal/timers:512:7)","type":"CAUGHT_ERROR"},"isEditorMode":"false","isPreview":false,"level":"info","message":{"dbconnectorRegion":"us-west-2","environment":"production","errorMessage":"The user aborted a request.","estimatedResponseSizeBytes":0,"isEditorMode":false,"isLatestConnectorVersion":false,"isPreview":false,"orgSid":"org_50df3e8a9ce342b5b5195a53f340ab31","queryName":"query_1020372","queryResponseTimeMs":120000,"resourceName":"a8b5ab6c-f50b-4797-a649-68303f6007ff","resourceType":"databricks","shardingEnabled":false,"success":false,"type":"QUERY_RESULT"},"orgSid":"org_50df3e8a9ce342b5b5195a53f340ab31","organization":{"id":1,"name":"user@company.com"},"page":{"embed":true,"folderId":5,"name":"Monitoring","uuid":"cda9f4c0-93b3-11ee-867d-bbb4d4178487"},"pid":48,"queryName":"appName","requestId":"354cd1dd-9a7c-4e39-8494-87750053c6b6","resourceName":"a8b5ab6c-f50b-4797-a649-68303f6007ff","resourceType":"databricks","retoolClientVersion":"3.52.1","timestamp":"2024-07-26T20:09:10.866Z","user":{"email":"email@domain.com","sid":"user_id"}}

A ResultSet was expected but not generated from query \"MERGE INTO %%%

We use databricks as our main resource.
How can I get the self-hosted retool to handle the query timeouts from DBX?

2

A few more log events:

node:17) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:17) [DEP0111] DeprecationWarning: Access to process.binding('http_parser') is deprecated.
{"level":"info","message":"RetoolDB credentials not found, skipping setup...","timestamp":"2024-07-30T14:21:49.030Z"}

sed: can't read ./dist/mobile/*.js: No such file or directory
3

So these errors were mainly due to the query timeout.
The reasons:

  1. Multiple queries were running at once.
  2. They were all running for more than the threshold.

Steps taken:

  1. Cached the results.
  2. Increased the timeout of queries.
  3. Changed the queries to manual loading & not automatically (this will start all queries as soon as user clicks on the app)