Hello!
We are using self-hosted Retool (kubernetes version) and doing a query to BigQuery.
When I tried to run this query in Google BQ Web Interface, it took 2-3 seconds to run and returned 72880 rows
Running this query in Retool "query library" took 30 seconds and returned "Unknown error" at the end
In retool pod logs I can see:
{"level":"info","message":{"http":{"method":"GET","request":{"time":200.7179617881775},"status_code":200,"url_base":"https://removed","url_path":"/api/checkHealth"},"type":"REQUEST_FINISH"},"pid":248,"requestId":"4afb413f-9f2a-4947-938a-de447ca4a2d7","timestamp":"2024-03-20T11:52:07.176Z"}
{"level":"info","message":"[Master] Worker 242 died (code null, signal SIGKILL). 2 workers left","timestamp":"2024-03-20T11:52:09.549Z"}
[Master] replacing worker
{"code":"ECONNRESET","errno":"ECONNRESET","level":"error","message":"Internal error occurred while making request to dbconnector request to http://localhost:3002/api/runQuery failed, reason: socket hang up","name":"FetchError","organization":{"id":1,"name":"removed"},"pid":248,"requestId":"dad5992f-47d0-44da-9ab3-0dce7362333c","stack":"FetchError: request to http://localhost:3002/api/runQuery failed, reason: socket hang up\n at ClientRequest.<anonymous> (/node_modules/node-fetch/lib/index.js:1501:11)\n at ClientRequest.emit (node:events:517:28)\n at ClientRequest.emit (node:domain:489:12)\n at Socket.socketOnEnd (node:_http_client:525:9)\n at Socket.emit (node:events:529:35)\n at Socket.emit (node:domain:489:12)\n at endReadableNT (node:internal/streams/readable:1368:12)\n at process.processTicksAndRejections (node:internal/process/task_queues:82:21)","timestamp":"2024-03-20T11:52:09.551Z","type":"system","user":{"email":"removed","sid":"removed"}}
{"level":"info","message":{"environment":"production","errorMessage":"Unexpected error occurred while running query","estimatedResponseSizeBytes":0,"isPreview":false,"orgSid":"remove","queryResponseTimeMs":39241,"resourceName":"76354244-833d-4af6-b77a-31be0e2e7121","resourceType":"bigquery","shardingEnabled":false,"success":false,"type":"QUERY_RESULT"},"organization":{"id":1,"name":"removed"},"pid":248,"requestId":"dad5992f-47d0-44da-9ab3-0dce7362333c","timestamp":"2024-03-20T11:52:09.552Z","user":{"email":"removed","sid":"removed"}}
{"level":"warn","msg":"Unexpected error occurred while running query","organization":{"id":1,"name":"removed"},"pid":248,"requestId":"dad5992f-47d0-44da-9ab3-0dce7362333c","stack":"M6: Unexpected error occurred while running query\n at /retool_backend/bundle/main.js:747:13883\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async S22 (/retool_backend/bundle/main.js:1017:25685)","timestamp":"2024-03-20T11:52:09.558Z","type":"MicroserviceError","user":{"email":"removed","sid":"removed"}}
Looks like Retool is killing own Worker after reaching some time limit (30s) (or memory limit?)
Can you please help with this?
Additionally I have specified DBCONNECTOR_QUERY_TIMEOUT_MS using chart, but it doesn't help
retool:
config:
dbConnectorTimeout: 120000
Is it something else I could do? Any ideas?
Thanks in advance
Oleksii