Issue - lambda times out before it finished running

💬 Queries and Resources
1

  • Goal: Wait for lambda to finish running.

  • Steps: Not sure what steps I can take.

  • Details: Hi, I have a lambda that gets triggered by a button in retool. Sometimes it needs to run for longer than 2 minutes. Due to retool default timeout, it times out after 2 minutes. I'd like to be able to increase that limit/find a work around so it properly finishes running (since it's success status triggers an event in retool). If it's relevant, my retool is cloud based. Thank you for your help in advance!

  • Screenshots:

    image
    image497×668 23.1 KB

    image
    image497×710 32.3 KB

1 Like
2

Hi @Shiri_Lempert, welcome to the forum! :wave:

From our doc on query timeouts:

"For [Cloud] users, the maximum duration before a query times out is 120 seconds (120000ms). Self-hosted users can set the DBCONNECTOR_QUERY_TIMEOUT_MS environment variable to change the maximum timeout value."

Maybe Self-hosted Retool would be a better fit for your needs.

3

Hi, is there no way to bypass it/ work around it? It's not an option for me to migrate to a self hosted environment over this issue

4

The only workaround that I can think of is triggering the Lambda function from a async Retool Workflow (no response block if we are worried about timeouts).

Screenshot 2024-10-11 at 11.42.00 AM
Screenshot 2024-10-11 at 11.42.00 AM889×479 46 KB

The tradeoff is that we wouldn't be able to attach a success event handler because the WF would have no response. Unless you have full control over the Lambda function and you can break it into steps, with the goal of making "loops" run each iteration in a new lambda invocation. However, this is me making an assumption that you can break the request down into multiple requests, ignore if it's not the case.

On the other hand, since we can take advantage of the async WF to make the request, maybe you are able to have Lambda report back to Retool when it's done processing.

You can store the status of the task in an external system like DynamoDB or S3 (or even SQS), and Retool can periodically poll the status by querying that external system.

Here is how that would look like:

  • Trigger the Lambda function from a Retool Workflow (you can trigger the WF from the App).
  • Lambda performs the long-running task and regularly updates the status of that task in an external storage system.
  • Retool periodically checks the status by querying that external system until the process completes.
1 Like