I'm pretty new to retool, and I'm struggling to setting up a scheduled workflow, which have to iterate over a large amount of data and batch operations.
According to the documentation, asynchronous workflows have up to 30h to succeed, while synchronous workflow have up to 15 minutes
So I made a workflow which is getting the count of entities and then pass it to a function which iterate entities by 200 and await the resolution of the current set operations before to pass to the next iteration
-Even if it's not adapted to my case, I tried to make a parallel process as explained here for a webhook implementation, but the workflow is still timedout after 15 minutes
Thanks for reaching out! I was able to reproduce the timeout you saw when manually triggering the workflow, but it worked when I triggered it via webhook. I think there is a limitation on how the workflow is run via the IDE that is capped at 15 minutes. I would test via webhook or scheduling, and it should meet the documented asynchronous runtime. Let me know if that works!
Hi @Taylor_M , sorry for not responding sooner! I just tried again, with a scheduled trigger, and I reached the timeout again when the workflow is triggered by schedule.
It seems like the asynchronous workflow is only possible when triggered by a webhook.
Good catch! I only tested the webhook previously. I just tested the schedule trigger and also hit the 15 minute limit. I am going to follow up with engineering and see what they have to say.
I'm facing a similar issue where I'm not able to get my workflow to run async (longer than 15min). I was able to create a simple test workflow with a wait block @ 17min (triggered by webhook from a retool app) and this worked. The workflow ran for 17min and didn't fail with 900000 ms error message, but it's still not working in my real-life workflow. I tried using webhooks with/without return blocks, scheduled triggers, removed retool DB queries, removed branches and global error handling, but it wasn't clear which part of my workflow was preventing it from running async. I've spent a lot of time testing in 15min test cycles, and think I've narrowed it down to one JS Code block (this code block iterates over >150 chunks of data from an API and feeds it through a pipeline line-by-line - this script is typically where the workflow fails), but I'm still not clear what the issue is and feel a bit blind.
What would be preventing a JS Code block from running asynchronously? Are you able to provide any more details on certain features or code that would prevent a workflow from running async for longer than 15min please?
As a feature request, I would like the ability to either know which queries/functions will only run synchronously, or the ability to set my intention for the workflow to be run asynchronously (like a boolean/flag) and then the workflow prevents me from implementing code or queries that would only run in the limited 15min synchronous style.
So what I learned since I posted here last is that it is not an async vs. sync workflow limit we are hitting here, but the JS code block limit, as it sounds like you have also figured out. A workflow should only run synchronously if called manually or from a webhook with a return block. So you should be running async but are likely hitting a JS code block limit of 15 minutes (the docs say 10 minutes now, but they used to say 15). One way to fix this is to break up the JS code block so it doesn't hit the timeout, and another is to move functions to be executed from function blocks instead of being called from the code block. If you are self-hosted, this limit can also be increased up to 24 days on the latest stable version, 3.148. https://docs.retool.com/workflows/concepts/limits#code-blocks
Ah ok, yeah the individual JS code block limit of 15min was what I originally understood and we've been managing around that by running longer jobs manually outside of retool when it fails, which isn't ideal. Then I read the updated wording for Workflow timeout in the documentation and it alluded to this limit being at the workflow level. The error line in the logs of the workflow also suggest the limit is at the workflow level "The workflow run exceeded the timeout of 900000 ms". Either way, it's good to confirm the 15min limit is at the block level.
It's a little difficult to carve up the script. I'm aggregating the streamed data on-the-fly as the logic relies on chronological order of timestamps on rows and uses specific occurrences of rows to know where to cut the data into different groups/buckets for aggregation - so at the end i am not retaining all the original data as it's too large. Handling the data in larger volumes at a time hits the heap memory limit. Processing each chunk separately (i.e. using the new loop block) has it's challenges because I need to retain some variables at a higher level than the iterations to store the current state of the aggregations or at least to know whether to aggregate a value or create a new bucket for it (so I don't aggr data from separate buckets together). Using the older loop block might be easier as it allows an embedded JS code block above the scope of the loop, but I believe this will also have the 15min limitation.
Anyway, I'll have another think about crafting a different approach with the new loop block and perhaps some multi-step functions?!, but before I re-write the entire script do you know if the new loop has a 15min on the entire block too, or now an individual limit on each of the iterations? The documentation here suggests 2min per iteration, but i thought I'd check as the documentation was slightly confusing to interpret for the workflow limits. Cheers, Adam
Sorry for the delay in responding. I wanted to do a little testing before I got back to you. From what I have seen, the new loop block behavior follows the description in the docs. Using the loop lambas, there isn't a 15 minute overall block timeout like when using the code blocks. The only timeouts you need to worry about are the iteration timeouts and the overall workflow timeout, which is the async vs. synch timeout distinction.
