I'm having serious issues with running a particular workflow that needs to deal with 1000s of operations. In essence I'm migrating our company old shop into Shopify, having retool as the ERP that binds everything together.
Workflows looks like the perfect way to create a one-off, idempotent, workflow to read the data from the old shop and import it into our retool db.
The problem I'm getting is that when I run this workflow, even just dealing with 500 records, I'm getting huge amount of errors, to the point that I'm unable to even open the db web video.
Hi! We migrated your instance to a different provider where we don't expect this error to show. Please let us know if you run into this again. Note: This does update your connection string. If you use the connection string to connect to your retooldb directly from outside of Retool, please update this.
Yes. There's a rate limit of 1k requests per minute for retoolDb that you might be exceeding. I would recommend spacing out your requests on your workflow.
As for exceeding 1Gb memory usage, let me see if I can find someone to help with this question.
Hi, @First4Figures, engineer on the Workflows team here. Unfortunately the 1GB memory limit is not configurable and applies per workflow run. We would recommend splitting up your data into smaller pieces if possible.
Good morning @himanshu and @ehe! Thank you for your feedback - Totally understand. I'll have a chat with your sales team, mostly around the limit of 5000 workflows which splitting could potentially cost us quite a bit more. Need to understand better the options.
- Totally understand. I'll have a chat with your sales team, mostly around the limit of 5000 workflows which splitting could potentially cost us quite a bit more. Need to understand better the options.