I have a workflow that inserts multiple records into Jobber (eg 500 per minute). I have another workflow with a webhook trigger on a Jobber update. When I launch the first workflow, the second workflow is triggered over 500 times in a short span. In the second workflow I have multiple queries into the retool db and it often times out with the error "Too many requests for RetoolDb".
Is there a way to temporarily shut off or disable the second workflow via a function?
Depending on the timing between the start and end, you could potentially orchestrate some kind of workflow queue maybe? Where the second workflows all go into a DB with their startTrigger data, and then the a single secondary workflow drains the DB gradually, with a pause between each run/bulk run? It might take work to figure out the optimal way to coordinate the work, but the "orchestration" workflow could set a flag in a DB saying it was running, so nothing else triggered it again while it's running, it runs on a loop until the DB is empty, and then sets the flag to false, maybe it waits 60 seconds at the end, checks the DB one last time, then sets itself to 0. Then you could control the pace.
That's a tricky one, @Shawn_Optipath. If I remember correctly, requests to RetoolDB are rate-limited at 1000 per minute.
There's no particularly straightforward way to dynamically disable a workflow, but you might be able to avoid hitting the rate limit by refactoring the workflow to perform bulk updates whenever possible and/or configuring a relatively long retry interval on the RetoolDB query.
If that doesn't work, @MikeCB's suggestion to implement a queue of sorts would likely work. I can also share a workaround for dynamically disabling the workflow, but it's not an ideal solution.
Thanks @Darren . Yes, I did refactor and it halved the db rates. I also added a loop with larger time intervals keeping the rates to just under 500 per minute for safe measure. This workaround will work for now, but the client has other projects that will continue to add strain.
Is the 1000 per minute rate imposed by Retool throttling Neon? If so, I might be able to convince the client to switch the db over directly to Neon. I've been dabbling with a Neon account and it might solve a lot of issues.
@MikeCB thanks for the input and apologies for not getting back. So many distractions at times
Yep, this is a limitation that we impose in order to maintain the health of our multi-tenant database architecture. It is possible for me to bump that number up a bit, but it may not be a sustainable solution. I'd definitely recommend looking into dedicated storage - either Neon or RDS - if you think the client may be open to it!
That makes complete sense. I will let you know if I need a little extra between now and when I convince them to dedicated storage. The retool db is a fantastic stepping stone though! Thanks @Darren