"Internal Error running a block" on loops (not timing out)

Workflows
1

I have often used Retool workflows to translate data between two systems, for example by calling up a table in SQL, then looping through each row to import into another system using their API. A simple example workflow is shown below.

However, starting this week, I am suddenly getting this error on my loop blocks:

"Internal Error running a block: Error: An internal server error occurred"

This happens immediately whenever I seem to go above even 3000 or so items in my initial query. It's not just with sql, this has happened when I've done a large GET from another API as well and then tried to loop through it. You can see the error in the picture as well.

I know this is not a timeout issue because I have the timeout on my loop blocks set to 10 minutes (600000 ms) and this error pops up after only a few seconds when a click play on the block.
This issue has also only been happening this week, as I've been able to use loops in Workflows for much larger sets of data in the past.

Does anybody know what this error is and how to avoid it? If I'm going to timeout (which I don't believe I would), then that's one thing, but I feel like the workflow shouldn't be deciding for me whether that's going to happen or not before I get the chance to test it myself.

image
image1223×974 59.3 KB

2

Hey @ilowe! There may be a race condition issue happening here. You can test this by switching the code looping through the data to add a delay:

const results = [];
async function delay(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

for (const [index, value] of query1.data.entries()) {
  const result = query10_lambda.trigger();
  await delay(220);
  results.push(result)
}

return await Promise.all(results)

A couple other loop condition troubleshooting steps that we generally recommend (though they may not be relevant to your issue):

  • Double check the data getting passed, return the value/index to test.
  • Using Promise.allSettled vs Promise.all in loops can be helpful when customers are seeing random failures in their loops and want a way to handle them.

Let me know if this helps at all or if you have any further questions!