Retool Workflows Call API with Batching

Hello,

I'm working with Workflows using a Loop Block with Parallel execution mode. I wanted to ask you guys, what is the best practice to call an API assuming that:

  1. I have large records (probably like 1K+ list of MD5 emails)
  2. Target API is quite slow to process.

Because I am currently fighting with timeout issue.
Please let me know your thoughts!

In general, your biggest concern will come from how often you send a request to your API, especiallyf it's a 3rd party API (not owned or controlled by you). Most APIs will have a rate-limit, smaller ones will have simple restrictions like max 10 requests/min. More popular APIs will have more complex rate-limits like max 10 requests/day and 2 requests/hour (you can make 10 requests per day at a frequency of 2 requests per hour, so it would take 5 hours to hit your daily limit). These larger APIs usually have endpoints specifically used for batch requests, if you look at the input for a single request then the input for a batch request is usually just an array of single requests. I'm guessing you're using an email service of some sort whos API is able to take an MD5 hash of an emails contents? Do you know if this API has a batch endpoint or if you have to send each email request individually? Do you know what the rate-limits are for the API or are you able to share what API you're using (if it's public)?

1 Like

Hey @bobthebear, thanks for the explanation. Unfortunately I don't know my platform API rate-limits.

Anyway seems like already have the solution and it worked fine :slight_smile:

since everything works you may not care, but just incase you can usually just google your APIs name + "rate limit" and get an answer:

  • "retool API rate limit"
  • "gmail API rate limit"
  • "openai batch API rate limit"
  • "retool email API rate limit"

or if you prefer, feel free to let me know here or in pm what API you're using and I can look it up and get the relevant links for you.

just answered this for somebody asking about Retools API rate limit specifically, so figured Id link it here:

but since you're asking about email, here's retools email service specifically :

2 Likes

Thankyou very much for letting me know this!

1 Like