Struggling with rate limits for API calls

💬 Workflows
1

Hi, I am trying to use workflows to fetch data from a REST API. Because the way the API a-was implemented, I needed to create another workflow to handle an API request. The latter workflow works without a problem. The workflow that fetches the data, fetches some data with a first call, which works. The data is a JSON array. Once I have this array, I need to loop through the array to fetch additional data for each item in the array. This where things go wrong. If I simply loop through the array and make the API calls, they quickly return with an error because rate limiting is in effect.
So I tried to write some javascript that inserts a delay after each call. The delay works, but no data is fetched from the API calls and no errors occur (data was fetched if I didn't use the delay until I hit upon the rate limit).
Below you can find the code that I used to create the loop with delays between each iteration.

function limiter(fn, wait) {
  let isCalled = false, calls = [];
  
  let caller = function() {
    if (calls.length && !isCalled){
      isCalled = true;
      calls.shift().call();
      setTimeout(function(){
        isCalled = false;
        caller();
      }, wait);
    }
  };
  
  return function(){
    calls.push(fn.bind(this, ...arguments));
    // this is needed for ES6 compatibility
    // let args = Array.prototype.slice.call(arguments);
    // calls.push(fn.bind.apply(fn, [this].concat(args)));
  
    caller();
  };
}
const doCall = limiter(value => {
  //console.log(value);
  //console.log(requestMailboxes_lambda);
  const result = requestMailboxes_lambda.trigger();
  results.push(result);
}, 600);
const results = [];
for (const [index, value] of joinOutputAccounts.data.entries()) {
  doCall(value);
}
return await Promise.all(results)

Any help would be appreciated.

2

Hey Tom!

You mentioned that you aren't getting any errors—do you mean no errors in your Workflow Logs, browser console or API Logs (if applicable)?

In your code, inside the doCall function, you trigger requestMailboxes_lambda for each item in the joinOutputAccounts.data array. However, it seems like you're trying to push the results into the results array without awaiting the actual API call. Perhaps this is causing issues?

I've modified your code a bit and added some comments, let me know if this makes a difference!

async function limiter(fn, wait) {

let isCalled = false;

const calls = [];

const caller = async function () {

if (calls.length && !isCalled) {

isCalled = true;

const call = calls.shift();

await call(); // Wait for the API call to complete

setTimeout(function () {

isCalled = false;

caller();

}, wait);

}

};

return async function () {

const args = arguments;

await new Promise((resolve) => {

calls.push(async () => {

await fn(...args);

resolve();

});

caller();

});

};

}

const doCall = limiter(async (value) => {

// Perform the API call here and push the result into results

const result = await requestMailboxes_lambda.trigger();

results.push(result);

}, 600);

const results = [];

for (const [index, value] of joinOutputAccounts.data.entries()) {

await doCall(value);

}

return results; // Return the results once all calls have completed