Error with returning all results for a cursor-based paginated API (infinite loop)

[jordanp]

Hi everyone!

It seems to be a common request on this forum, but I'd like to return all results for a cursor-based paginated API. The thread here: Returning all results for a cursor-based paginated API seems to be the best bet, but I can't seem to get it working on my app.

I've created two queries:

1. API Query to fetch my first n results
   

   

2. JavaScript Query, which calls the API Query as many times as we need to. In this case, we're using pagination_token as our cursor.

// Cursor-based pagination. This function recursively calls the API as long as the cursor from the previous request exists. 
const fetchAll = (pagination_token, records) => {

  // Base case: we've reached the end, and there are no more cursors.
  if (pagination_token == null) return records
  
  // Wrap the query result in a promise
  return new Promise(resolve => {
    return query9.trigger({
    	additionalScope: {
      	pagination_token
    	},
    	onSuccess: queryResult => {
        // Add the records from this query to all the records we have so far
        const newResults = records.concat(queryResult.records)
        
        return resolve(fetchAll(queryResult.pagination_token, newResults))
    	}
  	})
  })
}

return fetchAll(0, [])

Unfortunately, when I run the above JS query, my program gets stuck in an infinite loop. Could someone help debug?

Thank you!

[Kabirdas]

Hey @jordanp! Would you mind posting a screenshot of an example result you're getting from that API? Based on Twitter's pagination docs it looks like you should be receiving a next_token in the result that you might want to pass as your pagination_token, i.e. running something like

return resolve(fetchAll(queryResult.next_token, newResults))

[jordanp]

hey @Kabirdas! thanks so much for your help here. i actually got it to work thanks to some other Retool team members. i wasn't retrieving the next_token property correctly in the JS query

if you have any tips for this question: Group by functionality in table - #4 by ScottR -- let me know! no worries if not, however. this team is incredibly helpful regardless.