Filter through multiple pages of records based on API response

I have an API response that returns up to 100 records and if more exist, a pageKey will be returned. That pageKey is then added to the end of the initial request as a new parameter and you can continue looping through responses till no more pageKey exists.

The first call does not require the parameter, but every subsequent call requires it.

I'm struggling to figure out how to loop through the responses. I've attempted using a script through the success event handler, but it becomes an infinite loop.

Any help would be much appreciated.

Hey @MALeone41! Would something like this work for you?

https://community.retool.com/t/returning-all-results-for-a-cursor-based-paginated-api/3387

I tried that, but it's infinitely looping. Should I be putting that code in as an event trigger on success of the first request? Or should it go in the transformer?

Ah, interesting! Hmm, would you mind sharing some screenshots of your setup (e.g. your query)? There may just be a small piece of logic missing :slight_smile: I'd also be happy to hop in and take a look if that would be helpful (if so, feel free to DM me your subdomain!)

Here is what everything looks like so far.

A pageKey is returned from the initial API call if more than 100 items are returned. That pageKey is then used as a parameter to get the next 100 items.

I’ll test this code on my end with a similar API, thanks for sharing! As a quick sanity check, once you’ve reached the end of your data, the pageKey your API returns is exactly null correct? I’ve occasionally seen ‘’ or ‘null’ before, which never ends the loop with this exact code :smiling_face_with_tear:

I’m able to see it working following these docs!

https://docs.retool.com/docs/server-side-pagination-examples#cursor-based-pagination

Let me know about that null condition :slightly_smiling_face:

Hey Victoria,

I ended up using a different approach and I'm getting the info I need by using a simple while loop.

Thank you!

:muscle::muscle::muscle:

Awesome. Of course + write back in anytime!