Returning all results for a cursor-based paginated API

We’ve had a lot of customers ask how to return all the responses for a cursor-based paginated API, where:

  1. You make a request to an API
  2. API returns the first n results and a cursor
  3. cursor is used in a subsequent request to fetch the next n results

If you want to fetch all the results, create two queries. The first will be your API Query to fetch your results. The second is is a JavaScript Query, which calls the API Query as many times as we need to. In this case, we’re using offset as our cursor.

Note that offset is undefined-- that’s okay. We’ll be using the JS Query to inject the value through additionalScope (more info here).

Now, we want to have a JS Query, like so: https://repl.it/@AntonyBello/Recursive-Fetch#index.js

This query recursively calls the API Query as many times as we have offset returned to us (i.e., until we’ve returned all the data.)

1 Like