Hi everyone!
It seems to be a common request on this forum, but I'd like to return all results for a cursor-based paginated API. The thread here: Returning all results for a cursor-based paginated API seems to be the best bet, but I can't seem to get it working on my app.
I've created two queries:
-
API Query to fetch my first n results
-
JavaScript Query, which calls the API Query as many times as I need it to. In this case, I'm using next_token as my cursor.
// This function recursively calls the API as long as the cursor from the previous request exists.
const fetchAll = (next_token, records) => {
// Base case: we've reached the end, and there are no more cursors.
if (next_token == null) return records
// Wrap the query result in a promise
return new Promise(resolve => {
return query9.trigger({
additionalScope: {
next_token
},
onSuccess: queryResult => {
// Add the records from this query to all the records we have so far
const newResults = records.concat(queryResult.records)
return resolve(fetchAll(queryResult.next_token, newResults))
}
})
})
}
return fetchAll(0, [])
Unfortunately, when I run the above JS query, my program gets stuck in an infinite loop. Could someone help debug?
Thank you!
Hello,
Can you please try changing the function slightly like so:
const fetchAll = (next_token, records) => {
// Wrap the query result in a promise
return new Promise(resolve => {
// Base case: we've reached the end, and there are no more cursors.
if (next_token == null) {
return resolve(records)
}
query9.trigger({
additionalScope: {
next_token
},
onSuccess: queryResult => {
// Add the records from this query to all the records we have so far
const newResults = records.concat(queryResult.records)
resolve(fetchAll(queryResult.next_token, newResults))
}
})
})
}
return fetchAll(0, [])
@vangelov unfortunately -- the program still runs in an infinite loop. thank you for trying though!
do you have any idea what might cause this? one possibility is that it's never hitting the condition where next_token == null, but i doubt it.
Interesting. Can you post a screencast?
I tried setting up a Twitter api account, but for some reason it says my account is suspended..I've never used Twitter before 
If there's a way for you privately send me a temporary token I can test the query directly. I would understand if you're not comfortable doing this of course.
Actually I think I know what the problem is 
It's on this line: const newResults = records.concat(queryResult.records). Your queryResult does not have a records property. Use console.log to find the right one.
Here's a demo that gets all users from the example Postgres db. It uses offset/limit to imitate a cursor, but the principle is exactly the same.
Run the getAllUsers JS query to test it.
1 Like
thank you so much for your work on this!
I've actually figured it out thanks to @Tess!
the API and JS queries were configured as follows:
i simply wasn't using the right property names for my API query results. i hope this helps someone in the future!
