It seems to be a common request on this forum, but I'd like to return all results for a cursor-based paginated API. The thread here: Returning all results for a cursor-based paginated API seems to be the best bet, but I can't seem to get it working on my app.
JavaScript Query, which calls the API Query as many times as I need it to. In this case, I'm using next_token as my cursor.
// This function recursively calls the API as long as the cursor from the previous request exists.
const fetchAll = (next_token, records) => {
// Base case: we've reached the end, and there are no more cursors.
if (next_token == null) return records
// Wrap the query result in a promise
return new Promise(resolve => {
return query9.trigger({
additionalScope: {
next_token
},
onSuccess: queryResult => {
// Add the records from this query to all the records we have so far
const newResults = records.concat(queryResult.records)
return resolve(fetchAll(queryResult.next_token, newResults))
}
})
})
}
return fetchAll(0, [])
Unfortunately, when I run the above JS query, my program gets stuck in an infinite loop. Could someone help debug?
Can you please try changing the function slightly like so:
const fetchAll = (next_token, records) => {
// Wrap the query result in a promise
return new Promise(resolve => {
// Base case: we've reached the end, and there are no more cursors.
if (next_token == null) {
return resolve(records)
}
query9.trigger({
additionalScope: {
next_token
},
onSuccess: queryResult => {
// Add the records from this query to all the records we have so far
const newResults = records.concat(queryResult.records)
resolve(fetchAll(queryResult.next_token, newResults))
}
})
})
}
return fetchAll(0, [])
I tried setting up a Twitter api account, but for some reason it says my account is suspended..I've never used Twitter before
If there's a way for you privately send me a temporary token I can test the query directly. I would understand if you're not comfortable doing this of course.
It's on this line: const newResults = records.concat(queryResult.records). Your queryResult does not have a records property. Use console.log to find the right one.
Here's a demo that gets all users from the example Postgres db. It uses offset/limit to imitate a cursor, but the principle is exactly the same.
