Hello again!
I have labeled this as a bug although I might have misunderstood something.
I am still trying to get data with the scopus API. I can only access 25 entries at a time so what I can use is pagination to get the rest of the entries. What I need to do for that is send the cursor parameter set to "*" retrieve the @next
and submit this again until theres no pages left.
So far so good.
The problem I'm now facing is that when I use let cursor = "*"
the API call doesnt recognize this as valid and I get an error:
let cursorValue = "*"; // Initial cursor
// Trigger the first query to fetch the data
await searchScopus.trigger({
additionalScope: {
cursor: cursorValue, // Use the initial cursor
query: 'machine learning', // Your search term
count: 25 // Number of results per page
}
});
So I set a global variable called cursor:

this I cannot update by simply using cursor = "*"
so I used the setValue function:
let results = [];
let searchTerm = ''; // Your search term
let cursorValue = "*"; // Start with the initial cursor
cursor.setValue(cursorValue); // Set initial cursor
console.log(cursor.value)
let pageCount = 0;
const fetchPage = async () => {
// First API request with the initial cursor value
let data = await searchScopus.trigger({
additionalScope: {
cursor: cursor, // Use the cursor Retool variable here
query: searchTerm, // Pass the search term
count: 25 // Set items per page
}
});
//console.log('First API Response:', JSON.stringify(data, null, 2));
// Extract the results
let newResults = data['search-results'];
results.push(newResults);
// Get the next cursor value from the response and update the cursor Retool variable
let nextCursor = newResults.cursor['@next'];
if (nextCursor) {
// Update the cursor Retool variable
cursor.setValue(nextCursor); // Set the updated cursor value for the next iteration
// Set a delay before triggering the next request
setTimeout(fetchPage, 1000); // Delay next request by 1 second (or adjust as needed)
}
else {
// No more pages, so return the results
console.log('All pages fetched');
return results;
}
};
// Start the recursive pagination fetch
fetchPage();
here is my whole code.
As you can see, I tried to use the function somewhat recursively. I also the setTimeout function to try to update the cursor variable. Here is my issue, I want to update the cursor rather frequently to get to the next page unfortunately setValue seemingly only updates after the code has been executed, therefore this recursive code is caught in an infinite loop. As I said Im rather new to both retool as well as javascript hence I am not sure if this is intended or not.
Edit: I have found this but it doesn't work for me or at least I am not sure how to use it properly:
let cursorValue = "*"; // Start with the initial cursor
cursor.setValue(cursorValue).then((d)=>console.log(cursorValue)); // Set initial cursor
console.log("Cursor", cursor.value);
I'd appreciate clarification and/or workarounds!
Thank you in advance!