API pagination; variable call in API

💬 Queries and Resources
,
9

I have tinkered a bit and found a way to get data with the js code.
The issue I am now having is that setValue works asynchronously.
I tried to use await before and I found a thread which states I should use .then. Unfortunately I cannot seem to figure out how to make it work.
I also tried now to use await query.trigger() but this also doesn't seem to update my cursor state.

Here is my code:

let results = [];
let searchTerm = rt_publ_searchfield.value;  // Your search term
let cursorValue = "*"; // Start with the initial cursor
cursor.setValue(cursorValue); // Set initial cursor

let pageCount = 0;

const fetchPage = async () => {
  // First API request with the initial cursor value
  let data = await searchScopus.trigger({
    additionalScope: {
      cursor: cursor,   // Use the cursor Retool variable here
      query: searchTerm,      // Pass the search term
      count: 25               // Set items per page
    }
  });
 console.log(cursor.value)
  //console.log('First API Response:', JSON.stringify(data, null, 2));

  // Extract the results
  let newResults = data['search-results'];
  results.push(newResults);

  // Get the next cursor value from the response and update the cursor Retool variable
  let nextCursor = newResults.cursor['@next'];
  
  if (nextCursor != cursor.value) {
    // Update the cursor Retool variable
    //cursor.setValue(nextCursor); // Set the updated cursor value for the next iteration
     await query9.trigger()
    // Set a delay before triggering the next request
    setTimeout(fetchPage, 1000); // Delay next request by 1 second (or adjust as needed)

    
  }
  else {
    // No more pages, so return the results
    console.log('All pages fetched');
    return results;
  }
};

// Start the recursive pagination fetch
fetchPage();

And query9 is just:

let searchTerm = rt_publ_searchfield.value

let data = await searchScopus.trigger({
    additionalScope: {
      cursor: cursor,   // Use the cursor Retool variable here
      query: searchTerm,      // Pass the search term
      count: 25               // Set items per page
    }
  });

let newResults = data['search-results'];
let nextCursor = newResults.cursor['@next'];
await cursor.setValue(nextCursor)```

I am stuggling a lot here, so thank you for any tips and tricks you can give me!