Accessing query data within trigger().then does not work

App Building
1

I use .trigger to execute an API query and "then" to sequentially access to the values returned by the query.

Here an example:

query63.trigger().then
(
() => TemporaryStateCategoryUID.setValue(query63.data.id)
)

I figured out, that the data returned by query63.data.id is outdated when I try to access it. It doesn't reflect the data generated by the latest query63.trigger() call. If I use .then, I would expect that first Retool is executing the query query63 and then in a second step I can access the data created in query63 by using .then (resp. .data). However, it seems like the "storing" of the data generated by the query query63 happens somehow async in Retool and is not really part of the .trigger function.

2

Hi @Reboon!

I'm not seeing the synchronous behavior you're looking for with .then, but async/await syntax works for me. Can you give something like the below a shot? It logs the actual output from that particular query run once it completes.

async function triggerGetData() {
  let data = await getData.trigger();
  console.log(data);
}
triggerGetData();

-Justin

3

Hi @jmann

Thanks for the script. I was testing it and figured out the following behavior.

  1. This works. It is your script, but printing out the total row count.
async function triggerGetData() {
  let data = await GetDataFromLongRunningAPICall.trigger();
  console.log(data.total);
}
triggerGetData();
  1. This does not work. Here I am accessing the data of the query with .data after it's execution.
async function triggerGetData() {
  await GetDataFromLongRunningAPICall.trigger();
  console.log(GetDataFromLongRunningAPICall.data.total);
}
triggerGetData();

I would expect that the second script works as well since I want to access the data of the query after it has been prepare with .trigger().
I also verified whether the sync flow is correct. And yes it seems correct, first GetDataFromLongRunningAPICall is being executed (runs in my case like 6 seconds), after this has been completed the console.log runs. However, in the second query the data.total is empty.

I further tried a more complex version of the query.
As you can see in this query below, I am calling GetDataFromLongRunningAPICall same as before. But then I call QueryDataRunsAsWellAndWorksFine which fills the text component text1 with GetDataFromLongRunningAPICall.data.limit.
The same I do after 5 seconds with text2.setValue(GetDataFromLongRunningAPICall.data.limit);.

async function triggerGetData() {
  await GetDataFromLongRunningAPICall.trigger();
  QueryDataRunsAsWellAndWorksFine.trigger();
  await new Promise(r => setTimeout(r, 5000));
  text2.setValue(GetDataFromLongRunningAPICall.data.limit);
}
text1.setValue(0);
text2.setValue(0);
triggerGetData();

I would expect that text1 as well as text2 are being filled with the value from GetDataFromLongRunningAPICall.data.limit. But it is not, text1 is correctly filled. text2, however, contains the outdated data from GetDataFromLongRunningAPICall.trigger.
My conclusion so far is that Retool must somehow store a copy of GetDataFromLongRunningAPICall.data during run-time of a query and does not update this which is the reason why the data is outdated.

I also attached a link to a video that clearly shows the behavior.

image
image1882×707 130 KB

And here the JSON file of the app I was testing with in the video (the API call needs to be replaced accordingly).

Test Ticket 13318 - V2.json (23.8 KB)

4

@jmann were you able to reproduce my issue?
Here another simple example which I just encountered:

query88 (type JS Query):

query89.trigger().then
(
  () => utils.exportData(query89.data, 'test file', 'xlsx')
);

query89 (type: Query JSON with SQL):
select 'test rrrrrrrrrrrrrrrr'

If you run the above query query88, the export will be empty, because it can't access the data from query89 even though they were created before the export was called.

5

Actually I don't even think we need the async function to synchronously access the results after triggering a query, does just using await like in the below not work for you?

let data = await getData.trigger();
utils.exportData(data, 'test', 'xlsx');

In terms of the other example, like I saw originally, I'm not seeing the then syntax working, so let me know if I'm missing something on why just using await doesn't work here!

1 Like
6

I have tried your suggestion, and yes it does work indeed. But the problem is, the other way around it should work too which is clearly doesn't.
I made another example to illustrate the problem:

See here your code (first two lines), which does work properly. Now you see a third line, this line is accessing the data from getData by using *.data. This should work in my opinion, but it doesn't. *.data simply does not return any data (if the app was opened freshly and getData.trigger never ran before).

let data = await getData.trigger();
utils.exportData(data, 'test1', 'xlsx'); // this is working
utils.exportData(getData.data, 'test2', 'xlsx'); // this is not working

Or how do you see it? Shouldn't getData.data be available with data?
Because here another interesting example:

First you start this:

let data = await getData.trigger();
await new Promise(r => setTimeout(r, 30000));
utils.exportData(getData.data, 'test 10', 'xlsx');

While the above is still running, you start the below query:
utils.exportData(getData.data, 'test 11', 'xlsx');

What happens: The second query will finish right away and creates the Excel correlcty. The first query finishes within around 30 seconds and, surprise, the Excel won't create (because the query considers getData.data as empty, even though getData.trigger() ran before).

This proofs that getData.data is not available inside the query that started getData.trigger() but for all other queries, that started after getData.trigger() getData.data is available and filled with data.

Do you know what I mean?

1 Like
7

Hey @Reboon I see where you are going with it and yes it definitely is weird to behave like this. I am facing same issue and it took me a long time to realize what can go wrong because even in inspection I can see data but my code is failing.

So had you get any workaround for this let me know.

@Kabirdas / @victoria if you guys can take a look and see about this.

image
image1281×410 82.1 KB

8

I'm getting something very similar.

I too get issues with these different ways of accessing the data:

let data = await getData.trigger();
console.log(data);
console.log(getData.data);

The last line only works if I put a pause before accessing getData.data.

Weirdly my issue though is that I'm paginating through an API (Jira, not that it matters). I'm running getData.trigger() perhaps five times, and when I set data with let data = getData.trigger(), even though I'm running this five times, the data never changes from the first time it was set, so effectively I'm always getting the first page of results. I know Jira is sending back the right data because this causes an infinite loop and the first five calls are slow and the subsequent ones (with a startAt=600 and up) come back very fast, but still give me the first page of results.

Any ideas? I've tried using the API resource directly (calling getData.trigger()) and I've also tried re-instantiating a copy of the query each time I call for a page, using Object.assign({}, getData), as recommended by GPT-4, but regardless, the let data = await getData.trigger() always returns the first value.

Any thoughts?

9

hello try onSuccess params:

  query1.trigger({
    additionalScope: { i: i }, // This is where we override the `i` variable
    // You can use the argument to get the data with the onSuccess function
    onSuccess: function (data) {
      runQuery(i + 1);
    },
    onFailure: function (error) {
      // Update the Errors text
      errors += "Found error at line " + i.toString() + ":  " + error + "\n\n";
      Errors.setValue(errors);
      runQuery(i + 1);
    },
  });
10

Just want to push topic up. I've also struggle with the same issue.
let res = await query.trigger() workaround works for me too, but I want to know, why await + query.data way doesn't)

Thanks

11

I believe it's because Retool is a big React app and that's how setState in React works.

setState() does not always immediately update the component. It may batch or defer the update until later. This makes reading this.state right after calling setState() a potential pitfall.

It's the same reason that setting a variable and immediately reading from it without checking Keep variable references inside the query in sync with your app doesn't work. As your Javascript runs it doesn't defer to React state updates by default and so calling query.data will just fetch whatever version of that object exists in memory while the state setter that works underneath the hood may or may not be done mutating it.

Assigning the query result to a variable as shown above is going to be the quickest and simplest way of immediately using query data.

1 Like
12

In my side, the data is undefined.

1 Like
13

Encountered the same issue. Around 4 hours of debugging until I found this thread. The explanation by @kschirrmacher seems valid. But this should be part of the documentation imo. Its hard enough to debug retools own idiosyncrasies without thinking about React-behaviour.

There is some documentation about how to access .trigger() results here: Run JavaScript | Retool Docs - but its doesn eg cover cases where I do not only want to access the single .data property, but maybe also .metadata, etc.

1 Like