In edit mode, my app populates the Other Tasks / Tickets section with all the tickets and tasks if I open the custom component iframe code and add a space and then delete the space. Somehow this forces a refresh after all the queries finish, I think. I'm not changing the script, so I don't think my script is bad?
==
However, when I go into Preview mode, I can't use the same trick, so my preview is just empty and never populates. Any ideas how to trigger the custom component to refresh?
Hey @nl-setech!
Could you share the code you're using for your custom component? I've seen similar situations before and there's an internal bug report (which I can report back here on) but the resolution can still differ vastly depending on how the component is set up.
Also, are you seeing the component load properly on initial page load? Or does this only happen when passing new data?
Well, at the moment, it has stopped with this behavior. I have no idea why. I really haven't changed anything in particular, although for another problem that I can't get past, I'm changing, changing it back, changing, changing it back, and on and on ad nauseum.
I'm fairly certain it will happen again, as I have done nothing to address the problem, so when it happens, I'll post more information.
As for what I can remember of the problem, the component would not render on initial page load in either the preview or edit mode. I could only get it to render in edit mode, because I could modify the custom component code to force it to refresh. I don't mind sharing my code privately or sharing snippets publicly, but in this case, you might need the whole bit.
So the problem has returned again, and here is some possibly-related, possibly-not information...
I happened to notice these things:
#1
I am having either queryAllOpenTickets or another query that just seem to hang. Then, queryTaskSecondaryResources and queryTicketSecondaryResources fail, until for some reason, they rerun a third time.
Do queries that have inputs from multiple other queries run as soon as one query completes, or do they wait until all the queries they depend on are complete before running?
For example, if QueryA has inputs from QueryB and QueryC, does it run as soon as QueryB completes, or does it wait for both QueryB and QueryC to run?
#2
That query that seems to hang doesn't seem to ever time out, even though I have the default 10000 ms in the advanced settings.
I might just have some missing dependency? @Kabirdas - you mentioned a bug, are you thinking there's something else causing load problems, or possibly dependencies out of whack?
Well, that was easier to straighten out than I thought it would be. Can't figure out how to get the 2 that fail not to run in the beginning, but at any rate, they rerun on their own later and succeed.
My app's still not loading the custom components though.
The important thing, I think, is that the data is available in the arrays that are used to populate my custom component. So, I guess the dependencies aren't the problem?
Maybe the problem is that it takes so long for all my queries to request data from my external database that the Retool custom component doesn't know when to refresh?
Okay, that theory seemed to be correct. I found that custom components can be forced to refresh with the reload option, so I triggered a Javascript query after all the data is available that had in it "myCustomComponent.reload()"
I also added the new query as a button event handler, and that works every time.
Glad you were able to get things working!
Queries set to "Run automatically when inputs change" will view every one of their inputs separately and run whenever one of them recalculates. So, like you noticed, there's a good chance that's what you're seeing here.
It can also be that the query is running before one of the inputs has been populated causing it to hang, possibly before it even attempts to reach your DB/API. To remedy that, it can be helpful to disable the query if one of the inputs is nullish.
That being said, I'm curious to know what kind of resource you're using for queryAllOpenTickets. The query issue looks like it's a separate one - with some more context, I can see if we have an existing issue for it or file a new one since it's unclear what's happening with the query.
The bug I mentioned is specifically around the rerender logic for custom components. Your custom component should update whenever a dynamic value passed to its model does.
How many queries do getOtherItemsList and getStartedItemsList reference? Are they the values you're passing to your model?
If so, you might try having them be JavaScript queries as well. That, again, gives you more control over when they fire, so that you can make sure they're only being calculated when all their dependencies won't cause any errors.
That's just to give other options though, I think your idea to use reload() is a great way to get consistency without needing so much restructuring.
queryAllOpenTickets is a REST API to an external database. Actually, all of the queries in my app are REST API queries to the same external database.
I have a module that I plan on using in two apps - it does all of the data collection and formatting, based on a bunch of options the end user can choose to isolate the tickets and tasks they want to view. The module has 7 different REST API queries. Of those, 5 don't have any dependencies, but 2 of them require another query to complete to get part of the input data. Those are the 2 that repeatedly fail, until they don't. After the queries finish, there are a bunch of transformers that filter, sort and format the data. The outputs of the series of transformers finally populate the module outputs, which are used by getOtherItemsList and getStartedItemsList are the app side. But those two transformers do very little. All the heavy lifting is done by the module.
The two that hang don't have any dependencies. The 5 without dependencies should be able to run in parallel, but that seems not to work very well. I'm not sure why. As you mention, I am now using a Javascript query to control the order. I disabled the "run-on-page-load" feature of all the REST API queries, and just call them from the Javascript query. All I had to do was add an "await" to my query triggers. It kind-of takes forever to load now, but it seems pretty responsive once it's loaded, so I think it will do.
The part that confused me was that the 2 that had dependencies seemed to be running even though I triggered them in my Javascript query. I think because they had inputs they depended on, the input was something when the module loaded, so it triggered anyway? Not sure. Doesn't seem to break anything though.
If you await the dependencies using Promise.all and have the queries without dependencies run concurrently, do you see similar behavior?
I'm wondering if transformers running early might be a part of the issue (since they recalculate as aggressively as possible). I'll see if I can reproduce similar behavior on my end. That's part of the thinking behind having them be JavaScript queries too.
The fact the queries are running early is odd as well, the places to usually check are if they're "Set to run automatically when inputs change", set to run on page load, or attached so some other event handler. It sounds like you've gone through those checks already but listing them here just in case (and if someone else stumbles on this thread 
).
Would also be curious to see if there's any more information in the network tab of your browser's debug console regarding the queries that are hanging. That might be useful information!
That's much better - around 7 seconds to finish queries instead of 20+, when I start the non-dependent ones in parallel like you recommended. Seems to work.
The secondary resource queries depend on the other queries, and rerun when the first ones finish. They are setup to run when the inputs change, and are disabled when the inputs are null, but still seem to run in the beginning. I'm not too worried about it, though. Maybe the inputs are an empty array instead of null, or something like that.
@nl-setech that could very well be. You might also want to check if the way you're accessing the data might throw an error if it's not populated. For instance, if you've set "Disable query" to {{ !query.data.inputs }} the transformer will error if query.data is null causing the entire transformer to return null (falsey) instead of true.
