Hi all , Are queries running on client side server ? Because as I see closing the app 's page interrupt all queries running.
I am trying to run a query that synchronises data from a ecommerce website to Retool database . It takes a while to run.is there possibility to run queries in the background all allow to continue working on the page . The page is litteraly frozen when the query is running and interupt when closing the page?
It's a big issue need recommendations please / Thank you 
I haven't really tried much but as far as the language goes you have a few options.
Hey , never used Webworkers before , its it a hard task to implement with retool data queries ?
In theory, no it wouldn't be too difficult since web workers have access to fetch() and such. normally, you put a .js file on the server which contains the logic you want to do in the background. you would then use it like const myWorker = new Worker("worker.js"); with myWorker.postMessage() and myWorker.onmessage to send and receive stuff from/to the main thread and the worker.
heres' a web worker example
i guess there's like an official 'workaround' that they named embedded workers, if it works it looks like you can basically inline the worker.js file and not have to worry about uploading it to the server in right spot. i've never done this before though, especially in retool
Well I guess I'm about to dive into web workers then. I guess it wouldn't work on a Retool Cloud environement right ?
I'd he happy to know if someone tried it on retool yet
if you try the embedded workers, it wouldn't matter if you were on Retool Cloud or not but it might require using a Custom Component so you can add tags. you might be able to upload the worker file to the retool cloud server if you have access to source control or maybe with the Retool API or SSH?
those are about the only options i can think of. the file has to be located at the same location as the app (same origin i think) so i don't think storing it somewhere then fetching() it would work.
Still trying will give feedback on this
May I ask what the query is doing in the background? When I think about things in Retool happening in the background, my first thought is that it is probably a Workflow use case for an app.
@Ecommfox, @pyrrho probably has the best solution for this use case if 'data sync' process only needs to run on page load. If you have it running every hour or on page load for every user including external, you might need to continue trying to get a background task/worker running. all the sub tiers besides Enterprise, have a monthly limit on the number of successful workflow runs that can be used. for Business tier, you have 500/mo. so, it's something to consider before implementing and finding out you blow through 500 in a week 
.
forgot to answer this. JS Queries, and JS in general, is ran in a sandboxed environment separate from your project. Workflows are ran the same, sandboxed, but they are performed on a separate server.
It's actually a great solution I haven't though of imediately. I had originally one JS query chaining multiple POST api queries (which I call task ) , the idea was to free some bandwidth and allow the user to continue navigating the app or eventually close the web app;
But it raises other challenges, the user could trigger new tasks so they would have to be pedning while the other task finishes.
@bobthebear in the workflow a POST api query would be triggered repeadtedly for each row of a table when a user clicks an 'Import' button.
Users have roughly from 1500 to 7000 product row to import could this be done with a workflow , each 'task' counting as one worflow? If each APi call counts as a runs it won't be possible ;
AFAIK, the entirety of the workflow process is one run. This is true whether or not the whole process runs to the end or errors out at some point in the middle.
If your workflow is using a Loop component to call a single API endpoint for each row in your data, the whole workflow is only performing one run regardless of 1 or 1000 loops within.
thank you @pyrrho @bobthebear seems like a solution then!
Eventually would still need to upgrade
Not to belabor the point, but this is a great use case for a Retool DB of "queued" requests while in the app. I have a process which must be run in sequence but can take several minutes to complete. I have a workflow node which accesses this kind of DB for follow-up requests after a user has kicked off the process (with their initial request within the app).
Multiple users are able to populate the queue from their app instance and they all have visibility into their queued requests via a table. One issue with this is that very rarely the workflow gets "stuck" and I have to "kick" the queue to keep processing. I facilitated this with a special admin button that triggers the workflow manually 
Why does the queue get stuck?
The Api I am querying throws an ECONNRESET err more than often especially with POST requests. wouldn't it be a solution to implement an automatic 'retry on error' if the queue gets stuck?
I'm not entirely sure, but I believe it has to do with a webhook coming into the workflow from a separate system that processes the user request. It seems like the trigger is sent but not received by Retool, however, I've only seen this when the triggering system is involved so I'm inclined to think it is on their end and not Retool's.
I also haven't dug in very far to try to build out an error resolution. We are migrating away from the systems involved it so it seemed like a bit of extraneous work that was only handling a few edge cases. "Kicking the queue" suffices, but automatic retries would certainly be the right solution within a workflow depending on the system/API.