I'm running into a problem where, after submitting data to a Databricks table, I want to display that new data in a table, which initially displayed cached data.
My current setup is:
Databricks Submit Query
Event handler of the submit query triggers a JS query called "Invalidate_cache"
"Invalidate_cache" query runs with this code "return Promise.resolve(My_Query.invalidateCache())"
Event handler of that js query triggers "My_Query.trigger()"
I'd say about 20% of the time this results in My_Query being run from the cache instead of being run "fresh". Any idea why that happens / suggestions to improve my code?
How are you managing the execution order of the event handlers? The order in the UI is irrelevant - they are executed asynchronously. To make sure they all run in the order you want, you need to handle it differently (e.g. here).
Currently, the submit query runs, in its event handler I run a js script containing the invalidateCache() function on success and in the event handler of that js script, I trigger the new query load "on success". So I though in this case they would not run in parallel.
I'm not super familiar with asynch functions / promises so I wasn't sure how to implement it in my case. I tried putting this in the event handler of the submit query, but it did not work:
The way you are describing it (a series of queries chained by event handlers) should also work, but it is also possible that some of these events would return a promise before the action they initiate completes.
Could you share some code and screenshots of your setup without exposing anything you don't want to share? It may help myself and other forum members to help with the issue you are running into.
