Best practice for realtime/event-based apps?

I just discovered Retool and am considering changing strategy from a homegrown React web app I'm building. However, the application is heavily event-based and the server pushes content to the client via websockets. I could move the data through Redis or Postgres instead, but I still need a mechanism to notify the client to update itself. Assume 1 event every 5 seconds on average and that needs to update, say, 3 tables in the UI. Of course I could theoretically set up polling every second but seems a bit wasteful.

Is that a bad use of the platform? I also have normal crud operations and the overall capabilities of Retool are really quite a good match.

If it's not a horrible match, then what is best way to do this? e.g., build a custom websocket component that handles external events? Set up some polling? Thx.

2 Likes

Following. I'm curious if this can be done.

Use Workflows for notifications if need be but why would there be updates every 5 seconds? Just curious... yes CRUD is supported very well on this platform!

I just skimmed the workflow functionality. That seems like it's for server-side processing. I think my needs are purely client-based. I need to update tables and some other data when an event occurs in an application server I already have. I just need something to tell the UI to update itself from data source(s).

I just saw this post about the topic:

3 Likes

Interesting post about the websocket, I might give that a try sometime.

It might be a bit wast of resources, but it is currently possible to schedule a resource query to update x amount of time.

Yeah, also thought about polling my app server or redis which will return null if no updates. It's not as bad as it sounds though it hurts the soul a bit.

I really configure something similar successfully.

I configure jsprintmanager on my app.

Everytime when I open the app it will prompt like this
image

And if I click to print label, it will run the callback which I defined in Retool until the printer really printed out the label.
I guess it must use something like websocket.

I will try this one if I have some similar need.

It seem EventSource is another good choice.

Workflows currently can't push live updates to apps, though it is something that has been requested before. I've added this thread to the internal tracking ticket for the feature and can let you know here if it does end up being included! :pray:

1 Like

@Taka_Binancio Let us know if you end up moving your React app over to Retool!

Iv been using ably for achieving realtime updates across all app instances. It works very good. Super fast updates in the 500ms range I think. It is just like pusher but for me was easier to integrate. Pusher was making it hard to allow the app to send out the data updates. Which probably means it's more secure.

1 Like

@Shawn_Crocker , can you explain a bit more on how you achieved this with Ably. Seems to be an interesting solution.

When you go through the ably setup docs, they provide a JavaScript SDK which I added into retool(just a link to copy and paste). Then, you setup a function in either your company wide preloaded JavaScript or an app level preloaded JavaScript. The ably doc details it but it is basically a function that you run on page load from a Js query. Running the function connects the retool app to the ably channel defined in the preloaded function. Then, from any script or Js query, you can post to the channel. You are able to include JSON data with your post. Any app instance will receive this posted data and when it is recieved, you can choose to do whatever you want with it. In my case, I choose to load it into a state variable. There are a few more intricacies but that is a basic summary.

@mbruijnpff Sorry for such a late reply. I'm always missing my retool community notifications for some reason.

1 Like

But you still have to poll Ably to see if anything has come in right? As far as I know there is no mechanism for a Retool app to receive a push.

If I am wrong here and retool can receive a push of some sort, could you be more specific on how you wire up Ably to your state var?

@bradlymathews there is no polling. I will double check the wiring today to be certain I'm handing out accurate details. It been awhile its been working flawlessly and I have not needed to interact with it

@bradlymathews

  • So, you add this link into your libraries.


'https://cdn.ably.com/lib/ably.min-1.js'

The new library adds in some special functions specific to ably.

You need the 'const anyName = new Ably.Realtime.Promise('youAblyAuthKey');'.

You need a variable setup that represents a specific ably channel as well. You can see I have a few. The channel can be called anything and does not need to be setup before hand. It is created simply by it existing when the preload initializes. These channels allow you to organize different data structures and control what happens on the recieving end when the data pushes into the app.

You also need a function that you will call in your app when it is time to propagate data to all app instances. This function references a specific channel variable and returns it.

  • then you can setup the initializing JS query in your app.

I have my query set like,
await drawingsSync().subscribe('drawingProgress', message => {
if(!!message.data) activeItems.setIn([message.data.catalogId], message.data.inProgress);
});

I'm using an await just so that I can pause the app on load until I'm certain the ably connection is fully established. It connects faster then I can perceive so probably not really necessary to have the await.
So, it subscribes to one of the channels setup in the preload. You could have is subscribe to multiple channels. This function is what actually gets triggered when something is sent over the channel. The message has a .data which will hold the JSON if you choose to send JSON. In my case, in the trigger function, I check that there is actually data in the .data object and then add it into a temp state.

  • Then you can send data to the channel from any script using a special function.


I have hightlighted this function.
drawingsSync().publish('drawingUpdate', updatedItem);

The drawingsSync() function is a function that was created in the preload that returns one of the channel variables.

So, that how it is hooked up. I don't completely understand how it is possible for it to work though. What I am always wondering is, "how the hell is that function setup in the JS page load query getting triggered when I am not directly triggering it?". There is clearly something magical happening within the ably library. But, it does work and it does work with great stability. I have never had it not work.

Something to note, unlike pusher, the app that sends out the data to a channel will also receive the same data that it sends.

4 Likes