Managing Token Expiration in a Scheduled API Workflow on Retool Cloud

[ellenhelena]

Hello,
I've created a scheduled workflow to fetch data via API calls. The workflow requires obtaining an initial access token and then using a refresh token for subsequent requests, for as long as the initial access token has not expired.

Workflow Structure
Scheduled trigger. The workflow is triggered at intervals (e.g., 8:00, 8:15, 8:30).
Obtain the initial access token (POST request to /connect/token). The response includes: access_token, refresh_token, expires_in (duration of 1200 seconds), timestamp.
subsequent API calls. : Use the refresh_token for authentication.

The first run at 8:00 works fine. However, at 8:15, the workflow runs again and starts with the first codeblock the ‘obtain_access_token’, even if the access token of the previous run did not expire (1200s). This behavior is possible, but actually not allowed. The refresh token should be used instead.

I noticed that authentication can be handled directly at the Resource level with an Auth Workflow, but this feature is available for self-hosted Retool, we are on Retool Cloud.

Another option would be to encrypt the token and store it in a database. This way, I could check its expiration and conditionally request a new token as the first step of the workflow. However, I would like to avoid this if possible.

I am not sure which options I have in Retool Cloud to handle this. Any help/suggestions would be much appreciated.

[stefancvrkotic]

Hey Ellenhelena,

You can set up your resource and authentication within it.

What you're describing sounds like a custom auth flow - I've attached an example how you could execute on it.

Once token expires from the initial authentication a refresh flow is automatically triggered.

End result being whatever is stored in BEARER_TOKEN can be used directly in the resource.

Depending on your set-up, you could leverage different authentication flows. Hope this helps!

[ellenhelena]

Thanks a lot. However, I actually did build an AUTH Workflow, similar to your example, when finished I could not run it as we are on Retool Cloud, it was blocked by Retool.
Auth Worflows unfortunately can only be used on Retool Self Hosted.

Looking for any alternative options within Retool.

[stefancvrkotic]

If you're using OAUTH clicking "Share credentials between users" should allow you to do so using some service or individual credentials.

Definitely a weird pattern if you're trying to have a service only workflow resource.

Potentially adding some service credentials that do not expire within configuration variables and using those instead of access/refresh token?

[ellenhelena]

The API doesn’t support static credentials or long-lived tokens, and I don’t use OAuth, so I need to manage access and refresh tokens. Any other approaches within Retool Cloud that might work? I guess the solution was within the AUTH Workflow, which I cannot use on Retool Cloud...

[ellenhelena]

Found this in de docs, might be useful;
Manage configuration variables | Retool Docs

I'll look into it, and hopefully figure it out.

[Tess]

Hi @ellenhelena,

How is it going?

Custom authentication would be a good solution here, but unfortunately, we don't support user based authentication or custom auth with refresh flows in Workflows yet. This is regardless of self hosted or Cloud infrastructure. I will follow up here if our team ships a feature to support these types of authentication in Workflows

[ellenhelena]

Thanks for your reply!

I experimented with using configuration variables and created some to use within workflows, Manage configuration variables | Retool Docs.

However, updating the values requires the Retool Configurations API (as values cannot be updated dynamically without it), which isn’t available for Retool Cloud users. As the documentation states:
"Organizations on the Enterprise plan running on Retool Cloud or self-hosted Retool versions 3.42 and later can use the Retool API to manage configuration variables."

Unfortunately, this means I haven’t been able to solve the issue yet. Since I also cannot use AUTH workflows, and configuration variables don’t fit this use case, it seems I’ll need to set up an external solution to securely store and update the tokens outside of Retool. I’ll need to discuss internally to determine the best approach for us.

If you have any alternative suggestions, please let me know