Self-served OpenAI-compatible support

Hi, thank you for amazing no-code platform.

I'm MLOps Engineer so I want to use retool as admin or playground panel.
Because my environment uses many self-served llm, with openai specification compatible apis, I want to use them on retool. For example, in python, OpenAI SDK offers "base_url" option on OpenAI class to use another endpoint as a source.

I found out that retool only offers official providers(OpenAI, Cohere, etc), so self-served llm feature is lack.

So, can you check this?

Hello @Seungmin_Heo!

Interesting use case. You are correct that our current AI inegration tools only offer official providers as options.

For a self hosted LLM, if you have an available endpoint from your server, in theory you can make a REST API resource in Retool and use that to make Queries to the endpoint as you would with any other REST API on the internet :thinking:

This way you can send all the necessary headers, add information to the body of the request and if needed set up auth for the resource.

Are you looking to integrate Retool into your LLMs website/platform? In this case you could embed a Retool app inside of your code as outlined in our docs via our Retool JS SDK!

We haven't worked with any self-self LLM hosting users as far as I am aware so any further details on how we can best help integrate with Retool would be appreciated :slightly_smiling_face:

Sorry for late reply!

As you said, we are using JS script or simple rest api resources to use self-served llm apis. But important thing that we are considering is chat ui component supporting SSE-based streaming option. Because we have to test on multi-turn task environment, We need chat component as important aspect.

Another important point is, the reason why we want to adopt retool as admin source is we do not have frontend engineer and have no plan to hire them. Retool as no-code platform is perfect choice for our use cases.

So, If you consider this feature to be integrated in Retool, we look forward to it :slight_smile: