Hi
I would like to have the my AI workload (for both Agent and regular Retool AI) using the Google Vertex AI API endpoint rather than the Gemini Developer API (which is the default for Retool). Reason being to ensure we can use some of the additional features like prompt caching, observability, etc.
I am having difficulty, as the inbuilt paths for the Google schema don't line up with the Vertex AI API, and I can't seem to get the OpenAI compatibility for Vertex AI (and ideally aren't using a compatibility feature).
Any suggestions?
Hey @SF33, the best option right now is to set up a lightweight reverse proxy server that maps a supported schema to the Vertex endpoints. Essentially something like,
/v1/chat/completions -> /v1/projects/{project}/locations/{location}/publishers/google/models/gemini-pro:streamGenerateContent
so..... if possible, how about a FR for Retool Middleware? 
probly not feasible. idk how useful it'd be but being able to re-route internal requests could be kinda neat (role or environment based rerouting or external mapping i guess?)
Well, you can actually do this with Retool Workflows if you really want to... but Workflows isn't really designed for this to be an optimal user experience.
I've set up a workflow that does some endpoint mapping, and then provided that workflow's webhook URL as a Custom AI Provider URL, and it will work.
Generic Retool middleware is a cool idea, and it makes sense, but in full transparency, not something we plan on working on right now.
1 Like