Hi,
I was wondering if there was an option to be able to choose the embedding model/location for Vectors? Or if someone can help by clarifying my thoughts...
The way I understand it, we can make use of our own LLMs (like in Azure OpenAI), but that for the Vectors side of things, you're still passing the data to the OpenAI stack. Is that correct?
The reason this is important in our use-case is that we don't want the information in the Vector to leave our "control". With Azure OpenAI, we are hosting our own model within our tenant, and so any data is being contained as we need it to be. However if Vectors is sending data to OpenAI to be embedded first (as per your documentation here : Retool Vectors | Retool Docs), then it seems like we're losing control over it.
Is there anything we can do to point the embedding function at our own model in the same way we can choose which AI service we want to use?
Thanks,
Dave
1 Like
I did something like what you're trying to do using Azure Assistants API. It gives us a lot of flexibility as we can store our data the way we want. Then with the help of Code Interpreter, we can load them to Panda data frames. Once we do that, the sky is the limit.
Here's a short demo of it
Drop me a message if you need a walkthrough.
Hi @dcsearle, Thanks for the question!
Yes, when you add a URL or Document to Retool Vectors, Retool uses OpenAI's embeddings API to convert the text into chunks of embeddings (arrays of vectors/floats), which get stored in the vector database in Retool Vectors.
Retool Vectors uses the OpenAI format for storing data in the Retool Vectors' vector database.
Then you can use your Retool AI Resource and include relevant context from your Retool Vectors, just by selecting that Vector.
You can use Azure OpenAI's APIs directly through Retool AI, as well as others.
That's separate from how Retool Vectors works.
Retool Vectors always uses OpenAI for the Embeddings API calls. If you've configured your own OpenAI API key then the traffic will go through your own agreement with OpenAI.
I've heard of the capability to insert the OpenAI-formatted embeddings directly into Retool Vector's database. You could use a Langchain library in javascript or python, (or with embeddings APIs directly) all within Retool Workflows. You'd split and convert your text chunks into embeddings, then put them directly in the db. I don't have an example of that. Maybe someone in the community will help us out 
