My goal:
Retrieve more documents from Retool Vectors when using the Retool AI resource.
Issue:
I have a Retool AI query set up with the "generate text" action.
I have the "Use Retool Vectors" selected, and it is referencing a retool vector I have set up with ~200 documents.
However, when i Run the query, I can see it only references 1 or very few of the documents.
Is there any way to control the sensitivity of document retrieval, or have more insights in to the actual chunks it's matching on?
This was working with more documents being retrieved not too long ago but it recently has only been grabbing a few documents and it's significantly degrading the usefulness of the app for my stakeholders!
Thank you for the feedback, let me double check with the AI team if any changes have been made that would have altered the scope of how many documents an AI Query would pull from a Retool Vector.
They asked me to ask you "Are they using any label filtering?" and they are wondering if you have changed the model being used in the query.
My goal:
Retrieve more documents from Retool Vectors when using the Retool AI resource.
Issue:
I have a Retool AI query set up with the "generate text" action.
I have the "Use Retool Vectors" selected, and it is referencing a retool vector I have set up with ~200 documents.
However, when i Run the query, I can see it only references 1 or very few of the documents.
Is there any way to control the sensitivity of document retrieval, or have more insights in to the actual chunks it's matching on?
This was working with more documents being retrieved not too long ago but it recently has only been grabbing a few documents and it's significantly degrading the usefulness of the app for my stakeholders!
Nope, no metadata filtering or label filtering.
One thing I did notice that seems like a bug that i suspect is causing issues,
the chunking process when inserting in to a retool vector seems to be broken.
For instance, I have a 10,000 word document being inserted as a document in to a Retool Vector, and it's not getting chunked at all.
Whereas, 3 months ago when I did this, it was getting chunked in to 10/20/30 separate chunks.
This has definitely hurt my RAG app's performance and may be the reason the query with vectors is only pulling one or a few documents as context.
I can relay this to our engineering team and see if anything may have changed with how data is inserted into Retool Vectors and the data chunking process.
The chunking process did change! The TLDR is that we expanded the chunking window.
From our engineers: " We recently changed how things were chunked to make chunks way closer to the max size allowed by the AI providers APIs. This means chunks of english language text are usually around 30kb, not 1-2k characters.
If your document is not significantly larger than 30kb, you're only gonna end up with 1 chunk. I'm looking at uploads of 2.5mb files and it's chunked into ~80 pieces"
This should not have any effect on the effectiveness of the model to find the correct solution.
In terms of the original question on the number of documents inside of vectorsContext, I can keep looking into that. Did you change the temperature on the AI Query?
I just tested out a question with the same model as you are using with a single document in a Vector and it gave me 15 items inside of vectorsContext so I am very surprised you are getting so few items. Maybe the question is extremely specific and the agent is not able to find a lot of context?
