Upsert to Vector Store with dynamic metadata

Hi retool community,

I'd need to loop through a set of database records and then Upsert each as its own document with unique metadata to a vector store. However, it appears like the Upsert functionality does not allow for dynamic metadata content like it does for the document content.

Is this a known limitation or am I missing something? Thanks!

Ya I'm not sure why it's that way, annoyingly the query will still run successfully even if you do try and reference some outside variable:

It actually inserts/updates the document too, but if the metadata is invalid it overwrites whatever was in there with null or undefined... i'm not sure which

unfortunately the only workaround I've found is to have a seperate query for every metadata value and use a JS Query to run logic and decide which 'upsert' query to use. it's not a very useful workaround and i don't suggest it unless you only have a couple possibilities just because it'd be a mess/nightmare to look at, let alone keep updated for any changes made.

I posted about this too :frowning_face:

1 Like

ahhhh known bug, thanks for linking the thread! I wonder if the filters could be used as a workaround.... u'd have to know all possible metadata values, but at least u can put JS in that field :rofl:

This has been a popular topic recently! Thanks for the bump, @Steve7771. :+1: I've updated our internal ticket tracking this request and will provide an update here as soon as I can.

2 Likes

Hey @Steve7771 @bobthebear @khill-fbmc — Alex here from the Retool team :wave:

Some good news: I found a workaround for this issue. Posted it here in the other thread: Vector DB Dynamic Filtering - #5 by alex

Hope this helps!

1 Like

Hi Darren,

Thanks for sharing the workaround.

Any idea how far out before this bug ticket is picked up? I just want to save myself the time implementing the workaround if it's around the corner.

Thanks!

Welcome to the community, @kyle_little! I don't have any concrete timeline, which might be an indicator to move forward with the workaround if you really need this functionality. :sweat_smile:

As soon as I have more info, I'll be sure to share here.