Embed PDFs with images graphs and tables in a Vector Database

[mascaritas]

dear community

Is there a recommendation for a service where I can embed PDFs with additional elements than just the text? This would help to create more comprehensive agents for a support chat or similar application.

As I understand it, one can also use external vector databases, if retool vectors don’t offer this feature

Best Greetings

[Paulo]

Hi @mascaritas, you can definitely add images to the PDF exporter, here is how:

Upload to Retool Storage in signature component

Since you already have access to the image on base64 format form the step above, to prevent the image of the signature from being public, we could just reference the output of the JS query, and create the image from it on the PDF using Markdown:  Source: base64 - Hard Code Markdown Images - Stack Overflow

Alternatively, @mstevenson shared a TPA that can help you create more complex PDFs:

PDF Exporter in Retool

we have fully integrated CraftMyPDF into our retool application, it works really well once you have your templates setup. I would recommend it if you need to automate the creation of PDF documents. We use it to automatically complete sales quotes and merge with product datasheets to give the sales staff a complete customer ready quote on the fly. https://app.craftmypdf.com/

[mascaritas]

Dear Paulo

thanks a lot for your suggestions, def something I will also use for other workflows. However what I wanted to ask: how can one embed pdf's including their contained images into a vector-database or better yet retool vectors ?

Or does it require the use of another external service such as pinecone and embedding method X (which I am not familira with)

I hope this makes sense now as a question

Cheers

[Paulo]

Great question! If you are using Retool AI Document Actions, our PDF parser is only parsing raw text for now, so this may not be possible if you need the AI to reference those images.

On the other hand, the PDFs, including their images, could be stored as base64 data. We could then send that data to a LLM but I'm not sure if it would be able to provide accurate information about the underlying text and images on each file.

[mascaritas]

I will test this integration here, it seem to read manuals and tech documents particularly well:

EyeLevel | RAG for Ecommerce

[Paulo]

Hi @mascaritas, how did the integration go?

[mascaritas]

It is a great tool to ingest and understand multimodal pdfs.
I have made a prototype and gave it a 60 page interior design guideline (all with graphics and sketches) which I can ask now to design our house.

I ended up connecting it to a CustomGPT via OpenAPI Action Schema. If you want to have a look I can DM the link.

Best

[Paulo]

I would love to take a look at it. Thank you!

[mascaritas]

I checked and can also provide a preview like this.

how-to:
Just ask it a question like this or any other interior related:
I have two sofas, how shall I put them in a small room? And use the codeword "Ask Buddy" in case it does not trigger custom action.

https://chatgpt.com/g/g-67b0e4e481d48191a56ebb19942455b4-interior-buddy

curios to see what you find...

[Paulo]

It may be a private workspace:

[Jeje]

Just adding +1 here as that would be a really powerful addition to the retool vectors