Retool Vectors - Adding URLs not working

Discussion
1

Hi. I am trying to add URLs to Retool vectors. However, it keeps saying that No URLs found. That is very surprising as it was working a while back.

2

Looks like we should have a fix for this released in one of the upcoming releases. Thank you for reporting!

3

Can you share more context on this fix? I'm having a similar but potentially unrelated issue, where Retool only indexes a handful of URLs, when there are definitely dozens more URLs available to be identified in the query. Is this related at all?

4

This would be fixing vector that are breaking for long URLs. Are you getting any error messages? @hstan

5

Not really, my issue is a bit more encompassing.

My use case:

  • Enter a top level URL (https://amplitude.com) and index all available site data
  • Create vector embeddings of all site data
  • Run a prompt query to OpenAI API with the vector embeddings to generate answers to specific questions about the site content

The issue:

  • When I submit a top level URL, the result is often:
    • Incomplete - the fetch only finds 5-10 URLs, when I expect it to find dozens
    • Empty - no results are returned at all (even if the site allows robots.txt)
    • Hanging - fetch hangs for 15-20 mins, with no or limited results
    • Inconsistent - the vectors fetch returns very different URLs for the same top level URL entry

I want to like the vectors feature but it's really hard to rely on it for any production use cases so far.

If this is beyond scope of this thread I'll open a new topic for discussion.

6

Added a new post for discussion here. I appreciate any support! Retool URL vectors - unexpected behavior with fetching URLs and inconsistent results

1 Like