A problem I'm observing with my use case - when creating a URL vector, the vector is crawling way more data than I need it to from the source URLs. So much so, that I'm noticing a material degrading in the quality and consistency of the AI generation responses that use these vectors.
For example, the response is returning perhaps the first answer it finds in some linked URL, but not necessarily the best answer that I know is on the URLs I've directly provided.
I would much rather create a vector with ONLY my specific URLs, not all links found at these URLs. Is there a way to disable this?
I know it's possible to exclude specific URL paths, but I would rather categorically exclude any URLs that are not my direct input.
Didn't realize you had posted 
