I have been unable to setup our documentation website to be crawled by Retool's AI Vectors. Our website (docs.redeam.io) does not return an SEO-friendly webpage (a version that the Retool crawler can interpret) unless the User-Agent header provided in the request is a known SEO service (e.g. googlebot), or a particular query parameter is provided in the request to each page (?_escaped_fragment_=).
I would like to have the ability to customize the user-agent header sent by Retool's web crawler, and / or the ability to add custom query parameters to send along with the request to each crawled web page.