Bug in Chat Component w/ AI - Won't stream and doesn't auto-scroll

[MikeCB]

I've just started using the Chat component with Retool AI - amazing for quickly throwing together a chat bot! I'm on Retool cloud fwiw.

As I've been working with it though, I've run into some bugs or inconsistencies.

I'm using Retool AI with GPT4o and my own API key. When I enable streaming, it still comes as 1 chunk displayed after a long-ish delay. When I check the console, I can actually see that the result comes back as streamed data (an array of string basically), but it doesn't actually stream back. It did when I first set it up, but after a couple uses it stopped.

I've toggled streaming on and off, it doesn't seem to fix it. Any ideas how to get this working?

Another "bug" or something that would be nice to control would be to scroll to the bottom, or the the lowest point of new text when the AI response comes in. Right now if you've got a few entries in the chat, it can look like you never get a response because you need to scroll down to realize the AI has answered. Is there any workaround for this too?

** One final thing, maybe more of a feature request, but it would be great to set the chat to empty on page load. I tried using a query to do it on page load with chat1.clearHistory() && chat1.clearValue(), but it needs the component to load first, so I need to use a delay, which still results in a weird experience where the content flashes and then gets erased.

[Tess]

Hey @MikeCB Thanks for sharing your feedback! Sorry to bug you for more details, but any chance you can share a recording/screenshots of the first two issues?
​

I haven't been able to reproduce this issue with streaming, but it's definitely something I want to flag to our team. Same for the scrolling! That sounds buggy, but so far, I haven't run into this issue

It does look like always starting the chat component with a blank slate will be a feature request. I'll let you know when that ships

[MikeCB]

Hi @Tess,

Here is an example of the streaming issue.

You can see based on the loading icon in the input box, that it's expecting a streaming response (when not streaming, it shows in the chat itself). Here is the setting to confirm.

Here's a screenshot of the request payload and response.

I can't reproduce the scrolling issue right now either, so maybe that was temporary or fixed. Thanks for looking into it, let me know if you need any other info!