SSE Connection then Close

🤝 Discussion
1

I want to share my experience, see what people say.

  1. My goal: Talkdesk has a Live Queries API that streams data. I only need a chunk of this data periodically, so a stream isn't necessary. The data is only available to stream.
  2. Issue: Retool doesn't handle SSE events gracefully. EventSource wasn't working in workflows although some folks say it does.
  3. Steps I've taken to troubleshoot: Followed documentation till it said to create a custom react component. I didn't want to do that so I found a different way.
  4. Additional info: (Cloud or Self-hosted, Screenshots) Cloud
2

This is what I came up with. I removed some of the code relative to my needs. I open the stream, store the data, then close the stream. Let me know if you have any questions. I needed to import the Axios library.

const cancelToken = axios.CancelToken;
const source = cancelToken.source();
async function fetchStreamAndCapture() {
  try {
    const response = await axios({
      method: 'get',
      url: "stream/url",
      responseType: 'stream',
      cancelToken: source.token
    });

    let dataChunks = [];

    return new Promise((resolve, reject) => {
      response.data.on('data', (chunk) => {
        dataChunks.push(chunk);
        resolve(dataChunks.toString());
      });
      response.data.on('end', () => {
        console.log('Stream ended successfully.');
        resolve();
      });
      response.data.on('close', () => {
        console.log('Stream closed.');
        resolve();
      });
      response.data.on('error', (err) => {
        console.error('Error during streaming:', err);
        reject(err);
      });
    });
  } catch (error) {
    if (axios.isCancel(error)) {
      console.log('Request cancelled:', error.message);
    } else {
      console.log('Error:', error);
      throw error;
    }
  }
}

const streamString = await fetchStreamAndCapture()
  .then(data => {
    return data;
  }).catch(error => {
    console.log('Failed', error);
    return error;
  });

source.cancel('Operation cancelled by user.');

const searchSubstring = "data:";

const startIndex = streamString.indexOf(searchSubstring);
if (startIndex !== -1) {
  const result = JSON.parse(streamString.slice(startIndex + searchSubstring.length));
  return result
} else {
  console.log("Substring not found.");
}
3

You need to cancel the connection otherwise the workflow stays open indefinitely, although you do get the desired response even when it stays open.