OpenAI Whisper expecting UploadFile received <class 'str'>

[antoine.meilliez]

Hello,

The form suggested by the OpenAI integration for Whisper is casting file as a string while OpenAI API is expected an UploadFile object.

Is there a recommended way to use this integration with existing audio files or ones recorded via the microphone component?

Best regards,

[Tess]

Hi @antoine.meilliez

Thanks for reaching out!

Hmm, I see what you mean We've seen similar issues with other APIs, which we've solved through Justin's workaround here. I don't quite have it working for OpenAI yet, but I'll do some more testing & follow up

[Tess]

Hi there! I wanted to check in to let you know I've attached a bug internally for this

[Tess]

For simply transcribing audio, I did find the Whisper API (https://transcribe.whisperapi.com), which appears to accept uploaded files or file urls. If using this is an option to you, I might try uploading the base64 to something like s3 -> getting a signed url -> and then sending that url to this API

[yordis]

I am facing some similar issue:

but with the OpenAI Files API

Using the Workflow, I load some data from Retool PG database, and create a string. But it is failing right now because it is expecting a file as the value of the file input.

[Fri July 7th 2023 01:33:03.535 am] --- Running query: startTrigger ---
[Fri July 7th 2023 01:33:03.535 am] Evaluating JS query: startTrigger
[Fri July 7th 2023 01:33:03.572 am] --- Successfully finished running query: startTrigger ---
[Fri July 7th 2023 01:33:03.572 am] --- Running query: trainingData ---
[Fri July 7th 2023 01:33:03.989 am] --- Successfully finished running query: trainingData ---
[Fri July 7th 2023 01:33:03.989 am] --- Running query: processData ---
[Fri July 7th 2023 01:33:03.989 am] Evaluating JS query: processData
[Fri July 7th 2023 01:33:03.995 am] --- Successfully finished running query: processData ---
[Fri July 7th 2023 01:33:03.995 am] --- Running query: uploadFile ---
[Fri July 7th 2023 01:33:04.352 am] Error evaluating uploadFile: {"error":{"message":"The browser (or proxy) sent a request that this server could not understand.","type":"server_error","param":null,"code":null}}
[Fri July 7th 2023 01:33:04.352 am] --- Failed running query: uploadFile ---
[Fri July 7th 2023 01:33:04.352 am] --- Hit failure ---

[bce9722]

Hi @Tess
Is there an ETA on the bug fix?

[Tess]

Hi @bce9722, unfortunately I don't have an eta for this one Would you be able to use a Rest API resource type instead of the Open AI integration? There's a similar REST API solution here!

[Christopher_Menzel]

is there a solution already?

[Tess]

Hi @Christopher_Menzel does the REST solution described here work for you?

I don't have an eta for the bug fix yet, but it's still in our queue!

[Robert_Paillasson]

Hey there,

Thanks to @abusedmedia i solved this by doing :

You have to pass the value and not the base64data to make it work !

[Chris_Bailey]

Hey there,

I've been trying to achieve the same thing - at first using the built in connector and then via the REST API query.

I'm using an audio file that is stored in Retool Storage, but for some reason this keeps throwing an error. I've tried a number of different versions of this and gone down some rabbit holes, so was just wondering if anyone here could see where I'm going wrong and why this isn't working

Any help would be massively appreciated

[Mike_M]

Hi @Chris_Bailey, I tried replicating your setup and found no issues with sending an ogg from Retool Storage to the OpenAI transcriptions endpoint. I was able to replicate your error by "accidentally" including a space in the file key:

This would line up with the error saying that it is missing the file input. Hope this helps!