S3 upload not working in embed

Hello, I'm having an issue with S3 upload in Retool embed. The retool app is embedded using the react-retool package. When uploading a file, the request to

retool.our-domain.com/api/sign_s3_upload?

returns with the following error

{"error":true,"message":"no access key id given"}

The upload works when I open the app in Retool. Is there maybe any additional configuration we might be forgetting to set?

Thank you for any help, also happy to provide any additional info.

Tom