Hi there, I'm creating an ETL process where I retrieve data from S3 bucket, transform/cleanse it, and pass it to our system via API endpoint. We've encountered some scaling issue so the API endpoint is now only accepting json file on a form data body. I've tried using Blob object but no luck with my implementation there.
I was trying this on a retool app as well but no luck as I'm just getting this error:
Failed to execute 'postMessage' on 'Window': FormData object could not be cloned.
However, if I do the following it works:
1. create the file in retool
2. download/save to local
3. upload it in upload component in retool
4. use the file in the endpoint
Obviously, this process won't work as we're receiving the files every day and automating it is the ideal course. Any ideas on how to deal with the json file creation in workflows that I can pass in an API endpoint?
Hey @jocen! Can you share how you were doing this in the app when you were getting the 'postMessage' error? This sounds feasible, happy to dig in and help!
Hi @joeBumbaca,
Since it was quite an urgent task, I resorted to the following:
Extract and Transform the data in retool/js
Create file in Retool Storage
Pull the data from the newly created file
Upload the file through the API.
It's working fine with this implementation but there are extra parts that could've been trimmed down. Would be happy to know if there's a better implementation than that one. Here's what I did for creating the file in app - js query:
var dat = dataQuery
var res = []
dat.forEach(x => {
res.push({
param1: x.val1,
param2: x.val2,
param3: x.val3,
param4: x.val4
})
})
var b64 = JSON.stringify(res)
const blob = new Blob([b64], { type: "application/json" })
return blob
This returns nothing and running the endpoint results to this one:
