Upload signatures to Baserow

gscc · December 19, 2022, 8:55am

Hello, I have a question now.
It's just that when I try to upload a signature to Baserow, I don't know how to stuff the data into the body of the query.

The signature does not provide a File attribute and I am trying to convert the base64 to a File object but I am having other problems.

function sign2File(base64) {
  let b = atob(base64),
    bLen = b.length,
    bArr = new Uint8Array(bLen);
  while (bLen--) {
    bArr[bLen] = b.charCodeAt(bLen);
  }
  return new File([bArr], "sign.png", { type: "image/png" });
}
await upload.trigger({
  additionalScope: {
    file: sign2File(signature.value),
  },
});

The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined

Or maybe I do.

await upload.trigger({
  additionalScope: {
    file: atob(signature.value),
  },
});

error: "ERROR_INVALID_FILE", detail: "No file has been provided or the file is invalid."

Or

await upload.trigger({
  additionalScope: {
    file: { data: atob(signature.value) },
  },
});

error: "ERROR_INVALID_FILE", detail: "No file has been provided or the file is invalid."

How do I convert it to meet my needs?

By the way, I used Baserow's built-in upload function and only saw that he uploaded a very simple formData.

jonathanbredo · December 19, 2022, 6:03pm

Hey @gscc!
It looks like you're inputting the component to the REST endpoint - that won't fly!
If you input atob(file.value[0]), you might get a step further :-).

gscc · December 21, 2022, 12:24am

No, I'm not using File input; I'm using Signature's components.
My problem is that even if I convert the signature to a File object in the normal way, it won't transfer.
Incidentally, even when I tested uploading images using the File input component, it still did not upload properly.

jonathanbredo · December 21, 2022, 4:12pm

Alright, I figured out where the gap might be; looks like it's due to malformatted Content-Type submitted by Retool.

This request from Postman is accepted by Baserow;

And this is the request replicated in Retool;

I think we need to loop in the support team have a look.

gscc · December 23, 2022, 6:02am

Is there an official person who can help me with this issue?

victoria · December 23, 2022, 6:30pm

I’ve been summoned!

Hello Could you share a screenshot of your entire query in Retool? Do you have a Content-Type specified anywhere?

Also, it would be extra helpful if I could step into your app to take a look. If this sounds okay with you, feel free to DM me a link to your app! If not, no worries at all.

gscc · December 28, 2022, 8:07am

I tried to use camera to upload photos this time, but I got the same result.

{
  "request": {
    "url": "https://.../api/user-files/upload-file/?_=1672214600217",
    "method": "POST",
    "body": {
      "_overheadLength": 143,
      "_valueLength": 251,
      "_valuesToMeasure": [],
      "writable": false,
      "readable": true,
      "dataSize": 0,
      "maxDataSize": 2097152,
      "pauseStreams": true,
      "_released": true,
      "_streams": [],
      "_currentStream": null,
      "_insideLoop": false,
      "_pendingNext": false,
      "_boundary": "--------------------------951425178139047818271962",
      "_events": {},
      "_eventsCount": 1
    },
    "headers": {
      "User-Agent": "Retool/2.0 (+https://docs.tryretool.com/docs/apis)",
      "Authorization": "---sanitized---",
      "X-Retool-Forwarded-For": "59.124.22.27",
      "x-datadog-trace-id": "1908630530762329236",
      "x-datadog-parent-id": "2383080416677400499",
      "x-datadog-sampling-priority": "1",
      "ot-baggage-requestId": "undefined",
      "content-type": "multipart/form-data; boundary=--------------------------918648142937068518671229"
    }
  },
  "response": {
    "data": {
      "error": "ERROR_INVALID_FILE",
      "detail": "No file has been provided or the file is invalid."
    },
    "headers": {
      "date": [
        "Wed, 28 Dec 2022 08:03:22 GMT"
      ],
      "content-type": [
        "application/json"
      ],
      "content-length": [
        "91"
      ],
      "connection": [
        "keep-alive"
      ],
      "allow": [
        "POST, OPTIONS"
      ],
      "referrer-policy": [
        "same-origin"
      ],
      "vary": [
        "Origin"
      ],
      "x-content-type-options": [
        "nosniff"
      ],
      "x-frame-options": [
        "DENY"
      ],
      "strict-transport-security": [
        "max-age=63072000; preload"
      ],
      "cf-cache-status": [
        "DYNAMIC"
      ],
      "server-timing": [
        "cf-q-config;dur=5.9999874792993e-06"
      ],
      "report-to": [
        "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=KWfx08IjYsqSQtsBr6Y6vxcs4RRbnsF8b9gYzDf%2FM1dyHfOP9ZEKLvysszqtCFYdUl6aX8m4hqLxfqF%2BlniC%2FHTvtuPWVBkdSV%2F2cpr%2F1OyiDinj77Hg%2BBXzvaNuwazBzQ%3D%3D\"}],\"group\":\"cf-nel\",\"max_age\":604800}"
      ],
      "nel": [
        "{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}"
      ],
      "server": [
        "cloudflare"
      ],
      "cf-ray": [
        "7808c76da9d7c5e9-SEA"
      ],
      "alt-svc": [
        "h3=\":443\"; ma=86400, h3-29=\":443\"; ma=86400"
      ]
    },
    "status": 400,
    "statusText": "Bad Request"
  }
}

victoria · December 28, 2022, 10:20pm

Thank you for sharing that! To double check, it looks like your API accepts an image/png Content-Type, is that correct? If so, your request from Retool will need to specify that in the query headers! Currently in your query, you don't have a Content-Type header specified so maybe we can try that next.

Second question, do you know if you're uploading a JPG or JPEG? Currently, we don't base64 encode JPEGs that are sent with an image/jpg Content-Type. We do base64 encode JPEGs that are sent with an image/jpeg Content-Type.

gscc · December 30, 2022, 3:14am

I tried to add content-type to the header and got this tragedy.

gscc · December 31, 2022, 4:52pm

I tested it with normal JavaScript.
The request below works fine for uploading files.

But I have absolutely no way to make the same request using Retool's query.

victoria · January 5, 2023, 6:28am

Got it, thank you for sharing that! If it’s possible in Postman/similar services, it should be possible in Retool. I think our next step is to recreate the working external query as closely as possible in Retool (even if that means hardcoding certain values). Would it be alright if I stepped into your app to try a few configurations? If so, DMing me a link to your app would be perfect. If not, no worries at all! We can work together here

jonathanbredo · January 5, 2023, 6:51pm

@victoria the problem still is that when uploading a file from Retool, you can only specify the file data and the file name, not the content-type. This is manually set by Retool as application/octet-stream, as shown in my examples from Retool and Postman respectively :-).

Most file handling services will validate that the submitted file's content-type is equal to the expected file type by the extension of the file name, and then that it produces a valid file of that specific format. The Retool file submission seems to be built with the (faulty) expectation that these file services will base it solely on the file name.

Does this make sense? If not, we can jump on a quick call and I can show you :-).

Jonathan

victoria · January 17, 2023, 3:04pm

Ahh, I see. Thanks, Jonathan Will track this is a bug. Ideally, it would be best to specify the File type when uploading?

jonathanbredo · January 17, 2023, 3:26pm

Exactly! If the object looked like this instead, it would solve this issue;

{
    data: binary string,
    filename?: string,
    type?: string
}

victoria · January 23, 2023, 9:47pm

Just wanted to follow up here. I spoke with my team about this and it generated some questions!

1. Why is the Form Data type being used over something like Raw?

2. Do either of y'all have a screenshot of your Postman input?

jonathanbredo · January 24, 2023, 1:20pm

It's very common for APIs to expect files uploaded as multipart/form-data. The "why" here is a tad out of place, and you already support this method halfway, just lacking the last element. A big reason why a lot of APIs are designed this way, is that you have an RFC standardised way of transmitting file type and file name, instead of putting it somewhere non-standardised, in a raw request.

Postman just looks like this;

Notice the third column - that's the one not existing on Retool.

victoria · January 30, 2023, 10:26pm

https://gifs.intercomcdn.com/c-nWeWlKnS4AAAAC/noted-duly-noted.gif(image larger than 4 MB)

victoria · January 30, 2023, 10:26pm

Got it, thank you so much for taking the time to spell this out for me! Filed internally + will post in this thread with updates

pybanaszak · February 27, 2024, 2:34pm

Hello, any update on this issue ? I'm also stuck with this default header issue.
Any workaround ?
Thanks

steveaustin · March 4, 2024, 11:02pm

@victoria Any updates here? We're also stuck on this issue with the signature component. Any help would be appreciated!