Uploading Large Files on Cloud Retool

đź’¬ Queries and Resources
1

  1. My goal: I’m trying to upload quite large files (100-250MB) files. I don’t want to save these in storage, just immediately pass them through to an external API. (say,a POST to example.com/upload_json (currently self-hosted, so no file size limits there). A request body would be sourced from {{ bigFileUploader.value }} and look something like this:
    [{
    ”base64Data”:

    "LFF1Z <250 MB of Base 64 encoded data here> sLCwsLCwNCg==",
    “name”: "test_small.csv",
    “sizeBytes” : 555555,
    “type”: "text/csv"
    }]

  2. Issue: This approach seems to work fine for medium sized files (~70 MB). However, on larger files (e.g. 225 MB) this seems to fail silently.

  3. Steps I've taken to troubleshoot: Isolating the issue, I guess?

  4. Additional info: (Cloud or Self-hosted, Screenshots) This is on Cloud. The file uploader’s “limit” is at 250MB. I’m using an isolated page with just the upload drop box and manually testing what would be a triggered query.

What can I do to work around this? Do I need to do uploading entirely off of Retool or could I use a custom react component?

3

Sent this response on another thread recently and think it applies. Let me know if any Q’s

So the issue you’re running into is that there’s a 40mb max upload for files that pass through retool. You could use S3 as your resource because it uses signed URLs making the upload occur through your browser. hope that helps!