Retool has about a 50MB upload limit using its various File components. You can use fetch() to send larger files, but you have to convert them to blobs first and there are practical limits on file sizes there as well.
The solution is to use the browser's File Input type. But for that you need a Custom component and they can be difficult to integrate into Retool; getting it to look like the same UI and doing the proper communications between Custom components and Retool.
This tutorial shows you how to accomplish this.
Here is the finished app. You can't even tell that there is a Custom component in there.
First we need to add the custom component, called ccUploadtoS3, and throw in this code. it is heavily commented so you can follow what it is doing:
<style>
* {
font-family: 'Inter', sans-serif;
font-weight: 500;
font-size: 12px;
line-height: 16px;
}
body {
margin: 0;
}
.button-container {
display: flex;
align-items: center;
gap: 8px;
margin-top: 12px;
}
.button-disabled {
background-color: #bbbbbb;
color: #ffffff;
cursor: not-allowed !important;
}
.button {
border: none;
border-radius: 4px;
padding: 8px 16px;
font-weight: 500;
font-size: 12px;
cursor: pointer;
outline: none;
appearance: none;
user-select: auto;
}
.button-main {
background-color: #3C92DC;
color: #ffffff;
}
.button-main:hover {
background-color: #4DA3ED;
}
</style>
<!-- This is the HTML or the component -->
<div class="button-container">
<button class="button button-main"onclick="document.getElementById('file-input').click()" id="select-button">Step 1. Select a File</button>
<input class="button button-main" value="Step 1." type="file" id="file-input" style="display:none"/>
<button class="button button-disabled" id="prep-button">Step 2. Prepare File</button>
<button class="button button-disabled" id="upload-button">Step 3. Upload</button>
<button class="button button-disabled" id="reset-button">Reset</button>
</div>
<script type="text/babel">
// Throw all of our DOM elements into variables to make them a tad easier to use later on.
const fileInput = document.getElementById("file-input");
const prepButton = document.getElementById("prep-button");
const uploadButton = document.getElementById("upload-button");
const resetButton = document.getElementById("reset-button");
const selectButton = document.getElementById("select-button");
const log = document.getElementById("log");
const reader = new FileReader();
// Get a hook into the Model and inject model.signedUrl into the uploader
let uploader;
window.Retool.subscribe((model) => uploader = uploadData.bind(null, model.signedUrl));
// Change Handler for the File input
fileInput.addEventListener("change", () => {
// Load the file into memory
reader.readAsArrayBuffer(fileInput.files[0])
// Send the file name back to Retool
updateFileName(fileInput.files[0].name)
// Update the Status so the user is informed
updateStatus("Ready to Prep File")
// Manage Button state
disableButton(selectButton)
enableButton(resetButton)
enableButton(prepButton)
});
prepButton.addEventListener("click", () => {
// Run query that gets S3's signed URL
window.Retool.triggerQuery('getSignedURL')
// Update the Status so the user is informed
updateStatus("Ready to Upload")
// Manage Button state
disableButton(prepButton)
enableButton(uploadButton)
});
// Click handler for the Upload button
uploadButton.addEventListener("click", () => {
// Double check we have a file to upload (optional)
if (fileInput.files.length > 0) {
// Update the Status do the user is informed
updateStatus("Uploading...")
// Upload the file to S3
uploader(reader.result, fileInput.files[0].type);
// Manage Button state
disableButton(uploadButton)
}
});
// Click handler for the reset button
resetButton.addEventListener("click", () => {
// Clear File input
fileInput.value = null
// Tell Retool there is no longer a file in play
updateFileName(null)
// Update the Status do teh user is informed
updateStatus("")
// Manage Button state
disableButton(resetButton)
enableButton(selectButton)
disableButton(prepButton)
disableButton(uploadButton)
});
async function uploadData (url, data, type) {
// url was injected earlier and comes from the getSignedUrl query
const request = new Request(url, {
method: "PUT",
headers: {
"Content-Type": type,
},
body: data,
});
// This does the uploading
const response = await fetch(request);
// Update the Status do the user is informed
updateStatus("Uploading is Complete...")
}
function updateStatus(status) {
// Updates the model so that Retool can see it.
window.Retool.modelUpdate({ status: status})
// Fires a query over in Retool that updates the status
window.Retool.triggerQuery('onStatusChange')
}
function updateFileName(fileName) {
// Same as above, we cannot send additionalScope to the query from a Custom component
// so this technique is a workaround. See onFileNameChange query for the rest of the story
window.Retool.modelUpdate({ filename: fileName })
window.Retool.triggerQuery('onFileNameChange')
}
function enableButton(btn) {
//Just some DOM stuff to change the button style
btn.disabled=false
btn.classList.remove("button-disabled");
btn.classList.add("button-main");
}
function disableButton(btn) {
btn.disabled=true
btn.classList.add("button-disabled");
btn.classList.remove("button-main");
}
</script>
This is the Modal to use:
{
"signedUrl": {{getSignedURL?.data === null ? "" : getSignedURL.data.signedUrl}},
"filename": "",
"status": ""
}
Now add two temp state vars, currentFile and uploadStatus.
We need two communication handlers. These are triggered within ccUploadtoS3 to send data back to Retool.
Make one called onFileNameChange with is one line of code
currentFile.setValue(ccUploadtoS3.model.filename)
And make one called onStatusChange.
uploadStatus.setValue(ccUploadtoS3.model.status)
Drop in Text component that will show you the file name. Set its value to ##### {{currentFile.value}}.
And we need some way to get status back from ccUploadtoS3, so add an Alert component and set its title to Upload Status and its Description to {{uploadStatus.value}}
We need a query that will get the Signed URL from S3 which is needed to send a file there (don't ask me why, just following directions!) You will need to properly set up the S3 Resource, but that is beyond the scope of this tutorial - there are other resources for that. But the query looks like this:
Finally, assuming you will be doing something after the file is uploaded add a Button and set its Disabled property to {{!uploadStatus.value.includes('Complete')}}. This will enable the button only if ccUploadtoS3 has uploaded the file. Add whatever handler to the button you need for your application.
You now should have a fully functional S3 uploader with some UI tricks.
- Click the Select a File button
- A file picker will pop up for you to select a file.
- The file name is sent back to Retool.
- The file name is displayed.
- Status will update
- The Select button will disable
- The Prepare button will enable.
- Reset button will enable
- Click the Prepare button.
- S3 is prepped for the file.
- Status will update
- The Prepare button will disable
- The Upload button will enable.
- Now you click the Upload button.
- Status will update.
- File will upload.
- Status will update again.
- Upload Button will disable.
- Process File button will enable.
- You can click the Reset button any time after the file selection and all will reset and roll back to the beginning of the cycle.
Possible Improvements
I would really like to be able to run getSignedURL when the file is selected like so:
fileInput.addEventListener("change", () => {
reader.readAsArrayBuffer(fileInput.files[0])
updateFileName(fileInput.files[0].name)
window.Retool.triggerQuery('getSignedURL')
});
The problem is the file name is not updated in Retool fast enough and getSignedURL will be run with no, or the previously used, file name. I tried a sprinkling of Async/Await but I can't seem to get things reliably working.