I am trying to upload large documents to Google Drive via retool. Browse for local file(s) -> select google folder -> iterate through imported files -> send a POST query to Google to get a resumable upload URI -> send a PUT query with the returned URI from POST response ->move to next file in array.
My problem is that I cannot seem to find a way to wait until metadata has updated before firing the PUT. Google API sends the URI back in the header of its response instead of the body and there doesn't seem to be a way to pass that new metadata into onSuccess. So my PUT URI is outdated and fails. I've played around with promises but they have no produced different results.
Are you able to share what your query currently looks like? You may need to enable the following feature on the "Advanced" tab of your JS query in order to get the updated metadata after calling .trigger()
It appears "Keep variable references inside..." has solved part of the problem. But, the first call is pulling a non-updated version and so every value after that is still mismatched.
Here is the jsQuery code. Our team is currently evaluating retool, so we're not fully familiar with all aspects of retool. Please let me know if there's something about this approach that could be done in simpler / more efficient way.
I believe that this is an oddity about they way that .trigger() works, so you are passing the "old" state of the model into the onSuccess. I think the best way around this would be to do something like this:
So we are waiting for query9 to finish before calculating the value to pass to putFile() rather than calculating it when query9 is running, if that makes sense
This seems hopeful! I'm still getting an error, but it may be some other error in my call because the URI's are matching. The error is 400 "Not Found" so its hard to glean what the error might be.
How does retool handle local uploads via File Button? I had assumed the string values stored in the values array were base64, but that's just been an assumption. As of now i'm directly injecting that string value into the body of my PUT query with Content-Transfer-Encoding : base64 in the header.
Yup, that is correct. The .values property of the File Button is base64 of the file contents, so you should be good to do it that way.
You may want to check in the left panel to confirm that the request is being formed correctly, and all of the interpolated values are actually making it in
So the outstanding error after resolving the async issue with .trigger(). I'm not sure where the error was, I think it had something do to with how I was pulling the URI substring into the query.
Files are moving from retool into google drive! But...something about the files is corrupted. They are retaining their names, types, and sizes... but fail to open, or open blank. Not sure you'd know the cause. So close, yet so far away haha.
Here is the code currently running:
const values = fileInput1.value;
const fileInfo = fileInput1.files;
const folderId = state4.value;
/**
Iterate through uploaded files, merge value and file into a single object,
and call a POST and PUT query to Google Drive to upload files to a specifc
Google Drive folder.
- POST query is a resumeable upload. Metadata is sent to server and
a URI (upload_id) is returned via the query's response header in Locatiion[0].
- TODO: upload_id may be something that can be specified so that async
isses can be avoided.
- A PUT Query is then called via promise (.then(()=>...)) so that the Query's
header has time to be updated with the reponse header's URI link. Normally
the next query could be called interally via the query's onSuccess(data) function,
but the URI is located in the response'sheader instead of in the response's body.
- Instead of passing the entire URI in addtionalScope, a substring of the URI containing
the upload_id is passed. Retool's Query force inputs the base URL of the api, so the
entire URI cannot be passed as a single string. Instead we generate it by using the
api key and upload_id key.
BUGS: While data transfer to specific folders has been achieved, the files are currently
corrupted and not opening. File size is correct on Google's size, file names and
types are retained, but files cannot open (images) or open blank (excel files).
*/
function getFiles(value, files){
var fileInfo = [];
for(var i = 0; i < values.length; i++){
fileInfo[i] = {'name': files[i].name,'type': files[i].type, 'data': value[i], 'size': files[i].size}
postFile(folderId, fileInfo[i]);
}
}
function postFile(folderId, fileInfo){
query9.trigger({
additionalScope: {
fileType: fileInfo.type,
size: fileInfo.size,
name: fileInfo.name,
parents: [folderId]
},
onSuccess: function(data){
},
onFailure: function(error){
console.log(error);
}
}).then(() => {
putFile(query9.metadata.headers.location[0], fileInfo)
})
}
function putFile(uploadId, fileInfo){
console.log("ENTERING PUTFILE");
query14.trigger({
additionalScope : {
upload_id: uploadId.substring(uploadId.indexOf('upload_id')+10),
size: fileInfo.size,
data: fileInfo.data
},
onSuccess: function(data){
console.log("FILE UPLOADED");
},
onFailure: function(error){
console.log(error.data.url);
}
})
}
return getFiles(values, fileInfo);
One thing I noticed is that you declare const fileInfo at the top of your JS query, then have a locally-scoped var fileInfo in the getFiles function. I am not certain that this would cause issues, but it is definitely a possibility.
Also in getFiles you are referencing value[i] instead of values[i] -- Which I believe is the reason why the files in drive are missing their contents
I finally found the issue! Content-Transfer-Encoding in the PUT header was the culprit! Switched it to Content-Encoding and the Google finally decoded the base64 properly. I realized what was going on after opening an uploaded file in a text editor and noticed the contents were an untouched base64 string. Thanks for all your help!
I should note that this header key:value pair was temporarily missing when I screenshotted the PUT Query.
haha.
