Apply getResource operation to each element of an array

💬 Queries and Resources
1

My goal: make a gallery of images which are stored in AWS S3. Only S3 paths are given.
Issue: given an array of s3 paths (within the bucket), it 's not possible to use them directly in an image gallery, and there's no apparent way to make an array of presigned URLs or download each element of that array. Only single-object s3 operations exist. Not possible to call resource query for each element of an array form transformer, too.

3

Hi @pavelgur,

To start, is the issue that there is no query for an s3 resource that can request and receive an array of pre-signed URLs?

Can you share screenshots of what you have tried? Are you getting an error message or only able to receive one URL at a time per query?

If the issue is a bottleneck on getting a single pre-signed image URL verses an array of all the URLs, you may need to loop through queries of single object s3 operations. Might need a workflow for that operation to get each URL to store in Retool/locally. May need to re-format the data in s3 from an array to Retool.

Could you give me more details on what you tried to do with the array form transformer?

4

The issue is that this operation takes in single key

If it did receive an array of keys, and produce an array of values instead of just one, that would solve my case. It's really important because without that functionality, I have to proxy that request with a backend call which triples the latency.

5

Hi @pavelgur,

I believe this is a hard limitation of Retool's ability to loop through making an async calls to get the image data from the keys :thinking:

At first I was thinking a JS Query along the lines of

const keys = s3PathsArray;  // Your array of S3 keys
const urls = [];

for (let key of keys) {
  const url = await getPresignedUrl.trigger({ additionalScope: { key } });
  urls.push(url.data);
}

return urls;

But this may not work as i am guessing you have already tried. One option would be to give a workflow the array and then run and use a workflow loop block to get the image URLs. Then you could put those URLs into a Retool DB table and have the frontend grab the image data from there :face_with_monocle:

I can definitely raise this issue with engineering to see if this would be a limitation we could solve with a feature request or if looping through async calls in the frontend is farther down our roadmap :saluting_face:

6

I need a solution, fast. It's a 5 minute fix on the engineering side, just loop though Keys field and it's done. Workflows and DB is an overkill.

7

Ok, I will make a feature request for this.

9

Hi @pavelgur,

I asked one of our engineers about this functionality limitation and they said to me that it should be possible to achieve what you are looking to do by " in JavaScript either by triggering an S3 query with additionalScope or calling a Workflow that loops over webhook params"