How to read in a CSV file from S3

New to Retool and trying to have it read a CSV on S3 into a table. Currently my CSV content type on read in is binary/octet-stream. IsJson is false.

For the table component Data I've tried both {{ S3_read_file.data }} and {{ S3_read_file.data.Body }} (eg body in the octet-stream format is "xlfioefi") Both return Error: Invalid Value.

I did see this post re rendering S3 octet-stream images as base64 for the image component. Is there an equivalent solution for reading S3 CSVs into tables?

Hey there @ks23ever. The first step here is understanding what format your Retool query is returning. Can you open the property browser (left panel) and see what the query's properties are / send a screenshot?

(Apologies if you've done this already)

Hey, I am having the same issue. This is what my query is returning. I just want to display the contents of my CSV files in a table, as I can normally when I manually upload a CSV to Retool. Any ideas?

Hey @happylinen — it looks like the body of this query return is empty? If you look in the model browser (left panel) does it appear to be empty there as well?

I'm not sure where to find the model browser - could you send a screenshot of how I can access this?

The file itself is not empty. As I can run the 'Download file from S3' query and it downloads the file including all of it's contents. It's just a standard CSV with about 10,000 rows of data.

No problem, the model browser is the left panel opened in the top middle of the editor, and it's really useful for visualizing the data inside components, labeled by the correct syntax to select it.


In my test app, when I read a CSV from S3 this is the format I see which includes the contents of the CSV like this:\

Even returning {{ query.data.body}} of a CSV file does not populate a table (yes, it previews the CSV data when hovering and is in the side panel as yours is above but it doesn't fill the table object with the data). I'm also not sure how to display a CSV file from the bucket (GCP).

Hey @griffxbio — No worries, sorry that was a little confusing. If your CSV is returning as a single string from S3 and you want to display it in a table, you can use the built in library Papa.parse to parse it into a format the table accepts (either an array of objects or object of arrays). In the case of my last screenshot, we will want to pass the data to Papa.parse() and add a configuration objet to allow for headers. The code for this could look like this:

Papa.parse(readFileFromS3.data.Body,{header:true})

Alternatively, you could add this to the queries transformer like this:

2 Likes

This is a massive help @Chris-Thompson - It could be me, but this use of Papa to expose the csv to a table from a cloud bucket would be a very helpful addition to the documentation! Apologies if I missed it. ty

The next breakthrough will be being able to exporting tables as they are displayed and also preserving column order. :wink:

Hey @griffxbio — absolutely, that makes sense, I've added a request for the docs team to add this!