Getting random 192 MiB OOM errors when parsing CSV files from GCS

💬 App Building
,
1

We're seeing intermittent Code exceeded the 192MiB memory limit (OutOfMemoryError) errors when processing CSV files from a GCS resource.

The files are around 14.3 MB, and the GCS response body is around 20 million Base64 characters.

To isolate the issue, we simplified the code so it only:

  1. Reads the CSV from GCS

  2. Decodes and parses it incrementally

  3. Counts the rows

  4. Discards each row after processing

The strange part is that the exact same file with the exact same code can sometimes pass and sometimes hit the 192 MiB limit.

  1. Is there any way to stream or chunk-read a GCS file in Retool?
  2. Is it expected that the same code/file can sometimes pass and sometimes OOM?
  3. What's the recommended way to handle files around this size?
2

Summary

Intermittent 'Code exceeded the 192MiB memory limit (OutOfMemoryError)' errors occur when JavaScript reads and parses a ~14.3 MB CSV from a GCS resource (a ~20 million-character Base64 response body); the identical code and file sometimes succeed and sometimes fail, and there is a question of how to stream/chunk-read large GCS files.

AI Response

The 192MiB ceiling is a fixed per-execution memory budget for code running in Retool's sandbox, and a ~20M-character Base64 body plus the decoded buffer plus the parsed rows can sit right at that edge — which is why the same code/file passes or fails depending on when garbage collection runs. Retool JavaScript does not expose true streaming for a resource response, so the response body is materialized in memory before your code can process it. The practical workaround is to avoid holding the whole payload at once: request the object in byte ranges / smaller pieces and decode-and-parse each chunk incrementally rather than decoding the entire Base64 body first, or offload the heavy parsing to a Workflow (which has higher, configurable memory limits) or to your own backend. Because the workload is essentially at the memory boundary, reducing peak memory (chunked reads, releasing intermediate variables, not retaining decoded strings) is the reliable fix rather than expecting consistent success at this size.

Sources

:bookmark: Unable to upload large files without running out of memory
A recent solved topic covering the same out-of-memory behavior when moving large files through a resource, with accepted guidance on avoiding the memory ceiling.
:bookmark: Chunking Base64 Files for REST Uploads in Retool with JavaScript
A solved topic demonstrating how to chunk large Base64 file data in Retool JavaScript instead of processing it all at once, directly addressing the streaming/chunk-reading question.

The Community Team is testing out a new automation. Let us know if it's helpful (or not) by leaving a :heart:, :+1:, or :-1:. Or by marking this post as the "Solution"! Let us know if you have any feedback here. :rocket: