I'm working with Workflows and using a Loop Block with Parallel execution mode. I have an array as input, where each object contains a fileName + fileType
Each loop iteration reads a file from Google Cloud Storage (GCS) using the fileName.fileType as the key, and the output is an array of base64-encoded files.
My question: Does the Parallel mode guarantee that the output array keeps the same order as the input array? Or is there a chance that the files are processed at different speeds and return in a different order?
In Parallel execution mode, Retool does not guarantee that the output array will maintain the same order as the input array. Since tasks run asynchronously, some files might take longer to process than others, causing the results to return in a different order.
Potential Order Issues in Parallel Mode:
If some files are larger, they may take longer to process.
Network latency can cause certain files to return later.
Google Cloud Storage (GCS) might have variable response times.
How to Ensure Order is Maintained:
Option 1: Map the Results Back to Their Index eg inputArray.map((file, index) => ({ ...file, originalIndex: index })). In the loop, process each file normally. After the loop finishes, sort the results back by originalIndex before returning the final output.
Option 2: Use Sequential Execution Instead
If order is critical and performance isn't an issue, switch from Parallel to Sequential execution mode in the Loop Block. This guarantees each file is processed in order, but will be slower.
The documents give you a basic understanding of this.
When running a loop in Parallel mode, we don't make any guarantees about the order of execution but the output array will always be a direct mapping of the input. This means that the order is preserved.
