Based on the Python code I'm guessing your are building a Workflow, correct?
It's important to keep in mind that the Workflow Editor can only keep track of 258MB of data in memory (see the Workflow usage limits). Assuming that you're converting more than one images to bytes, you're memory usage is going to ramp up pretty quickly.
I don't know the Google Vision API, but I would recommend to use image URLs instead of byte arrays to save up on space in the Workflow.
Hope this helps!