Unable to get random bytes for temporary file: No such file or directory

[isdanni]

I have a workflow that clones the single branch main of a github repo using a fine-grained access token, makes some changes to the data folder(.txt files only) and commits the result.

It has been running smoothly until this week I noticed there's an error when cloning the repo: Error: Unable to get random bytes for temporary file: No such file or directory.

I checked the memory usage and /tmp folder is at 1% usage, which means this is not a memory problem, as I've been particularly careful with file size.

[isdanni]

The git version is 2.39.2.

[Darren]

Hey @isdanni! Thanks for reaching out. That sounds like a cool workflow. I'm going to assume that you're running Retool on Ubuntu - let me know if that's not the case.

This particular issue isn't something I'm familiar with but some research suggests it might be related to the presence (or absence) of the /dev/random and /dev/urandom files. They should be created on boot but it's worth double checking. There's a more detailed explanation here, including instructions on how to regenerate these files if necessary.

Let me know if that works for you!

[isdanni]

Thanks for looking into this! It looks like it's Debian GNU/Linux 12(bookworm):

Here's the output of ls:

Output of pwd: /tmp

If you go one level up and run ls:

bin/ dev/ etc/ lib/ lib64/ proc/ tmp/ usr/

[Darren]

Can you try running the command cd /dev && ls | grep random?

[isdanni]

The output of cd /dev && ls | grep random:

[Darren]

Great - thanks for checking that. I would typically expect to see a file called urandom also show up in this list.

Can you run the following commands in sequence and then try to execute the workflow?

mknod -m 644 /dev/random c 1 8
mknod -m 644 /dev/urandom c 1 9
chown root:root /dev/random /dev/urandom

[isdanni]

Thanks for your quick response!

mknod -m 644 /dev/random c 1 8  # output: mknod: /dev/random: File exists

mknod -m 644 /dev/urandom c 1 9   # output: mknod: /dev/urandom: Read-only file system 

# running /dev/random only since cannot create /urandom:
chown root:root /dev/random  # output: chown: chaning ownership of `/dev/random`: Read-only file system

I suspect it's either corrupt .git or network access since I'm cloning with git clone -b master --single-branch --depth=1 https://{fine_grained_access_token}@github.com/{username}/{repo}.git; Using SSH would be better but it's a bit difficult to do with PAT and non-root access on Debian.

Update

Running which git, the output is /usr/bin/git

[Darren]

Can you also run chown root:root /dev/urandom? And try running the workflow afterwards.

[isdanni]

@isdanni

mknod -m 644 /dev/urandom c 1 9

Running chown root:root /dev/urandom only gives chown: cannot access /dev/urandom: No such file or directory.

But if running both mknod -m 644 /dev/urandom c 1 9 && chown root:root /dev/urandom the output is mknod: /dev/urandom: Read-only file system.

[Darren]

Hmm that's definitely odd. I'm still leaning towards it being an issue with randomness due to the original error message that you shared:

@isdanni

It has been running smoothly until this week I noticed there's an error when cloning the repo: Error: Unable to get random bytes for temporary file: No such file or directory.

Are you confident that the error is being thrown while cloning and not at another step of the process? One similar error that I've seen while doing research was resolved by running git config --global core.fscache false in order to clear the cache of any potentially corrupted files, completing the original transaction, and then reenabling the cache. It's worth a shot!

[isdanni]

Yes agree, actually git config --global .. is one of the first few commands I have tried but got the same error; also tried commands like git config --global http.version HTTP/1.1 && git config --global http.postBuffer 128m && git config --global pack.packSizeLimit 50m && git config --global pack.windowMemory 50m && git config --global core.compression because in a few threads the issues were solved because the repo size is too big or buffer size. Doesn't work in my case because the repo I'm trying to clone is very small, does not contain any model or log files: just Python scripts and a few txts.

The issue does happen during cloning, which I agree is very weird because unable to get random bytes/invalid index-pack output mostly happens at git push from issues I've seen.

Is there an option to maybe update/choose workflow image? I think there might be a patch causing this in Debian.

[Darren]

I'm reading back through the thread and actually want to return to your first post. I should have asked this earlier - what kind of deployment are you running? Docker? Kubernetes? Helm? I'm also curious to know which version of Retool you're running.

Depending on the specifics, it may be helpful to check how much disk space is available for the code-executor container, in particular - assuming your deployment is set up with one. If present, this is where all workflows are executed. As such, we may also want to verify randomness generation in that specific context.