Increase file upload size on self hosted, internal server error

Hey all,

Ok, so I'm trying to upload videos to Vimeo, like a few other posts I've seen. Didn't see a solid solution though. I'm on a self hosted install, created with the AWS auto installer.

What is odd is that I get a straight-up "Internal server error" when I upload a file that is too big (~74MB is enough to fail) in the rest query. Works great with small enough files though. In the docker logs I get the following:

{"message":{"type":"UNCAUGHT_ERROR","stack":"PayloadTooLargeError: request entity too large\n    at readStream (/snapshot/retool_development/node_modules/raw-body/index.js:155:17)\n    at getRawBody (/snapshot/retool_development/node_modules/raw-body/index.js:108:12)\n    at read (/snapshot/retool_development/node_modules/body-parser/lib/read.js:77:3)\n    at jsonParser (/snapshot/retool_development/node_modules/body-parser/lib/types/json.js:134:5)\n    at bodyParser (/snapshot/retool_development/node_modules/body-parser/index.js:109:5)\n    at /snapshot/retool_development/node_modules/dd-trace/packages/datadog-instrumentations/src/router.js\n    at bodyParser (/snapshot/retool_development/node_modules/dd-trace/packages/datadog-shimmer/src/shimmer.js)\n    at Layer.handle [as handle_request] (/snapshot/retool_development/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/snapshot/retool_development/node_modules/express/lib/router/index.js:317:13)\n    at /snapshot/retool_development/node_modules/express/lib/router/index.js:284:7\n    at Function.process_params (/snapshot/retool_development/node_modules/express/lib/router/index.js:335:12)\n    at next (/snapshot/retool_development/node_modules/express/lib/router/index.js:275:10)\n    at compression (/snapshot/retool_development/node_modules/compression/index.js:220:5)\n    at /snapshot/retool_development/node_modules/dd-trace/packages/datadog-instrumentations/src/router.js\n    at compression (/snapshot/retool_development/node_modules/dd-trace/packages/datadog-shimmer/src/shimmer.js)\n    at Layer.handle [as handle_request] (/snapshot/retool_development/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/snapshot/retool_development/node_modules/express/lib/router/index.js:317:13)\n    at /snapshot/retool_development/node_modules/express/lib/router/index.js:284:7\n    at Function.process_params (/snapshot/retool_development/node_modules/express/lib/router/index.js:335:12)\n    at next (/snapshot/retool_development/node_modules/express/lib/router/index.js:275:10)\n    at /snapshot/retool_development/backend/transpiled/server/modules/loggerMiddleware.js\n    at /snapshot/retool_development/node_modules/dd-trace/packages/datadog-instrumentations/src/router.js"},"level":"error","timestamp":"2022-09-25T02:21:11.950Z"}

I've added CLIENT_MAX_BODY_SIZE: 5000M (I know that's huge, hoping to manage some big video files...) to the https-portal section in the docker-compose.yml. I've removed and restarted the compose file, but I still get the internal server error. I have also configured the https-portal section and the env file to use my subdomain, that's not working either and I wonder if that's related somehow?

Also, I'd get a complete crash when uploading a too big file until I added a swap disk. Would have to reboot the server.

Any help would be awesome! I assume I'm missing something?

Hi @Drew!

At the moment there is a hard-coded 50MB limit to requests sent through the backend that our dev team is looking into, I can let you know here when we have an update. The only way around this I'm aware of at the moment is to make the request directly from your browser so it avoids the backend entirely.

@Kabirdas Ah, gotcha. Thought I was going crazy. Yeah, I guess I'll take a look at a custom iframe solution in the meantime. Thanks!

1 Like