Unable to get on-prem docker containers to work/connect

Self Hosted Retool
1

Hello. I've followed the docker curl commands to pull down the retool on-premise images. I've done this on Ubuntu and macOS. I can see macOS might not be supported and that might be another story. But on the Ubuntu side it looks like it "should" work, but I'm not being able to connect into it.

Here is the output form a curl git session. I cannot connect into port 3000. How can I troubleshoot this?

I've tried this about 5-6+ times on Ubuntu so far.

==========

xxxxx@gm-laptop:~/work/git/retool$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/tryretool/retool-onpremise/master/local-trial)"

INFO: setting up install location...

  • /home/xxxxx/retool

INFO: downloading...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 131 100 131 0 0 385 0 --:--:-- --:--:-- --:--:-- 384
100 56682 0 56682 0 0 67158 0 --:--:-- --:--:-- --:--:-- 67158

INFO: unpacking...
Archive: master.zip
f81347cd0a0bf1a5e24b608170d3a72d938ce452
creating: retool-onpremise-master/
inflating: retool-onpremise-master/.gitignore
extracting: retool-onpremise-master/Dockerfile
extracting: retool-onpremise-master/Dockerfile.local
inflating: retool-onpremise-master/README.md
inflating: retool-onpremise-master/app.json
creating: retool-onpremise-master/cloudformation/
inflating: retool-onpremise-master/cloudformation/README.md
inflating: retool-onpremise-master/cloudformation/fargate.private.yaml
inflating: retool-onpremise-master/cloudformation/fargate.yaml
inflating: retool-onpremise-master/cloudformation/retool.yaml
inflating: retool-onpremise-master/docker-compose-local-trial.yml
inflating: retool-onpremise-master/docker-compose.yml
inflating: retool-onpremise-master/docker.env.template
inflating: retool-onpremise-master/docker.env.testing
inflating: retool-onpremise-master/docker.env.trial
inflating: retool-onpremise-master/docker_setup
inflating: retool-onpremise-master/get-docker-compose.sh
inflating: retool-onpremise-master/get-docker.sh
inflating: retool-onpremise-master/heroku.yml
creating: retool-onpremise-master/heroku/
extracting: retool-onpremise-master/heroku/Dockerfile
inflating: retool-onpremise-master/install.sh
creating: retool-onpremise-master/keys/
inflating: retool-onpremise-master/keys/README.md
creating: retool-onpremise-master/kubernetes/
inflating: retool-onpremise-master/kubernetes/retool-container.yaml
inflating: retool-onpremise-master/kubernetes/retool-jobs-runner.yaml
inflating: retool-onpremise-master/kubernetes/retool-postgres.yaml
inflating: retool-onpremise-master/kubernetes/retool-secrets.template.yaml
inflating: retool-onpremise-master/local-trial
creating: retool-onpremise-master/retool/
inflating: retool-onpremise-master/retool/README.md
inflating: retool-onpremise-master/stop-local-trial
creating: retool-onpremise-master/testing/
inflating: retool-onpremise-master/testing/Dockerfile
inflating: retool-onpremise-master/testing/generate.js
inflating: retool-onpremise-master/testing/package-lock.json
inflating: retool-onpremise-master/testing/package.json
inflating: retool-onpremise-master/testing/retool-test.sh
inflating: retool-onpremise-master/testing/wait-for-it.sh
inflating: retool-onpremise-master/update_retool.sh
creating: retool-onpremise-master/userData/
inflating: retool-onpremise-master/userData/userData.env

INFO: setting environment variables

INFO: docker found!

INFO: docker-compose found!
[sudo] password for xxxxx:
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.test-runner: 'profiles'

INFO: updating Retool!
Pulling postgres (postgres:11.12)...
11.12: Pulling from library/postgres
778066204fb7: Pull complete
df352e900918: Pull complete
a7242d94fa18: Pull complete
7583c6f08560: Pull complete
eb70ff3ea1b2: Pull complete
d1409f9a52f6: Pull complete
174aa3d8763d: Pull complete
ee06562012c0: Pull complete
cfabba7d913e: Pull complete
2ea1f79b5724: Pull complete
83521fbf3fb9: Pull complete
d07f3841107d: Pull complete
8952bdfb1398: Pull complete
Digest: sha256:13e23e5f97a42286c5703691e2fbad0806970bd13121fe7c28f32349ee0cf917
Status: Downloaded newer image for postgres:11.12

INFO: running Retool! This can take up to 5 minutes
Building db-connector
Step 1/2 : FROM tryretool/backend:latest
latest: Pulling from tryretool/backend
b4d181a07f80: Already exists
f9bc4241c5e2: Pull complete
db558db6dd4b: Pull complete
b411e2bdc9f1: Pull complete
b982c8eeb328: Pull complete
69b02a53a3d4: Pull complete
f671c955b1a7: Pull complete
a5ce7f697aec: Pull complete
ab6b7445e6df: Pull complete
cb9098055743: Pull complete
7f6548daff00: Pull complete
8a0dcbc5f3dc: Pull complete
b2054ac232b3: Pull complete
bb5d70e1abf8: Pull complete
8bccfd81f7e3: Pull complete
f8876b23f89f: Pull complete
e60e8df1733d: Pull complete
783102ba2843: Pull complete
aabeaedd5906: Pull complete
c2cdfd3a9ad8: Pull complete
d1e0bfc42114: Pull complete
48dd0605a8cd: Pull complete
902186744039: Pull complete
d80f3151bc87: Pull complete
f02da33e58cc: Pull complete
94ce61844711: Pull complete
83789739e402: Pull complete
Digest: sha256:11627b62287badf583066bae1290611f2712083c724e14d3081dba2f5ea849d1
Status: Downloaded newer image for tryretool/backend:latest
---> f9dce11ea606
Step 2/2 : CMD ./docker_scripts/start_api.sh
---> Running in 6872e2978bc1
Removing intermediate container 6872e2978bc1
---> 24ecb9f98d9f
Successfully built 24ecb9f98d9f
Successfully tagged retoolonpremise_db-connector:latest
WARNING: Image for service db-connector was built because it did not already exist. To rebuild this image you must use docker-compose build or docker-compose up --build.
Building db-ssh-connector
Step 1/2 : FROM tryretool/backend:latest
---> f9dce11ea606
Step 2/2 : CMD ./docker_scripts/start_api.sh
---> Using cache
---> 24ecb9f98d9f
Successfully built 24ecb9f98d9f
Successfully tagged retoolonpremise_db-ssh-connector:latest
WARNING: Image for service db-ssh-connector was built because it did not already exist. To rebuild this image you must use docker-compose build or docker-compose up --build.
Building jobs-runner
Step 1/2 : FROM tryretool/backend:latest
---> f9dce11ea606
Step 2/2 : CMD ./docker_scripts/start_api.sh
---> Using cache
---> 24ecb9f98d9f
Successfully built 24ecb9f98d9f
Successfully tagged retoolonpremise_jobs-runner:latest
WARNING: Image for service jobs-runner was built because it did not already exist. To rebuild this image you must use docker-compose build or docker-compose up --build.
Building api
Step 1/2 : FROM tryretool/backend:latest
---> f9dce11ea606
Step 2/2 : CMD ./docker_scripts/start_api.sh
---> Using cache
---> 24ecb9f98d9f
Successfully built 24ecb9f98d9f
Successfully tagged retoolonpremise_api:latest
WARNING: Image for service api was built because it did not already exist. To rebuild this image you must use docker-compose build or docker-compose up --build.
Creating retoolonpremise_db-ssh-connector_1 ...
Creating retoolonpremise_postgres_1 ...
Creating retoolonpremise_db-connector_1 ...
Creating retoolonpremise_postgres_1
Creating retoolonpremise_db-connector_1
Creating retoolonpremise_postgres_1 ... done
Creating retoolonpremise_jobs-runner_1 ...
Creating retoolonpremise_api_1 ...
Creating retoolonpremise_jobs-runner_1
Creating retoolonpremise_api_1 ... done

-- !! RETOOL IS BOOTING !! --
Check out your BROWSER for some awesomeness!!! up>

Navigate to: http://localhost:3000/auth/signup or [publically_accessible_ip]:3000/auth/signup
To STOP Retool, run: /home/xxxxx/retool/retool-onpremise/stop-local-trial
To RESTART Retool, run: /home/xxxxx/retool/retool-onpremise/local-trial

Retool was installed in ~/retool. It will run in the background until you manually stop it. If Retool stops you can restart it without losing your data.
xxxxx@gm-laptop:~/work/git/retool$ telnet localhost 3000
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
xxxxx@gm-laptop:~/work/git/retool$ ifconfig|grep 192
inet 192.168.32.1 netmask 255.255.240.0 broadcast 192.168.47.255
inet 192.168.16.1 netmask 255.255.240.0 broadcast 192.168.31.255
inet 192.168.1.175 netmask 255.255.255.0 broadcast 192.168.1.255
xxxxx@gm-laptop:~/work/git/retool$ telnet 192.168.1.175 3000
Trying 192.168.1.175...
telnet: Unable to connect to remote host: Connection refused
xxxxx@gm-laptop:~/work/git/retool$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9b9316b592b6 retoolonpremise_db-connector "docker-entrypoint.s…" 12 minutes ago Up 12 minutes 3000-3002/tcp retoolonpremise_db-connector_1
90b528dd4d33 retoolonpremise_db-ssh-connector "docker-entrypoint.s…" 12 minutes ago Up 12 minutes 3000-3002/tcp retoolonpremise_db-ssh-connector_1
xxxxx@gm-laptop:~/work/git/retool$ docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6defd7a3ac02 retoolonpremise_jobs-runner "docker-entrypoint.s…" 13 minutes ago Exited (1) 12 minutes ago retoolonpremise_jobs-runner_1
0af4d47f705a retoolonpremise_api "docker-entrypoint.s…" 13 minutes ago Exited (0) 12 minutes ago retoolonpremise_api_1
9b57648c84e4 postgres:11.12 "docker-entrypoint.s…" 13 minutes ago Exited (1) 13 minutes ago retoolonpremise_postgres_1
9b9316b592b6 retoolonpremise_db-connector "docker-entrypoint.s…" 13 minutes ago Up 13 minutes 3000-3002/tcp retoolonpremise_db-connector_1
90b528dd4d33 retoolonpremise_db-ssh-connector "docker-entrypoint.s…" 13 minutes ago Up 13 minutes 3000-3002/tcp retoolonpremise_db-ssh-connector_1
03ec2ab118cd 957a5cad6339 "/bin/bash -c 'curl …" 5 weeks ago Exited (1) 5 weeks ago objective_antonelli
8817f2f57ec7 24e198a7c33a "/bin/bash -c 'curl …" 5 weeks ago Exited (1) 5 weeks ago priceless_morse
67f8dc3761e1 directus/directus:latest "entrypoint" 6 weeks ago Exited (1) 6 weeks ago directus-local
96bdbf46d9c9 mariadb:latest "docker-entrypoint.s…" 6 weeks ago Exited (0) 3 weeks ago mariadb-local
db14d97ef9a2 node-app-image "docker-entrypoint.s…" 6 weeks ago Exited (1) 3 weeks ago node-app
074e42f44829 microsoft/mssql-server-linux:2017-GA "/bin/sh -c /opt/mss…" 2 months ago Exited (137) 6 weeks ago mssql1
3ab1fa1c9f99 portainer/portainer-ce "/portainer" 2 months ago Exited (2) 3 weeks ago portainer
xxxxx@gm-laptop:~/work/git/retool$ docker container ls -a|grep retool
6defd7a3ac02 retoolonpremise_jobs-runner "docker-entrypoint.s…" 13 minutes ago Exited (1) 13 minutes ago retoolonpremise_jobs-runner_1
0af4d47f705a retoolonpremise_api "docker-entrypoint.s…" 13 minutes ago Exited (0) 13 minutes ago retoolonpremise_api_1
9b57648c84e4 postgres:11.12 "docker-entrypoint.s…" 13 minutes ago Exited (1) 13 minutes ago retoolonpremise_postgres_1
9b9316b592b6 retoolonpremise_db-connector "docker-entrypoint.s…" 13 minutes ago Up 13 minutes 3000-3002/tcp retoolonpremise_db-connector_1
90b528dd4d33 retoolonpremise_db-ssh-connector "docker-entrypoint.s…" 13 minutes ago Up 13 minutes 3000-3002/tcp retoolonpremise_db-ssh-connector_1
xxxxx@gm-laptop:~/work/git/retool$ docker exec -ti 9b9316b592b6 /bin/bash
retool_user@9b9316b592b6:/retool_backend$ telnet
bash: telnet: command not found
retool_user@9b9316b592b6:/retool_backend$ nc
bash: nc: command not found
retool_user@9b9316b592b6:/retool_backend$ ifconfig
bash: ifconfig: command not found
retool_user@9b9316b592b6:/retool_backend$ ps
bash: ps: command not found
retool_user@9b9316b592b6:/retool_backend$ ls -la
total 460140
drwxrwxrwx 1 retool_user retool_user_group 4096 Aug 5 20:12 .
drwxr-xr-x 1 root root 4096 Aug 11 16:34 ..
drwxr-xr-x 3 retool_user retool_user_group 4096 Aug 5 20:11 common
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 dbconnector
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 denodo
drwxrwxrwx 1 retool_user retool_user_group 40960 Aug 5 20:12 dist
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 docker_scripts
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 env
-rw-r--r-- 1 root root 72794506 Aug 5 16:37 instantclient-basic-linux.x64-18.3.0.0.0dbru.zip
drwxr-xr-x 1 retool_user retool_user_group 4096 Aug 5 20:11 jdbcdrivers
drwxr-xr-x 2 retool_user retool_user_group 20480 Aug 5 20:11 migrations
drwxr-xr-x 1152 root root 36864 Aug 5 16:41 node_modules
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 oracle
-rw-r--r-- 1 retool_user retool_user_group 11609 Aug 5 20:11 package.json
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 phantom
-rwxr-xr-x 1 retool_user retool_user_group 397689246 Aug 5 20:11 retool_backend
drwxr-xr-x 3 retool_user retool_user_group 4096 Aug 5 20:11 server
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 templates
-rw-r--r-- 1 retool_user retool_user_group 517354 Aug 5 20:11 yarn.lock
retool_user@9b9316b592b6:/retool_backend$ netstat
bash: netstat: command not found
retool_user@9b9316b592b6:/retool_backend$ exit
xxxxx@gm-laptop:~/work/git/retool$ docker exec -ti 90b528dd4d33 /bin/bash
retool_user@90b528dd4d33:/retool_backend$ telnet
bash: telnet: command not found
retool_user@90b528dd4d33:/retool_backend$ netstat
bash: netstat: command not found
retool_user@90b528dd4d33:/retool_backend$ ifconfig
bash: ifconfig: command not found
retool_user@90b528dd4d33:/retool_backend$ ps
bash: ps: command not found
retool_user@90b528dd4d33:/retool_backend$ /bin/ps
bash: /bin/ps: No such file or directory
retool_user@90b528dd4d33:/retool_backend$ ls
autogen_ssh_keys dbconnector dist env jdbcdrivers migrations oracle phantom server yarn.lock
common denodo docker_scripts instantclient-basic-linux.x64-18.3.0.0.0dbru.zip keys node_modules package.json retool_backend templates
retool_user@90b528dd4d33:/retool_backend$ ls -la
total 460148
drwxrwxrwx 1 retool_user retool_user_group 4096 Aug 11 16:34 .
drwxr-xr-x 1 root root 4096 Aug 11 16:34 ..
drwxr-xr-x 2 root root 4096 Aug 10 05:34 autogen_ssh_keys
drwxr-xr-x 3 retool_user retool_user_group 4096 Aug 5 20:11 common
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 dbconnector
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 denodo
drwxrwxrwx 1 retool_user retool_user_group 40960 Aug 5 20:12 dist
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 docker_scripts
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 env
-rw-r--r-- 1 root root 72794506 Aug 5 16:37 instantclient-basic-linux.x64-18.3.0.0.0dbru.zip
drwxr-xr-x 1 retool_user retool_user_group 4096 Aug 5 20:11 jdbcdrivers
drwxrwxr-x 2 node node 4096 Aug 10 17:36 keys
drwxr-xr-x 2 retool_user retool_user_group 20480 Aug 5 20:11 migrations
drwxr-xr-x 1152 root root 36864 Aug 5 16:41 node_modules
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 oracle
-rw-r--r-- 1 retool_user retool_user_group 11609 Aug 5 20:11 package.json
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 phantom
-rwxr-xr-x 1 retool_user retool_user_group 397689246 Aug 5 20:11 retool_backend
drwxr-xr-x 3 retool_user retool_user_group 4096 Aug 5 20:11 server
drwxr-xr-x 2 retool_user retool_user_group 4096 Aug 5 20:11 templates
-rw-r--r-- 1 retool_user retool_user_group 517354 Aug 5 20:11 yarn.lock
retool_user@90b528dd4d33:/retool_backend$ ssh
bash: ssh: command not found

==========

Thanks,

-gregg

2

Hi @ChicagoGregg ! :wave:

Could you share the output of sudo docker-compose ps ? This will tell us if any containers are failing to start. Can you also check if there are any errors within sudo docker-compose logs --tail 500 ? This should include any container errors that were thrown on startup!

3

I was unable to start using the self-hosted -> local machine easy install script.
The retool-onpremise_api_1 is failing to start. The last part on the logs is:

Database migrations are up to date.

Database migrations are up to date.

Database migrations are up to date.

Setting http and https agent maxSockets to 25

Setting http and https agent maxSockets to 25

Setting http and https agent maxSockets to 25

{"message":"Not configuring Sentry...","level":"info","timestamp":"2022-05-17T15:36:36.037Z"}

{"message":"Not configuring StatsD...","level":"info","timestamp":"2022-05-17T15:36:36.039Z"}

Running node v12.22.2

{"message":"Not configuring Sentry...","level":"info","timestamp":"2022-05-17T15:36:36.044Z"}

ARGV: [ '--max_old_space_size=5120', '--max-http-header-size=80000' ]

{"message":"Not configuring StatsD...","level":"info","timestamp":"2022-05-17T15:36:36.046Z"}

Running node v12.22.2

ARGV: [ '--max_old_space_size=5120', '--max-http-header-size=80000' ]

{"message":"Not configuring Sentry...","level":"info","timestamp":"2022-05-17T15:36:36.053Z"}

{"message":"Not configuring StatsD...","level":"info","timestamp":"2022-05-17T15:36:36.055Z"}

Running node v12.22.2

ARGV: [ '--max_old_space_size=5120', '--max-http-header-size=80000' ]

{"namespace":"memoryUsage","pid":57,"message":{"pid":57,"Memory Stats for process":{"rss":"0 MB","heapTotal":"176.525312 MB","heapUsed":"161.356144 MB","percentageHeapUsed":"91.41%"},"CPU Usage % for process":"21.74%","Memory Stats for container":{"freeSystemMemory":"84.160512 MB","totalSystemMemory":"2085.158912 MB","percentageFreeMemory":"4.04%"}},"level":"info","timestamp":"2022-05-17T15:36:47.584Z"}

{"message":"Initialized general rate limiter: 60 attempts every 60 seconds","level":"info","timestamp":"2022-05-17T15:36:52.244Z"}

{"message":"Initialized invite rate limiter: 50 attempts every 86400 seconds","level":"info","timestamp":"2022-05-17T15:36:52.252Z"}

Tue, 17 May 2022 15:36:52 GMT body-parser deprecated bodyParser: use individual json/urlencoded middlewares at ../snapshot/retool_development/backend/transpiled/server/app.js:0:0

{"message":"Initialized general rate limiter: 60 attempts every 60 seconds","level":"info","timestamp":"2022-05-17T15:36:52.358Z"}

{"message":"Initialized invite rate limiter: 50 attempts every 86400 seconds","level":"info","timestamp":"2022-05-17T15:36:52.364Z"}

Tue, 17 May 2022 15:36:52 GMT body-parser deprecated bodyParser: use individual json/urlencoded middlewares at ../snapshot/retool_development/backend/transpiled/server/app.js:0:0

{"message":"Initialized general rate limiter: 60 attempts every 60 seconds","level":"info","timestamp":"2022-05-17T15:36:52.443Z"}

{"message":"Initialized invite rate limiter: 50 attempts every 86400 seconds","level":"info","timestamp":"2022-05-17T15:36:52.452Z"}

Tue, 17 May 2022 15:36:52 GMT body-parser deprecated bodyParser: use individual json/urlencoded middlewares at ../snapshot/retool_development/backend/transpiled/server/app.js:0:0

Tue, 17 May 2022 15:36:52 GMT body-parser deprecated undefined extended: provide extended option at ../snapshot/retool_development/node_modules/body-parser/index.js:105:29

Tue, 17 May 2022 15:36:52 GMT body-parser deprecated undefined extended: provide extended option at ../snapshot/retool_development/node_modules/body-parser/index.js:105:29

Tue, 17 May 2022 15:36:52 GMT body-parser deprecated undefined extended: provide extended option at ../snapshot/retool_development/node_modules/body-parser/index.js:105:29

./docker_scripts/start_api.sh: line 10:    57 Killed                  ./retool_backend```
4

Hi @jlvega !

It seems like this could potentially be an issue with running out of memory, do you know what the current memory limit is on your local docker?

5

Hi Ben, just before it crashes these are the stats:

CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS

06452de423d0 retool-onpremise_postgres_1 0.26% 5.922MiB / 1.942GiB 0.30% 49.9kB / 352kB 298MB / 25.5MB 11
14f79d44b7e6 retool-onpremise_jobs-runner_1 2.00% 61.09MiB / 1.942GiB 3.07% 51.2kB / 4.99kB 292MB / 331MB 35
2e4042523573 retool-onpremise_api_1 98.29% 840.8MiB / 1.942GiB 42.28% 162kB / 23.2kB 12.5GB / 532MB 57
50174999e681 retool-onpremise_db-ssh-connector_1 37.05% 222.4MiB / 1.942GiB 11.18% 726B / 0B 3.72GB / 120MB 9
5d3d3074d4d2 retool-onpremise_db-connector_1 34.68% 185.3MiB / 1.942GiB 9.32% 586B / 0B 737MB / 17.1MB 7

6

Hi @jlvega ,

Could you check the amount of memory that you have provisioned to the containers within the Docker Desktop settings? This Stackoverflow post gives a brief summary of how to update this value!

7

Hi) Facing similar isuue with self-hosted retool on ubuntu server. Run sudo docker-compose ps and got this :

image
image1059×158 28.4 KB

seems like retool-onpremise-api-1 and retool-onpremise-api-1 are not working.

1 Like
8

same issue popping up for me as well.

9

Hi @Shritwik_Bhaduri, are you able to share the container logs for your instance after attempting to start the containers?