SSL Certificate Setups not working (Letsencrypt or Manual)

Self Hosted Retool
1

I am following the instructions for Self-Hosted Retool for Let's Encrypt, and then when that did not work, using a manual certificate.

When I setup for Let's Encrypt, I make the two edits from the documentation and run docker-compose up -d. It pulls the NGINX container, and that starts, but it does not generate any certificates. I can see the traffic being allowed to the internet, but it does nothing on the https-portal container's logs. When I look at the conf file in that container, it does not have any definition for an SSL connection. No certs, no 443 settings. Only port 80. And, port 80 does not pass through. (I can see it's defined when I do a docker ps).

I reverted back to the unedited docker-compose.yml and followed the Manual Certificate instructions. I put the crt and key files into the certs directory, I create an nginx.conf file as per the instructions, and the nginx container just restarts every so often. It never comes up enough so allow me to look at the logs. I do not see any traffic coming from that container. So I don't really see anything helpful to troubleshoot the issue. (I'm not a docker expert, but I can at least get around, restart things, access exec mode to view logs, etc...)

I'd settle for Let's Encrypt working. It seems like there should be more instructions than just setup DNS and change local to production in the yml file. Perhaps there is still need to create an nginx.conf file. I don't know. Any help would be appreciated.

2

re: Let's Encrypt, is your FQDN resolving to a public IP address? Our installation is internal (FQDN resolves to private IP), and we had use the manual method.

Doesn't help with your troubleshooting, but may reassure you that you're on the right path.

3

hi @summitrad,

I'm having the same issues with the manual certificate instructions. The nginx container doesn't start. Where can I see the logs for that container?
Haven't tried the Letsencrypt way, as I have a certificate already.

4

Hi there,

Thank you all for reaching out!

It would be helpful to see the https-portal logs & the docker-compose.yml file

5

So, I'm starting with the Let's Encrypt here. While I have a perfectly serviceable cert (several actually). I thought I would try the "easy" button first.

I will say, when I try the Let's Encrypt option, the https-portal container starts. But it doesn't really do anything. It doesn't log anything. The nginx config that is in that container only has a binding for port 80. (creating an nginx config file is only part of the manual cert instructions).

docker ps returns the configuration showing port mappings to the container, but inside the container, it is only port 80.

docker.env
# Set node environment to production
NODE_ENV=production

# Set the JWT secret for the API server
JWT_SECRET=randomstring

# Set and generate postgres credentials
#
# These variables can be changed to point to a database you host yourself
POSTGRES_DB=hammerhead_production
POSTGRES_USER=retool_internal_user
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_PASSWORD=

## If you need Google SSO
# CLIENT_ID={GOOGLE CLIENT ID}
# CLIENT_SECRET={GOOGLE CLIENT SECRET}

## If you wish for Retool to be hosted on a server with a public IP address, then you can use these configs to run the nginx container
#HOSTNAME=https://retool.summitrad.com
#HOSTNAME=retool.summitrad.com
#DOMAINS=http://retool.summitrad.com -> http://api:3000
DOMAINS=retool.summitrad.com -> http://api:3000

## License key
COOKIE_INSECURE=true
#LICENSE_KEY=SSOP_7bfc14b1-41d0-4138-a695-433a4d187c2a
LICENSE_KEY=NOPE

As you can see, only the basic changes spelled out in the Let's encrypt instructions.

docker-compose.yml:

version: '2'
services:
api:
build:
context: ./
dockerfile: Dockerfile.local
env_file: ./docker.env
environment:
- SERVICE_TYPE=MAIN_BACKEND
- DB_CONNECTOR_HOST=http://db-connector
- DB_CONNECTOR_PORT=3002
- DB_SSH_CONNECTOR_HOST=http://db-ssh-connector
- DB_SSH_CONNECTOR_PORT=3002
networks:
- frontend-network
- backend-network
- db-connector-network
- db-ssh-connector-network
depends_on:
- postgres
- db-connector
- db-ssh-connector
command: bash -c "./docker_scripts/wait-for-it.sh postgres:5432; ./docker_scripts/start_api.sh"
links:
- postgres
ports:
- '3000:3000'
volumes:
- ./keys:/root/.ssh
- ssh:/retool_backend/autogen_ssh_keys

jobs-runner:
build:
context: ./
dockerfile: Dockerfile.local
env_file: ./docker.env
environment:
- SERVICE_TYPE=JOBS_RUNNER
networks:
- backend-network
depends_on:
- postgres
command: bash -c "chmod -R +x ./docker_scripts; sync; ./docker_scripts/wait-for-it.sh postgres:5432; ./docker_scripts/start_api.sh"
links:
- postgres

db-connector:
build:
context: ./
dockerfile: Dockerfile.local
command: bash -c "./retool_backend"
env_file: ./docker.env
environment:
- SERVICE_TYPE=DB_CONNECTOR_SERVICE
networks:
- db-connector-network
restart: on-failure

db-ssh-connector:
build:
context: ./
dockerfile: Dockerfile.local
command: bash -c "./docker_scripts/generate_key_pair.sh; ./retool_backend"
env_file: ./docker.env
environment:
- SERVICE_TYPE=DB_SSH_CONNECTOR_SERVICE
networks:
- db-ssh-connector-network
volumes:
- ssh:/retool_backend/autogen_ssh_keys
- ./keys:/retool_backend/keys
restart: on-failure

postgres:
image: 'postgres:9.6.5'
env_file: docker.env
networks:
- backend-network
- db-connector-network
volumes:
- data:/var/lib/postgresql/data

Uncomment below to use nginx container to handle the frontend & SSL certification

https-portal:
image: tryretool/https-portal:latest
ports:
- '80:80'
- '443:443'
links:
- api
restart: always
env_file: ./docker.env
environment:
STAGE: 'production' # <- Change 'local' to 'production' to use a LetsEncrypt signed SSL cert
networks:
- frontend-network
networks:
frontend-network:
backend-network:
db-connector-network:
db-ssh-connector-network:

volumes:
ssh:
data:
user-data:

As you can see, the only change is to set the environment: STAGE: to 'production'

Everything else is as it was when I installed the retool demo, then upgraded to an enterprise key when we purchased our license.

The only other thing I've done is to not include passwords or my key in this post.

6

Here is the NGINX log directory for the https-portal container:
root@fbc26697a10b:/var/log/nginx# ls -lha
total 8.0K
drwxr-xr-x 1 root root 4.0K Apr 30 2018 .
drwxr-xr-x 1 root root 4.0K Jul 2 2018 ..
lrwxrwxrwx 1 root root 11 Apr 30 2018 access.log -> /dev/stdout
lrwxrwxrwx 1 root root 11 Apr 30 2018 error.log -> /dev/stderr

There's nothing in the logs at all.

Here is the base nginx.conf in /etc/nginx:

root@fbc26697a10b:/etc/nginx# cat nginx.conf
# This file will be compiled into /etc/nginx/nginx.conf

user nginx;
worker_processes 1;

pid /var/run/nginx.pid;

events {
** worker_connections 1024;**
}

http {
** include /etc/nginx/mime.types;**
** default_type application/octet-stream;**

** log_format main '$remote_addr - $remote_user [$time_local] "$request" '**
** '$status $body_bytes_sent "$http_referer" '**
** '"$http_user_agent" "$http_x_forwarded_for"';**

** access_log off;**

** access_log /var/log/nginx/access.log main;**

** sendfile on;**

** keepalive_timeout 65;**

** gzip on;**
** server_tokens off;**

** server_names_hash_max_size 512;**

** include /etc/nginx/conf.d/*.conf;**

** server {**
** listen 80 default_server;**
** server_name _;**
** return 444;**
** }**
}

As you can see, there's not 443 bindings here.

Here is the only file in /etc/nginx/conf.d (retool.summitrad.com.conf)

root@fbc26697a10b:/etc/nginx/conf.d# cat retool.summitrad.com.conf
server {
** listen 80;**
** server_name retool.summitrad.com;**

** location / {**
** return 301 https://$server_name$request_uri;**
** }**

** location /.well-known/acme-challenge/ {**
** alias /var/www/default/challenges/;**
** try_files $uri =404;**
** }**

}

As you can see here as well, there is no 443 binding. Only the port 80 binding.

Here is the port configuration for the https-portal container.
fbc26697a10b tryretool/https-portal:latest "/init" 19 hours ago Up 19 hours 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp retool-onpremise-https-portal-1

As you can see, the ports are configured for both 80 and 443 here. (It's just the default from the docker-compose.yml).

Unlike with the manual configuration, there is NO mention in the docs of needing to create an nginx.conf file for Let's encrypt.

7

@summitrad I got it working with my own certs. Probably the best and easiest option.
My nginx container would not start because it could not find the certs in the folder. It was just a typo in the path.

1 Like
8

Ok, thank you for the feedback. I will try it with my manual cert again. Thanks for the info. I'll make sure to check and re-check my paths. I will update the thread with my progress.

9

I go through, I setup the basics (very careful to not make typos in my nginx location nor my certs location. Also, with my certs and key file names and extension.

This is the status that I get after about 1 minute.

5f09ac38a1be tryretool/https-portal:latest "/init nginx-debug -…" 4 minutes ago Up 2 seconds 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp retool-onpremise-nginx-1

But, generally, this is what it does:

5f09ac38a1be tryretool/https-portal:latest "/init nginx-debug -…" 3 minutes ago Restarting (0) 7 seconds ago retool-onpremise-nginx-1

It just keeps rebooting itself. I cannot exec bash on the container. I cannot do anything with it actually. ReTool itself works OK on port 3000. NO issues there.

This is the section of the docker-compose.yml changed for using a manual cert.

nginx:
** image: tryretool/https-portal:latest**
** ports:**
** - '80:80'**
** - '443:443'**
** command: [nginx-debug, "-g", "daemon off;"] # Improve error logging in the container**
** volumes:**
** - ./nginx:/etc/nginx/conf.d**
** - ./certs:/etc/nginx/certs**
** links:**
** - api**
** depends_on:**
** - api**
** restart: always**
** env_file: ./docker.env**
** environment:**
** STAGE: 'local' # <- Change 'local' to 'production' to use a LetsEncrypt signed SSL cert**
** networks:**
** - frontend-network**

I let everything else alone, and updated, adjusted the lines from the instructions.

Here is my simple nginx.conf

server {
** listen 80;**
** server_name XXXXXXXXXX; # <- Change this to your subdomain**

** location / {**
** return 301 https://$host$request_uri;**
** }**
}
server {
** listen 443 ssl;**
** server_name XXXXXXXX; # <- Change this to match server_name above**
** ssl_certificate /etc/nginx/certs/summit-wildcard-24.crt; # <- Change this to your .crt file name**
** ssl_certificate_key /etc/nginx/certs/summit-wildcard-24.key; # <- Change this to your .key file name**

** location / {**
** proxy_pass http://XXXX:3000;**
** }**
}

Those are the actual names of the certificate. (Doesn't matter if I use a single name cert, a SAN cert, or a Wildcard cert, the nginx container behaves exactly the same way.)

Do I need the encrypted key, or the unencrypted key file? (I have both, but I've only tried the encrypted key file)

I have verified the permissions are correct for the yml file, the env file , the certs directory and files, as well as the nginx directly and files. Everything appears just as it should be.

I've tried it with and without the additional debug command and it behaves the same way. The only difference is , without the debug option, the command column just say "/init" when I issue a docker ps.

(Sorry about the XXXXX's.. if I try to use any real urls it tells me I cannot post the link)

10

I just answered my own question. I finally found the correct command to view the logs on the nginx container when I could not run bash. (Sorry, it's been way too long since I've done much with docker in production.) It did not like the encrypted key file that openssl exported. I had to use the UNENCRYPTED key file, and then the container started and stayed running.

Thanks for being my sounding board on all of this.