Using cloud_sql_proxy from google activly both on self host and on resources

Queries and Resources
,
1

This is a magical tool for self hosting on GCP and conneciton to the DB. I have used this as a "sidecar" in the deployment of retool, and its working well!

The problem comes to when I am to connect to a different DB. A different instances and project. The proxy can handle this Connect using the Cloud SQL Auth proxy  |  Cloud SQL for MySQL  |  Google Cloud, just giving it a different port.

So I would believe that if I added a localhost 5433(the next iteration port number that is given to the next instance) in the resource of a postgressql, things would techincally just work out of the box.

But unfortunately not, I get ECONNREFUSE on port 5433. I do get password error on 5432 though, which is facinating.

I have executed inside the container of the "main backend" and seen that the localhost 5433 is acctually receivable with tcp, ref this article: Testing connectivity from a container | by Nuno Aguiar | Medium.

So what might block the connection itself? What would be a good hostname to use to be able to access the cloudproxy passage?

2

Hi @snorreentur,

It sounds like you are trying to access two databases simultaneously, one on port 5432 and 5433 is that correct?

I believe this means you will need to run two instances of cloud_sql_proxy as well, but it seems like you are only running one instance. Can you confirm?

3

Hi thanks for the suggestion! But from my understanding the cloudproxy supports multiple instances: GitHub - GoogleCloudPlatform/cloud-sql-proxy: A utility for connecting securely to your Cloud SQL instances

I have added two instance connection names, and setting up the server works, in the form that it connects to the database retool needs on 5432. Adding a second instance just propagates the port number for that instance to 5433.

It runs as a sidecar so i think I cannot run more than one. Inside poddetails you can see these to containers

image
image1738×380 31.7 KB

I have confirmed that the port itself is open inside the retool worker containers, through TCP checks, that has this cloud proxy sidecard attached.

  - name: cloud-sql-proxy
    image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.4.0
    args: 
      - "--structured-logs"
      - "--port=5432"
      - "instance connection name 1"
      - "instance connection name 2"

But trying to address the proxy connection through a resource inside the retool UI, fails with the information above. I add localhost and the 5433 port as a postgressql server, with the right username and password.

Since the server itself is able to connect, how can a resource try to connect? Do I have to buypass the container itself or something?

4

from my understanding the cloudproxy supports multiple instances

Ah yes you are correct, you should be able to proxy two instances with a single cloud_sql_proxy

Next thing I would try is to swap the order of the two instances, so in your example make "instance connection name 2" the first argument so it receives connections through localhost:5432. If you can connect to instance 1 but cannot connect to instance 2 in that case, that suggests the problem is with the DB.

5

So swapping the instances helped me understand further what the problem is as i did not see the same errors in the log earlier.

When executing from 5432 I found out that the service account executing the query did not have access to the project. So when that was added it worked. Now I changed back the order and that worked as well, so back in business because of a service account addition

But the error feedback was not optimal and I would love a better feedback, together with some better documentation on this part