Retool container installation causes EC2 instance problems

Self Hosted Retool
1

Hey there

We have Retool installed on an EC2 container. When changing some configuration in the docker.env file and taking the containers down and then trying to docker-compose up, I noticed that something seemed to cause Docker(?) to crash and now am unable to ssh into the EC2 (even after successful reboots).

It's likely that we will have to mount the disk to a "repair" instance of EC2, in order to try and backup the data. Is there a way to get/find the app json files from the disk? That way I can just delete the instance and start up a fresh one. It is concerning though, as this is the 2nd time this kind of fatal crash has occurred.

2

Hey @will_root!

Would you mind sharing your container logs so we can check for any particular errors there to find why your instance may be crashing? It could also be helpful to see the configuration of your docker.env file.

In case we don't identify and remedy the crash itself - the state of your apps are saved in the page_saves table of your on-prem Postgres DB. If possible, I'd recommend actually migrating the DB entirely to your fresh instance as outlined in our docs.

3

Problem was I couldn't get into the container at all.

Managed to get the db data back after some (read: a lot) trial and error. The gist of it is there seems to be some problem with the AWS Ubuntu Kernel and Docker, which is causing kernel panic - not sure how I'm going to resolve this in a new container. Pleased that it doesn't seem to be related to Retool, but worried that it's quite pervasive to an EC2 instance in general.

Now that I have my db backed up. Would it be possible to share how the apps are saved? I'd like to just start fresh with new everything and just import .json apps, and update them from there. I imagine i'd need to join the pages and page_saves tables on the latest pageId and id? Is there anything else that i'd need to do?

4

Hey @will_root, sorry about the late reply but have you had any luck here? This isn't really a supported use case for Retool and I'm not sure if there are necessary references not contained within those tables that would break how they get imported into your instance. To start I would suggest moving over the data and excluding any foreign keys (particularly user ids) but it's a bit of a shot in the dark. Curious to know what more you've found.

5

Hey @Kabirdas, I managed to recover the data.

I ended up looking at the output from when you download an app and did the following to create a json object that I could just copy/paste into a json file:

WITH cte_max AS (
    SELECT
        "pageId",
        max(id) AS id
    FROM
        page_saves
    GROUP BY
        "pageId"
)
SELECT
    p.name,
    jsonb_build_object('uuid',
    p.uuid,
    'page',jsonb_build_object('id',ps."pageId",'data',ps.data)
        )
FROM
    page_saves ps
    JOIN cte_max c ON ps."pageId" = c."pageId"
    join pages p on p.id = ps."pageId"
        AND ps.id = c.id;

Everything's up and running again. FWIW, I also think the original problem might also arise from removing Google auth keys in the docker.env and restarting the containers. It didn't seem to like that very much.

1 Like
6

Ahh I see, thank you for sharing the query here and also calling out the issue with the Google auth key! :slightly_smiling_face: