Sudden slow queries performance since some weeks back

  • Goal: Load dashboard fast

  • Details:
    The slowdown is noticed since around 2 weeks back. Our DB servers are based in SG (asia).

SELECT 
    "Charges".*,
    "User"."address"
FROM 
    "Charges"
JOIN 
    "User" ON "Charges"."user_id" = "User"."id"
ORDER BY 
    "Charges"."createdAt" DESC;

The above query completes in 490s for 31k rows.
Same SQL DB query completes 1.6s locally

Select "User"."name","User"."address"
FROM "User"
WHERE "User"."id" = {{ text29.value }};

This single row query takes 10s on retool, 7ms from our dev env.

VPNing to US seems to improve on query speed, so it might be similar to the EU slowdown incident but now for asia.