I have "moved" the SQL server and database from Australia East (as we are based in NZ) to US West 2 (to align with Retool) and I've seen a significant improvement: Now that query is taking under 500ms. The powershell script is now taking a longer time (250ms vs 50ms) due to - presumably - the fact that the azure datacentre is now much further away from me.
In summary, I think problem is solved 