Public apps should respect release versions without you needing to change the link as long as you make sure that the new version you've created is live by publishing the release:
It sounds like you may have already done this though would mind sharing what version of Retool you're using? Have you also tried accessing your public link in an incognito window or a separate browser to check for any caching issues?
URL query parameters not being supported in public apps is a bit of a soft statement, some of them might work but that behavior isn't guaranteed. The intended use case for public apps typically aligns with users who wouldn't be using these parameters. If you find yourself relying on these parameters either working or not working please let us know! It may be that there's a bug (as seems to be the case in this thread) or a potential feature request.
Thanks for the update! Just to add context : i think this is really needed if you want to make embeds reliable. Without it, it's impossible to QA new "embedded" version before sending them to end-users, so it makes it quite unsafe. If there any way we can upvote this feature request? Otherwise long term we will have to drop retool for this usecase.