Ok its official, I have to say I do not like the new feature. It is very slow and so my app is basically broken while I am editing the JS. I now need to edit the JS in a text editor and then copy/paste it in to test it.
I would rather edit it and click the save button, but not have the window close when I do (see Scripts and Styles dialog closes when clicking off the window losing all your changes)
This is very much like my issue with live loading a custom component (How to prevent custom component from reloading on every change?) but at least that is fairly performative.