Ah, I was worried that was the case! At the moment, custom columns of type modal don't have the same functionality as a standalone modal. As a general workaround, you can have your column be a button that runs modal1.open(), where modal1 is a standalone modal who is hidden from the canvas (so it can only be opened from the table).
Let me know if something like this might work for you, or if you need any clarification getting this set up!