I believe you can go into the Plotly Interactions layout and remove the flags on the "clickmode" property.
By default they should be set to "select+event" which means that the selection flag is enabled as well as an event trigger for what do if something is selected.