First off, what a ridiculous argument. Would you rather be dead or deal with some harassment?
Why would the solution be to remove everyone's right to self defense if a sub group is still legally struggling with it? They're essentially saying the solution to the problem is to make everyone have it rather than fix it.
Additional reading: