Great personal insight noticing that your anxiety is triggering unreasonable reactions to people.
As a counselor who specializes in both anxiety and relationships, I've noticed that anxiety can drive irritability, as well as controlling and demanding behavior.
Anger is a secondary emotion. (Anger can be anything from irritability to rage.) There is some sort of other emotional pain or discomfort under anger. Anxiety is one of the emotions that can be turned into anger.
When we release the inner tension at the anger level, we do nothing to underdress the real underlying emotion--so it's just ready and waiting to be triggered again.
I explain all this more fully in my anxiety workbook: