For women's polo shirts try the Port Authority brand that is available on Amazon. I'm not out at work, but I made the switch to these and some straight leg women's khaki pants over 6 months ago and nobody at work has noticed. The shirt is cut in a way that makes it look like it could potentially be a men's slim fit style.
This Port Authority shirt is available in a large variety of colors, but each color is a different listing on Amazon. The link provided is to the Navy colored shirt.