I cannot personally vouch for this because I know nothing. However, interestingly enough, I have a friend who has a MA that is basically the history of fashion design, so I asked sent her your question.
From her e-mail to me:
"Fashion history is a huge subject but not a single subject so it's hard to recommend anything without knowing what you want to do -- is it art or is it business or is it archaeology, do you want to know about medieval fashion or modern fashion, etc. The good histories are all textbooks and stupid expensive and hard to find in libraries."
Then she said she has been recommending this lately to people who think fashion history is dumb. It's popular book that's not specialist oriented.
[Tim Gunn's Fashion Bible: The Fascinating History of Everything in Your Closet(https://www.amazon.com/Tim-Gunns-Fashion-Bible-Fascinating-ebook/dp/B0064CL138)]