You can use foobar2000 with the waveform seekbar plugin. I use this as my music player.
There's also resonic player for just navigating through the filesystem itself: https://resonic.at/player - I use this as my default audio player when clicking files.
I usually just use Ableton, but I tried out this random free DAW that didn't have a sample library and came across Resonic Player which was pretty neat, also has a pro version with some extra functions but I havent looked at that.
You need to have a master "samples" directory somewhere. You can put aliases to other directories in it for easy navigation, but if you want to have a searchable directory it's better to have the samples actually in there. One advantage of collecting samples into one big folder is that if you are ever missing a sample, you can just tell ableton to search your samples folder and it's pretty much guaranteed to be in there.
So yes, it requires moving and re-linking but it really only takes a few seconds to fix a particular ableton project, just click the orange warning at the bottom of the screen and search for the files.
While I'm here I should recommend you check out Resonic Player, which is an amazing, lightning-fast audio player that lets quickly audition and manage samples (windows only).
Resonic Pro is my main audio player. I use it every day, as I go through hundreds of tracks. It is very fast and lightweight. There is a free version as well, which is great. The Pro version has more features, of course, which is the one I use.
I use Resonic Pro for cleanup duty. It plays samples practically instantly. Del key brings up a confirm dialog, you can hit Del-Enter really easily, or disable the confirmation under advanced options . You can assign a shortcut key to play the next file since everything moves up one when you delete.
If it's clear the whole folder is garbage you can delete folders from the browser pane. I have a meta pane set up to the right so I can verify sample rate, bit depth, etc. Very fast especially when deciding what to do with free sample packs.
The free version might be all you need. Pro version adds the meta panel, ASIO playback, high-quality MIDI playback (free version sounds fine though for evaluation), REX file support, BWF, unlimited channel files (up from 8), up to 192kHz (from 96), advanced looping, and some other stuff I don't think is necessary for clean up duty. https://resonic.at/product-comparison
I'm currently testing a couple of stand-alone sample browser apps:
Sononym: Very powerful and easy to use; but it auto-categorises samples and it's currently impossible to change the categories or add new ones. If it decides your vocal sample is a snare then you're stuck with it. And if often refuses to categorise at all. It does automatically create keywords for each sample based on file name/path (which is useful). Hopefully the categorisation issue will change soon! But it has a great feature that will search for "similar" samples based on a variety of parameters. Free trial for 30 days. I'll buy it once the category issue is fixed ($99). Uses a lot of RAM (400MB or so with 100k samples). https://www.sononym.net/
Resonic: Two versions, free and pro. I'm testing the Free version which lacks metadata and searching, but it's a fast sample browser. Offers several different sample data view modes (waveform, spectrum, etc.). I've asked for a Pro trial (the download page offers a 30 day trial if you contact them), will update once I've had a go with the metadata/searching functions. Get the free version, there's nothing to lose. Pro version is 69 euros. https://resonic.at/
There are others I think. The Loopmasters Cloud app will do something similar.
Might give it a look see. Screen images look a lot like Media Monkey, which I have for lots of options as needed. For simple music playing, I've been running Resonic - https://resonic.at/player but it doesn't have playlists, so it plays based on whats in the folder. No integration, no bells and whistles, just plays the files. To each their own.
Also, a search on KVR turned up these purpose-designed programs: IcedAudio AudioFinder (Mac, $69), Audio Ease Snapper 2 (Mac, $79), and Resonic Player (Windows XP, Vista, 7, 8, free).