Get a book containing all his stories. I recommend this one, its pretty inexpensive and look fantastic, comes with a case and bookmarker. Then if you wanna read everything, or look online for suggestions, you will have all of them at hand!
As far as which collection to get, this is a good choice right here.
https://www.amazon.ca/Complete-Fiction-Lovecraft-Knickerbocker-Classics-ebook/dp/B00UZ2G8PY