I would give someone starting out ye Penguin Classics collection, The Thing on the Doorstep and Other Weird Stories, as it contains a near-perfect blend of the kinds of things HPL penned, some of the dream-like phantasies such as "The White Ship" and "The Quest of Iranon," the very strange and wonderful early classic "The Music of Erich Zann," the gruesome horrors of "Pickman's Model," "The Dunwich Horror" and "The Thing on the Doorstep," and then those two very different short novels, At the Mountains of Madness and The Case of Charles Dexter Ward.
I have all the Penguin Lovecrafts and I gotta agree. That one is the best cross-section.
I started my wife on The Dunwich Horror for the very reason it's controversial among Lovecraft scholars: there are some very un-Lovecraftian heroics in it. Next up, she's getting The Color Out of Space. Fun time's over.