Radiohead is one of those bands that, in theory, I should absolutely love. However, I've really never been able to get into them as a whole. "The Bends" is a great album. "Kid A" was...interesting. "Hail the the Thief", while a great vinyl release, never really captured my attention. I think if I had a buddy who was really into them, and could help me see their greatness, I could probably be converted.
Back to the topic at hand...
Joy Division is another great discography to own. It's pretty easy to do with their "official" releases (Unknown Pleasures, Closer, Still), but you should seek out "Substance", their "unreleased" first album "Warsaw", and any Peel Sessions releases you can get your hands on.
Another intersting collection would be Tool. Starting with their first EP "Opiate" through their latest release "10,000 Days", they have evolved lyrically, musically, and philosophically - but still continue to rock yer face. It you took the best of Rush and Metallica, mixed it with a touch of NIN, and served it up raw, you'd have Tool. Don't expect 4/4 time, songs about "nothin' but a good time", a shit-kickin' beat, because these guys like to mix it up and keep you thinking. Start at the beginning ("Opiate") and move forward.
Someone already mentioned The Pixies, right? They are another beginning-to-end must have. Grab "Doolittle" then go get "Come on, Pilgrim / Surfer Rosa". This will get you hooked. The rest starts to get a bit produced and sometimes feel a bit unfinished, but still is better than the competition.
I'd admit it....I am a huge R.E.M fan. Sure, they've had some clunkers since the mid-90's, but their early catalog is phenomenal. "Murmur" and "Reckoning" are sheer brilliance. "Fables..." is a low-key classic, but "Life's Rich Pageant" is a country-inspired rocker. You *have* to buy "Dead Letter Office" simply for the "Chronic Town" EP (included). Just the afore mentioned releases alone qualify you as genius. With the exception of "Automatic..." and "Accelerate", the rest are debatable (and in the case of "Around the Sun", forgettable). However, as a whole, it's a great collection to have.
