Since you're not familiar with many pieces, I'd recommend you get a collection. My first rec would be the Living Stereo box:
http://www.amazon.com/Living-Stereo-Box-Set/dp/B009J3K4MI/ref=sr_1_2?ie=UTF8&qid=1363097132&sr=8-2&keywords=living+stereo+collectionYou'd get all types of good classical music: symphonies, concertos, lieder, overtures, etc. And cheaper than buying singles ~$1.50 disc if you get the used set. While these recordings may not be the absolute BEST recordings of the respective pieces, but in my experience very few of them are duds. Here's the CD listing (by conductor or soloist, for whatever reason...):
1 Various Living Stereo Sampler
2 MUNCH, CHARLES/Saint-Sans: Symphony No. 3; Debussy: La Mer; Ibert: Escales
3 MUNCH, CHARLES/Ravel: Daphnis et Chlo
4 REINER, FRITZ/ Strauss: Also sprach Zarathustra; Ein Heldenleben
5 REINER, FRITZ/Bartk: Concerto for Orchestra;Music for Strings, Percussion & Celesta; Hungarian Sketches
6 HEIFETZ, JASCHA /Beethoven: Violin Concerto; Mendelssohn: Violin Concerto
7 CLIBURN, VAN /Tchaikovsky: Piano Concerto No. 1; Rachmaninoff: Piano Concerto No. 2
8 FIEDLER, ARTHUR/Gershwin: Rhapsody in Blue; Concerto in F; An American in Paris; Variations..
9 REINER, FRITZ /Mussorgsky: Pictures at an Exhibition
10 PRICE, LEONTYNE/Leontyne Price: Arias
11 RUBINSTEIN, ARTHUR Chopin: Ballades & Scherzos
12 MONTEUX, PIERRE/Tchaikovsky: Symphony No. 6 Pathtique
13 GOULD, MORTON/Brass & Percussion
14 HEIFETZ, JASCHA/Sibelius, Prokofiev, Glazunov: Violin Concertos
15 MUNCH, CHARLES/Berlioz: Requiem 2CD
17 MUNCH, CHARLES/Ravel: Bolro; La Valse; Rapsodie espagnole; Debussy: Images for Orchestra
18 PIATIGORSKY, GREGOR/Dvork; Walton: Cello Concertos
19 REINER, FRITZ/ Dvork: New World Symphony
20 REINER, FRITZ/Rimsky-Korsakov: Scheherazade
21 RUBINSTEIN, ARTHUR /Brahms: Piano Concerto No. 1
22 FIEDLER, ARTHUR/ Offenbach: Gat parisienne
23 CLIBURN, VAN/Rachmaninoff: Piano Concerto No. 3; Prokofiev: Piano Concerto No. 3
24 FIEDLER, ARTHUR/Hi-Fi Fiedler
25 HEIFETZ, JASCHA/Brahms; Tchaikovsky: Violin Concertos
26 MONTEUX, PIERRE/Franck:Symphony in D Minor;Stravinsky: Petrouchka
27 MUNCH, CHARLES/Beethoven: Symphonies Nos. 5 & 6
28 MUNCH, CHARLES/Berlioz: Symphonie Fantastique; Love Scene From Romeo & Juliet
29 REINER, FRITZ/Strauss: Scenes from Elektra & Salome
30 REINER, FRITZ/ Mahler: Symphony No. 4
31 RUBINSTEIN, ARTHUR Chopin: Piano Concertos
32 STOKOWSKI, LEOPOLD/ Rhapsodies
33 GOULD, MORTON/Copland: Billy The Kid & Rodeo; Grofe: Grand Canyon Suite
34 MOFFO, ANNA/Arias from Faust; La Bohme; Dinorah; Carmen; Turandot; Semiramide; Lakm
35 REINER, FRITZ/ Respighi: Pines Of Rome; Fountains Of Rome & Debussy: La Mer
36 REINER, FRITZ/Vienna
37 MUNCH, CHARLES/Mendelssohn: Symphonies Nos. 4 & 5; Octet: Scherzo
38 FIEDLER, ARTHUR/Pops Caviar
39 RUBINSTEIN, ARTHUR/Beethoven: Sonatas (Moonlight; Les Adieux; Pathetique; Appassionata)
40 HEIFETZ, JASCHA/Bruch: Violin Concerto No. 1; Scottish Fantasy & Vieuxtemps: Violin Concerto No. 5
41 LANZA, MARIO Mario! Lanza At His Best; Vagabond King Highlights
42 FOX, VIRGIL/ Encores
43 LEINSDORF, ERICH/Puccini: La Bohme 2CD
45 LEINSDORF, ERICH/Puccini: Madama Butterfly
47 MOFFO, ANNA/Verdi: La Traviata 2CD
49 BJORLING, JUSSI/Puccini: Turandot 2CD
51 MUNCH, CHARLES/Schubert: Symphonies "The Great" & "Unfinished"
52 REINER, FRITZ/ Strauss: Don Quixote & Don Juan
53 HEIFETZ, JASCHA /Bach: Concerto for Two Violins in D Minor
54 BREAM, JULIAN/Popular Classics for Spanish Guitar
55 PRICE, LEONTYNE/Albeniz: Navarra, Iberia; Falla: El amor
56 RUBINSTEIN, ARTHUR/Saint-Saens: Piano Concerto No. 2; Franck: Symphonic Variations;
57 MUNCH, CHARLES/Berlioz: Harold In Italy; The Roman Carnival Overture; Benvenuto Cellini Overture
58 REINER, FRITZ/Mahler: Das Lied Von Der Erde
59 REINER, FRITZ/Strauss: Symphonia Domestica; Suite From Le Bourgeois Gentilhomme
60 CLIBURN, VAN/Schumann: Piano Concerto In A minor; Beethoven: Piano Concerto No. 5, Op. 73 "Emperor"
BTW, I have heard of some people having problems with the quality of the discs. You may consider buying a new set for a little more money so you could return it if there are manufacturing defects.