In my experience the SB3 tops out at 24bit/48kHz. I've heard that so far only the Transporter can handle 24/96.
I have no idea if iTunes can handle 24/96 files. I don't use iTunes to manager my music. I have all my music arranged into directories of my own choosing and point the SlimServer to the top directory. From the SB3, I can then just "browse music folder" and find any file I want, including 24/96 WAV files.
-Rob