If you want to call your disc a Compact Disc and have it play on a standard CD player, then yes, you most definitely do need to have it in the 16 bit, 44.1 KHz PCM format specified by the Redbook. That what defines a CD, if it is anything else it is no longer a CD and will not play on a standard CD player.
But Redbook only specifies what goes on the actual CD, not how it is recorded or mastered. Lots of studios record at 24/96 thoses days, some at 24/192. I believe Chesky has a recorder with a 6
MHz sample rate. But the digital signal is then converted to 16/44.1 PCM for the manufacturing process, to comply with the Redbook specifications. XRCDs are one example of this, Chesky CDs are another. Their specific processes are different, but they both end up with 16 bit, 44.1 KHz PCM actually on the CD, just like a standard mass market CD.
BTW, what you're asking for already exists, it's called SACD and DVD-Audio.
