You can certainly use an all in one. Lots of people even use basic netbooks as their music servers. Music serving and decoding is a trivial task for today's PC's.
I don't know exactly what you have in mind, but on the simple level the best sound will be obtained if your music server is used only for that - and not to do other computer related tasks.
Depending on how high a sound quality you are looking for, you can start doing things like shutting off unecessary processes on the server, using ASIO, Kernel streaming or WASAPI drivers, and getting a specialized USB DAC. I don't know if you are interested on that level, but there's lots of information on the net about how to get optimum sound from a PC, if that's where you want to go.