I use these speakers with the XR45 and the Sharp SD-SG11. I got two pair when Crutchfield closed them out, since I needed a low cost replacement for some very old speakers, plus another pair. I still haven't gotten around to buying the Dennis Murphy crossover parts and redoing them, but I will someday when I get time (sigh).
I don't find them too harsh from either of these digital amps. But that may be personal taste in sound. However, they do have a high frequency rise. Have you tried trimming back the treble by 1 - 2 dbs? The Panasonics do this at 20Khz which may soften it a bit. Also, these amps take a while to burn in - like a couple of months of continuous playing. It does make things sound better. If you haven't had the XR50 that long, you probably haven't yet heard what it can do.
I do like the Polks, for what they are. They disappear more than draw attention to themselves. A sub would be nice to add. I hope to move up to better stuff someday, maybe some Europas? Got to move up in steps and keep the $ low.
Hope you find great sound soon!
Ben