Is your server on a nas, or a dedicated computer?
And, are your music files attached to the BDP, or elsewhere? All aspect that seem to matter...
RoonServer is running on a Mac Mini (Mid 2011), running El Capitan (10.11.5) w/ 8 GB RAM and booting from an external FW800 SSD (I find this part important with Roon, which needs a fast drive for it's database, et al.).
The music files are my iTunes library on a WD MyCloud mounted via SMB, connected to the same Ethernet switch as the Mac Mini. The BDP-1 is on the end of an Airport Express as a wireless bridge.
I've setup RoonServer to "watch" the iTunes/iTunes Media/Music folder, as opposed to mounting as an "iTunes Library", which I've had challenges with in the past. I use "Playlist Export" (see Apple App Store for Mac) to export selected iTunes playlists as M3U files into a folder located within the same watched folder. RoonServer picks these up, automatically, and add them to my library.
Note: I've worked around the latest Mac OS X issues with SMB by manually disabling SMB 3 signing, which was killing performance on NAS shares (well documented on Apple's support site).