The HDMI specification provides a basic overview on how the audio is transferred, if interested, figure 5.2 in version 1.3 is a good high level picture. As I lightly mentioned, the video frame rate is used to recover the audio clock for most designs, well at least all designs i've looked into, and this can impact jitter but also the amount of available audio data bandwidth. A typical example for most TVs today display at 120Hz so a blu-ray that is 24fps will have each video frame sent 5 times, which provides more bandwidth to send the audio data. It's likely same clock oscillator is used at 120Hz and 60Hz, but the jitter introduced at the clock recovery could be less if the frame rate is 60Hz, but now with half the data bandwidth. With all the possible options, HDMI might not be the best option for us audiophiles.
As with the noise, anything is possible, but since HDMI encodes every pixel and with HDCP enabled every pixel is also encrypted the audio data will arrive safely. Which is why I would say it's unexpected to be noisy, but maybe of lower quality or less detail. In the past I had a hdmi switch that did add video artifacts sometimes, the audio did sound have a few pops too, but i was so annoyed with the white squares on my screen, i didn't have time to be annoyed with the pops. I quick reset of the hdmi switch did the trick.
Jim