fr338 ,
you found your problem right there . 2 outlets on the wall . Those can create a daisy chain grounding effect . One of the outlets might and normally will measure differently to Earth ground . Cheap very old outlets been used thousands of times over a home owners usage . I see ground loop city at your house my friend . One of your outlets might not even have a ground connection .
Best is to install a dedicated 20 amp breaker at your electrical service panel . Don't sweat it because these parts are relatively cheap . You might need an electrician to wire it up . 10ga. Romex wire and at minimum 10 foot deep copper ground rod . Go to a hardware store and get a hospital grade outlet . Use a minimum 6ga. pure copper ground conductor to wire in your ground rod to the service panel . Best to CAD weld or braze the ground rod connection . Use a 5/8ths inch diameter or larger ground rod . No clamps either on the ground rod . So what your equipment is looking for is a "clean - solid - direct" path to Earth ground . Buy a 5 outlet power strip and plug into your new "audio only" outlet . Plug your audio system into this ONLY . I would open this power strip and silver solder any friction fit connections . Get rid of the on and off switching capability per outlet . Wire the thing in direct . Again you might have to buy polarized power cords . Keep this in mind . There are numerous ways to improve your Earth ground .
So to answer your question you have a ground loop . I see your yellow icon smacking his head . Don't sweat it . You were looking for power and spotted the other outlet . Even if you don't "hear" a hum in your system , adding a dedicated 20 amp line and good Earth ground will lower the noise floor on your audio only power line . Which your amps and so on are connected to . You just lowered the noise floor of your entire audio system .
Depending on your house wiring , those 2 outlets might be on different breakers . You'd definitely have a ground loop there . You'd have differences in potential to Earth ground across the outlets .
I would remove the so called ultimate outlet and the other power filter . Run all your equipment direct . These devices are sonic band-aids . They attempt to correct improper wiring/grounding that most homes have . They also increase the impedance on your power line . Something you do not want . Also these items add measurable noise to your power line . Again , not at my house ......

Let me know how this turns out for you . Be careful doing this .
cityjim