Put a sound source 10 feet in front of you, a bell for example. Close your eyes, somebody rings it.
You point to where you think the sound is. You chose that direction because each ear heard the same volume, and both ears heard it at the exact same time.
You point directly in front of you, yes. Same amplitude to both ears, same delay. Gotcha.
Move the bell to 45 degrees off axis, to the right. Ring again.
You point to the bell, and again, you made that decision because your right ear heard a louder signal than your left, and, your right ear heard it first.
Amplitude is much stronger at the right ear, and the delay is much shorter at the right ear, also more direct sound vs radiated sound is heard. The density of early and late reflections, and the associated comb filtering and room gain effects are masked by the louder direct sound.
At the same time, left ear hears a lower amplitude, with a longer delay, and a more diffuse sound. There will be a higher ratio of early and late reflections, and the associated effects I described earlier.
The brain combines and interpolates all this (independant L/R) information, and allows us to localize where the sound is eminating from, and from how far away.
Now, same bell, recorded. Center the pan control, the image is centered...you point to the middle of the speaker setup.
Pan it...rotate the pan control to the right, do so until the bell is at that 45 degree position.
This is not equivalent. Now, you are pointing to a spot based entirely on the difference in amplitude between right and left. But that decision is not made with ANY time difference information, strictly amplitude.
You adapted to the localization information you were presented, not to the localization parameters that should have been present for a bell in that location.
Same bell, recorded how? It sounds like you are describing a mono source. You mention center the pan (singular) control.
In the case of a mono signal panned to 45 deg. off axis to the right, then I agree, you would get only amplitude differences from the L/R speakers, but no delay between L/R (assuming you are sitting in a near field listening environment, free of room reflections, or with headphones).
However, in a stereo recording, you have a L and R signal that are assigned to 2 separate channels, and panned hard R and hard L. If you record with a stereo mic configuration pointing directly forward, and you move your "bell" 45 deg, off axis, it will play back as your ears would have heard it in your first example.
I totally get what you are saying about amplitude, and delay between the L/R hearing, but in the second example, with an amplitude
only change, then you are talking about a mono signal.
Cheers