Primary concern folks raise is, as you mentioned, getting the port too close to the barrier (wall). IMO that concern involves the overall bass response of any speaker that will go up when placed near a barrier. Note that typical ported speakers tend to have a bulge in the bass frequency response to compensate for fast drop off at lower frequencies and close placement to barriers exaggerate that bulge even more. Note that most either have not considered this effect or disagree with my opinion.
I've calculated the minimum acceptable distance (gap) between rear ported speakers and the barrier (wall) based simply on comparative cross-sectional areas of the port versus the area defined by the perimeter of the speaker multiplied by the gap. If you use a cross-sectional area of the overall gap as 10 times that of the port (which should minimize the affect of the barrier), you'll answer will be under 1 inch for whatever speaker you pick (big speakers would have bigger perimeter and tend to have larger ports). So unless you're putting speakers inside a bookcase or such, I can't imagine the dreaded nearby wall affecting the port.
Regardless, good imaging (system/speaker dependent) requires space behind the speakers to develop depth of soundstage, so speakers should be well away from barriers anyway. This also plays well into trying to minimize the listening room adding to the recording. Most experts recommend pulling speakers away from all barriers (walls/ceiling) so that first reflections are delayed by at least 0.11 milliseconds so the mind can separate/ignore those reflections which requires an additional 13 feet more reflected versus direct travel distance (about 6 feet from walls). So either way optimal setup would negate worries about gaps between rear port and walls.
The secondary concern of front versus rear ports is the phase response of the sound coming out of the port versus that of the woofer (it will always be 180 degrees out of phase minus a small delay for the sound to travel from back of driver to the port). The question is, do you want that sound to be more (front port) or less (rear port) prominent?
Ported design is valid, but care must be taken in floor-standers to not locate the port at near the opposite end from the woofer (as is often seen). The result is to create a giant flute that simple port design doesn't account for.
