I think it has to do with how "air tight" the elevator is. What you are saying makes perfect sense and you have no argument there from me, but most elevators have ventilation, fans, AC units, etc and are far from air tight. According to a friend who works on them, some even purposely move air through the car from the shaft as the car moves so the car doesn't create as much air pressure in the shaft which causes noise on other floors at the doors. Basically, instead of pushing the air out of the way with the car, they allow some of the air to pass through the car at a slower rate.
I have a few RC helicopters here, just no elevators handy, haha.
~T.J.