Actually, with almost all players advanced beyond 5.0, the second serve swing is faster than any first serve swings.
Why? Because you CAN. Second serve swings, as you mentioned, are led with the edge of the racket, more streamlined and aero.
First serve swings, usually flat or FAST topspin serves, the racket is more open at contact, so the rackethead slows down or never accelerates up to the speed of a spin oriented second serve.