Yeah that's what's a bit frustrating, after the scouting report his weapon speed is 1 while mine is 1.15, so it's slightly faster. As far as him getting to attack first, I get it he is less encumbered (at least that's what I've been finding in my tests). If you want to swing first you want your Total Equipment Weight to be really low, basically lower than your opponent. In this case he was lower so he got to go first.

It's possible that the location I was attacking happened to be the location he was defending. Anyway not sure, just puzzling.