Let’s use a metaphor.
You have a shot gun.
When you shoot the gun, you can seriously wound or kill other people. But the gun will also kick back, and that can REALLY hurt. It can leave a big ass bruise. Hell, if you hold it wrong, and the gun is big enough, you could break your shoulder. Maybe you even shoot yourself in the foot every now and then.
But you are still the one holding the gun.
Because sexist societies (like racist, homophobic and other bigoted ones) set up a rigid code of what is and isn’t acceptable, and punishes those who transgress regardless of their privilege. The rules may be more restrictive for the oppressed group, but they are there for everyone.
To give an example. My stepdad came from South Africa during Aparthide. Despite being white, the police would regularly target him for the crime of having black friends, liking black music, being a hippy, etc. He could have stopped doing those things a hell of a lot more easily than his black friends could have stopped being black, but it didn’t stop the rigid rules of the revolting country beating the crap out of him for transgressing what ‘white’ was supposed to mean.