This was the biggest one for me as well. Buffing strokes, the right angle, and easing off the pressure have made my face much happier.

To the OP, like everyone else said, you just gotta keep at...