Yes, you're pressing too hard. Use a lighter touch. You should be thinking about removing the lather from your skin -- that's how light the touch should be.

For my own experience (with a DE...