The "trick" is to keep shaving! :) Four shaves is nothing really, you are barely starting.
I had "eureka" moments around my 6-7th shave, finding a good blade angle that made the razor glide way...