I have a similar issue, and heres how I handle it:

Put your fingers on one side of the bump, pull the skin and do a VERY LIGHT pass with your razor. Now rinse and repeat pulling the skin the other...