What does the alum block feel like when you rub it on? If it stings worse than the non-red parts, it's probably razor burn.

Also, have you considered the soap/cream/brush you're using? Could...