I always rinse and so far have remembered to keep the spine facing the faucet just in case I get my razor to close to it.
I like the fact that the water keeps the blade warm to the touch on my face.