Why should we avoid these phrases at all?

What's wrong with using words which came from religion?

This is purely my own little theory, but it seems to me that it's only those who have grown up with religion and then have had to undergo some painful, life changing process of giving it up, that feel the need to rid every sign of religion from their lives and their vocabulary.

I grew up nonreligious (with the exception of a few years of mild CofE stuff in early childhood) and I've never felt using religious words or phrases were a problem in the slightest. Just because I don't actually believe in God doesn't mean I can't use the English language and all it's religious hangovers. I don't believe in Thor either but I don't mind calling Thursday Thursday.