No one talks about it. But this one factor could make you work for ever and forget retirement living.