A Natural Bias for Language Generation Models