A consistent and flexible framework for deep matrix factorizations