Trainable back-propagated functional transfer matrices