Time-Series-Informed Closed-loop Learning for Sequential Decision Making and Control