However,extrinsic rewards may be insufficiently informative to encourage an agent to explore and understand its environment, particularly in partially observed settings where the agent has a limited view of its environment.

Recent work has shown that including the optimization problem as alayer in the model training pipeline results in predictions oftheunobservedparameters thatlead tohigher decision quality.

Amongst those functions, the simplest are single-index modelsf(x) = ϕ(x θ), where the labels are generated by an arbitrary non-linear scalar link functionϕ applied to an unknown one-dimensional projectionθ of the input data.

Itturns out, somewhat surprisingly, that this is not the case even for the most standard algorithms that minimizetheempiricalrisk.

Our analysis reveals that the Polyak step-size adapts toany directional smoothness to obtain the tightest possible convergence rate.