We revisit the finite-armed linear bandit model by Nelson et al. (2022), where contexts and rewards are governed by a finite hidden Markov chain. Nelson et al. (2022) approach this model by a reduction to linear contextual bandits; but to do so, they actually introduce a simplification in which rewards are linear functions of the posterior probabilities over the hidden states given the observed contexts, rather than functions of the hidden states themselves. Their analysis (but not their algorithm) also does not take into account the estimation of the HMM parameters, and only tackles expected, not high-probability, bounds, which suffer in addition from unnecessary complex dependencies on the model (like reward gaps). We instead study the more natural model incorporating direct dependencies in the hidden states (on top of dependencies on the observed contexts, as is natural for contextual bandits) and also obtain stronger, high-probability, regret bounds for a fully adaptive strategy that estimates HMM parameters online. These bounds do not depend on the reward functions and only depend on the model through the estimation of the HMM parameters.

Choosing where to search matters. A time-tested path in the quest for new products or processes isthrough experimental optimization.

We exploit the richstructures ofthereformulated problem anddevelopastochastic primal-dual algorithm, SVRPDA-I, to solve the problem efficiently.

Our result shed light on the difference between Adam and (stochastic) gradient descent from a theoretical perspective.

Specifically, NTK-based analysis requires that the network weights stay very close to their initialization inthe "node-wise" `2 distancethroughoutthetraining.

Amongst those functions, the simplest are single-index modelsf(x) = ϕ(x θ), where the labels are generated by an arbitrary non-linear scalar link functionϕ applied to an unknown one-dimensional projectionθ of the input data.

Recent findings demonstrate that modern neural networks trained by full-batch gradient descent typically enter a regime called Edge of Stability (EOS).

To explain the decision of any regression and classification model, we extend the notion ofprobabilistic sufficient explanations (P-SE).

Let (t) betheparametersofmodel(4) trained by Gradient Descent(2)withquadraticloss.