Itturns out, somewhat surprisingly, that this is not the case even for the most standard algorithms that minimizetheempiricalrisk.

Our analysis reveals that the Polyak step-size adapts toany directional smoothness to obtain the tightest possible convergence rate.

M-step mainly consists of the weighted sum of partial derivative. Thus the complexity of regular E-step takes O(Nkd), and O(Nk(L+d)) for OT-based E step where L is the number of iterations of Sinkhorn's algorithm. The complexity of M-step takes O(Nkd).

In online multiclass classification a learner has to repeatedly predict the label that corresponds to a feature vector. Algorithms in this setting have a wide range of applications ranging from predicting the outcomes ofsport matches torecommender systems.

However, this assumption does not always hold in practice.