We propose the algorithms for online convex optimization which lead to cumulative squared constraint violations of the form $\sum\limits_{t=1}^T\big([g(x_t)]_+\big)^2=O(T^{1-\beta})$, where $\beta\in(0,1)$. Previous literature has focused on long-term constraints of the form $\sum\limits_{t=1}^Tg(x_t)$. There, strictly feasible solutions can cancel out the effects of violated constraints. In contrast, the new form heavily penalizes large constraint violations and cancellation effects cannot occur. Furthermore, useful bounds on the single step constraint violation $[g(x_t)]_+$ are derived. For convex objectives, our regret bounds generalize existing bounds, and for strongly convex objectives we give improved regret bounds. In numerical experiments, we show that our algorithm closely follows the constraint boundary leading to low cumulative violation.

Continuous allocation tasks are a class of problems where an agent needs to distribute a limited amount of resources over a set of entities at each time step.

This paper explores the realm of infinite horizon average reward Constrained Markov Decision Processes (CMDPs).

We theoretically establish that our algorithms almost surely converge to locally optimal policies of our safe optimization framework.

However, an emerging issue in allocation problems is data privacy.

In many applications, learning systems are required to process continuous non-stationary data streams.

Many articles in the literature solve the CMDP with an unknown environment.