Country
OnlineConvexOptimization withContinuousSwitchingConstraint
In many sequential decision making applications, the change of decision would bring an additional cost, such as the wear-and-tear cost associated with changing server status. To control the switching cost, we introduce the problem of online convex optimization with continuous switching constraint, where the goal is to achieve a small regret given a budget on the overall switching cost. We first investigate the hardness of the problem, and provide a lower bound of orderΩ( T)whentheswitchingcostbudgetS = Ω( T),andΩ(min{T/S,T}) whenS = O( T), where T is the time horizon. The essential idea is to carefully design an adaptive adversary, who can adjust the loss function according to thecumulative switchingcostofthe playerincurredso farbasedonthe orthogonal technique. We then develop a simple gradient-based algorithm which enjoys the minimax optimal regret bound.
Information-TheoreticSafeExplorationwith GaussianProcesses
Acommon approach istoplace aGaussian process prior on the unknown constraint and allowevaluations only inregions that are safe with high probability. Most current methods rely on a discretization of the domain and cannot be directly extended to the continuous case. Moreover, the way in which they exploit regularity assumptions about the constraint introduces an additional critical hyperparameter.
1 Theoreticalanalysis 1.1 Graphicalillustrationsofkeyequations Fig. 1illustrateskeyequationsinthemaintextaswellasinthesupplementarymaterials. (a)physicalspace (b)neuralspace
The biggerµ is,thebetter the error correction. For the set of( x) that form a group, a matrix representationM( x) is equivalent to another representation M( x)if there exists an invertible matrixP such that M( x)=PM( x)P 1 for each x. A matrix representation is reducible if it is equivalent to a block diagonal matrix representation, i.e., we can find a matrixP, such thatPM( x)P 1 is block diagonal for every x. IfM is block-diagonal,M =diag(Mk,k=1,...,K), with nonequivalentblocks,andeachblock Mkcannotbefurtherreduced,thenthematrixelements (Mkij( x)) are orthogonal basis functions of x. Such orthogonality relations are proved by Schur [15] for finite group, and by Peter-Weyl for compact Lie group [13].