Behaviour cloning (BC) is a classic algorithm to imitate expert demonstrations [7], which uses supervised learning to greedily match the expert behaviour at demonstrated expert states. Due to environmentstochasticity,covariateshift,andpolicyapproximationerror,theagentmaydriftaway from the expert state distribution and ultimately fail to mimic the demonstrator [8].

In the following, we provide a brief overview of Riemannian geometry and constant curvature manifolds, specifically the Poincaré ball and the hypersphere models. Sphere In the two-dimensional settingd = 2, we rely on polar coordinates to parametrize the sphere S2. In the following subsection we remind that this regularization term can also be motivated from an estimator'svarianceperspective. 5 D.2 Frobeniusnorm Hutchinson'sestimator Hutchinson'sestimator(Hutchinson,1990)isasimple waytoobtain a stochastic estimate ofthetrace ofamatrix. The variance of this estimator thus depends on the Frobenius norm of the vector's field Jacobian Thenγ(tn) is also a Cauchy sequence by Equation 16. So for every sequence (tn) in (a,b) that converges tob, we have that(γ(tn)) converges top.

By replacing the linear operation with our approximated one in transformers, we can achieve up to 2.7 peak memory reduction with almost no accuracy drop and enables up to6.4 larger batch size.

Thegeometry isrepresented as a zero level-set of a neural network, while the neural renderer, derived from therendering equation, iscapable of(implicitly) modeling awide setoflighting conditions and materials.

The point of the highest Top1 accuracy is represented by five-pointed star for bothapproaches. In addition, as shown in Fig. r3, theoptimal balance coefficients forMobileNetV3 onCIFAR10, MobileNetV3 onCIFAR100 and ResNet50 on CIFAR100 are 5, 10 and 10 respectively. In the logits distillation, the main loss coefficient is 0.8 and distillation coefficient is 0.2. In the feature distillation, the feature distillation coefficient of each stage is 0.02. ForCIFAR100, we train all 4 networks utilizing standard training setting and do the same test above.

We define this previously overlooked problem asnetwork loafing. As social loafing will ultimately cause the low individual productivity and the reduced overall performance, network loafing will also hinder the performance of a given residual network and its sub-networks.

In this paper, we consider whether learned optimization can help overcome these problems.