We present a complete mechanistic description of the algorithm learned by a minimal non-linear sparse data autoencoder in the limit of large input dimension. The model, originally presented in arXiv:2209.10652, compresses sparse data vectors through a linear layer and decompresses using another linear layer followed by a ReLU activation. We notice that when the data is permutation symmetric (no input feature is privileged) large models reliably learn an algorithm that is sensitive to individual weights only through their large-scale statistics. For these models, the loss function becomes analytically tractable. Using this understanding, we give the explicit scalings of the loss at high sparsity, and show that the model is near-optimal among recently proposed architectures. In particular, changing or adding to the activation function any elementwise or filtering operation can at best improve the model's performance by a constant factor. Finally, we forward-engineer a model with the requisite symmetries and show that its loss precisely matches that of the trained models. Unlike the trained model weights, the low randomness in the artificial weights results in miraculous fractal structures resembling a Persian rug, to which the algorithm is oblivious. Our work contributes to neural network interpretability by introducing techniques for understanding the structure of autoencoders. Code to reproduce our results can be found at https://github.com/KfirD/PersianRug .

That was the message delivered by Agility Robotics and Boston Dynamics during their respective opening and closing keynotes at the inaugural Robotics Summit & Showcase, produced by The Robot Report and WTWH Media in Boston. Agility Robotics CEO and co-founder Damion Shelton updated attendees on its Cassie bipedal robot. Boston Dynamics co-founder and CEO Marc Raibert quickly discussed the wheel-leg hybrid robot Handle, which he said we'll hear more about in 2019 with a real application, while focusing more on the Atlas bipedal and SpotMini quadruped robots. Raibert conducted a live demo of SpotMini (watch below) where the robot traversed a small obstacle and picked up a soda can and handed it to Raibert. Neither company claims legged robots are a fit for every application. "If we evolved with wheels, I'm sure our environments would be good for wheels, too," Shelton said.