Wefindthatmeaningful hierarchical localfeatures can be learned despite the fact that these objectives operate on global instancelevelfeatures. Finally,we study the phenomenon offeaturesuppressionamong competing features shared across augmented views, such as "color distribution" vs"objectclass".

Neural Information Processing Systems http://nips.cc/

Although the popular MNIST dataset [LeCun et al., 1994] is derived from the NIST database [Grother and Hanaoka, 1995], the precise processing steps for this derivation have been lost to time. We propose a reconstruction that is accurate enough to serve as a replacement for the MNIST dataset, with insignificant changes in accuracy. We trace each MNIST digit to its NIST source and its rich metadata such as writer identifier, partition identifier, etc. We also reconstruct the complete MNIST test set with 60,000 samples instead of the usual 10,000. Since the balance 50,000 were never distributed, they can be used to investigate the impact of twenty-five years of MNIST experiments on the reported testing performances. Our limited results unambiguously confirm the trends observed by Recht et al. [2018, 2019]: although the misclassification rates are slightly off, classifier ordering and model selection remain broadly reliable. We attribute this phenomenon to the pairing benefits of comparing classifiers on the same digits.

Thermodynamic computing offers a potential route to energy-efficient computation. Unlike digital or quantum computing, which must at considerable energetic cost overpower or suppress thermal noise, thermodynamic computing is designed to use thermal noise as a source of energy. Physical devices whose states evolve under Langevin dynamics can be engineered to perform computations as they relax toward thermal equilibrium. Because these computations are carried out by the natural dynamics of the system, such devices can in principle operate with very low energy overhead, approaching fundamental thermodynamic limits [1-6]. A key challenge for thermodynamic computing is to identify algorithms that make efficient use of thermodynamic hardware and that reproduce the algebraic and machine-learning operations done digitally. Recent work has shown that thermodynamic computers can solve linear algebra problems, such as matrix inversion, in thermodynamic equilibrium [4, 5]. The advantage of equilibrium operation is that the computer's degrees of freedom obey the Boltzmann distribution, which depends in a precise way on the computer's potential energy. By choosing this potential energy appropriately, therefore, we can specify the desired computation.

With the rise of modern deep learning, neural networks have become an essential part of virtually every artificial intelligence system, making it difficult even to imagine different models for intelligent behavior. In contrast, nature provides us with many different mechanisms for intelligent behavior, most of which we have yet to replicate. One of such underinvestigated aspects of intelligence is embodiment and the role it plays in intelligent behavior. In this work, we focus on how the simple and fixed behavior of constituent parts of a simulated physical body can result in an emergent behavior that can be classified as cognitive by an outside observer. Specifically, we show how simulated voxels with fixed behaviors can be combined to create a robot such that, when presented with an image of an MNIST digit zero, it moves towards the left; and when it is presented with an image of an MNIST digit one, it moves towards the right. Such robots possess what we refer to as ``morphological cognition'' -- the ability to perform cognitive behavior as a result of morphological processes. To the best of our knowledge, this is the first demonstration of a high-level mental faculty such as image classification performed by a robot without any neural circuitry. We hope that this work serves as a proof-of-concept and fosters further research into different models of intelligence.

We introduce inductive biases for positive signalling and positive listening, which ease this problem. In a simple one-step environment, we demonstrate how these biases ease the learning problem.

The authors introduce a new version of the MNIST data set that they call QMNIST, which is the result of a very thoughtful and systematic analysis of existing materials used to build the original MNIST. I am impressed by the meticulous investigation carried out to recover the precise processing steps needed to generate MNIST examples from the original NIST images. The QMNIST data set is thus the product of a very accurate reconstruction of the original process (though the authors note that some minor discrepancies are still present). The authors then investigate whether the performance of popular classification methods measured using the new QMNIST test set actually differ from that measured on the original MNIST test set. Overall, I think this research is well conducted and presented in a very clear way.