We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of the Fisher matrix. In particular, we show a specific example that employs a simple and efficient reparametrization of the neural network weights by implicitly whitening the representation obtained at each layer, while preserving the feed-forward computation of the network. Such networks can be trained efficiently via the proposed Projected Natural Gradient Descent algorithm (PRONG), which amortizes the cost of these reparametrizations over many parameter updates and is closely related to the Mirror Descent online learning algorithm. We highlight the benefits of our method on both unsupervised and supervised learning tasks, and showcase its scalability by training on the large-scale ImageNet Challenge dataset.

Liquid argon time projection chambers are often used in neutrino physics and dark-matter searches because of their high spatial resolution. The images generated by these detectors are extremely sparse, as the energy values detected by most of the detector are equal to 0, meaning that despite their high resolution, most of the detector is unused in a particular interaction. Instead of representing all of the empty detections, the interaction is usually stored as a sparse matrix, a list of detection locations paired with their energy values. Traditional machine learning methods that have been applied to particle reconstruction such as convolutional neural networks (CNNs), however, cannot operate over data stored in this way and therefore must have the matrix fully instantiated as a dense matrix. Operating on dense matrices requires a lot of memory and computation time, in contrast to directly operating on the sparse matrix. We propose a machine learning model using a point set neural network that operates over a sparse matrix, greatly improving both processing speed and accuracy over methods that instantiate the dense matrix, as well as over other methods that operate over sparse matrices. Compared to competing state-of-the-art methods, our method improves classification performance by 14%, segmentation performance by more than 22%, while taking 80% less time and using 66% less memory. Compared to state-of-the-art CNN methods, our method improves classification performance by more than 86%, segmentation performance by more than 71%, while reducing runtime by 91% and reducing memory usage by 61%.

We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of the Fisher matrix. In particular, we show a specific example that employs a simple and efficient reparametrization of the neural network weights by implicitly whitening the representation obtained at each layer, while preserving the feed-forward computation of the network. Such networks can be trained efficiently via the proposed Projected Natural Gradient Descent algorithm (PRONG), which amortizes the cost of these reparametrizations over many parameter updates and is closely related to the Mirror Descent online learning algorithm. We highlight the benefits of our method on both unsupervised and supervised learning tasks, and showcase its scalability by training on the large-scale ImageNet Challenge dataset.

The complex events observed at the NOvA long-baseline neutrino oscillation experiment contain vital information for understanding the most elusive particles in the standard model. The NOvA detectors observe interactions of neutrinos from the NuMI beam at Fermilab. Associating the particles produced in these interaction events to their source particles, a process known as reconstruction, is critical for accurately measuring key parameters of the standard model. Events may contain several particles, each producing sparse high-dimensional spatial observations, and current methods are limited to evaluating individual particles. To accurately label these numerous, high-dimensional observations, we present a novel neural network architecture that combines the spatial learning enabled by convolutions with the contextual learning enabled by attention. This joint approach, TransformerCVN, simultaneously classifies each event and reconstructs every individual particle's identity. TransformerCVN classifies events with 90\% accuracy and improves the reconstruction of individual particles by 6\% over baseline methods which lack the integrated architecture of TransformerCVN. In addition, this architecture enables us to perform several interpretability studies which provide insights into the network's predictions and show that TransformerCVN discovers several fundamental principles that stem from the standard model.

A robotic surgeon is set to be tested aboard the International Space Station (ISS) – and could one day independently perform surgery on humans in space. After years of support and sponsorship from NASA, scientists in Nebraska have developed a robot called MIRA, short for'miniaturized in vivo robotic assistant'. In 2024, the miniature surgical robot will blast off towards the space station, where it will demonstrate its ability to cut simulated tissue. Scientists claim it could one day repair an astronaut's ruptured appendix during a mission to Mars, or remove shrapnel from a soldier injured by an explosive thousands of miles away. The ISS (pictured) floats in low Earth orbit at an altitude of 254 miles.

Anyone who has ever had to brush long hair will know that trying to get the knots out can be a nightmare. But mathematicians have now proved what many have suspected for some time – that the key to freeing the tangles is beginning at the ends and moving upwards the roots. Harvard researchers created a model that simulated two helically entwined filaments (similar to a strand of DNA) to represent a tangle of hair, and analysed different ways of'brushing' it so the hairs became free. Their results, published in the journal Soft Matter, revealed short brush strokes that start at the'free' end of the hair and move towards the'clamped' end are most effective. Experiments and simulations show the'tine' (representing a prong of the brush) moving along the double helix from the clamped end towards the free end'Using this minimal model, we study the detangling of the double helix via a single stiff tine (prong) that moves along it, leaving two untangled filaments in its wake,' said Plumb-Reyes, a graduate student at SEAS. 'We measured the forces and deformations associated with combing and then simulated it numerically.'

Using data collected in a global survey during 2019, the report analysed a sample of 151 fintechs and incumbents across 33 countries to paint a rich picture of how artificial technology is being developed and deployed within the financial services sector. While 77% of respondents noted that they expect AI to become an essential business driver across the financial services industry in the near term, the report found that the way incumbents and fintechs are leveraging AI technologies differ in a number of ways. A higher share of fintechs tend to be creating AI-based products and services, employing autonomous decision-making systems, and relying on cloud-based systems. Whereas incumbents appear to focus on "harnessing AI to improve existing products. This might explain why AI appears to have a higher positive impact on fintechs' profitability."

Delta may be known for its airplanes, but a new and surprisingly dexterous exoskeleton may be their next product to take off. The suit, called the Guardian XO, is a relatively small full-body exoskeleton that the company envisions will be used for heavy duty construction and commercial applications that requires brute strength. In a demonstration of the all-electric suit at CES in Las Vegas - the first ever public demo of the device - Delta and its partner Sarcos Robotics showed off the exoskeleton's capabilities. The demonstrator - a moderately sized young man by the name of Ben - strapped himself into the suit in just a couple minutes and started the first trial. 'It's a pretty comfortable machine, I can move around as if I wasn't wearing this,' said Ben who told the audience that he had only been training with the suit for about four months.

We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of the Fisher matrix. In particular, we show a specific example that employs a simple and efficient reparametrization of the neural network weights by implicitly whitening the representation obtained at each layer, while preserving the feed-forward computation of the network. Such networks can be trained efficiently via the proposed Projected Natural Gradient Descent algorithm (PRONG), which amortizes the cost of these reparametrizations over many parameter updates and is closely related to the Mirror Descent online learning algorithm. We highlight the benefits of our method on both unsupervised and supervised learning tasks, and showcase its scalability by training on the large-scale ImageNet Challenge dataset.

We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of the Fisher matrix. In particular, we show a specific example that employs a simple and efficient reparametrization of the neural network weights by implicitly whitening the representation obtained at each layer, while preserving the feed-forward computation of the network. Such networks can be trained efficiently via the proposed Projected Natural Gradient Descent algorithm (PRONG), which amortizes the cost of these reparametrizations over many parameter updates and is closely related to the Mirror Descent online learning algorithm. We highlight the benefits of our method on both unsupervised and supervised learning tasks, and showcase its scalability by training on the large-scale ImageNet Challenge dataset.