Deep Learning
How Many Samples are Needed to Estimate a Convolutional Neural Network?
Simon S. Du, Yining Wang, Xiyu Zhai, Sivaraman Balakrishnan, Russ R. Salakhutdinov, Aarti Singh
A widespread folklore for explaining the success of Convolutional Neural Networks (CNNs) is that CNNs use a more compact representation than the Fully-connected Neural Network (FNN) and thus require fewer training samples to accurately estimate their parameters. We initiate the study of rigorously characterizing the sample complexity of estimating CNNs.
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks
The curvature is a key geometric property of the loss landscape, which is characterized by the Hessian matrix or approximations such as the Gauss-Newton (GN) matrix, and strongly influences the convergence of gradient-based optimization methods. In the realm of deep learning, where models often have millions of parameters, understanding the geometry of the optimization landscape is essential to understanding the effectiveness of training algorithms.
Integrating GNN and Neural ODEs for Estimating Non-Reciprocal Two-Body Interactions in Mixed-Species Collective Motion
Analyzing the motion of multiple biological agents, be it cells or individual animals, is pivotal for the understanding of complex collective behaviors. With the advent of advanced microscopy, detailed images of complex tissue formations involving multiple cell types have become more accessible in recent years. However, deciphering the underlying rules that govern cell movements is far from trivial.
The Download: what's next for electricity, and living in the conspiracy age
Plus: Donald Trump wants to outlaw individual states' right to regulate AI The International Energy Agency recently released the latest version of the World Energy Outlook, the annual report that takes stock of the current state of global energy and looks toward the future. It contains some interesting insights and a few surprising figures about electricity, grids, and the state of climate change. Let's dig into some numbers . This article is from The Spark, MIT Technology Review's weekly climate newsletter. Everything is a conspiracy theory now. Our latest series " The New Conspiracy Age " delves into how conspiracies have gripped the White House, turning fringe ideas into dangerous policy, and how generative AI is altering the fabric of truth.