Deep Learning
ATD: Augmenting CP Tensor Decomposition by Self Supervision
Existing tensor decomposition objectives (e.g., Frobenius norm) are designed for fitting raw data under
MAgNet: Mesh Agnostic Neural PDE Solver
Agnostic Neural PDE Solver (MAgNet) is able to make accurate predictions across a variety of PDE simulation datasets and compares favorably with existing baselines.
On the Stability and Scalability of Node Perturbation Learning
The immense success of deep learning in recent years has revived interest in the backpropagation algorithm (known simply as "backprop") as a learning mechanism in the brain [
Residual Pathway Priors for Soft Equivariance Constraints Marc Finzi New York University Greg Benton New York University Andrew Gordon Wilson New York University
A disadvantage of hard coding these restrictions is that this prior knowledge may not match reality. A scene may have long range non-local interactions, rotation equivariance may be violated by a preferred camera angle, or a dynamical system may occasionally have discontinuous transitions. In particular, symmetries are delicate. A small perturbation like adding wind breaks the rotational symmetry of a pendulum, and bumpy or tilted terrain could break the translation symmetry for locomotion.
Focal Attention for Long-Range Interactions in Vision Transformers Jianwei Y ang
Recently, Vision Transformer and its variants have shown great promise on various computer vision tasks. The ability of capturing local and global visual dependencies through self-attention is the key to its success.
Delayed Gradient Averaging: Tolerate the Communication Latency in Federated Learning
Federated Learning is an emerging direction in distributed machine learning that enables jointly training a model without sharing the data.