Country
Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes Jerry Y ao-Chieh Hu Dennis Wu
We study the optimal memorization capacity of modern Hopfield models and Kernelized Hopfield Models (KHMs), a transformer-compatible class of Dense Associative Memories.
GEPS: Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning
We demonstrate the versatility of our approach for both fully data-driven and for physics-aware neural solvers.
A Unified Framework for Rank-based Loss Minimization
The empirical risk function is the cornerstone of machine learning.
CORNN: Convex optimization of recurrent neural networks for rapid inference of neural dynamics Fatih Dinc Department of Applied Physics Stanford University Stanford, CA94305 Adam Shai
A promising way to extract computational principles from these large datasets is to train data-constrained recurrent neural networks (dRNNs).
Offline Multitask Representation Learning for Reinforcement Learning
We theoretically investigate offline multitask low-rank RL, and propose a new algorithm called MORL for offline multitask representation learning.
The geometry of hidden representations of large transformer models
By analyzing the intrinsic dimension (ID) and neighbor composition, we find that the representations evolve similarly in transformers trained on protein language tasks and image reconstruction tasks.