Technology
On the Double Descent of Random Features Models Trained with SGD
We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/overparameterized regime. In this work, we derive precise non-asymptotic error bounds of RF regression under both constant and polynomial-decay step-size SGD setting, and observe the double descent phenomenon both theoretically and empirically. Our analysis shows how to cope with multiple randomness sources of initialization, label noise, and data sampling (as well as stochastic gradients) with no closedform solution, and also goes beyond the commonly-used Gaussian/spherical data assumption. Our theoretical results demonstrate that, with SGD training, RF regression still generalizes well for interpolation learning, and is able to characterize the double descent behavior by the unimodality of variance and monotonic decrease of bias. Besides, we also prove that the constant step-size SGD setting incurs no loss in convergence rate when compared to the exact minimum-norm interpolator, as a theoretical justification of using SGD in practice.
Learning on the Edge: Online Learning with Stochastic Feedback Graphs
The framework of feedback graphs is a generalization of sequential decisionmaking with bandit or full information feedback. In this work, we study an extension where the directed feedback graph is stochastic, following a distribution similar to the classical Erdลs-Rรฉnyi model. Specifically, in each round every edge in the graph is either realized or not with a distinct probability for each edge.
Xi tests China's reach by blocking already-done Meta deal
Xi tests China's reach by blocking already-done Meta deal The Manus decision comes just weeks before China's Xi Jinping and the U.S. president are scheduled to meet at a high-profile summit. Meta cut the deal for Manus as part of its effort to catch up with rivals such as Alphabet's Google, OpenAI and Anthropic. China has sought for years to exert influence over business deals beyond its home turf. Still, its decision to press Meta Platforms to unwind a $2 billion acquisition of AI startup Manus marks a step unlike anything it's tried before. The country's powerful state planner decreed Monday that the deal must be canceled -- four months after it was sealed.