Goto

Collaborating Authors

 Education


Eventually LIL Regret: Almost Sure $\ln\ln T$ Regret for a sub-Gaussian Mixture on Unbounded Data

arXiv.org Machine Learning

We prove that a classic sub-Gaussian mixture proposed by Robbins in a stochastic setting actually satisfies a path-wise (deterministic) regret bound. For every path in a natural ``Ville event'' $E_α$, this regret till time $T$ is bounded by $\ln^2(1/α)/V_T + \ln (1/α) + \ln \ln V_T$ up to universal constants, where $V_T$ is a nonnegative, nondecreasing, cumulative variance process. (The bound reduces to $\ln(1/α) + \ln \ln V_T$ if $V_T \geq \ln(1/α)$.) If the data were stochastic, then one can show that $E_α$ has probability at least $1-α$ under a wide class of distributions (eg: sub-Gaussian, symmetric, variance-bounded, etc.). In fact, we show that on the Ville event $E_0$ of probability one, the regret on every path in $E_0$ is eventually bounded by $\ln \ln V_T$ (up to constants). We explain how this work helps bridge the world of adversarial online learning (which usually deals with regret bounds for bounded data), with game-theoretic statistics (which can handle unbounded data, albeit using stochastic assumptions). In short, conditional regret bounds serve as a bridge between stochastic and adversarial betting.


Contributor: Rob Reiner reshaped how California understands and invests in children

Los Angeles Times

Things to Do in L.A. Hollywood director Rob Reiner engineered Proposition 10, a 1998 tobacco tax that created First 5 California, generating more than $11 billion for early childhood programs statewide. This is read by an automated voice. Please report any issues or inconsistencies here . After his tragic death Sunday, the world remembers Rob Reiner as a cinematic force -- and he was one, as an unforgettable presence on the ambitious 1970s sitcom "All in the Family" and later as the director of beloved films. I came to know him differently: as a restless thinker who transformed his own life story into bold public policy, reshaping how California understands and invests in its youngest children.


Becoming a Centenarian

The New Yorker

Like The New Yorker, I was born in 1925. Somewhat to my surprise, I decided to keep a journal of my hundredth year. The author, who was born on December 17, 1925, notes that the magazine's first issue came out ten months before he did. Old age is no joke, but it can feel like one. You look everywhere for your glasses, until your wife points out that you're wearing them. I turn a hundred this year. People act as though this is an achievement, and I suppose it is, sort of. Nobody in my family has lived this long, and I've been lucky. I'm still in pretty good health, no wasting diseases or Alzheimer's, and friends and strangers comment on how young I look, which cues me to cite the three ages of man: Youth, Maturity, and You Look Great. On the other hand, I've lost so many useful abilities that my wife, Dodie, and I have taken to calling me Feebleman. Look, up in the sky! No, it's Dodie doesn't want me to know how old she is, but she's nearly three decades younger than I am, and I become ...


AI might not be coming for lawyers' jobs anytime soon

MIT Technology Review

AI might not be coming for lawyers' jobs anytime soon Generative AI might have aced the bar exam, but an LLM still can't think like a lawyer. When the generative AI boom took off in 2022, Rudi Miller and her law school classmates were suddenly gripped with anxiety. "Before graduating, there was discussion about what the job market would look like for us if AI became adopted," she recalls. So when it came time to choose a speciality, Miller--now a junior associate at the law firm Orrick--decided to become a litigator, the kind of lawyer who represents clients in court. She hoped the courtroom would be the last human stage. "Judges haven't allowed ChatGPT-enabled robots to argue in court yet," she says.


An Efficient Variant of One-Class SVM with Lifelong Online Learning Guarantees

arXiv.org Machine Learning

We study outlier (a.k.a., anomaly) detection for single-pass non-stationary streaming data. In the well-studied offline or batch outlier detection problem, traditional methods such as kernel One-Class SVM (OCSVM) are both computationally heavy and prone to large false-negative (Type II) errors under non-stationarity. To remedy this, we introduce SONAR, an efficient SGD-based OCSVM solver with strongly convex regularization. We show novel theoretical guarantees on the Type I/II errors of SONAR, superior to those known for OCSVM, and further prove that SONAR ensures favorable lifelong learning guarantees under benign distribution shifts. In the more challenging problem of adversarial non-stationary data, we show that SONAR can be used within an ensemble method and equipped with changepoint detection to achieve adaptive guarantees, ensuring small Type I/II errors on each phase of data. We validate our theoretical findings on synthetic and real-world datasets.


Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation

arXiv.org Machine Learning

For four decades statistical physics has been providing a framework to analyse neural networks. A long-standing question remained on its capacity to tackle deep learning models capturing rich feature learning effects, thus going beyond the narrow networks or kernel methods analysed until now. We positively answer through the study of the supervised learning of a multi-layer perceptron. Importantly, (i) its width scales as the input dimension, making it more prone to feature learning than ultra wide networks, and more expressive than narrow ones or ones with fixed embedding layers; and (ii) we focus on the challenging interpolation regime where the number of trainable parameters and data are comparable, which forces the model to adapt to the task. We consider the matched teacher-student setting. Therefore, we provide the fundamental limits of learning random deep neural network targets and identify the sufficient statistics describing what is learnt by an optimally trained network as the data budget increases. A rich phenomenology emerges with various learning transitions. With enough data, optimal performance is attained through the model's "specialisation" towards the target, but it can be hard to reach for training algorithms which get attracted by sub-optimal solutions predicted by the theory. Specialisation occurs inhomogeneously across layers, propagating from shallow towards deep ones, but also across neurons in each layer. Furthermore, deeper targets are harder to learn. Despite its simplicity, the Bayes-optimal setting provides insights on how the depth, non-linearity and finite (proportional) width influence neural networks in the feature learning regime that are potentially relevant in much more general settings.


Google Translate is now better at translating slang terms and idioms using AI

Engadget

Google has also introduced a new speech-to-speech translation feature for headphones. Google is rolling out new Gemini-assisted functionality to Search and its Translate app. It says its AI can now provide more natural and accurate text translations for phrases that have more nuanced meanings. Translate will now take slang terms and colloquial expressions into consideration rather than provide sometimes unhelpful direct translations. The latest update to its text translation feature is rolling out first in the US and India, translating between English and just under 20 other languages, including German, Spanish, Chinese and Arabic.


Robot Talk Episode 137 – Getting two-legged robots moving, with Oluwami Dosunmu-Ogunbi

Robohub

Claire chatted to Oluwami Dosunmu-Ogunbi from Ohio Northern University about bipedal robots that can walk and even climb stairs. Oluwami Dosunmu-Ogunbi (Wami) is an Assistant Professor in the Mechanical Engineering Department at Ohio Northern University. Her research focuses on controls with applications in bipedal locomotion and engineering education. She is the first Black woman to receive a PhD in Robotics at the University of Michigan. During her Ph.D., she developed the Biped Bootcamp technical document, which she is transforming into an undergraduate curriculum --introducing students to bipedal robotics while providing advanced coursework for juniors and seniors.


Supervised Learning of Random Neural Architectures Structured by Latent Random Fields on Compact Boundaryless Multiply-Connected Manifolds

arXiv.org Machine Learning

This paper introduces a new probabilistic framework for supervised learning in neural systems. It is designed to model complex, uncertain systems whose random outputs are strongly non-Gaussian given deterministic inputs. The architecture itself is a random object stochastically generated by a latent anisotropic Gaussian random field defined on a compact, boundaryless, multiply-connected manifold. The goal is to establish a novel conceptual and mathematical framework in which neural architectures are realizations of a geometry-aware, field-driven generative process. Both the neural topology and synaptic weights emerge jointly from a latent random field. A reduced-order parameterization governs the spatial intensity of an inhomogeneous Poisson process on the manifold, from which neuron locations are sampled. Input and output neurons are identified via extremal evaluations of the latent field, while connectivity is established through geodesic proximity and local field affinity. Synaptic weights are conditionally sampled from the field realization, inducing stochastic output responses even for deterministic inputs. To ensure scalability, the architecture is sparsified via percentile-based diffusion masking, yielding geometry-aware sparse connectivity without ad hoc structural assumptions. Supervised learning is formulated as inference on the generative hyperparameters of the latent field, using a negative log-likelihood loss estimated through Monte Carlo sampling from single-observation-per-input datasets. The paper initiates a mathematical analysis of the model, establishing foundational properties such as well-posedness, measurability, and a preliminary analysis of the expressive variability of the induced stochastic mappings, which support its internal coherence and lay the groundwork for a broader theory of geometry-driven stochastic learning.


Defining the Scope of Learning Analytics: An Axiomatic Approach for Analytic Practice and Measurable Learning Phenomena

arXiv.org Machine Learning

Learning Analytics (LA) has rapidly expanded through practical and technological innovation, yet its foundational identity has remained theoretically under-specified. This paper addresses this gap by proposing the first axiomatic theory that formally defines the essential structure, scope, and limitations of LA. Derived from the psychological definition of learning and the methodological requirements of LA, the framework consists of five axioms specifying discrete observation, experience construction, state transition, and inference. From these axioms, we derive a set of theorems and propositions that clarify the epistemological stance of LA, including the inherent unobservability of learner states, the irreducibility of temporal order, constraints on reachable states, and the impossibility of deterministically predicting future learning. We further define LA structure and LA practice as formal objects, demonstrating the sufficiency and necessity of the axioms and showing that diverse LA approaches -- such as Bayesian Knowledge Tracing and dashboards -- can be uniformly explained within this framework. The theory provides guiding principles for designing analytic methods and interpreting learning data while avoiding naive behaviorism and category errors by establishing an explicit theoretical inference layer between observations and states. This work positions LA as a rigorous science of state transition systems based on observability, establishing the theoretical foundation necessary for the field's maturation as a scholarly discipline.