Deep Learning
AI voice chat sucks. This startup thinks it's cracked it
PCWorld reports that Thinking Machines, founded by ex-OpenAI executive Mira Murati, has developed new AI voice interaction models that enable real-time conversations with interruptions and visual cue recognition. The technology uses a dual-AI system with a fast interaction model and background model for complex tasks, employing a multi-stream, micro-turn approach. This advancement could transform AI voice chat from current CB radio-style turn-taking into natural human-like conversations, though the technology remains in research phase. Voice chatting with today's AI can feel as stilted as an old-school CB radio exchange, where you're forced to take turns as you talk. "Hey ChatGPT, let's talk about the movies!
ChatGPT is 20/month, but one AI platform gives you GPT, Claude, and Gemini for a year for 30
When you purchase through links in our articles, we may earn a small commission. You can get access to ChatGPT, Claude, and Gemini through ChatOn AI Assistant for just $30. Juggling AI subscriptions can get expensive fast. A single AI subscription can cost hundreds per year, and using multiple tools only drives the price higher. That's part of why ChatOn AI Assistant has been gaining attention recently.
Trump heads to China to spread the gospel of American tech while emulating Xi Jinping on AI
Donald Trump is heading to China this week, and if his guest list is any clue, he wants to discuss technology with Xi Jinping. Donald Trump is heading to China this week, and if his guest list is any clue, he wants to discuss technology with Xi Jinping. Donald Trump is heading to China this week. If his guest list is any clue, he wants to discuss technology with Xi Jinping, though perhaps after the war in Iran. On Monday, news broke that outgoing Apple CEO, Tim Cook, as well as SpaceX and Tesla CEO, Elon Musk, would join the US president.
Digg is back again, this time to aggregate AI news
Digg is back again and has taken on yet another form: A website that aggregates news about artificial intelligence. Digg's job is to find that signal and bring it to you. AI is just the beginning, he said, calling it the noisiest, fastest-moving space on the internet. He promised that more verticals are coming, but he didn't say when Digg will start aggregating news about other topics. At the moment, the website follows 1,000 people directly involved in AI research, investing and media, built from X's social graph.
Daybreak is OpenAI's response to Anthropic's Claude Mythos
OpenAI has just launched Daybreak, a cybersecurity initiative that's clearly the company's competitor to Anthropic's Project Glasswing . If you'll recall, Glasswing uses Anthropic's unreleased AI model, Claude Mythos Preview, to provide its clients' cyber defense needs. It's been promising, so far: Mozilla revealed in April that Mythos helped it find and patch 271 vulnerabilities in the latest release of the Firefox browser. OpenAI says Daybreak uses its various AI models, including its specialized security agent Codex. In its announcement, the company explained that Daybreak is built around the premise that cyber defense should be built into software from the start and not just revolve around finding and fixing vulnerabilities.
Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control
Li, Bolian, Wang, Yifan, Ding, Yi, Lochab, Anamika, Grama, Ananth, Zhang, Ruqi
Reinforcement learning (RL) has enabled complex reasoning abilities in large language models (LLMs). However, most RL algorithms suffer from performance saturation, preventing continued gains as RL training scales. This problem can be characterized by the collapse of entropy, a key diagnostic for exploration in RL. Existing attempts focus on preventing entropy collapse through regularization or clipping. However, their resulting entropy curves often exhibit instability in the long term, which hinders performance gains. In this paper, we introduce Entrocraft, a simple rejection-sampling approach that realizes user-customized entropy schedule by biasing the advantage distributions. Entrocraft requires no objective regularization and is advantage-estimator-agnostic. Theoretically, we relate per-step entropy change to the advantage distribution under minimal assumptions. This explains the behavior of existing RL and entropy-preserving methods. Entrocraft also enables a systematic study of entropy schedules, which reveals that linear annealing, which starts high and decays to a slightly lower target, performs best. Empirically, Entrocraft addresses performance saturation, significantly improving generalization, output diversity, and long-term training. It enables a 4B model to outperform an 8B baseline, sustains improvement for up to 4x longer before plateauing, and raises pass@K by 50% over the baseline.
A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering
Wang, Zhanliang, Xiao, Jiancong, Jin, Ruochen, Yang, Shu, Hou, Bojian, Shen, Li
Calibration measures whether a model's predicted confidence aligns with its empirical accuracy, and is central to the reliable deployment of large language models (LLMs) in high-stakes domains such as medicine and law. While much recent work focuses on improving LLM calibration, the equally important question of how to evaluate it in realistic settings remains underdeveloped. Open-ended question answering (QA), the most common deployment setting for modern LLMs, is where existing evaluation methods fall short: logit-based metrics need restricted output formats and internal probabilities; verbalized confidence is self-reported and often overconfident; and sampling-based methods rely on task-specific extraction rules without a clear finite-sample target. We introduce Sem-ECE (Semantic-Sampling Expected Calibration Error), a calibration evaluation framework for open-ended QA that samples answers from the model, groups them into semantic classes, and uses the resulting frequencies as confidence. We study two estimators within this framework: Sem$_1$-ECE, the same-sample self-consistency score, and Sem$_2$-ECE, a held-out variant that separates answer selection from confidence evaluation. We prove both are asymptotically unbiased, and further show that they agree on easy questions but diverge on hard ones with Sem$_2$ achieving strictly smaller calibration error, so their gap also serves as a diagnostic for question difficulty. Experiments on three open-ended QA benchmarks across five leading commercial LLMs match our theoretical predictions and show that Sem-ECE outperforms verbalized confidence and existing sampling-based methods, while complementing logit-based evaluation when internal probabilities are unavailable.
Posterior Concentration of Bayesian Physics-Informed Neural Networks for Elliptic PDEs
Unlike a standard PINN--which produces an approximate Deep neural networks (DNNs) or multi-layer perceptronssolution by minimizing a PDE-residual loss and thus yields (MLPs) offer various inherent advantages over traditionalonly a point estimate, failing to quantify uncertainty inapproaches of scientific computing and data analysis, suchduced by noisy or limited data, a Bayesian PINN returns a as finite element methods, wavelets and kernel methods, full posterior distribution over solutions by combining the which are often hampered by the irregular and nonlinearuncertain information from the likelihood (data) and the data structures and the high input dimensions. In contrast, DNNs are capable of approximating a rich class of functions prior. Bayesian neural networks, originating in the seminal works of MacKay (MacKay, 1995) and Neal (Neal, 1995), with aforementioned complexities and can also easily en-have been extensively studied over the past three decades codes additional complex physical structures, such as sym- (Lampinen & Vehtari, 2001; Titterington, 2004; Graves, metry and other invariant structures.
Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity
This paper investigates the learning theory of Transformer networks for regression tasks on the compact Euclidean domain $[0,1]^d$ and $d$-dimensional compact Riemannian manifolds. We propose a novel constructive approximation framework for Transformers that builds local approximations of the target function and aggregates them into a global approximation via softmax partition of unity. This approach leverages the attention mechanism to achieve spatial localization through affine transformations of the input. The softmax activation plays a crucial role in aggregating local approximations to a global output. From an approximation perspective, we prove that a dense Transformer equipped with only two encoder blocks and standard single-hidden-layer point-wise feed-forward networks can achieve a uniform $\varepsilon$-approximation error for $α$-Hölder continuous functions with $α\in (0,1]$ using $\mathcal{O}(\varepsilon^{-d/α})$ total parameters. Building upon this approximation guarantee, we establish a near minimax-optimal generalization error bound of order $\mathcal{O}\big(n^{-\frac{2α}{2α+d}} \log n\big)$ for the empirical risk minimizer, where $n$ is the training data size. The Transformer architecture studied in this paper is dense, shallow and wide, and employs softmax activation and sinusoidal positional encodings, closely reflecting practical implementations.
Local LMO: Constrained Gradient Optimization via a Local Linear Minimization Oracle
Richtárik, Peter, Gruntkowska, Kaja, Li, Hanmin
We design Local LMO - a new projection-free gradient-type method for constrained optimization. The key algorithmic idea is to replace the global linear minimization oracle over the constraint set used by Frank-Wolfe (FW) with a local linear minimization oracle over the intersection of the constraint set and a "small" ball centered at the current iterate. In particular, when minimizing $f:\mathbb{R}^d\to \mathbb{R}$ over a constraint $\emptyset\neq\mathcal{X}\subseteq\mathbb{R}^d$, Local LMO performs the iteration \[x_{k+1}\in \arg\min_{z\in\mathcal{X}\cap\mathcal{B}(x_{k},t_k)}\langle\nabla f(x_{k}), z \rangle,\] where $x_0\in\mathcal{X}$, and $t_k>0$ is a suitably chosen radius which can be interpreted as an effective stepsize. While designed as an alternative to FW, Local LMO is perhaps best viewed as a generalization of Gradient Descent (GD) rather than a modification of FW. Indeed, it is easy to see that Local LMO reduces to GD in the unconstrained setting and, more generally, to GD restricted to an affine subspace if the constraint $\mathcal{X}$ is affine. We prove that this simple algorithmic scheme transfers the known (unaccelerated) convergence rates of Projected Gradient Descent (PGD) to the projection-free world in several important regimes, some of which are beyond the reach of FW. In contrast to FW theory, i) our guarantees hold without requiring the feasible set $\mathcal{X}$ to be bounded, ii) our theory does not require the "curvature" assumption, which allows us to establish a standard sublinear rate for convex functions with bounded gradients, iii) we obtain a linear rate in the smooth strongly convex regime. Furthermore, we obtain sharp sublinear rates in the smooth convex and non-convex regimes, in the $(L_0,L_1)$-smooth convex regime, and in stochastic and non-differentiable settings.