Goto

Collaborating Authors

 Deep Learning


Finance Minister Katayama says G7 will discuss AI defense standards

The Japan Times

Finance Minister Satsuki Katayama speaks during an interview on Monday. The Group of Seven nations will discuss standards on artificial intelligence security and defense, Finance Minister Satsuki Katayama has said. Speaking in a recent interview, Katayama said that financial institutions "need to decide the order of priority for fixing their systems," in order to prepare for the possibility of advanced AI models detecting a large number of vulnerabilities in their systems. She added that the G7 nations, which include Japan, will discuss related criteria and work together to tackle cyberattacks. State-of-the-art AI models, such as Claude Mythos, developed by U.S. startup Anthropic, are believed to be highly proficient in identifying system vulnerabilities. Katayama has been negotiating with the United States to ensure that major financial institutions in Japan have access to these technologies.


OpenAI reportedly wants all AI companies to give the US government a stake in their businesses

Engadget

Sam Altman is in talks with the US government in a bid to clear political hurdles, says the Financial Times. OpenAI's Sam Altman has reportedly been in talks with the US government to ensure his company's path towards achieving its goals remains free of political hurdles. According to the Financial Times, Altman has suggested giving the government a five percent stake in the company, in order to share the spoils of the AI boom with the public. But his idea doesn't only involve OpenAI: Under his proposal, other top AI companies like Google, Anthropic, xAI and Meta would have to agree to give the government a similar stake in their businesses. AI companies like Anthropic and OpenAI have recently encountered roadblocks from the US government when it came to releasing their latest AI models.


Adaptive parallel reasoning: the next paradigm in efficient inference scaling

AIHub

What if a reasoning model could decide when to decompose and parallelize independent subtasks, how many concurrent threads to spawn, and how to coordinate them based on the problem at hand? We provide a detailed analysis of recent progress in the field of parallel reasoning, especially adaptive parallel reasoning. Disclosure: this post is part landscape survey, part perspective on adaptive parallel reasoning. One of the authors (Tony Lian) co-led ThreadWeaver ( Lian et al., 2025), one of the methods discussed below. The authors aim to present each approach on its own terms. Recent progress in LLM reasoning capabilities has been largely driven by inference-time scaling, in addition to data and parameter scaling ( OpenAI et al., 2024; DeepSeek-AI et al., 2025). Models that explicitly output reasoning tokens (through intermediate steps, backtracking, and exploration) now dominate math, coding, and agentic benchmarks.


OpenAI 'in early talks to give 5% stake to US government'

The Guardian

OpenAI CEO Sam Altman has been in talks about public ownership with Donald Trump, according to the report. OpenAI CEO Sam Altman has been in talks about public ownership with Donald Trump, according to the report. OpenAI'in early talks to give 5% stake to US government' OpenAI is reportedly in early stage talks to give a 5% stake in the ChatGPT developer to the US government as artificial intelligence companies attempt to smooth relations with Donald Trump's administration. The OpenAI chief executive, Sam Altman, has argued that giving the US public a financial stake in the company is the best way to share the benefits of AI, according to the Financial Times, which cited two unnamed people familiar with the discussions. The proposal would also involve other US AI companies giving a similar stake to the government, the FT reported, although it is not clear yet whether companies such as Anthropic, Google and Meta would agree to the plan.


Neural Network-Based Estimation of Time-Dependent Parameters in AR(p) Processes

arXiv.org Machine Learning

We investigate a forecasting framework based on a simple discrete-time dynamic model with coefficients varying in time. The parameters of the model are recovered within a deep learning framework, which makes it possible to retain a transparent parametric structure while simultaneously accounting for complex and nonstationary patterns in the observed phenomenon. Our analysis covers two specifications of the noise process. Besides the standard Gaussian setting, we also consider Laplace-distributed noise, which can offer a more adequate description in the presence of heavier tails and sharper local fluctuations. For both cases, we formulate the predictive scheme of the model and analyze the associated uncertainty quantification, including the construction of prediction intervals. The results illustrate that a relatively simple model, when combined with time-dependent parameter estimation, can serve as a mathematically tractable and practically flexible tool for forecasting complex dynamics under different noise assumptions. The general model is stated for TVAR($p$), while the prediction-interval formulas and the numerical experiments are developed for the TVAR(1) case.


Function-Counting Theory for Low-Dimensional Data Structures

arXiv.org Machine Learning

The success of deep learning models in classification and regression is widely attributed to the low-dimensional structure that real-world data tend to exhibit, despite their high-dimensional representation. This work attempts to provide a mathematical framework for binary classification on low-dimensional data, building on Cover's (1965) function-counting theory. With our framework, we aim to address the question of how the low-dimensional structure of the data affects the classification capabilities of learning models. Cover's theory relies on a general position assumption that blinds it to the underlying data structure. We refine this assumption to account for the low-dimensionality of the data and derive dichotomy counts that reflect the data structure. We further extend Cover's separation capacity and problem of generalization to the low-dimensional setting, enabling the impact of the underlying data structure on both to be analyzed.


Deep Multitask Learning for Mixed-Type Outcomes with Shared Sparsity

arXiv.org Machine Learning

Most existing multitask learning approaches are limited by their reliance on task-specific loss functions tailored to the scale and type of each outcome. When outcomes differ across tasks, these losses are generally not directly comparable, which makes it difficult to formulate a unified objective and may limit information sharing across tasks. We propose a multitask transformation framework in which task-specific responses may differ through unknown monotone transformations. Motivated by high-dimensional biological applications in which the predictor dimension may diverge with the sample size while only a common subset of predictors is informative, we consider shared sparsity across tasks. Under this framework, we estimate the target functions and identify important predictors by optimizing a smoothed rank-based criterion with a group-Lasso penalty, implemented through a multitask deep neural network with a shared first layer. We establish the nonasymptotic excess-risk bounds, and variable-selection consistency for the proposed estimator. Simulation studies show that the proposed method achieves competitive prediction and variable-selection performance compared with competing approaches. Analyses of gene-expression studies with continuous, binary, and mixed outcomes further illustrate that the proposed method improves prediction and identifies biologically meaningful shared predictors.


Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization

arXiv.org Machine Learning

Transformer-based large models have demonstrated remarkable generalization abilities across different tasks by leveraging a context-aware attention module for in-context learning. With richer context, transformers adapt more effectively to the current use case without any parameter updates. However, the quadratic computational and memory complexity with respect to context length significantly slows data processing in softmax transformers. Linear transformers were proposed to address this issue by reducing the complexity to linear dependence on context length, but the design and understanding of the feature mapping in linear attention, from a theoretical viewpoint, remain unclear. In this paper, we investigate the approximation and generalization abilities of linear transformers under a two-staged sampling process from domain generalization. We show that linear transformers perform in-context learning as learning a mapping from context distributions to response functions. A dimension-independent convergence rate is obtained for our generalization analysis, which also exhibits the tradeoff between the regularities of data distributions and latent features. Guided by our theoretical framework, we propose a new perspective on activation and loss design for linearizing pretrained softmax large language models.


Prototype Language Models

arXiv.org Machine Learning

Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc. Standard language models generate tokens through a dense network pathway, causing training data's influence to be distributed across parameters rather than organized along explicit, traceable components. We introduce a prototype language model architecture, Prototypes for Interpretable Sequence Modeling (PRISM), that forms each prediction via a sparse, non-negative mixture of learned prototypes, trained with clustering objectives that anchor each prototype to coherent neighborhoods of training examples. Across architectures from 130M to 1.6B parameters trained on up to 50B tokens, prototype language models either surpass or remain within 2.5 percentage points on average downstream accuracy of matched dense baselines. We show that sparse prototype structure localizes curvature in the loss landscape, yielding a more tractable Hessian and enabling training data attribution that is ~500x faster than post hoc baselines when consuming equivalent memory. Calibrating linear prototype controllers can improve downstream accuracy by roughly 3 points while tracing those corrections back to training neighborhoods, and targeted prototype suppression can remove model behaviors without finetuning or measurable loss in generation quality.


Active-GRPO: Adaptive Imitation and Self-Improving Reasoning for Molecular Optimization

arXiv.org Machine Learning

Scientific reasoning is an increasingly important capability of large language models, yet improving the robustness and efficiency of training such reasoning remains a key open challenge. We study this problem in instruction-based molecular optimization, where answer-only supervised fine-tuning (SFT) collapses multi-step reasoning and reinforcement learning with verifiable rewards (RLVR) suffers from sparse feedback. Reference-guided Policy Optimization (RePO) mitigates both by anchoring policy updates to dataset-provided references, but its effectiveness is tightly coupled to reference quality: weak or misaligned references impose a performance ceiling. To overcome this ceiling, we propose active reasoning, a paradigm in which the policy actively decides, on a per-instance basis, when to imitate a reference and when to reinforce its own discoveries, while continuously upgrading what it imitates. We instantiate this paradigm as Active Group Relative Policy Optimization (Active-GRPO), realized through two coupled mechanisms: active imitate-reinforce and active referencing. The former performs imitation learning when the reference still outperforms the policy's own candidates, and shifts to self-improvement via reinforcement learning once the policy has generated molecules that surpass the reference. The latter continuously upgrades the reference itself by replacing it with the best policy-generated candidate discovered so far, progressively raising the imitation target and ensuring that reference guidance remains informative--rather than restrictive--throughout training. Across TOMG-Bench MOLOPT, Active-GRPO improves average SR Sim from 0.0959 for GRPO and 0.1665 for RePO to 0.1773 under matched three-seed evaluation, with statistically significant gains on LogP, MR, and QED.