Goto

Collaborating Authors

 Deep Learning


AWS CEO Matt Garman Doesn't Think AI Should Replace Junior Devs

WIRED

The head of Amazon Web Services has big plans to offer AI tools to businesses, but says that replacing coders with AI is "a non-starter for anyone who's trying to build a long-term company." Amid the breathless coverage and relentless AI hype of recent years, one of the world's biggest tech companies--Amazon--has been notably absent. Matt Garman, the CEO of Amazon Web Services, is looking to change that. At the recent AWS re:Invent conference, Garman announced a bunch of frontier AI models, as well as a tool designed to let AWS customers build models of their own. That tool, Nova Forge, allows companies to engage in what's known as custom pretraining--adding their data in the process of building a base model--which should allow for vastly more customized models that suit a given company's needs. Sure, it doesn't quite have the sexiness of a Sora 2 announcement, but that's not Garman's goal: He's less interested in mass consumer use of AI and more interested in enterprise solutions that'll integrate AI into all of AWS's offerings--and have a material impact on a corporate P&L. For this week's episode of, I caught up with Garman after AWS re:Invent to talk about what the company announced, whether he feels behind in the AI race, how he thinks about managing huge teams (and managing internal dissent), and why he's not convinced that AI is (or should be) the great job thief of our era. We always start these conversations with some very quick questions, like a warmup. If AWS had a mascot, what would it be? We have a big S3 bucket sometimes that goes around, so we'll call it that. Sorry, what is an S3 bucket? An S3 bucket is like a thing that you store your S3 objects in, but we actually have a large foam big bucket that walks around and actually looks like a paint bucket. So you do have a mascot. Well, S3 has a bucket, it has a mascot. It's probably the closest we have, and I like it. What's the most expensive mistake you've ever made? Personally, the most expensive mistake I ever made was playing basketball too long and I tore my Achilles. So that cost me about nine months of being able to walk. I probably should have known that into my thirties I was well past basketball-playing age.


Despite soaring valuation, uncertainty clouds the outlook for OpenAI

The Japan Times

Three years after ChatGPT made OpenAI the leader in artificial intelligence and a household name, rivals have closed the gap and some investors are wondering if the sensation has the wherewithal to stay dominant. Investor Michael Burry, made famous in the film The Big Short, recently likened OpenAI to Netscape, which ruled the web-browser market in the mid-1990s only to lose to Microsoft's Internet Explorer. OpenAI is the next Netscape, doomed and hemorrhaging cash, Burry said recently in a post on X, formerly Twitter. In a time of both misinformation and too much information, quality journalism is more crucial than ever. By subscribing, you can help us get the story right.


OpenAI's Chief Communications Officer Is Leaving the Company

WIRED

Hannah Wong told staff she is moving on to her “next chapter.” The company will be running an executive search to find a replacement, according to a memo.


Advancing Machine Learning Optimization of Chiral Photonic Metasurface: Comparative Study of Neural Network and Genetic Algorithm Approaches

arXiv.org Machine Learning

Chiral photonic metasurfaces provide unique capabilities for tailoring light-matter interactions, which are essential for next-generation photonic devices. Here, we report an advanced optimization framework that combines deep learning and evolutionary algorithms to significantly improve both the design and performance of chiral photonic nanostructures. Building on previous work utilizing a three-layer perceptron reinforced learning and stochastic evolutionary algorithm with decaying changes and mass extinction for chiral photonic optimization, our study introduces a refined pipeline featuring a two-output neural network architecture to reduce the trade-off between high chiral dichroism (CD) and reflectivity. Additionally, we use an improved fitness function, and efficient data augmentation techniques. A comparative analysis between a neural network (NN)-based approach and a genetic algorithm (GA) is presented for structures of different interface pattern depth, material combinations, and geometric complexity. We demonstrate a twice higher CD and the impact of both the corner number and the refractive index contrast at the example of a GaP/air and PMMA/air metasurface as a result of superior optimization performance. Additionally, a substantial increase in the number of structures explored within limited computational resources is highlighted, with tailored spectral reflectivity suggested by our electromagnetic simulations, paving the way for chiral mirrors applicable to polarization-selective light-matter interaction studies.


A Nonparametric Statistics Approach to Feature Selection in Deep Neural Networks with Theoretical Guarantees

arXiv.org Machine Learning

This paper tackles the problem of feature selection in a highly challenging setting: $\mathbb{E}(y | \boldsymbol{x}) = G(\boldsymbol{x}_{\mathcal{S}_0})$, where $\mathcal{S}_0$ is the set of relevant features and $G$ is an unknown, potentially nonlinear function subject to mild smoothness conditions. Our approach begins with feature selection in deep neural networks, then generalizes the results to H{ö}lder smooth functions by exploiting the strong approximation capabilities of neural networks. Unlike conventional optimization-based deep learning methods, we reformulate neural networks as index models and estimate $\mathcal{S}_0$ using the second-order Stein's formula. This gradient-descent-free strategy guarantees feature selection consistency with a sample size requirement of $n = Ω(p^2)$, where $p$ is the feature dimension. To handle high-dimensional scenarios, we further introduce a screening-and-selection mechanism that achieves nonlinear selection consistency when $n = Ω(s \log p)$, with $s$ representing the sparsity level. Additionally, we refit a neural network on the selected features for prediction and establish performance guarantees under a relaxed sparsity assumption. Extensive simulations and real-data analyses demonstrate the strong performance of our method even in the presence of complex feature interactions.


Multiclass Graph-Based Large Margin Classifiers: Unified Approach for Support Vectors and Neural Networks

arXiv.org Machine Learning

While large margin classifiers are originally an outcome of an optimization framework, support vectors (SVs) can be obtained from geometric approaches. This article presents advances in the use of Gabriel graphs (GGs) in binary and multiclass classification problems. For Chipclass, a hyperparameter-less and optimization-less GG-based binary classifier, we discuss how activation functions and support edge (SE)-centered neurons affect the classification, proposing smoother functions and structural SV (SSV)-centered neurons to achieve margins with low probabilities and smoother classification contours. We extend the neural network architecture, which can be trained with backpropagation with a softmax function and a cross-entropy loss, or by solving a system of linear equations. A new subgraph-/distance-based membership function for graph regularization is also proposed, along with a new GG recomputation algorithm that is less computationally expensive than the standard approach. Experimental results with the Friedman test show that our method was better than previous GG-based classifiers and statistically equivalent to tree-based models.


CoDeQ: End-to-End Joint Model Compression with Dead-Zone Quantizer for High-Sparsity and Low-Precision Networks

arXiv.org Machine Learning

While joint pruning--quantization is theoretically superior to sequential application, current joint methods rely on auxiliary procedures outside the training loop for finding compression parameters. This reliance adds engineering complexity and hyperparameter tuning, while also lacking a direct data-driven gradient signal, which might result in sub-optimal compression. In this paper, we introduce CoDeQ, a simple, fully differentiable method for joint pruning--quantization. Our approach builds on a key observation: the dead-zone of a scalar quantizer is equivalent to magnitude pruning, and can be used to induce sparsity directly within the quantization operator. Concretely, we parameterize the dead-zone width and learn it via backpropagation, alongside the quantization parameters. This design provides explicit control of sparsity, regularized by a single global hyperparameter, while decoupling sparsity selection from bit-width selection. The result is a method for Compression with Dead-zone Quantizer (CoDeQ) that supports both fixed-precision and mixed-precision quantization (controlled by an optional second hyperparameter). It simultaneously determines the sparsity pattern and quantization parameters in a single end-to-end optimization. Consequently, CoDeQ does not require any auxiliary procedures, making the method architecture-agnostic and straightforward to implement. On ImageNet with ResNet-18, CoDeQ reduces bit operations to ~5% while maintaining close to full precision accuracy in both fixed and mixed-precision regimes.


Interpretable Hypothesis-Driven Trading:A Rigorous Walk-Forward Validation Framework for Market Microstructure Signals

arXiv.org Machine Learning

We develop a rigorous walk-forward validation framework for algorithmic trading designed to mitigate overfitting and lookahead bias. Our methodology combines interpretable hypothesis-driven signal generation with reinforcement learning and strict out-of-sample testing. The framework enforces strict information set discipline, employs rolling window validation across 34 independent test periods, maintains complete interpretability through natural language hypothesis explanations, and incorporates realistic transaction costs and position constraints. Validating five market microstructure patterns across 100 US equities from 2015 to 2024, the system yields modest annualized returns (0.55%, Sharpe ratio 0.33) with exceptional downside protection (maximum drawdown -2.76%) and market-neutral characteristics (beta = 0.058). Performance exhibits strong regime dependence, generating positive returns during high-volatility periods (0.60% quarterly, 2020-2024) while underperforming in stable markets (-0.16%, 2015-2019). We report statistically insignificant aggregate results (p-value 0.34) to demonstrate a reproducible, honest validation protocol that prioritizes interpretability and extends naturally to advanced hypothesis generators, including large language models. The key empirical finding reveals that daily OHLCV-based microstructure signals require elevated information arrival and trading activity to function effectively. The framework provides complete mathematical specifications and open-source implementation, establishing a template for rigorous trading system evaluation that addresses the reproducibility crisis in quantitative finance research. For researchers, practitioners, and regulators, this work demonstrates that interpretable algorithmic trading strategies can be rigorously validated without sacrificing transparency or regulatory compliance.


Evaluating Singular Value Thresholds for DNN Weight Matrices based on Random Matrix Theory

arXiv.org Machine Learning

This study evaluates thresholds for removing singular values from singular value decomposition-based low-rank approximations of deep neural network weight matrices. Each weight matrix is modeled as the sum of signal and noise matrices. The low-rank approximation is obtained by removing noise-related singular values using a threshold based on random matrix theory. To assess the adequacy of this threshold, we propose an evaluation metric based on the cosine similarity between the singular vectors of the signal and original weight matrices. The proposed metric is used in numerical experiments to compare two threshold estimation methods.


Optimized Architectures for Kolmogorov-Arnold Networks

arXiv.org Machine Learning

Efforts to improve Kolmogorov-Arnold networks (KANs) with architectural enhancements have been stymied by the complexity those enhancements bring, undermining the interpretability that makes KANs attractive in the first place. Here we study overprovisioned architectures combined with sparsification to learn compact, interpretable KANs without sacrificing accuracy. Crucially, we focus on differentiable sparsification, turning architecture search into an end-to-end optimization problem. Across function approximation benchmarks, dynamical systems forecasting, and real-world prediction tasks, we demonstrate competitive or superior accuracy while discovering substantially smaller models. Overprovisioning and sparsification are synergistic, with the combination outperforming either alone. The result is a principled path toward models that are both more expressive and more interpretable, addressing a key tension in scientific machine learning.