Goto

Collaborating Authors

 howard


A Comprehensive Experimental Characterization of Mechanical Layer Jamming Systems

Gumowski, Jessica, Digumarti, Krishna Manaswi, Howard, David

arXiv.org Artificial Intelligence

Organisms in nature, such as Cephalopods and Pachyderms, exploit stiffness modulation to achieve amazing dexterity in the control of their appendages. In this paper, we explore the phenomenon of layer jamming, which is a popular stiffness modulation mechanism that provides an equivalent capability for soft robots. More specifically, we focus on mechanical layer jamming, which we realise through two-layer multi material structure with tooth-like protrusions. We identify key design parameters for mechanical layer jamming systems, including the ability to modulate stiffness, and perform a variety of comprehensive tests placing the specimens under bending and torsional loads to understand the influence of our selected design parameters (mainly tooth geometry) on the performance of the jammed structures. We note the ability of these structures to produce a peak change in stiffness of 5 times in bending and 3.2 times in torsion. We also measure the force required to separate the two jammed layers, an often ignored parameter in the study of jamming-induced stiffness change. This study aims to shed light on the principled design of mechanical layer jammed systems and guide researchers in the selection of appropriate designs for their specific application domains.


Gradual Forgetting: Logarithmic Compression for Extending Transformer Context Windows

Dickson, Billy, Tiganj, Zoran

arXiv.org Artificial Intelligence

Most approaches to long-context processing increase the complexity of the transformer's internal architecture by integrating mechanisms such as recurrence or auxiliary memory modules. In this work, we introduce an alternative approach that modifies the input representation itself, rather than the transformer architecture. Inspired by cognitive models of human memory, our method applies a scale-invariant logarithmic compression to the input tokens. The resulting compressed representation is processed by a standard, unmodified transformer, preserving architectural simplicity. We evaluate this approach on the WikiText-103 and PG-19 language modeling benchmarks, showing a reduction in perplexity compared to uncompressed baselines. Moreover, performance improves consistently with longer compressed temporal contexts, showing that input-level logarithmic compression is a simple and effective way to extend a transformer's long-range memory.


Improved and Generalized Upper Bounds on the Complexity of Policy Iteration

Neural Information Processing Systems

Given a Markov Decision Process (MDP) with $n$ states and $m$ actions per state, we study the number of iterations needed by Policy Iteration (PI) algorithms to converge to the optimal $\gamma$-discounted optimal policy. We consider two variations of PI: Howard's PI that changes the actions in all states with a positive advantage, and Simplex-PI that only changes the action in the state with maximal advantage. We show that Howard's PI terminates after at most $ O \left( \frac{ n m}{1-\gamma} \log \left( \frac{1}{1-\gamma} \right)\right) $ iterations, improving by a factor $O(\log n)$ a result by Hansen et al. (2013), while Simplex-PI terminates after at most $ O \left( \frac{n^2 m}{1-\gamma} \log \left( \frac{1}{1-\gamma} \right)\right) $ iterations, improving by a factor $O(\log n)$ a result by Ye (2011). Under some structural assumptions of the MDP, we then consider bounds that are independent of the discount factor~$\gamma$: given a measure of the maximal transient time $\tau_t$ and the maximal time $\tau_r$ to revisit states in recurrent classes under all policies, we show that Simplex-PI terminates after at most $ \tilde O \left( n^3 m^2 \tau_t \tau_r \right) $ iterations.


SIMPOL Model for Solving Continuous-Time Heterogeneous Agent Problems

Salguero, Ricardo Alonzo Fernández

arXiv.org Artificial Intelligence

This paper presents SIMPOL (Simplified Policy Iteration), a modular numerical framework for solving continuous-time heterogeneous agent models. The core economic problem, the optimization of consumption and savings under idiosyncratic uncertainty, is formulated as a coupled system of partial differential equations: a Hamilton-Jacobi-Bellman (HJB) equation for the agent's optimal policy and a Fokker-Planck-Kolmogorov (FPK) equation for the stationary wealth distribution. SIMPOL addresses this system using Howard's policy iteration with an *upwind* finite difference scheme that guarantees stability. A distinctive contribution is a novel consumption policy post-processing module that imposes regularity through smoothing and a projection onto an economically plausible slope band, improving convergence and model behavior. The robustness and accuracy of SIMPOL are validated through a set of integrated diagnostics, including verification of contraction in the Wasserstein-2 metric and comparison with the analytical solution of the Merton model in the no-volatility case. The framework is shown to be not only computationally efficient but also to produce solutions consistent with economic and mathematical theory, offering a reliable tool for research in quantitative macroeconomics.


Neural Policy Iteration for Stochastic Optimal Control: A Physics-Informed Approach

Kim, Yeongjong, Kim, Yeoneung, Kim, Minseok, Cho, Namkyeong

arXiv.org Artificial Intelligence

We propose a physics-informed neural network policy iteration (PINN-PI) framework for solving stochastic optimal control problems governed by second-order Hamilton--Jacobi--Bellman (HJB) equations. At each iteration, a neural network is trained to approximate the value function by minimizing the residual of a linear PDE induced by a fixed policy. This linear structure enables systematic $L^2$ error control at each policy evaluation step, and allows us to derive explicit Lipschitz-type bounds that quantify how value gradient errors propagate to the policy updates. This interpretability provides a theoretical basis for evaluating policy quality during training. Our method extends recent deterministic PINN-based approaches to stochastic settings, inheriting the global exponential convergence guarantees of classical policy iteration under mild conditions. We demonstrate the effectiveness of our method on several benchmark problems, including stochastic cartpole, pendulum problems and high-dimensional linear quadratic regulation (LQR) problems in up to 10D.


Lower Bound on Howard Policy Iteration for Deterministic Markov Decision Processes

Asadi, Ali, Chatterjee, Krishnendu, de Raaij, Jakob

arXiv.org Artificial Intelligence

Deterministic Markov Decision Processes (DMDPs) are a mathematical framework for decision-making where the outcomes and future possible actions are deterministically determined by the current action taken. DMDPs can be viewed as a finite directed weighted graph, where in each step, the controller chooses an outgoing edge. An objective is a measurable function on runs (or infinite trajectories) of the DMDP, and the value for an objective is the maximal cumulative reward (or weight) that the controller can guarantee. We consider the classical mean-payoff (aka limit-average) objective, which is a basic and fundamental objective. Howard's policy iteration algorithm is a popular method for solving DMDPs with mean-payoff objectives. Although Howard's algorithm performs well in practice, as experimental studies suggested, the best known upper bound is exponential and the current known lower bound is as follows: For the input size $I$, the algorithm requires $\tildeΩ(\sqrt{I})$ iterations, where $\tildeΩ$ hides the poly-logarithmic factors, i.e., the current lower bound on iterations is sub-linear with respect to the input size. Our main result is an improved lower bound for this fundamental algorithm where we show that for the input size $I$, the algorithm requires $\tildeΩ(I)$ iterations.


Deep reinforcement learning with time-scale invariant memory

Kabir, Md Rysul, Mochizuki-Freeman, James, Tiganj, Zoran

arXiv.org Artificial Intelligence

The ability to estimate temporal relationships is critical for both animals and artificial agents. Cognitive science and neuroscience provide remarkable insights into behavioral and neural aspects of temporal credit assignment. In particular, scale invariance of learning dynamics, observed in behavior and supported by neural data, is one of the key principles that governs animal perception: proportional rescaling of temporal relationships does not alter the overall learning efficiency. Here we integrate a computational neuroscience model of scale invariant memory into deep reinforcement learning (RL) agents. We first provide a theoretical analysis and then demonstrate through experiments that such agents can learn robustly across a wide range of temporal scales, unlike agents built with commonly used recurrent memory architectures such as LSTM. This result illustrates that incorporating computational principles from neuroscience and cognitive science into deep neural networks can enhance adaptability to complex temporal dynamics, mirroring some of the core properties of human learning.


Money for nothing: is universal basic income about to transform society?

The Guardian

When Elinor O'Donovan found out she had been randomly selected to participate in a basic income pilot scheme, she couldn't believe her luck. In return for a guaranteed salary of just over 1,400 ( 1,200) a month from the Irish government, all the 27-year-old artist had to do was fill out a bi-annual questionnaire about her wellbeing and how she spends her time. "It was like winning the lottery. I was in such disbelief," she says. The income, which she will receive until September 2025, has enabled her to give up temping and focus instead on her art.


Utilizing Weak-to-Strong Consistency for Semi-Supervised Glomeruli Segmentation

Zhang, Irina, Denholm, Jim, Hamidinekoo, Azam, Ålund, Oskar, Bagnall, Christopher, Huix, Joana Palés, Sulikowski, Michal, Vito, Ortensia, Lewis, Arthur, Unwin, Robert, Soderberg, Magnus, Burlutskiy, Nikolay, Qaiser, Talha

arXiv.org Artificial Intelligence

Accurate segmentation of glomerulus instances attains high clinical significance in the automated analysis of renal biopsies to aid in diagnosing and monitoring kidney disease. Analyzing real-world histopathology images often encompasses inter-observer variability and requires a labor-intensive process of data annotation. Therefore, conventional supervised learning approaches generally achieve sub-optimal performance when applied to external datasets. Considering these challenges, we present a semi-supervised learning approach for glomeruli segmentation based on the weak-to-strong consistency framework validated on multiple real-world datasets. Our experimental results on 3 independent datasets indicate superior performance of our approach as compared with existing supervised baseline models such as U-Net and SegFormer.


For hard-hit tech workers, AI is a silver lining

Los Angeles Times

For the thousands of tech workers recently laid off in California and across the country, the future may not be as bleak as it looks right now: Many are likely to retrain fairly quickly for new jobs in the burgeoning field of artificial intelligence. The massive rounds of layoffs at tech giants and many smaller companies were largely the result of stricter investor demands -- what managers saw as over-hiring during the pandemic and a stock market that rewarded those personnel cuts. But the industry also was clearing the way to focus on AI, which is expected to revolutionize computer-related technology and work in the years ahead -- even as it displaces jobs, previously handled by humans, in areas as varied as coding and background acting. Not only is AI taking over more standard computer programming once done entirely by humans, it is also starting to spur waves of new applications -- and with them, jobs, both tech and non-tech, in a wide range of industries, including in Southern California. "What we're seeing is a lot of tech companies are actually monetizing the AI solution," said Jenn Longnion, the Los Angeles-based founder of See & Free Consulting, which helps businesses grow sustainably.