Reinforcement learning has been successful across several applications in which agents have to learn to act in environments with sparse feedback. However, despite this empirical success there is still a lack of theoretical understanding of how the parameters of reinforcement learning models and the features used to represent states interact to control the dynamics of learning. In this work, we use concepts from statistical physics, to study the typical case learning curves for temporal difference learning of a value function with linear function approximators. Our theory is derived under a Gaussian equivalence hypothesis where averages over the random trajectories are replaced with temporally correlated Gaussian feature averages and we validate our assumptions on small scale Markov Decision Processes. We find that the stochastic semi-gradient noise due to subsampling the space of possible episodes leads to significant plateaus in the value error, unlike in traditional gradient descent dynamics. We study how learning dynamics and plateaus depend on feature structure, learning rate, discount factor, and reward function. We then analyze how strategies like learning rate annealing and reward shaping can favorably alter learning dynamics and plateaus. To conclude, our work introduces new tools to open a new direction towards developing a theory of learning dynamics in reinforcement learning.

Wasserstein barycenters have become popular due to their ability to represent the average of probability measures in a geometrically meaningful way. In this paper, we present an algorithm to approximate the Wasserstein-2 barycenters of continuous measures via a generative model. Previous approaches rely on regularization (entropic/quadratic) which introduces bias or on input convex neural networks which are not expressive enough for large-scale tasks. In contrast, our algorithm does not introduce bias and allows using arbitrary neural networks. In addition, based on the celebrity faces dataset, we construct Ave, celeba!

Kenya's Sabastian Sawe has become the first man to run a marathon in under two hours, winning the London Marathon in 1:59:30. Ethiopia's Tigst Assefa defended her London Marathon crown on Sunday, breaking her own world record. The 31-year-old, who has never lost a marathon, smashed the world record by 65 seconds. Yomif Kejelcha of Ethiopia stayed on Sawe's heels for most of the 42.195km course before fading down the final stretch to take second in his marathon debut with 1:59:41, while Jacob Kiplimo of Uganda won bronze in 2:02:28. All three finished under Kiptum's previous record time.

We propose a novel neural network-based approach to modeling protein dynamics using an implicit representation of a protein's surface in 3D and time. Our method utilizes the zero-level set of signed distance functions (SDFs) to represent protein surfaces, enabling temporally and spatially continuous representations of protein dynamics. Our experimental results demonstrate that our model accurately captures protein dynamic trajectories and can interpolate and extrapolate in 3D and time. Importantly, this is the first study to introduce this method and successfully model large-scale protein dynamics. This approach offers a promising alternative to current methods, overcoming the limitations of first-principles-based and deep learning methods, and provides a more scalable and efficient approach to modeling protein dynamics. Additionally, our surface representation approach simplifies calculations and allows identifying movement trends and amplitudes of protein domains, making it a useful tool for protein dynamics research. Codes are available at https://github.com/Sundw-818/DSR,

The tech titans are slated to duke it out in court. The tech titans are slated to duke it out in court. Musk and Altman's bitter feud over OpenAI to be laid bare in court Tesla chief believes Altman broke company's founding agreement - and legal battle promises to be explosive T he bitter rivalry between two of the tech world's most powerful men arrives in court this week, as Elon Musk's lawsuit against Sam Altman and OpenAI heads to trial in Oakland, California. The case is set to feature some of the biggest names in Silicon Valley, and its outcome could affect the course of the AI boom. Musk's suit, filed in 2024, focuses on the formative years of OpenAI when he, Altman and others co-founded the artificial intelligence company as a nonprofit with a grand purpose.

Ductless AC systems get smart tech features--and the growing pains that come with them. App works well, with voice assistant support. Ductless mini-split air-conditioners have risen in popularity dramatically in recent years. One study now pegs their domestic market share at more than 40 percent compared to larger-scale HVAC units--and the smaller systems are even more popular in Asia and Europe than in the US. Mini-splits make a compelling climate control solution for a number of reasons.

What are Russia's gains from the Iran war? 'We are not losers; we are winners' Ukrainian officials say Russian attacks in several regions have killed at least five people and damaged a ship in the port of Odesa - as Moscow claimed to have intercepted more than 200 Ukrainian drones. A Russian drone attack killed two men on Saturday in Ukraine's northeastern Sumy region, according to Governor Oleh Hryhorov. He said civilians were hit in Bilopil close to the Russian border. In the southern region of Kherson, Governor Oleksandr Prokudin said Russian shelling wounded seven people. Further east, Russian forces launched more than 700 attacks on 50 settlements in the Zaporizhia region over the past 24 hours, killing two people and injuring four, according to Governor Ivan Fedorov.