An important use case is offered by vehicular ground users are static and have known locations. The same wireless networks, in which UABSs serve as relays authors in [28] extended their previous work by considering between vehicular users and the network, enabling the users multiple UABSs. Unlike these previous works, in this paper, to upload data collected by on-board sensors [5]-[11]. Such we consider traffic conditions characterized by vehicular users user-generated data are collected by the network, and then with a priori unknown locations and we move beyond conventional forwarded to other vehicles by means of BSs or road side meta-RL by accounting for the constraint that simulators units (RSUs). Being able to offer stronger, possibly line-ofsight for previous traffic configurations cannot be revisited. The (LoS), links to vehicles as compared to (static) ground rest of the paper is organized as follows. The system model BSs, UABSs can support demanding vehicle-to-everything and the problem formulation are described in Section II.

Reinforcement Learning is a subfield of Machine Learning in which an agent explores an environment to learn how to perform specific tasks by taking actions with a good outcome and avoiding those with a bad one. A reinforcement learning model will learn from its experiences and will identify which actions lead to the best rewards. In reinforcement learning, the agent takes action based on the state of the environment, and the environment will return the reward and the next state. The agent employs a trial and error method to learn. It initially takes random actions and identifies which actions lead to long-term rewards over time.

We develop a regression based primal-dual martingale approach for solving finite time horizon MDPs with general state and action space. As a result, our method allows for the construction of tight upper and lower biased approximations of the value functions, and, provides tight approximations to the optimal policy. In particular, we prove tight error bounds for the estimated duality gap featuring polynomial dependence on the time horizon, and sublinear dependence on the cardinality/dimension of the possibly infinite state and action space. From a computational point of view the proposed method is efficient since, in contrast to usual duality-based methods for optimal control problems in the literature, the Monte Carlo procedures here involved do not require nested simulations.

We propose a new class of deep reinforcement learning (RL) algorithms that model latent representations in hyperbolic space. Sequential decision-making requires reasoning about the possible future consequences of current behavior. Consequently, capturing the relationship between key evolving features for a given task is conducive to recovering effective policies. To this end, hyperbolic geometry provides deep RL models with a natural basis to precisely encode this inherently hierarchical information. However, applying existing methodologies from the hyperbolic deep learning literature leads to fatal optimization instabilities due to the non-stationarity and variance characterizing RL gradient estimators. Hence, we design a new general method that counteracts such optimization challenges and enables stable end-to-end learning with deep hyperbolic representations. We empirically validate our framework by applying it to popular on-policy and off-policy RL algorithms on the Procgen and Atari 100K benchmarks, attaining near universal performance and generalization benefits. Given its natural fit, we hope future RL research will consider hyperbolic representations as a standard tool.

Autonomous agents that act with each other on behalf of humans are becoming more common in many social domains, such as customer service, transportation, and health care. In such social situations greedy strategies can reduce the positive outcome for all agents, such as leading to stop-and-go traffic on highways, or causing a denial of service on a communications channel. Instead, we desire autonomous decision-making for efficient performance while also considering equitability of the group to avoid these pitfalls. Unfortunately, in complex situations it is far easier to design machine learning objectives for selfish strategies than for equitable behaviors. Here we present a simple way to reward groups of agents in both evolution and reinforcement learning domains by the performance of their weakest member. We show how this yields ``fairer'' more equitable behavior, while also maximizing individual outcomes, and we show the relationship to biological selection mechanisms of group-level selection and inclusive fitness theory.

Fostered by novel analytical techniques, digitalization and automation, modern bioprocess development provides high amounts of heterogeneous experimental data, containing valuable process information. In this context, data-driven methods like machine learning (ML) approaches have a high potential to rationally explore large design spaces while exploiting experimental facilities most efficiently. The aim of this review is to demonstrate how ML methods have been applied so far in bioprocess development, especially in strain engineering and selection, bioprocess optimization, scale-up, monitoring and control of bioprocesses. For each topic, we will highlight successful application cases, current challenges and point out domains that can potentially benefit from technology transfer and further progress in the field of ML.

While there is a long line of for Markov games (MGs) almost exclusively related works on the theoretical understanding of singleagent build on the "optimism in the face of uncertainty" RL with general function approximation (Jiang et al., (OFU) principle. This work focuses on a different 2017; Sun et al., 2019; Wang et al., 2020; Jin et al., 2021a; approach of posterior sampling, which is Du et al., 2021; Dann et al., 2021), the theory of MARL celebrated in many bandits and reinforcement with general function approximation is substantially less learning settings but remains under-explored for explored. In this paper, we aim to explore this topic in MGs. Specifically, for episodic two-player zerosum the context of two-player zero-sum Markov games (MGs) MGs, a novel posterior sampling algorithm (Shapley, 1953; Littman, 1994). is developed with general function approximation. Theoretical analysis demonstrates that the The goal of learning in a two-player zero-sum MG is to posterior sampling algorithm admits a T -regret learn the Nash equilibrium at which the policy of each bound for problems with a low multi-agent decoupling player maximizes her own cumulative rewards, provided coefficient, which is a new complexity that the policies of other agents are fixed.

We present a family $\{\hat{\pi}\}_{p\ge 1}$ of pessimistic learning rules for offline learning of linear contextual bandits, relying on confidence sets with respect to different $\ell_p$ norms, where $\hat{\pi}_2$ corresponds to Bellman-consistent pessimism (BCP), while $\hat{\pi}_\infty$ is a novel generalization of lower confidence bound (LCB) to the linear setting. We show that the novel $\hat{\pi}_\infty$ learning rule is, in a sense, adaptively optimal, as it achieves the minimax performance (up to log factors) against all $\ell_q$-constrained problems, and as such it strictly dominates all other predictors in the family, including $\hat{\pi}_2$.

For the most comfortable, human-aware robot navigation, subjective user preferences need to be taken into account. This paper presents a novel reinforcement learning framework to train a personalized navigation controller along with an intuitive virtual reality demonstration interface. The conducted user study provides evidence that our personalized approach significantly outperforms classical approaches with more comfortable human-robot experiences. We achieve these results using only a few demonstration trajectories from non-expert users, who predominantly appreciate the intuitive demonstration setup. As we show in the experiments, the learned controller generalizes well to states not covered in the demonstration data, while still reflecting user preferences during navigation. Finally, we transfer the navigation controller without loss in performance to a real robot.

With the recent advances in mobile energy storage technologies, electric vehicles (EVs) have become a crucial part of smart grids. When EVs participate in the demand response program, the charging cost can be significantly reduced by taking full advantage of the real-time pricing signals. However, many stochastic factors exist in the dynamic environment, bringing significant challenges to design an optimal charging/discharging control strategy. This paper develops an optimal EV charging/discharging control strategy for different EV users under dynamic environments to maximize EV users' benefits. We first formulate this problem as a Markov decision process (MDP). Then we consider EV users with different behaviors as agents in different environments. Furthermore, a horizontal federated reinforcement learning (HFRL)-based method is proposed to fit various users' behaviors and dynamic environments. This approach can learn an optimal charging/discharging control strategy without sharing users' profiles. Simulation results illustrate that the proposed real-time EV charging/discharging control strategy can perform well among various stochastic factors.