Reinforcement learning is one of the hottest fields to be in right now, with concrete applications growing at an incredibly rapid pace, from beating video games to robotics. At its essence, reinforcement learning (RL) deals with decision making --i.e. it attempts to answer the question of how an agent should act in a given environment. Loosely speaking, all of RL comes down to either finding or evaluating a policy, which is just a way of behaving. For example, a policy could be a playing strategy in chess. A policy takes a state -- in the chess example, the position of all the pieces on the board -- and assigns an action to it. For example, given the state of your chess board, your policy might ask you to move your queen forward.

Welcome to a reinforcement learning tutorial. In this part, we're going to focus on Q-Learning. Q-Learning is a model-free form of machine learning, in the sense that the AI "agent" does not need to know or have a model of the environment that it will be in. The same algorithm can be used across a variety of environments. For a given environment, everything is broken down into "states" and "actions."

Generative adversarial imitation learning (GAIL) demonstrates tremendous success in practice, especially when combined with neural networks. Different from reinforcement learning, GAIL learns both policy and reward function from expert (human) demonstration. Despite its empirical success, it remains unclear whether GAIL with neural networks converges to the globally optimal solution. The major difficulty comes from the nonconvex-nonconcave minimax optimization structure. To bridge the gap between practice and theory, we analyze a gradient-based algorithm with alternating updates and establish its sublinear convergence to the globally optimal solution. To the best of our knowledge, our analysis establishes the global optimality and convergence rate of GAIL with neural networks for the first time.

Combinatorial optimization (CO) is the workhorse of numerous important applications in operations research, engineering and other fields and, thus, has been attracting enormous attention from the research community for over a century. Many efficient solutions to common problems involve using hand-crafted heuristics to sequentially construct a solution. Therefore, it is intriguing to see how a combinatorial optimization problem can be formulated as a sequential decision making process and whether efficient heuristics can be implicitly learned by a reinforcement learning agent to find a solution. This survey explores the synergy between CO and reinforcement learning (RL) framework, which can become a promising direction for solving combinatorial problems.

In this paper, as a study of reinforcement learning, we converge the Q function to unbounded rewards such as Gaussian distribution. From the central limit theorem, in some real-world applications it is natural to assume that rewards follow a Gaussian distribution , but existing proofs cannot guarantee convergence of the Q-function. Furthermore, in the distribution-type reinforcement learning and Bayesian reinforcement learning that have become popular in recent years, it is better to allow the reward to have a Gaussian distribution. Therefore, in this paper, we prove the convergence of the Q-function under the condition of $E[r(s,a)^2]<\infty$, which is much more relaxed than the existing research. Finally, as a bonus, a proof of the policy gradient theorem for distributed reinforcement learning is also posted.

Adversarial Machine Learning (AML) is emerging as a major field aimed at the protection of automated ML systems against security threats. The majority of work in this area has built upon a game-theoretic framework by modelling a conflict between an attacker and a defender. After reviewing game-theoretic approaches to AML, we discuss the benefits that a Bayesian Adversarial Risk Analysis perspective brings when defending ML based systems. A research agenda is included.

Do you remember that scene in Walt Disney's Bambi where the titular fawn learns to stand up and walk under its own power? It's a charming vignette in the movie, showcasing a skill that plenty of baby animals -- from pigs to giraffe to, yes, deer -- pick up within minutes of their birth. Over the first few hours of life, these animals rapidly refine their motor skills until they have full control over their own locomotion. Humans, who learn to stand holding onto things at around seven months and who begin walking at 15 months, are hopelessly sluggish by comparison. Guess what the latest task that robots have beaten us at?

Do you remember that scene in Walt Disney's Bambi where the titular fawn learns to stand up and walk under its own power? It's a charming vignette in the movie, showcasing a skill that plenty of baby animals -- from pigs to giraffe to, yes, deer -- pick up within minutes of their birth. Over the first few hours of life, these animals rapidly refine their motor skills until they have full control over their own locomotion. Humans, who learn to stand holding onto things at around seven months and who begin walking at 15 months, are hopelessly sluggish by comparison. Guess what the latest task that robots have beaten us at?

The field of robotics took one step forward--followed by another, then several more--when a robot called Rainbow Dash recently taught itself to walk. The four-legged machine only required a few hours to learn to walk backward and forward, and turn right and left while doing so. Researchers from Google, UC Berkeley and the Georgia Institute of Technology published a paper on the ArXiv preprint server describing a statistical AI technique known as deep reinforcement learning they used to produce this accomplishment, which is significant for several reasons. Most reinforcement learning deployments take place in computer-simulated environments. Rainbow Dash, however, used this technology to learn to walk in an actual physical environment.

Many current behavior generation methods struggle to handle real-world traffic situations as they do not scale well with complexity. However, behaviors can be learned off-line using data-driven approaches. Especially, reinforcement learning is promising as it implicitly learns how to behave utilizing collected experiences. In this work, we combine policy-based reinforcement learning with local optimization to foster and synthesize the best of the two methodologies. The policy-based reinforcement learning algorithm provides an initial solution and guiding reference for the post-optimization. Therefore, the optimizer only has to compute a single homotopy class, e.g.\ drive behind or in front of the other vehicle. By storing the state-history during reinforcement learning, it can be used for constraint checking and the optimizer can account for interactions. The post-optimization additionally acts as a safety-layer and the novel method, thus, can be applied in safety-critical applications. We evaluate the proposed method using lane-change scenarios with a varying number of vehicles.