I am currently taking a course in the verification of cyber-physical systems. When I say that, think formal and probabilistic verification of state machines for safety. It's a graduate course and the professor wants us all to do a large project. Anything that somewhat relates to the course material is fair game. I thought about mixing it together with machine learning.

\begin{abstract} We model individual T2DM patient blood glucose level (BGL) by stochastic process with discrete number of states mainly but not solely governed by medication regimen (e.g. insulin injections). BGL states change otherwise according to various physiological triggers which render a stochastic, statistically unknown, yet assumed to be quasi-stationary, nature of the process. In order to express incentive for being in desired healthy BGL we heuristically define a reward function which returns positive values for desirable BG levels and negative values for undesirable BG levels. The state space consists of sufficient number of states in order to allow for memoryless assumption. This, in turn, allows to formulate Markov Decision Process (MDP), with an objective to maximize the total reward, summarized over a long run. The probability law is found by model-based reinforcement learning (RL) and the optimal insulin treatment policy is retrieved from MDP solution.

Samuel Kaski) - One of the core questions in machine learning at the moment is how to interact with humans. We turn this question into a probabilistic modelling problem, and model both the user and the task to drive the interaction. The solutions need combinations of probabilistic modelling, reinforcement learning and approximate Bayesian computation. We are looking for a postdoc who already masters some of these and offer an opportunity to learn the rest and work with us on this exciting bleeding-edge problem. Antti Oulasvirta) - The position offers an exciting opportunity to learn about and work on applications of machine learning methods and computational models of cognition, perception, and behavior in interactive systems.

A new academic paper, Machine Learning for Trading, is the first conclusive study that shows success from having a machine learning-based trading strategy. The author, Gordon Ritter, Adjunct Professor in the Mathematics in Finance Program, New York University, constructed an artificial system which he knew would admit a profitable strategy, to see if a machine would find it. Newsweek is hosting an AI and Data Science in Capital Markets conference in NYC, Dec. 6-7. In order to train a machine learning algorithm to behave as a rational risk-averse investor required appropriate reinforcement learning, specifically a mathematical technique called Q-learning (playing some sort of game where you are trying to maximise the reward function that may occur at several periods in the future). The machine learning agent found and exploited arbitrage opportunities in the presence of transaction costs in a simulated market proof of concept.

We are looking for a Machine Learning Researcher with a specialised focus on Reinforcement and Active Learning. The candidate will have a sound understanding of modern machine learning, deep learning, probabilistic modelling techniques and expertise in Reinforcement and Active Learning and their applications in real-world problems. You will have the opportunity to contribute to this high performing team who seek to apply their knowledge in the high impact field of improving human's capability in drug discovery. If this challenge and opportunity excites you, please email your CV and a covering letter to careers@benevolent.ai

A new academic paper, Machine Learning for Trading, is the first conclusive study that shows success in having a machine learning-based trading strategy. The author, Gordon Ritter, Adjunct Professor in the Mathematics in Finance Program, New York University, constructed an artificial system which he knew would admit a profitable strategy, to see if a machine would find it. In order to train a machine-learning algorithm to behave as a rational risk-averse investor required appropriate reinforcement learning, specifically a mathematical technique called Q-learning (playing some sort of game where you are trying to maximise the reward function that may occur at several periods in the future). The machine learning agent found and exploited arbitrage opportunities in the presence of transaction costs in a simulated market proof of concept. Ritter explained: "I was really trying to answer the question, does machine learning have any application to trading at all, or no application; sort of a binary question. Can machine learning be applied to the problem of trading? "I reasoned that in a system that I know admits a profitable trading strategy, because I constructed it that way, can the machine find it.

Experience replay is a key technique behind many recent advances in deep reinforcement learning. Allowing the agent to learn from earlier memories can speed up learning and break undesirable temporal correlations. Despite its wide-spread application, very little is understood about the properties of experience replay. How does the amount of memory kept affect learning dynamics? Does it help to prioritize certain experiences? In this paper, we address these questions by formulating a dynamical systems ODE model of Q-learning with experience replay. We derive analytic solutions of the ODE for a simple setting. We show that even in this very simple setting, the amount of memory kept can substantially affect the agent's performance. Too much or too little memory both slow down learning. Moreover, we characterize regimes where prioritized replay harms the agent's learning. We show that our analytic solutions have excellent agreement with experiments. Finally, we propose a simple algorithm for adaptively changing the memory buffer size which achieves consistently good empirical performance.

A new academic paper, Machine Learning for Trading, is the first conclusive study that shows success in having a machine learning-based trading strategy. The author, Gordon Ritter, Adjunct Professor in the Mathematics in Finance Program, New York University, constructed an artificial system which he knew would admit a profitable strategy, to see if a machine would find it. In order to train a machine-learning algorithm to behave as a rational risk-averse investor required appropriate reinforcement learning, specifically a mathematical technique called Q-learning (playing some sort of game where you are trying to maximise the reward function that may occur at several periods in the future). The machine learning agent found and exploited arbitrage opportunities in the presence of transaction costs in a simulated market proof of concept. Ritter explained: "I was really trying to answer the question, does machine learning have any application to trading at all, or no application; sort of a binary question. Can machine learning be applied to the problem of trading? "I reasoned that in a system that I know admits a profitable trading strategy, because I constructed it that way, can the machine find it.

At the bottom of the background page, there is a more extensive list of resources on AI safety.

This paper proposes a general method for improving the structure and quality of sequences generated by a recurrent neural network (RNN), while maintaining information originally learned from data, as well as sample diversity. An RNN is first pre-trained on data using maximum likelihood estimation (MLE), and the probability distribution over the next token in the sequence learned by this model is treated as a prior policy. Another RNN is then trained using reinforcement learning (RL) to generate higher-quality outputs that account for domain-specific incentives while retaining proximity to the prior policy of the MLE RNN. To formalize this objective, we derive novel off-policy RL methods for RNNs from KL-control. The effectiveness of the approach is demonstrated on two applications; 1) generating novel musical melodies, and 2) computational molecular generation. For both problems, we show that the proposed method improves the desired properties and structure of the generated sequences, while maintaining information learned from data.