A graphical model is a structured representation of the data generating process. The traditional method to reason over random variables is to perform inference in this graphical model. However, in many cases the generating process is only a poor approximation of the much more complex true data generating process, leading to suboptimal estimations.

There are mainly two notable types of SGMs which are inherent for different learning problems.

Sequential data in our everyday life is often hierarchically structured.

One approach toward automatic grading is to learn an agent that interacts with a student's program and explores states indicative of errors via reinforcement learning. However, existing work on this approach only provides binary feedback of whether a program is correct or not, while students require finer-grained feedback on the specific errors in their programs to understand their mistakes. In this work, we show that exploring to discover errors can be cast as a meta-exploration problem.

We present the problem of reinforcement learning with exogenous termination. We define the Termination Markov Decision Process (TerMDP), an extension of the MDP framework, in which episodes may be interrupted by an external non-Markovian observer.

These RL algorithms are designed with a flat Markovian view of the environment in the form of a "state, action, reward, and next state" interface [

Markov processes over denumerable products of spaces, describing large systems of automata.

Our second contribution is to introduce an action semantic inductive bias and implement it with a technique named Entity-Bound Action Layer.

In recent years, robust Markov decision processes (MDPs) have emerged as a prominent modeling framework for dynamic decision problems affected by uncertainty.