Offline imitation from observations aims to solve MDPs where only task-specific expert states and task-agnostic non-expert state-action pairs are available. Offline imitation is useful in real-world scenarios where arbitrary interactions are costly and expert actions are unavailable. The state-of-the-art'DIstribution Correction Estimation' (DICE) methods minimize divergence of state occupancy between expert and learner policies and retrieve a policy with weighted behavior cloning; however, their results are unstable when learning from incomplete trajectories, due to a non-robust optimization in the dual domain. To address the issue, in this paper, we propose Trajectory-Aware Imitation Learning from Observations (TAILO). TAILO uses a discounted sum along the future trajectory as the weight for weighted behavior cloning. The terms for the sum are scaled by the output of a discriminator, which aims to identify expert states. Despite simplicity, TAILO works well if there exist trajectories or segments of expert behavior in the task-agnostic data, a common assumption in prior work. In experiments across multiple testbeds, we find TAILO to be more robust and effective, particularly with incomplete trajectories.

The goal is to show that under some situations, one of these problems is easy and the other is hard. To show that optimization can be harder than sampling, the construction hides the solution of an NP-hard problem as a small bump in a mostly flat function. Thus, approximate sampling is easy (the distribution is mostly uniform), but optimization would result in solving an NP-hard problem. To show that sampling can be harder than optimization, the construction amplifies the number of solutions of an NP-hard problem and plants an additional simple solution, and then encodes this into a function that is flat in many places, but has bumps at every possible solution of the NP-hard problem. Optimization is as easy as finding the planted simple solution, but, intuitively, sampling requires finding many of the hard solutions.

Offline imitation from observations aims to solve MDPs where only task-specific expert states and task-agnostic non-expert state-action pairs are available. Offline imitation is useful in real-world scenarios where arbitrary interactions are costly and expert actions are unavailable. The state-of-the-art'DIstribution Correction Estimation' (DICE) methods minimize divergence of state occupancy between expert and learner policies and retrieve a policy with weighted behavior cloning; however, their results are unstable when learning from incomplete trajectories, due to a non-robust optimization in the dual domain. To address the issue, in this paper, we propose Trajectory-Aware Imitation Learning from Observations (TAILO). TAILO uses a discounted sum along the future trajectory as the weight for weighted behavior cloning.

The role of nurses is manifold. Beyond being caretakers, they often also take up the mantles of patient advocacy, patient education, administration, emotional support, and more. In Singapore's Tan Tock Seng Hospital (TTSH), nurses don an additional hat as innovators. The hospital is home to the Nursing Innovation Bunch (NIB), a group dedicated to creating novel solutions to address the day-to-day pain points identified by hospital staff. The NIB was established in 2020, and joins other innovation initiatives like the hospital's Centre for Healthcare Innovation Living Lab to bring the ideas of nurses, healthcare workers and other allied health professionals to life.

More and more people are becoming aware that truly smart things are already here and we are definitely seeing a massive trend of artificial intelligence being used across commercial products. This makes people anxious, especially after watching a couple of episodes of Westworld. And knowing there is an AI in your todo list or in Alexa device on your kitchen table doesn't help that feeling at all. Often we hear of the bright minds of our world talking about existential threats and dangers of AI in a vague manner. We talk about implications but we rarely sit down and actually talk through possible simple solutions how to prevent inevitable scenarios.

In Lifschitz, V. (Ed.), Artificial Intelligence and Mathematical Theory of Computation: Papers in Honor of John McCarthy, pp. 359–380. Academic Press.