Online reinforcement learning (RL) with sparse rewards poses a challenge partly because of the lack of feedback on states leading to the goal. Furthermore, expert offline data with reward signal is rarely available to provide this feedback and bootstrap online learning. How can we guide online agents to the right solution without this on-task data? Reward shaping offers a solution by providing fine-grained signal to nudge the policy towards the optimal solution. However, reward shaping often requires domain knowledge to hand-engineer heuristics for a specific goal. To enable more general and inexpensive guidance, we propose and analyze a data-driven methodology that automatically guides RL by learning from widely available video data such as Internet recordings, off-task demonstrations, task failures, and undirected environment interaction. By learning a model of optimal goal-conditioned value from diverse passive data, we open the floor to scaling up and using various data sources to model general goal-reaching behaviors relevant to guiding online RL. Specifically, we use intent-conditioned value functions to learn from diverse videos and incorporate these goal-conditioned values into the reward. Our experiments show that video-trained value functions work well with a variety of data sources, exhibit positive transfer from human video pre-training, can generalize to unseen goals, and scale with dataset size.

Amidst all the fast-paced technological innovations, contact centres continue to be at the frontline of delivering customer experience. "Even though businesses have identified different mechanisms to reach out to users such as mobile applications, notifications etc, users still reach out to the call center. Case in point, even when you are able to book a cab in under two minutes through the app, you will want to reach out to customer care if there is a problem," shares Sourabh Gupta, Co-Founder & CEO, Vernacular.ai, an AI-first SaaS business enhancing customer experience through intelligent voice conversations. However, Sourabh points out that innovation for contact centres has been overlooked and that's why today they are unable to offer the same convenience that the business provides digitally through other mediums. This gap has come to the fore amidst the pandemic.

The travel sector has arguably been slower than other industries to take up machine learning – a subset of the larger field of Artificial Intelligence – focusing on automation methods to learn and predict, from past data. Is it a cultural phenomenon? The travel industry is among most traditional of all in terms of its main selling point – the personalised, human-facing customer experience – and has struggled to come to terms with machines replacing human recommendation and action. Today's customer is seeking more answers, more quickly, from companies before and after buying products and services – and the modern traveller is no exception. Traditional travel firms need to move with the times and respond to customer expectations.