Recent advancements in instruction-tuning datasets have predominantly focused on specific tasks like mathematical or logical reasoning. There has been a notable gap in data designed for aligning language models to maintain topic relevance in conversations - a critical aspect for deploying chatbots to production. We introduce the CantTalkAboutThis dataset to help language models remain focused on the subject at hand during task-oriented interactions. It consists of synthetic dialogues on a wide range of conversation topics from different domains. These dialogues are interspersed with distractor turns that intentionally divert the chatbot from the predefined topic. Fine-tuning language models on this dataset helps make them resilient to deviating from the role assigned and improves their ability to maintain topical coherence compared to general-purpose instruction-tuned LLMs like GPT-4-turbo and Mixtral-Instruct. Additionally, preliminary observations suggest that training models on this dataset also enhance their performance on fine-grained instruction following tasks, including safety alignment.

A common pipeline in learning-based control is to iteratively estimate a model of system dynamics, and apply a trajectory optimization algorithm - e.g.~$\mathtt{iLQR}$ - on the learned model to minimize a target cost. This paper conducts a rigorous analysis of a simplified variant of this strategy for general nonlinear systems. We analyze an algorithm which iterates between estimating local linear models of nonlinear system dynamics and performing $\mathtt{iLQR}$-like policy updates. We demonstrate that this algorithm attains sample complexity polynomial in relevant problem parameters, and, by synthesizing locally stabilizing gains, overcomes exponential dependence in problem horizon. Experimental results validate the performance of our algorithm, and compare to natural deep-learning baselines.

Remember Tay? That's what I immediately fixed upon when Microsoft's new Bing started spouting racist terms in front of my fifth-grader. I have two sons, and both of them are familiar with ChatGPT, OpenAI's AI-powered tool. When Bing launched its own AI-powered search engine and chatbot this week, my first thought upon returning home was to show them how it worked, and how it compared with a tool that they had seen before. As it happened, my youngest son was home sick, so he was the first person I began showing Bing to when he walked in my office. I started giving him a tour of the interface, as I had done in my hands-on with the new Bing, but with an emphasis on how Bing explains things at length, how it uses footnotes, and, most of all, includes safeguards to prevent users from tricking it into using hateful language like Tay had done.

In 2021, Spiceworks reported survey results that revealed, "Almost one-third (31%) of the professionals surveyed said their organizations are now using artificial intelligence (AI), and 43% are exploring the technology. About 34% reported their companies had not deployed any AI projects." This and other surveys show that most companies are in early stages of AI adoption -- and they most likely have not yet thought about change management for their AI systems, and what it's going to take to keep their AI systems up, running and relevant. In 2016, Microsoft developed a chatbot called Tay. Tay was designed to learn from human interactions on social media.

Nowadays, much of humanity's hopes are placed in the development of Artificial Intelligence (AI), which is seen as a way to cure diseases, improve diagnostics or care for the environment. However, there are also many fears motivated by the possibility that the algorithms could end up escaping human control. In fact, some intellectual figures of the stature of the late physicist Stephen Hawking have reflected on the apocalyptic risk of these technologies, a warning that has been joined by others such as tech magnate Elon Musk and UN High Commissioner for Human Rights Michelle Bachelet. While the debate continues, here is a series of developments in recent years that will neither save the world nor bring about its demise, but rather serve to entertain us with the more curious side of AI. Can the sexual orientation of people be detected by their appearance?

It's guiding our internet search results, removing the guesswork from our online shopping experience, assisting us in selecting the next Netflix show and even in the ads we see on Facebook. Algorithms are nothing more than math and code. However, they are created by humans and rely on our data. Since humans are susceptible to error and prejudice, the algorithms they create may have errors too. Depending on who designs them, how they are built, and how they are actually used, these systems may be biased.

Last week, researchers at Facebook's parent company Meta released BlenderBot 3, a "publicly available chatbot that improves its skills and safety over time". The chatbot is built on top of Meta's OPT-175B language model, effectively the company's white-label version of the more famous GPT-3 AI. Like most state-of-the-art AIs these days, that was trained on a vast corpus of text scraped from the internet in questionable ways, and poured into a datacentre with thousands of expensive chips that turned the text into something approaching coherence. But where OPT-175B is a general-purpose textbot, able to do anything from write fiction and answer questions to generate spam emails, BlenderBot 3 is a narrower project: it can have a conversation with you. That focus allows it to bring in other expertise, though, and one of Meta's most significant successes is hooking the language model up to the broader internet.

Meta's AI research labs have created a new state-of-the-art chatbot and are letting members of the public talk to the system in order to collect feedback on its capabilities. The bot is called BlenderBot 3 and can be accessed on the web. BlenderBot 3 is able to engage in general chitchat, says Meta, but also answer the sort of queries you might ask a digital assistant, "from talking about healthy food recipes to finding child-friendly amenities in the city." The bot is a prototype and built on Meta's previous work with what are known as large language models or LLMS -- powerful but flawed text-generation software of which OpenAI's GPT-3 is the most widely known example. Like all LLMs, BlenderBot is initially trained on vast datasets of text, which it mines for statistical patterns in order to generate language.

Releasing a new artificial intelligence system that learns from people on the internet can be a risky proposition. Just ask Microsoft's "teen" chatbot Tay--except you can't, because Tay was taken down when it started reproducing sexist and racist remarks shortly after it launched in 2016. Meta apparently believes that AI can learn to do better. The company just announced BlenderBot 3, a much more advanced chatbot designed to learn through conversation without getting into the kind of trouble that derailed Tay. And the company has made it available for anyone to try out.

When Microsoft unleashed Tay, its AI-powered chatbot, on Twitter on 23 March 2016, the software giant's hope was that it would "engage and entertain people… through casual and playful conversation". An acronym for'thinking about you', Tay was designed to mimic the language patterns of a 19-year-old American girl and learn by interacting with human users on the social network. Within hours, things had gone badly wrong. Trolls tweeted politically incorrect phrases at the bot in a bid to manipulate its behaviour. Sure enough, Tay started spewing out racist, sexist and other inflammatory messages to its following of more than 100,000 users. Microsoft was forced to lock the @TayandYou account indefinitely less than a day later, but not before its creation had tweeted more than 96,000 times.