Estimation of a model's confidence on its outputs is critical for Conversational AI systems based on large language models (LLMs), especially for reducing hallucination and preventing over-reliance. In this work, we provide an exhaustive exploration of methods, including approaches proposed for open- and closed-weight LLMs, aimed at quantifying and leveraging model uncertainty to improve the reliability of LLM-generated responses, specifically focusing on dialogue state tracking (DST) in task-oriented dialogue systems (TODS). Regardless of the model type, well-calibrated confidence scores are essential to handle uncertainties, thereby improving model performance. We evaluate four methods for estimating confidence scores based on softmax, raw token scores, verbalized confidences, and a combination of these methods, using the area under the curve (AUC) metric to assess calibration, with higher AUC indicating better calibration. We also enhance these with a self-probing mechanism, proposed for closed models. Furthermore, we assess these methods using an open-weight model fine-tuned for the task of DST, achieving superior joint goal accuracy (JGA). Our findings also suggest that fine-tuning open-weight LLMs can result in enhanced AUC performance, indicating better confidence score calibration.

Weeks after Assassin's Creed Valhalla proposed that Norwich in the dark ages was an ideal place to pass the time during a global pandemic, Ubisoft is back with a much sunnier escapade in Greek mythology. Immortals Fenyx Rising may sound like a chewed-up heavy metal tape your dad found at a car boot sale, but it's actually a charming open-world adventure where you bounce around the heavens solving problems for the gods. The idea for Immortals struck the developers when they were researching ancient Greece for Assassin's Creed Odyssey, but they evidently found time for a few study breaks in The Legend of Zelda: Breath of the Wild, too. Clambering over mountains while keeping an eye on your stamina meter, gathering fruit and flowers to cook potions, delving into vaults to solve physics puzzles; let's just say there are a few nods. There's even a big red ethereal foe towering ominously over the centre of the map, waiting for you to save a quartet of far-flung deities before confronting him.

Ahead of Ubisoft's Forward gaming event, the company offered us some remote demos of two of its AAA releases this year. While my colleague had no issues playing Watch Dogs Legion, my substandard internet connection meant my session with Assassin’s Creed Valhalla was taxing. After losing its way with back-to-back-to-back releases in the early-to-mid ‘10s, 2017’s Egypt-based Origins was a return to form for the Assassin’s Creed series, followed a year later by the similarly good Odyssey, which mapped mainland Greece and its many Aegean islands.

The attraction, funded by local businesses, forms the centrepiece of Norwich's Christmas lights experience, with thousands of visitors stopping to take #mytunneloflight selfies.

One of the major benefits of our self-driving future is just how much more gosh-darn productive we're all going to be. Instead of wasting brain power driving our cars to work, we'll sit back and let the ride do it for us. Suddenly, that time can be harnessed for our leisure or, more likely, to cram even more work into our days. But what will happen to all the people like me, who get sick at the thought of sitting in a car? I'm not much of a car guy, because when I think about cars, my overriding memory is vomiting on the side of the highway.