When language models (LMs) are trained via reinforcement learning (RL) to generate natural language "reasoning chains", their performance improves on a variety of difficult question answering tasks. Today, almost all successful applications of RL for reasoning use binary reward functions that evaluate the correctness of LM outputs. Because such reward functions do not penalize guessing or low-confidence outputs, they often have the unintended side-effect of degrading calibration and increasing the rate at which LMs generate incorrect responses (or "hallucinate") in other problem domains. This paper describes RLCR (Reinforcement Learning with Calibration Rewards), an approach to training reasoning models that jointly improves accuracy and calibrated confidence estimation. During RLCR, LMs generate both predictions and numerical confidence estimates after reasoning. They are trained to optimize a reward function that augments a binary correctness score with a Brier score -- a scoring rule for confidence estimates that incentivizes calibrated prediction. We first prove that this reward function (or any analogous reward function that uses a bounded, proper scoring rule) yields models whose predictions are both accurate and well-calibrated. We next show that across diverse datasets, RLCR substantially improves calibration with no loss in accuracy, on both in-domain and out-of-domain evaluations -- outperforming both ordinary RL training and classifiers trained to assign post-hoc confidence scores. While ordinary RL hurts calibration, RLCR improves it. Finally, we demonstrate that verbalized confidence can be leveraged at test time to improve accuracy and calibration via confidence-weighted scaling methods. Our results show that explicitly optimizing for calibration can produce more generally reliable reasoning models.

Estimating the parameters of ordinary differential equations (ODEs) is of fundamental importance in many scientific applications. While ODEs are typically approximated with deterministic algorithms, new research on probabilistic solvers indicates that they produce more reliable parameter estimates by better accounting for numerical errors. However, many ODE systems are highly sensitive to their parameter values. This produces deep local maxima in the likelihood function -- a problem which existing probabilistic solvers have yet to resolve. Here we present a novel probabilistic ODE likelihood approximation, DALTON, which can dramatically reduce parameter sensitivity by learning from noisy ODE measurements in a data-adaptive manner. Our approximation scales linearly in both ODE variables and time discretization points, and is applicable to ODEs with both partially-unobserved components and non-Gaussian measurement models. Several examples demonstrate that DALTON produces more accurate parameter estimates via numerical optimization than existing probabilistic ODE solvers, and even in some cases than the exact ODE likelihood itself.

Two of Glasgow's leading scientists will develop cutting-edge Artificial Intelligence (AI) technology thanks to a £20 million UK Government cash boost. The Scottish projects, at the University of Glasgow and University of Strathclyde, are among fifteen innovative projects receiving the new Turing AI fellowships as part of the UK government's ambition to establish the UK as a world leader in AI and support researchers to scale up their innovations. Dr Antonio Hurtado, University of Strathclyde, received £1.16 million. He aims to meet the growing demand across the UK economy to process large volumes of data fast and efficiently, while minimising the energy required to do so. His AI technology will use laser light, similar to those used in supermarket checkouts, to perform complex tasks at ultrafast speed – from weather forecasting to processing images for medical diagnostics.

Computational biology and bioinformatics provide vast data gold-mines from protein sequences, ideal for Language Models (LMs) taken from Natural Language Processing (NLP). These LMs reach for new prediction frontiers at low inference costs. Here, we trained two auto-regressive language models (Transformer-XL, XLNet) and two auto-encoder models (Bert, Albert) on data from UniRef and BFD containing up to 393 billion amino acids (words) from 2.1 billion protein sequences (22- and 112-times the entire English Wikipedia). The LMs were trained on the Summit supercomputer at Oak Ridge National Laboratory (ORNL), using 936 nodes (total 5616 GPUs) and one TPU Pod (V3-512 or V3-1024). We validated the advantage of up-scaling LMs to larger models supported by bigger data by predicting secondary structure (3-states: Q3=76-84, 8-states: Q8=65-73), sub-cellular localization for 10 cellular compartments (Q10=74) and whether a protein is membrane-bound or water-soluble (Q2=89). Dimensionality reduction revealed that the LM-embeddings from unlabeled data (only protein sequences) captured important biophysical properties governing protein shape. This implied learning some of the grammar of the language of life realized in protein sequences. The successful up-scaling of protein LMs through HPC to larger data sets slightly reduced the gap between models trained on evolutionary information and LMs. The official GitHub repository: https://github.com/agemagician/ProtTrans

In 2015, a worried father asked Rhema Vaithianathan a question that still weighs on her mind. A small crowd had gathered in a basement room in Pittsburgh, Pennsylvania, to hear her explain how software might tackle child abuse. Each day, the area's hotline receives dozens of calls from people who suspect that a child is in danger; some of these are then flagged by call-centre staff for investigation. But the system does not catch all cases of abuse. Vaithianathan and her colleagues had just won a half-million-dollar contract to build an algorithm to help. Vaithianathan, a health economist who co-directs the Centre for Social Data Analytics at the Auckland University of Technology in New Zealand, told the crowd how the algorithm might work. For example, a tool trained on reams of data -- including family backgrounds and criminal records -- could generate risk scores when calls come in. That could help call screeners to flag which families to investigate.

Entrepreneurs need little convincing that technology is important, rapidly evolving, and likely to have a profound impact on their businesses. But keeping track of developments, and knowing where to focus one's attention, is anything but straightforward. Analysts at PricewaterhouseCoopers (pdf) say the impact of constant technological breakthroughs represent a "megatrend" – a change so big that "every business should develop an emerging technology strategy". They have highlighted eight key areas that all businesses should pay attention to. The artificial intelligence market is growing rapidly and forecast to be worth $36bn by 2025.

Entrepreneurs need little convincing that technology is important, rapidly evolving, and likely to have a profound impact on their businesses. But keeping track of developments, and knowing where to focus one's attention, is anything but straightforward. Analysts at PricewaterhouseCoopers (pdf) say the impact of constant technological breakthroughs represent a "megatrend" – a change so big that "every business should develop an emerging technology strategy". They have highlighted eight key areas that all businesses should pay attention to. The artificial intelligence market is growing rapidly and forecast to be worth 36bn by 2025.

Entrepreneurs need little convincing that technology is important, rapidly evolving, and likely to have a profound impact on their businesses. But keeping track of developments, and knowing where to focus one's attention, is anything but straightforward. Analysts at PricewaterhouseCoopers (pdf) say the impact of constant technological breakthroughs represent a "megatrend" – a change so big that "every business should develop an emerging technology strategy". They have highlighted eight key areas that all businesses should pay attention to. The artificial intelligence market is growing rapidly and forecast to be worth 36bn by 2025.

Our Watch-Bot watches what a human is currently doing, and uses our unsupervised learning model to detect the human's forgotten actions. Once a forgotten action detected (put-milk-backto-fridge in the example), it points out the related object (milk in the example) by the laser spot in the current scene. Andrew Dalton in Engadget called it "a sort of robo-sentry." Watch-Bot is designed to keep an eye on tasks in the home or office and remind you if one of those tasks is still not done--not with a beep, not with a soothing companion-like voice, but with a laser pointer to nab the object still needing attention. Evan Ackerman in IEEE Spectrum said Watch-Bot can independently learn your household activity patterns in order to come up with its unfinished task reminders. Core components of Watch-Bot are a 3D sensor (a Kinect, in this case), a camera that can pan and tilt, a laptop, and laser pointer, said IEEE Spectrum.

About one in four owners of premium cars would buy a self-driving vehicle, according to Volvo's chief executive, who has vowed to make the technology affordable. Håkan Samuelsson said Volvo had had a deluge of interest in its "Drive me" trial in London next year, when 100 drivers will test its new autonomous driving technology on motorways and major roads. The Swedish carmaker plans to start selling vehicles equipped with the technology as early as 2020. Volvo will test the technology in Gothenburg this year. It is also looking into conducting a trial in China, where congestion and road safety are major issues.