Despite possessing impressive skills, Large Language Models (LLMs) often fail unpredictably, demonstrating inconsistent success in even basic common sense reasoning tasks. This unpredictability poses a significant challenge to ensuring their safe deployment, as identifying and operating within a reliable "safe zone" is essential for mitigating risks. To address this, we present PredictaBoard, a novel collaborative benchmarking framework designed to evaluate the ability of score predictors (referred to as assessors) to anticipate LLM errors on specific task instances (i.e., prompts) from existing datasets. PredictaBoard evaluates pairs of LLMs and assessors by considering the rejection rate at different tolerance errors. As such, PredictaBoard stimulates research into developing better assessors and making LLMs more predictable, not only with a higher average performance. We conduct illustrative experiments using baseline assessors and state-of-the-art LLMs. PredictaBoard highlights the critical need to evaluate predictability alongside performance, paving the way for safer AI systems where errors are not only minimised but also anticipated and effectively mitigated. Code for our benchmark can be found at https://github.com/Kinds-of-Intelligence-CFI/PredictaBoard

Forecast combinations have flourished remarkably in the forecasting community and, in recent years, have become part of the mainstream of forecasting research and activities. Combining multiple forecasts produced from single (target) series is now widely used to improve accuracy through the integration of information gleaned from different sources, thereby mitigating the risk of identifying a single "best" forecast. Combination schemes have evolved from simple combination methods without estimation, to sophisticated methods involving time-varying weights, nonlinear combinations, correlations among components, and cross-learning. They include combining point forecasts and combining probabilistic forecasts. This paper provides an up-to-date review of the extensive literature on forecast combinations, together with reference to available open-source software implementations. We discuss the potential and limitations of various methods and highlight how these ideas have developed over time. Some important issues concerning the utility of forecast combinations are also surveyed. Finally, we conclude with current research gaps and potential insights for future research.

This book is intended to support data scientists and developers so they can quickly enter the area of text analytics and natural language processing. Thus, we put the focus on developing practical solutions that can serve as blueprints in your daily business. A blueprint, in our definition, is a best-practice solution for a common problem. It is a template that you can easily copy and adapt for reuse. For these blueprints we use production-ready Python frameworks for data analysis, natural language processing, and machine learning.

For a long time, it was an accepted truth that video games just didn't work on screen. Remember the quasi-cyberpunk 1993 Super Mario movie, starring Dennis Hopper? It was so bad that basically everyone involved with it has disavowed it. And TV? Kids of the 90s will remember the incredibly annoying voice of Sonic the Hedgehog on Saturday morning TV – or the permanent repeats of the Pokémon anime series – but other than that, the entertainment world never took games seriously. Now, though, things are different.

Spectators sit stacked one row behind another, all craning their necks to look down at the miniaturized soccer pitch below them. With bated breath, they watch as a tiny player gently bumps the ball up the pitch toward its opponent. The goal in sight, the attacker has two options -- evade its opponent by expertly moving the ball around it, or send a safer pass to a teammate outside the fray. Like any soccer star, the player chooses glory and begins to putter its feet back and forth to move the ball -- when it begins to lose balance. And down it falls, like a felled tree.

The first idea that likely springs to mind when you think about artificial intelligence and cars is autonomous driving. But the auto industry has many other uses for AI: Collecting and parsing safety data, and design, to name just two. Given the hype around the opportunities presented by intelligent machines, we might expect that segment of the industry to be booming. Like a host of other industries where leaders want to explore and employ AI but can't, there just aren't enough people to hire. Car companies globally aren't progressing AI projects nearly as fast as trends suggested they would two years ago, according to a report released last month by the Research Institute of Capgemini, a global consultancy firm.

It's a college major that sounds straight out of science fiction: Starting this fall, at least two U.S. schools are offering degrees focused on artificial intelligence. Carnegie Mellon University in Pittsburgh announced on May 10 that the school will launch a bachelor of science program in artificial intelligence this fall. "Specialists in artificial intelligence have never been more important, in shorter supply or in greater demand by employers," said Andrew Moore, dean of the School of Computer Science, in a statement. Students in the computer-science school can enter the degree program in their second year. The course of study will include the same computer science and math courses as other students in the school, but will "focus more on how complex inputs -- such as vision, language and huge databases -- are used to make decisions or enhance human capabilities," the statement says.

The Tesla Autopilot system was engaged when a Tesla Model S sedan was crushed as it rammed into a stopped truck at 60 MPH in Utah last week, the driver has told police. The driver luckily escaped with only a broken foot, though the car suffered extensive damage. When interviewed by police, the Tesla's 28-year-old driver "said that she had been using the'Autopilot' feature," and was looking at her phone shortly before the accident in South Jordan, near Salt Lake City, according to a statement Monday from South Jordan Police Sgt. Winkler told The Deseret News that the diver, who hasn't been identified by police, had entered an address into the car's GPS and was looking on her phone for possible alternate routes. She "looked up just as the accident was about to happen," he said.

Artificial Intelligence, which has suddenly caught the attention of the IT industry and the governments across the world, is facing a large skills shortage, a top Microsoft official has said. The fast-emerging field of Artificial Intelligence, which has suddenly caught the attention of the IT industry and the governments across the world, is facing a large skills shortage, a top Microsoft official has said. The Artificial Intelligence (AI) is also facing the challenge of appropriate use of data, group programme manager of Microsoft Learning Matt Winkler told PTI. "There is a pretty large skills shortage. Lots of folks are talking about it (AI). A lot of folks are very, very excited about it and then they want to go and make that real. And when they go to make that real, there's a really large skills shortage," Winkler said.

The National Transportation Safety Board is investigating a crash and fire involving a Telsa Model S car. Two teens died in Fort Lauderdale, Florida crash on Tuesday. The probe is not expected to involve Tesla's semi-autonomous Autopilot system. A Tesla sedan with a semi-autonomous Autopilot feature rear-ended a fire department truck at 60 mph (97 kph) apparently without braking before impact on May 11, 2018, but police say it's unknown if the Autopilot feature was engaged. SAN FRANCISCO -- A Tesla Model S that crashed into a stopped fire truck at 60 mph was operating in Autopilot mode, according to Utah police officials.