buettner
Consistent and Asymptotically Unbiased Estimation of Proper Calibration Errors
Popordanoska, Teodora, Gruber, Sebastian G., Tiulpin, Aleksei, Buettner, Florian, Blaschko, Matthew B.
Proper scoring rules evaluate the quality of probabilistic predictions, playing an essential role in the pursuit of accurate and well-calibrated models. Every proper score decomposes into two fundamental components -- proper calibration error and refinement -- utilizing a Bregman divergence. While uncertainty calibration has gained significant attention, current literature lacks a general estimator for these quantities with known statistical properties. To address this gap, we propose a method that allows consistent, and asymptotically unbiased estimation of all proper calibration errors and refinement terms. In particular, we introduce Kullback--Leibler calibration error, induced by the commonly used cross-entropy loss. As part of our results, we prove the relation between refinement and f-divergences, which implies information monotonicity in neural networks, regardless of which proper scoring rule is optimized. Our experiments validate empirically the claimed properties of the proposed estimator and suggest that the selection of a post-hoc calibration method should be determined by the particular calibration error of interest.
ASIC plan for AI snoops on insurance calls strains hearing
Australia's financial watchdog might be dreaming of the day when call centre surveillance software automatically catches crooked insurance sales staff. But there's still a way to go before AI-powered voice analytics can decipher the verbage that bubbles out of a sales boiler room. That's the reality check bowled up to regulators and industry rapidly spitballing prototypes of new regtech solutions as banks, insurers and auditors all trying to find ways to automatically detect bad behaviour without creating a profit sapping compliance cost sinkhole in the process. At a closely watched regtech forum late last month, ASIC outlined its findings from a trial of voice analytics software applied to a sample of life insurance sales calls. With a freshly sharpened set of teeth, the watchdog says it sees "great potential" in using voice analytics to automatically identify instances of potential misconduct in life insurance sales calls - but there's a catch.
You can't buy a self-driving BMW until 2021 (and that's a good thing)
At this point, if you're an automaker and you're not talking about autonomous cars, you might want to take a long hard look at your product roadmap. During a briefing at its Mountain View research campus, BMW talked about how it plans to bring a level 3 (autonomous driving in very specific circumstances where the driver should be ready to take over control) car to consumers in 2021 and deliver level 4 and 5 ride-hail vehicles to urban pilot programs the same year. Right now a lot of that strategy hinges on its partners while the automaker maintains the BMW brand. The varying degrees of autonomous vehicles the automaker is set to drop in 2021 are nothing new. BMW announced those plans way back in March.
Directions in AI Research and Applications at Siemens Corporate Research and Development
Buettner, Wolfram, Estenfeld, Klaus, Haugenederr, Hans, Struss, Peter
Many barriers exist today that prevent effective industrial exploitation of current and future AI research. These barriers can only be removed by people who are working at the scientific forefront in AI and know potential industrial needs. The Knowledge Processing Laboratory's research and development concentrates in the following areas: (1) natural language interfaces to knowledge-based systems and databases; (2) theoretical and experimental work on qualitative modeling and nonmonotonic reasoning for future knowledge-based systems; (3) application-specific language design, in particular, Prolog extensions; and (4) desi gn and analysis of neural networks. This article gives the reader an overview of the main topics currently being pursued in each of these areas.