We introduce RoMQA, the first benchmark for robust, multi-evidence, multi-answer question answering (QA). RoMQA contains clusters of questions that are derived from related constraints mined from the Wikidata knowledge graph. RoMQA evaluates robustness of QA models to varying constraints by measuring worst-case performance within each question cluster. Compared to prior QA datasets, RoMQA has more human-written questions that require reasoning over more evidence text and have, on average, many more correct answers. In addition, human annotators rate RoMQA questions as more natural or likely to be asked by people. We evaluate state-of-the-art large language models in zero-shot, few-shot, and fine-tuning settings, and find that RoMQA is challenging: zero-shot and few-shot models perform similarly to naive baselines, while supervised retrieval methods perform well below gold evidence upper bounds. Moreover, existing models are not robust to variations in question constraints, but can be made more robust by tuning on clusters of related questions. Our results show that RoMQA is a challenging benchmark for large language models, and provides a quantifiable test to build more robust QA methods.

Tuberculosis (TB), an infectious bacterial disease, is a significant cause of death, especially in low-income countries, with an estimated ten million new cases reported globally in $2020$. While TB is treatable, non-adherence to the medication regimen is a significant cause of morbidity and mortality. Thus, proactively identifying patients at risk of dropping off their medication regimen enables corrective measures to mitigate adverse outcomes. Using a proxy measure of extreme non-adherence and a dataset of nearly $700,000$ patients from four states in India, we formulate and solve the machine learning (ML) problem of early prediction of non-adherence based on a custom rank-based metric. We train ML models and evaluate against baselines, achieving a $\sim 100\%$ lift over rule-based baselines and $\sim 214\%$ over a random classifier, taking into account country-wide large-scale future deployment. We deal with various issues in the process, including data quality, high-cardinality categorical data, low target prevalence, distribution shift, variation across cohorts, algorithmic fairness, and the need for robustness and explainability. Our findings indicate that risk stratification of non-adherent patients is a viable, deployable-at-scale ML solution. As the official AI partner of India's Central TB Division, we are working on multiple city and state-level pilots with the goal of pan-India deployment.

Interpretable machine learning (IML) is concerned with the behavior and the properties of machine learning models. Scientists, however, are only interested in models as a gateway to understanding phenomena. Our work aligns these two perspectives and shows how to design IML property descriptors. These descriptors are IML methods that provide insight not just into the model, but also into the properties of the phenomenon the model is designed to represent. We argue that IML is necessary for scientific inference with ML models because their elements do not individually represent phenomenon properties; instead, the model in its entirety does. However, current IML research often conflates two goals of model analysis -- model audit and scientific inference -- making it unclear which model interpretations can be used to learn about phenomena. Building on statistical decision theory, we show that IML property descriptors applied on a model provide access to relevant aspects of the joint probability distribution of the data. We identify what questions such descriptors can address, provide a guide to building appropriate descriptors and quantify their epistemic uncertainty.

ML security has the same goal as all cybersecurity measures: reducing the risk of sensitive data being exposed. If a bad actor interferes with your ML model or the data it uses, that model may output incorrect results that, at best, undermine the benefits of ML and, at worst, negatively impact your business or customers. "Executives should care about this because there's nothing worse than doing the wrong thing very quickly and confidently," says Zach Hanif, vice president of machine learning platforms at Capital One. And while Hanif works in a regulated industry--financial services--requiring additional levels of governance and security, he says that every business adopting ML should take the opportunity to examine its security practices. Devon Rollins, vice president of cyber engineering and machine learning at Capital One, adds, "Securing business-critical applications requires a level of differentiated protection. It's safe to assume many deployments of ML tools at scale are critical given the role they play for the business and how they directly impact outcomes for users."

Welcome to insideBIGDATA's "Heard on the Street" round-up column! In this regular feature, we highlight thought-leadership commentaries from members of the big data ecosystem. Each edition covers the trends of the day with compelling perspectives that can provide important insights to give you a competitive advantage in the marketplace. We invite submissions with a focus on our favored technology topics areas: big data, data science, machine learning, AI and deep learning. Data is the new oil.

Historically, security has been a binary rule-based system in which the state is either 0 (this file is benign) or 1 (congratulations, you have a virus). Complex systems define these classes based on a set of rules. But, how can you face this challenge at scale with more than 450,000 new malware types identified per day and more than 1.3 billion malware types already out there? How can Security Operations Center(SOC) teams handle the explosion in new types and the sheer scale of attacks? Furthermore, security experts need to understand why a file or an event is classified as malware or as an anomaly.

Artificial Intelligence in the pharmaceutical industry has massive potential to transform drug discovery by accelerating research and timelines to make the drug more affordable and increase the likelihood of FDA approval. It's a powerful data mining tool based on massive pharmacological data and the machine learning process.AI enhances the success rate of the designed drug, helps physicians choose the preferred combination drug, helps consumers select the right medicine, and helps insurers and regulators generate a comprehensive prognosis by analyzing the various data injected by the operator. To explore the AI apps related to pharmaceuticals or the pharm industry, jump into the AI-integrated Blinx AI's AppStore and try the magic of AI https://bit.ly/3hHh4wj

Tesla will play a major role in a manslaughter trial this week over a fatal crash caused by a vehicle operating on autopilot, in what could be a defining case for the self-driving car industry. At the trial's heart is the question of who is legally responsible for a vehicle that can drive – or partially drive – itself. Kevin George Aziz Riad is on trial for his role in a 2019 crash. Police say Riad exited a freeway in southern California in a Tesla Model S, ran a red light and crashed into a Honda Civic, killing Gilberto Lopez and Maria Guadalupe Nieves-Lopez. Tesla's autopilot system, which can control speed, braking and steering, was engaged at the time of the crash that killed the couple, who were on their first date.

A Huawei executive said Thursday information and communications technologies, or ICT, will enable the digitalization of industry, spark innovation and make other industries green. The remarks were made at a session organized by the Global Innovation Hub (UGIH) of the United Nations Framework Convention on Climate Change (UNFCCC) at the ongoing 27th Conference of the Parties, or COP27, in Sharm El-Sheikh of Egypt. Referring to what is known as the "enabling effect", Philippe Wang, Huawei's Executive Vice President for the Northern Africa region, said ICT is "making other industries greener". "5G, Artificial Intelligence, data analytics, cloud computing – all these things will improve industrial processes in a way that cuts energy use, and lowers carbon emissions," he said. According to Philippe Wang, in the same way that ICT enables a smart streetlight to turn itself off when no one is around, 5G wireless base stations can automatically shut down when there is no data traffic, which saves energy.

Technology has played a pivotal role in transforming how businesses operate, for the better and for the worse. The development of semiconductors and the computing revolution that followed changed the business landscape dramatically in a few decades. The next transformational change in business will be brought about by artificial intelligence. Artificial Intelligence or AI tries to mimic human intelligence with computing power. Complex statistical operations are applied to huge amounts of data to train computers to learn some aspects of human intelligence.