Personal
A Neighbor-Searching Discrepancy-based Drift Detection Scheme for Learning Evolving Data
Gu, Feng, Lu, Jie, Fang, Zhen, Wang, Kun, Zhang, Guangquan
Uncertain changes in data streams present challenges for machine learning models to dynamically adapt and uphold performance in real-time. Particularly, classification boundary change, also known as real concept drift, is the major cause of classification performance deterioration. However, accurately detecting real concept drift remains challenging because the theoretical foundations of existing drift detection methods - two-sample distribution tests and monitoring classification error rate, both suffer from inherent limitations such as the inability to distinguish virtual drift (changes not affecting the classification boundary, will introduce unnecessary model maintenance), limited statistical power, or high computational cost. Furthermore, no existing detection method can provide information on the trend of the drift, which could be invaluable for model maintenance. This work presents a novel real concept drift detection method based on Neighbor-Searching Discrepancy, a new statistic that measures the classification boundary difference between two samples. The proposed method is able to detect real concept drift with high accuracy while ignoring virtual drift. It can also indicate the direction of the classification boundary change by identifying the invasion or retreat of a certain class, which is also an indicator of separability change between classes. A comprehensive evaluation of 11 experiments is conducted, including empirical verification of the proposed theory using artificial datasets, and experimental comparisons with commonly used drift handling methods on real-world datasets. The results show that the proposed theory is robust against a range of distributions and dimensions, and the drift detection method outperforms state-of-the-art alternative methods.
This seemingly ordinary podcast is anything but - can YOU tell what makes it so different?
It seems like every Z-list celebrity and reality star has a podcast these days, but can you tell what makes this one truly unique? Eagle-eyed viewers may have spotted that its co-host, Jake, appears a bit stiff and robotic - and it's not because he's camera shy. Jake is a fully artificial intelligence-generated avatar. Human podcast host Jakob Wredstrøm, who created Jake as a digital clone of himself, says his is the first AI podcast. Jakob Wredstrøm, the creator of the podcast Sound Connections, created an AI version of himself called Jake (pictured).
Measuring Social Norms of Large Language Models
Yuan, Ye, Tang, Kexin, Shen, Jianhao, Zhang, Ming, Wang, Chenguang
We present a new challenge to examine whether large language models understand social norms. In contrast to existing datasets, our dataset requires a fundamental understanding of social norms to solve. Our dataset features the largest set of social norm skills, consisting of 402 skills and 12,383 questions covering a wide set of social norms ranging from opinions and arguments to culture and laws. We design our dataset according to the K-12 curriculum. This enables the direct comparison of the social understanding of large language models to humans, more specifically, elementary students. While prior work generates nearly random accuracy on our benchmark, recent large language models such as GPT3.5-Turbo and LLaMA2-Chat are able to improve the performance significantly, only slightly below human performance. We then propose a multi-agent framework based on large language models to improve the models' ability to understand social norms. This method further improves large language models to be on par with humans. Given the increasing adoption of large language models in real-world applications, our finding is particularly important and presents a unique direction for future improvements.
Engadget Podcast: Microsoft's Surface and Windows head on Copilot AI PCs
Microsoft made some unusually major moves ahead of its Build developer conference: It announced a new Copilot initiative for powerful AI PCs, which will be led by the new Surface Pro and Surface Laptop. These machines are powered by Qualcomm's new Snapdragon X Plus and Elite chips, and they come with a special version of Windows 11 optimized for Arm mobile chips and AI. Basically, Microsoft is doing for PCs what Apple did with its M-series Macs four years ago. We still don't know how well these new machines will perform, but it sounds like Microsoft has certainly heard our complaints about Arm-based Windows devices. Listen below or subscribe on your podcast app of choice. If you've got suggestions or topics you'd like covered on the show, be sure to email us or drop a note in the comments! And be sure to check out our other podcast, Engadget News! Devindra: Hey everyone, this is Devindra here. I had a chance to chat with Pavan Davuluri, the head of Microsoft Windows and Devices, basically the team in charge of Surface and Windows. And we talked about the new Copilot Plus Surface PCs, the Surface Pro and the Surface Laptop, and the whole new Copilot Plus initiative in general. We've reviewed quite a few of the ARM based Windows PCs and you know, they have not worked out so well. So I think this could be different, at least from the benchmarks we've seen.
What Raisi's Death Means for the Future of Iran
I last interviewed Ebrahim Raisi, the ultra-hard-line President of Iran, during his début appearance at the United Nations, in 2022. He spoke belligerently and with such speed that the interpreter struggled to keep up. He was the same on the U.N. dais, where he furiously waved a photo of General Qassem Soleimani and demanded that Donald Trump be tried for ordering his assassination--a "savage, illegal, immoral crime"--in a U.S. drone strike, in 2020. Back home, Iran was in turmoil after nationwide protests erupted in response to the death, in police custody, of a twenty-two-year-old named Mahsa Amini. She had been arrested for improper hijab; too much hair was showing.
Do YOU think it sounds like Scarlett Johansson? ChatGPT's 'flirty' AI bot's voice is revealed - so, do you think it resembles the Hollywood A-lister?
Ever since Scarlett Johansson voiced an AI assistant in the sci-fi blockbuster'Her', many tech fans have dreamed of making that technology a reality. But it now seems that OpenAI may have pursued that dream too literally as they face accusations of deliberately copying Johansson's voice for ChatGPT's latest update. According to Ms Johansson's statement, the likeness is'so eerily similar to mine that close friends and news outlets could not tell the difference'. Following the allegations, OpenAI's'flirty' voice assistant has now been paused, yet tech fans have been weighing in on whether there really is a resemblance. So, do you think ChatGPT's AI voice sounds like Scarlett Johansson?
Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach
Jurenka, Irina, Kunesch, Markus, McKee, Kevin R., Gillick, Daniel, Zhu, Shaojian, Wiltberger, Sara, Phal, Shubham Milind, Hermann, Katherine, Kasenberg, Daniel, Bhoopchand, Avishkar, Anand, Ankit, Pîslar, Miruna, Chan, Stephanie, Wang, Lisa, She, Jennifer, Mahmoudieh, Parsa, Rysbek, Aliya, Ko, Wei-Jen, Huber, Andrea, Wiltshire, Brett, Elidan, Gal, Rabin, Roni, Rubinovitz, Jasmin, Pitaru, Amit, McAllister, Mac, Wilkowski, Julia, Choi, David, Engelberg, Roee, Hackmon, Lidan, Levin, Adva, Griffin, Rachel, Sears, Michael, Bar, Filip, Mesar, Mia, Jabbour, Mana, Chaudhry, Arslan, Cohan, James, Thiagarajan, Sridhar, Levine, Nir, Brown, Ben, Gorur, Dilan, Grant, Svetlana, Hashimoshoni, Rachel, Weidinger, Laura, Hu, Jieru, Chen, Dawn, Dolecki, Kuba, Akbulut, Canfer, Bileschi, Maxwell, Culp, Laura, Dong, Wen-Xin, Marchal, Nahema, Van Deman, Kelsie, Misra, Hema Bajaj, Duah, Michael, Ambar, Moran, Caciularu, Avi, Lefdal, Sandra, Summerfield, Chris, An, James, Kamienny, Pierre-Alexandre, Mohdi, Abhinit, Strinopoulous, Theofilos, Hale, Annie, Anderson, Wayne, Cobo, Luis C., Efron, Niv, Ananda, Muktha, Mohamed, Shakir, Heymans, Maureen, Ghahramani, Zoubin, Matias, Yossi, Gomes, Ben, Ibrahim, Lila
A major challenge facing the world is the provision of equitable and universal access to quality education. Recent advances in generative AI (gen AI) have created excitement about the potential of new technologies to offer a personal tutor for every learner and a teaching assistant for every teacher. The full extent of this dream, however, has not yet materialised. We argue that this is primarily due to the difficulties with verbalising pedagogical intuitions into gen AI prompts and the lack of good evaluation practices, reinforced by the challenges in defining excellent pedagogy. Here we present our work collaborating with learners and educators to translate high level principles from learning science into a pragmatic set of seven diverse educational benchmarks, spanning quantitative, qualitative, automatic and human evaluations; and to develop a new set of fine-tuning datasets to improve the pedagogical capabilities of Gemini, introducing LearnLM-Tutor. Our evaluations show that LearnLM-Tutor is consistently preferred over a prompt tuned Gemini by educators and learners on a number of pedagogical dimensions. We hope that this work can serve as a first step towards developing a comprehensive educational evaluation framework, and that this can enable rapid progress within the AI and EdTech communities towards maximising the positive impact of gen AI in education.
A machine learning framework for interpretable predictions in patient pathways: The case of predicting ICU admission for patients with symptoms of sepsis
Zilker, Sandra, Weinzierl, Sven, Kraus, Mathias, Zschech, Patrick, Matzner, Martin
Proactive analysis of patient pathways helps healthcare providers anticipate treatment-related risks, identify outcomes, and allocate resources. Machine learning (ML) can leverage a patient's complete health history to make informed decisions about future events. However, previous work has mostly relied on so-called black-box models, which are unintelligible to humans, making it difficult for clinicians to apply such models. Our work introduces PatWay-Net, an ML framework designed for interpretable predictions of admission to the intensive care unit (ICU) for patients with symptoms of sepsis. We propose a novel type of recurrent neural network and combine it with multi-layer perceptrons to process the patient pathways and produce predictive yet interpretable results. We demonstrate its utility through a comprehensive dashboard that visualizes patient health trajectories, predictive outcomes, and associated risks. Our evaluation includes both predictive performance - where PatWay-Net outperforms standard models such as decision trees, random forests, and gradient-boosted decision trees - and clinical utility, validated through structured interviews with clinicians. By providing improved predictive accuracy along with interpretable and actionable insights, PatWay-Net serves as a valuable tool for healthcare decision support in the critical case of patients with symptoms of sepsis.
World is ill-prepared for breakthroughs in AI, say experts
The world is ill-prepared for breakthroughs in artificial intelligence, according to a group of senior experts including two "godfathers" of AI, who warn that governments have made insufficient progress in regulating the technology. A shift by tech companies to autonomous systems could "massively amplify" AI's impact and governments need safety regimes that trigger regulatory action if products reach certain levels of ability, said the group. The recommendations are made by 25 experts including Geoffrey Hinton and Yoshua Bengio, two of the three "godfathers of AI" who have won the ACM Turing award – the computer science equivalent of the Nobel prize – for their work. The intervention comes as politicians, experts and tech executives prepare to meet at a two-day summit in Seoul on Tuesday. The academic paper, called "managing extreme AI risks amid rapid progress", recommends government safety frameworks that introduce tougher requirements if the technology advances rapidly.
AAAI-24 Awards
AAAI Awards were presented in February at AAAI-24 in Vancouver, Canada. Each year, the Association for the Advancement of Artificial Intelligence recognizes its members, esteemed members of the AI community, and promising students, with the following awards and honors. The AAAI Award for Artificial Intelligence for the Benefit of Humanity recognizes the positive impacts of artificial intelligence to protect, enhance, and improve human life in meaningful ways with long-lived effects. The winner of this year's award is Milind Tambe (Harvard University/Google Research). Milind has been recognized for "ground-breaking applications of novel AI techniques to public safety and security, conservation, and public health, benefiting humanity on an international scale."