Goto

Collaborating Authors

 Education


Math Word Problem Solving by Generating Linguistic Variants of Problem Statements

arXiv.org Artificial Intelligence

The art of mathematical reasoning stands as a fundamental pillar of intellectual progress and is a central catalyst in cultivating human ingenuity. Researchers have recently published a plethora of works centered around the task of solving Math Word Problems (MWP) $-$ a crucial stride towards general AI. These existing models are susceptible to dependency on shallow heuristics and spurious correlations to derive the solution expressions. In order to ameliorate this issue, in this paper, we propose a framework for MWP solvers based on the generation of linguistic variants of the problem text. The approach involves solving each of the variant problems and electing the predicted expression with the majority of the votes. We use DeBERTa (Decoding-enhanced BERT with disentangled attention) as the encoder to leverage its rich textual representations and enhanced mask decoder to construct the solution expressions. Furthermore, we introduce a challenging dataset, $\mathrm{P\small{ARA}\normalsize{MAWPS}}$, consisting of paraphrased, adversarial, and inverse variants of selectively sampled MWPs from the benchmark $\mathrm{M\small{AWPS}}$ dataset. We extensively experiment on this dataset along with other benchmark datasets using some baseline MWP solver models. We show that training on linguistic variants of problem statements and voting on candidate predictions improve the mathematical reasoning and robustness of the model. We make our code and data publicly available.


Exploring the MIT Mathematics and EECS Curriculum Using Large Language Models

arXiv.org Artificial Intelligence

We curate a comprehensive dataset of 4,550 questions and solutions from problem sets, midterm exams, and final exams across all MIT Mathematics and Electrical Engineering and Computer Science (EECS) courses required for obtaining a degree. We evaluate the ability of large language models to fulfill the graduation requirements for any MIT major in Mathematics and EECS. Our results demonstrate that GPT-3.5 successfully solves a third of the entire MIT curriculum, while GPT-4, with prompt engineering, achieves a perfect solve rate on a test set excluding questions based on images. We fine-tune an open-source large language model on this dataset. We employ GPT-4 to automatically grade model responses, providing a detailed performance breakdown by course, question, and answer type. By embedding questions in a low-dimensional space, we explore the relationships between questions, topics, and classes and discover which questions and classes are required for solving other questions and classes through few-shot learning. Our analysis offers valuable insights into course prerequisites and curriculum design, highlighting language models' potential for learning and improving Mathematics and EECS education.


In Search for a Generalizable Method for Source Free Domain Adaptation

arXiv.org Artificial Intelligence

Source-free domain adaptation (SFDA) is compelling because it allows adapting an off-the-shelf model to a new domain using only unlabelled data. In this work, we apply existing SFDA techniques to a challenging set of naturally-occurring distribution shifts in bioacoustics, which are very different from the ones commonly studied in computer vision. We find existing methods perform differently relative to each other than observed in vision benchmarks, and sometimes perform worse than no adaptation at all. We propose a new simple method which outperforms the existing methods on our new shifts while exhibiting strong performance on a range of vision datasets. Our findings suggest that existing SFDA methods are not as generalizable as previously thought and that considering diverse modalities can be a useful avenue for designing more robust models.


Robot Talk Episode 54 โ€“ Robotics and science fiction

Robohub

In this special live recording of the Robot Talk podcast at the Great Exhibition Road Festival, Claire chatted to Glyn Morgan (Science Museum), Bani Anvari (University College London) and Thrishantha Nanayakara (Imperial College London) to explore how our intelligent friends from the world of science fiction match up with state-of-the art robotics and artificial intelligence reality. Glyn Morgan is a curator of exhibitions at the Science Museum, most recently: "Science Fiction: Voyage to the Edge of Imagination" (open until August 20th). He also teaches a course on Science Fiction at Imperial College, and has published widely on many aspects of the genre writing for the Los Angeles Review of Books, the Royal Society, and the Science Fiction Research Association, amongst others. His research is interested in the interface between science fiction and other disciplines from history to psychology and beyond, and the ways science fiction can be used as a cognitive tool to help us understand ourselves and our society. Bani Anvari is a Full Professor of Intelligent Mobility at the Centre for Transport Studies in the Faculty of Engineering at University College London (UCL).


NCAA athlete claims she was scolded by AI over message about women's sports

FOX News

College volleyball player Macy Petty reacts to the U.S. House passing a bill that would ban biological males from competing in women's sports on'Fox News @ Night.' An NCAA volleyball player claims ChatGPT scolded her when she asked the artificial intelligence platform to shorten a tweet about the debate over transgender athletes participating in women's sports. "I was trying to explain [in the tweet] that I'm an NCAA athlete, and that it's important to champion the voice of female athletes and to stand up against this ideological war that's going on that's putting women in danger and taking away the opportunities for scholarships," Macy Petty told Fox News Digital in a phone interview Thursday, explaining it was "a lot of information to cram in one tweet." Petty said she is novice when it comes to using ChatGPT - OpenAI's wildly popular chatbot that can mimic human conversation based on prompts - and had seen an Instagram reel touting the importance of using the platform as the future of technology. After watching the reel, Petty said she was presented with a great opportunity to use the system: Allegedly asking ChatGPT to shorten a tweet on women's sports that had gone over the social media platform's character limit.


South Korea's Nationwide Effort for AI Semiconductor Industry

Communications of the ACM

Samsung Electronics and SK Hynix, the world's leading memory semiconductor companies, have launched investment and employment plans for their AI semiconductor and foundry businesses. Samsung Electronics is trying to develop next-generation AI semiconductor products by leveraging its strengths in mobile chipset design and memory manufacturing. Samsung develops its own neural processing units (NPUs) and integrates them into multiple processing platforms, including the Exynos mobile processor and the Exynos auto processor series. Once secretly focusing solely on their own product development, major companies are starting to open up relations with the academia and research communities to learn the latest AI technologies and engage with well-educated researchers. Another notable product development direction is putting AI computation logic into memories.


Welcome

Communications of the ACM

Welcome to the special section highlighting cutting-edge research and innovation emerging from East Asia and Oceania. Our region encompasses Southeast Asia, Oceania, and Asia-Pacific countries, including Japan and Korea. The articles in this section--designated as "Hot Topics" and "Big Trends"--aim to not only showcase technological advancements from this region, but also to strengthen research collaboration and communication with regions worldwide. This special section brings together some of the most innovative research in computer science and technology from this flourishing region. The articles cover a wide range of topics, from state-of-the-art developments in learning analytics, AI and machine learning, education, Big Data, neuromorphic computing, and blockchain technology, to applications in disease prediction and assistive devices.


ChatGPT may excel in States Medical Licensing Examination but falters in basic Linear Algebra

arXiv.org Artificial Intelligence

The emergence of ChatGPT has been rapid, and although it has demonstrated positive impacts in certain domains, its influence is not universally advantageous. Our analysis focuses on ChatGPT's capabilities in Mathematics Education, particularly in teaching basic Linear Algebra. While there are instances where ChatGPT delivers accurate and well-motivated answers, it is crucial to recognize numerous cases where it makes significant mathematical errors and fails in logical inference. These occurrences raise concerns regarding the system's genuine understanding of mathematics, as it appears to rely more on visual patterns rather than true comprehension. Additionally, the suitability of ChatGPT as a teacher for students also warrants consideration.


A Unified Approach to Controlling Implicit Regularization via Mirror Descent

arXiv.org Artificial Intelligence

Inspired by the remarkable success of deep neural networks, there has been significant interest in understanding the generalization performance of overparameterized models. Substantial efforts have been invested in characterizing how optimization algorithms impact generalization through their "preferred" solutions, a phenomenon commonly referred to as implicit regularization. In particular, it has been argued that gradient descent (GD) induces an implicit $\ell_2$-norm regularization in regression and classification problems. However, the implicit regularization of different algorithms are confined to either a specific geometry or a particular class of learning problems, indicating a gap in a general approach for controlling the implicit regularization. To address this, we present a unified approach using mirror descent (MD), a notable generalization of GD, to control implicit regularization in both regression and classification settings. More specifically, we show that MD with the general class of homogeneous potential functions converges in direction to a generalized maximum-margin solution for linear classification problems, thereby answering a long-standing question in the classification setting. Further, we show that MD can be implemented efficiently and under suitable conditions, enjoys fast convergence. Through comprehensive experiments, we demonstrate that MD is a versatile method to produce learned models with different regularizers, which in turn have different generalization performances.


Is Pre-training Truly Better Than Meta-Learning?

arXiv.org Artificial Intelligence

In the context of few-shot learning, it is currently believed that a fixed pre-trained (PT) model, along with fine-tuning the final layer during evaluation, outperforms standard meta-learning algorithms. We re-evaluate these claims under an in-depth empirical examination of an extensive set of formally diverse datasets and compare PT to Model Agnostic Meta-Learning (MAML). Unlike previous work, we emphasize a fair comparison by using: the same architecture, the same optimizer, and all models trained to convergence. Crucially, we use a more rigorous statistical tool -- the effect size (Cohen's d) -- to determine the practical significance of the difference between a model trained with PT vs. a MAML. We then use a previously proposed metric -- the diversity coefficient -- to compute the average formal diversity of a dataset. Using this analysis, we demonstrate the following: 1. when the formal diversity of a data set is low, PT beats MAML on average and 2. when the formal diversity is high, MAML beats PT on average. The caveat is that the magnitude of the average difference between a PT vs. MAML using the effect size is low (according to classical statistical thresholds) -- less than 0.2. Nevertheless, this observation is contrary to the currently held belief that a pre-trained model is always better than a meta-learning model. Our extensive experiments consider 21 few-shot learning benchmarks, including the large-scale few-shot learning dataset Meta-Data set. We also show no significant difference between a MAML model vs. a PT model with GPT-2 on Openwebtext. We, therefore, conclude that a pre-trained model does not always beat a meta-learned model and that the formal diversity of a dataset is a driving factor.