Generative AI
Effects of a Prompt Engineering Intervention on Undergraduate Students' AI Self-Efficacy, AI Knowledge and Prompt Engineering Ability: A Mixed Methods Study
Woo, David James, Wang, Deliang, Yung, Tim, Guo, Kai
Prompt engineering is critical for effective interaction with large language models (LLMs) such as ChatGPT. However, efforts to teach this skill to students have been limited. This study designed and implemented a prompt engineering intervention, examining its influence on undergraduate students' AI self-efficacy, AI knowledge, and proficiency in creating effective prompts. The intervention involved 27 students who participated in a 100-minute workshop conducted during their history course at a university in Hong Kong. During the workshop, students were introduced to prompt engineering strategies, which they applied to plan the course's final essay task. Multiple data sources were collected, including students' responses to pre- and post-workshop questionnaires, pre- and post-workshop prompt libraries, and written reflections. The study's findings revealed that students demonstrated a higher level of AI self-efficacy, an enhanced understanding of AI concepts, and improved prompt engineering skills because of the intervention. These findings have implications for AI literacy education, as they highlight the importance of prompt engineering training for specific higher education use cases. This is a significant shift from students haphazardly and intuitively learning to engineer prompts. Through prompt engineering education, educators can faciitate students' effective navigation and leverage of LLMs to support their coursework.
How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course
Scholl, Andreas, Kiesler, Natalie
This research paper contributes to the computing education research community's understanding of Generative AI (GenAI) in the context of introductory programming, and specifically, how students utilize related tools, such as ChatGPT. An increased understanding of students' use is mandatory for educators and higher education institutions, as GenAI is here to stay, and its performance is likely to improve rapidly in the near future. Learning about students' use patterns is not only crucial to support their learning, but to develop adequate forms of instruction and assessment. With the rapid advancement of AI, its broad availability, and ubiquitous presence in educational environments, elaborating how AI can enhance learning experiences, especially in courses such as introductory programming is important. To date, most studies have focused on the educator's perspective on GenAI, its performance, characteristics, and limitations. However, the student perspective, and how they actually use GenAI tools in course contexts, has not been subject to a great number of studies. Therefore, this study is guided by the following research questions: (1) What do students report on their use pattern of ChatGPT in the context of introductory programming exercises? and (2) How do students perceive ChatGPT in the context of introductory programming exercises? To address these questions, computing students at a large German university were asked to solve programming tasks with the assistance of ChatGPT as part of their introductory programming course. Students (n=298) provided information regarding the use of ChatGPT, and their evaluation of the tool via an online survey. This research provides a comprehensive evaluation of ChatGPT-3.5's application by novice programmers in a higher education context...
ThinK: Thinner Key Cache by Query-Driven Pruning
Xu, Yuhui, Jie, Zhanming, Dong, Hanze, Wang, Lei, Lu, Xudong, Zhou, Aojun, Saha, Amrita, Xiong, Caiming, Sahoo, Doyen
Large language models (LLMs) (Hadi et al., 2023; Brown et al., 2020; OpenAI, 2023; Touvron et al., 2023a,b; Scao et al., 2022; Reid et al., 2024) have emerged as a dominant paradigm in natural language processing, achieving state-of-the-art performance across various tasks. A key principle, the Scaling Law (Kaplan et al., 2020), suggests that LLMs exhibit emergent abilities as model size increases, enhancing their capacity to understand context and handle long sequences (Xiong et al., 2023). This capacity growth allows LLMs to generate coherent and contextually accurate responses and enables various downstream applications, such as document summarization (Zhang et al., 2019, 2024a), code generation (Chen et al., 2021b), and conversational AI (Bordes et al., 2016; OpenAI, 2022),. Despite their success in various applications, the generation of LLMs incurs significant expenses, which escalate with increasing model size and sequence length. Notably, both the training (Strubell et al., 2020; Hoffmann et al., 2022; Dong et al., 2024a) and inference (Ainslie et al., 2023) stages involve frequent generation by LLMs, further contributing to these costs. Consequently, efficient LLMs have gained popularity in recent years (Hu et al., 2021; Wan et al., 2023). To address these challenges, quantization (Frantar et al., 2022; Lin et al., 2024; Dettmers et al., 2024; Xu et al., 2023) and pruning methods (Frankle and Carbin, 2018; Blalock et al., 2020) are employed to reduce model size. Additionally, managing long sequences presents another cost due to the transformer attention mechanism.
Silicon Valley's Trillion-Dollar Leap of Faith
Tech companies like to make two grand pronouncements about the future of artificial intelligence. First, the technology is going to usher in a revolution akin to the advent of fire, nuclear weapons, and the internet. And second, it is going to cost almost unfathomable sums of money. Silicon Valley has already triggered tens or even hundreds of billions of dollars of spending on AI, and companies only want to spend more. Their reasoning is straightforward: These companies have decided that the best way to make generative AI better is to build bigger AI models.
AI startups swap independence for Big Tech's deep pockets
It's the case of the vanishing startup: Some of Silicon Valley's most promising names in the fast-developing generative artificial intelligence space are being gobbled up by or tied to the hip of U.S. tech giants. Short on funds, in the past few months promising companies such as Inflection AI or Adept have seen founders and key executives quietly exit the stage to join the world's dominant tech companies through discrete transactions. Critics believe these deals are acquisitions in all but name and have been especially designed by Microsoft or Amazon to avoid the attention of competition regulators, which the companies strenuously deny.
Synthetic Counterfactual Faces
Ramesh, Guruprasad V, Rosenberg, Harrison, Hooda, Ashish, Fawaz, Shimaa Ahmed Kassem
Computer vision systems have been deployed in various applications involving biometrics like human faces. These systems can identify social media users, search for missing persons, and verify identity of individuals. While computer vision models are often evaluated for accuracy on available benchmarks, more annotated data is necessary to learn about their robustness and fairness against semantic distributional shifts in input data, especially in face data. Among annotated data, counterfactual examples grant strong explainability characteristics. Because collecting natural face data is prohibitively expensive, we put forth a generative AI-based framework to construct targeted, counterfactual, high-quality synthetic face data. Our synthetic data pipeline has many use cases, including face recognition systems sensitivity evaluations and image understanding system probes. The pipeline is validated with multiple user studies. We showcase the efficacy of our face generation pipeline on a leading commercial vision model. We identify facial attributes that cause vision systems to fail.
Machine Unlearning in Generative AI: A Survey
Liu, Zheyuan, Dou, Guangyao, Tan, Zhaoxuan, Tian, Yijun, Jiang, Meng
Generative AI technologies have been deployed in many places, such as (multimodal) large language models and vision generative models. Their remarkable performance should be attributed to massive training data and emergent reasoning abilities. However, the models would memorize and generate sensitive, biased, or dangerous information originated from the training data especially those from web crawl. New machine unlearning (MU) techniques are being developed to reduce or eliminate undesirable knowledge and its effects from the models, because those that were designed for traditional classification tasks could not be applied for Generative AI. We offer a comprehensive survey on many things about MU in Generative AI, such as a new problem formulation, evaluation methods, and a structured discussion on the advantages and limitations of different kinds of MU techniques. It also presents several critical challenges and promising directions in MU research. A curated list of readings can be found: https://github.com/franciscoliu/GenAI-MU-Reading.
To accept or not to accept? An IRT-TOE Framework to Understand Educators' Resistance to Generative AI in Higher Education
Kalmus, Jan-Erik, Nikiforova, Anastasija
Since the public release of Chat Generative Pre-Trained Transformer (ChatGPT), extensive discourse has emerged concerning the potential advantages and challenges of integrating Generative Artificial Intelligence (GenAI) into education. In the realm of information systems, research on technology adoption is crucial for understanding the diverse factors influencing the uptake of specific technologies. Theoretical frameworks, refined and validated over decades, serve as guiding tools to elucidate the individual and organizational dynamics, obstacles, and perceptions surrounding technology adoption. However, while several models have been proposed, they often prioritize elucidating the factors that facilitate acceptance over those that impede it, typically focusing on the student perspective and leaving a gap in empirical evidence regarding educators viewpoints. Given the pivotal role educators play in higher education, this study aims to develop a theoretical model to empirically predict the barriers preventing educators from adopting GenAI in their classrooms. Acknowledging the lack of theoretical models tailored to identifying such barriers, our approach is grounded in the Innovation Resistance Theory (IRT) framework and augmented with constructs from the Technology-Organization-Environment (TOE) framework. This model is transformed into a measurement instrument employing a quantitative approach, complemented by a qualitative approach to enrich the analysis and uncover concerns related to GenAI adoption in the higher education domain.
Is Generative AI an Existential Threat to Human Creatives? Insights from Financial Economics
With the phenomenal rise of generative AI models (e.g., large language models such as GPT or large image models such as Diffusion), there are increasing concerns about human creatives' futures. Specifically, as generative models' power further increases, will they eventually replace all human creatives' jobs? We argue that the answer is "no," even if existing generative AI models' capabilities reach their theoretical limit. Our theory has a close analogy to a familiar insight in financial economics on the impossibility of an informationally efficient market [Grossman and Stiglitz (1980)]: If generative AI models can provide all the content humans need at low variable costs, then there is no incentive for humans to spend costly resources on content creation as they cannot profit from it. But if no human creates new content, then generative AI can only learn from stale information and be unable to generate up-to-date content that reflects new happenings in the physical world. This creates a paradox.
Websites accuse AI startup Anthropic of bypassing their anti-scraping rules and protocol
Freelancer has accused Anthropic, the AI startup behind the Claude large language models, of ignoring its "do not crawl" robots.txt Meanwhile, iFixit CEO Kyle Wiens said Anthropic has ignored the website's policy prohibiting the use of its content for AI model training. Matt Barrie, the chief executive of Freelancer, told The Information that Anthropic's ClaudeBot is "the most aggressive scraper by far." His website allegedly got 3.5 million visits from the company's crawler within a span of four hours, which is "probably about five times the volume of the number two" AI crawler. Similarly, Wiens posted on X/Twitter that Anthropic's bot hit iFixit's servers a million times in 24 hours.