Generative AI
Anthropic's newest Claude chatbot beats OpenAI's GPT-4o in some benchmarks
Anthropic rolled out its newest AI language model on Thursday, Claude 3.5 Sonnet. The updated chatbot outperforms the company's previous top-tier model, Claude 3 Opus, while working at twice the speed. Claude users (including those on free accounts) can check it out beginning today. Sonnet, which tends to be Anthropic's most balanced model, is the first release in the Claude 3.5 family. The company says Claude 3.5 Haiku (the fastest in each generation) and Claude 3.5 Opus (the most powerful) will arrive later this year.
We're Still Waiting for the Next Big Leap in AI
When OpenAI announced GPT-4, its latest large language model, last March, it sent shockwaves through the tech world. It was clearly more capable than anything seen before at chatting, coding, and solving all sorts of thorny problems--including school homework. Anthropic, a rival to OpenAI, announced today that it has made its own AI advance that will upgrade chatbots and other use cases. But although the new model is the world's best by some measures, it's more of a step forward than a big leap. Anthropic's new model, called Claude 3.5 Sonnet, is an upgrade to its existing Claude 3 family of AI models.
How generative AI could reinvent what it means to play
After a while, however, the repetitive chitchat (or threats) of a passing stranger forces you to bump up against the truth: This is just a game. It's still fun--I had a whale of a time, honestly, looting stagecoaches, fighting in bar brawls, and stalking deer through rainy woods--but the illusion starts to weaken when you poke at it. Video games are carefully crafted objects, part of a multibillion-dollar industry, that are designed to be consumed. You play them, you loot a few stagecoaches, you finish, you move on. It may not always be like that.
CollaFuse: Collaborative Diffusion Models
Allmendinger, Simeon, Zipperling, Domenique, Struppek, Lukas, Kรผhl, Niklas
In the landscape of generative artificial intelligence, diffusion-based models have emerged as a promising method for generating synthetic images. However, the application of diffusion models poses numerous challenges, particularly concerning data availability, computational requirements, and privacy. Traditional approaches to address these shortcomings, like federated learning, often impose significant computational burdens on individual clients, especially those with constrained resources. In response to these challenges, we introduce a novel approach for distributed collaborative diffusion models inspired by split learning. Our approach facilitates collaborative training of diffusion models while alleviating client computational burdens during image synthesis. This reduced computational burden is achieved by retaining data and computationally inexpensive processes locally at each client while outsourcing the computationally expensive processes to shared, more efficient server resources. Through experiments on the common CelebA dataset, our approach demonstrates enhanced privacy by reducing the necessity for sharing raw data. These capabilities hold significant potential across various application areas, including the design of edge computing solutions. Thus, our work advances distributed machine learning by contributing to the evolution of collaborative diffusion models.
How critically can an AI think? A framework for evaluating the quality of thinking of generative artificial intelligence
Zaphir, Luke, Lodge, Jason M., Lisec, Jacinta, McGrath, Dom, Khosravi, Hassan
Generative AI such as those with large language models have created opportunities for innovative assessment design practices. Due to recent technological developments, there is a need to know the limits and capabilities of generative AI in terms of simulating cognitive skills. Assessing student critical thinking skills has been a feature of assessment for time immemorial, but the demands of digital assessment create unique challenges for equity, academic integrity and assessment authorship. Educators need a framework for determining their assessments vulnerability to generative AI to inform assessment design practices. This paper presents a framework that explores the capabilities of the LLM ChatGPT4 application, which is the current industry benchmark. This paper presents the Mapping of questions, AI vulnerability testing, Grading, Evaluation (MAGE) framework to methodically critique their assessments within their own disciplinary contexts. This critique will provide specific and targeted indications of their questions vulnerabilities in terms of the critical thinking skills. This can go on to form the basis of assessment design for their tasks.
A Large Language Model Outperforms Other Computational Approaches to the High-Throughput Phenotyping of Physician Notes
Munzir, Syed I., Hier, Daniel B., Oommen, Chelsea, Carrithers, Michael D.
High-throughput phenotyping, the automated mapping of patient signs and symptoms to standardized ontology concepts, is essential to gaining value from electronic health records (EHR) in the support of precision medicine. Despite technological advances, high-throughput phenotyping remains a challenge. This study compares three computational approaches to high-throughput phenotyping: a Large Language Model (LLM) incorporating generative AI, a Natural Language Processing (NLP) approach utilizing deep learning for span categorization, and a hybrid approach combining word vectors with machine learning. The approach that implemented GPT-4 (a Large Language Model) demonstrated superior performance, suggesting that Large Language Models are poised to be the preferred method for high-throughput phenotyping of physician notes.
Fantastic Copyrighted Beasts and How (Not) to Generate Them
He, Luxi, Huang, Yangsibo, Shi, Weijia, Xie, Tinghao, Liu, Haotian, Wang, Yue, Zettlemoyer, Luke, Zhang, Chiyuan, Chen, Danqi, Henderson, Peter
Recent studies show that image and video generation models can be prompted to reproduce copyrighted content from their training data, raising serious legal concerns around copyright infringement. Copyrighted characters, in particular, pose a difficult challenge for image generation services, with at least one lawsuit already awarding damages based on the generation of these characters. Yet, little research has empirically examined this issue. We conduct a systematic evaluation to fill this gap. First, we build CopyCat, an evaluation suite consisting of diverse copyrighted characters and a novel evaluation pipeline. Our evaluation considers both the detection of similarity to copyrighted characters and generated image's consistency with user input. Our evaluation systematically shows that both image and video generation models can still generate characters even if characters' names are not explicitly mentioned in the prompt, sometimes with only two generic keywords (e.g., prompting with "videogame, plumber" consistently generates Nintendo's Mario character). We then introduce techniques to semi-automatically identify such keywords or descriptions that trigger character generation. Using our evaluation suite, we study runtime mitigation strategies, including both existing methods and new strategies we propose. Our findings reveal that commonly employed strategies, such as prompt rewriting in the DALL-E system, are not sufficient as standalone guardrails. These strategies must be coupled with other approaches, like negative prompting, to effectively reduce the unintended generation of copyrighted characters. Our work provides empirical grounding to the discussion of copyright mitigation strategies and offers actionable insights for model deployers actively implementing them.
SPL: A Socratic Playground for Learning Powered by Large Language Model
Zhang, Liang, Lin, Jionghao, Kuang, Ziyi, Xu, Sheng, Yeasin, Mohammed, Hu, Xiangen
Dialogue-based Intelligent Tutoring Systems (ITSs) have significantly advanced adaptive and personalized learning by automating sophisticated human tutoring strategies within interactive dialogues. However, replicating the nuanced patterns of expert human communication remains a challenge in Natural Language Processing (NLP). Recent advancements in NLP, particularly Large Language Models (LLMs) such as OpenAI's GPT-4, offer promising solutions by providing human-like and context-aware responses based on extensive pre-trained knowledge. Motivated by the effectiveness of LLMs in various educational tasks (e.g., content creation and summarization, problem-solving, and automated feedback provision), our study introduces the Socratic Playground for Learning (SPL), a dialogue-based ITS powered by the GPT-4 model, which employs the Socratic teaching method to foster critical thinking among learners. Through extensive prompt engineering, SPL can generate specific learning scenarios and facilitates efficient multi-turn tutoring dialogues. The SPL system aims to enhance personalized and adaptive learning experiences tailored to individual needs, specifically focusing on improving critical thinking skills. Our pilot experimental results from essay writing tasks demonstrate SPL has the potential to improve tutoring interactions and further enhance dialogue-based ITS functionalities. Our study, exemplified by SPL, demonstrates how LLMs enhance dialogue-based ITSs and expand the accessibility and efficacy of educational technologies.
Former OpenAI Chief Scientist Announces New Safety-Focused Company
Ilya Sutskever, a co-founder and former chief scientist of OpenAI, announced on Wednesday that he's launching a new venture dubbed Safe Superintelligence Inc. Sutskever said on X that the new lab will focus solely on building a safe "superintelligence"--an industry term for a hypothetical system that's smarter than humans. Sutskever is joined at Safe SuperIntelligence Inc. by co-founders Daniel Gross, an investor and engineer who worked on AI at Apple till 2017, and Daniel Levy, another former OpenAI employee. The new American-based firm will have offices in Palo Alto, Calif., and Tel Aviv, according to a description Sutskever shared. I am starting a new company: https://t.co/BG3K3SI3A1 Sutskever was one of OpenAI's founding members, and was chief scientist during the company's meteoric rise following the release of ChatGPT.
Adobe Says It Won't Train AI Using Artists' Work. Creatives Aren't Convinced
When users first found out about Adobe's new terms of service (which were quietly updated in February), there was an uproar. Adobe told users it could access their content "through both automated and manual methods" and use "techniques such as machine learning in order to improve [Adobe's] Services and Software." Many understood the update as the company forcing users to grant unlimited access to their work, for purposes of training Adobe's generative AI: Firefly. Late on Tuesday, Adobe issued a clarification: In an updated version of its terms of service agreement, it pledged not to train AI on its user content stored locally or in the cloud and gave users the option to opt-out of content analytics. Caught in the crossfire of intellectual property lawsuits, the ambiguous language used to previously update the terms shed light on a climate of acute skepticism among artists, many of whom over rely on Adobe for their work.