Generative AI (GAI) is impacting teaching and learning directly or indirectly across a range of subjects and disciplines. As educators, we need to understand the potential and limitations of AI in HCI education and ensure our graduating HCI students are aware of the potential and limitations of AI in HCI. In this paper, we report on the main pedagogical insights gained from the inclusion of generative AI into a 10 week undergraduate module. We designed the module to encourage student experimentation with GAI models as part of the design brief requirement and planned practical sessions and discussions. Our insights are based on replies to a survey sent out to the students after completing the module. Our key findings, for HCI educators, report on the use of AI as a persona for developing project ideas and creating resources for design, and AI as a mirror for reflecting students' understanding of key concepts and ideas and highlighting knowledge gaps. We also discuss potential pitfalls that should be considered and the need to assess students' literacies and assumptions of GAIs as pedagogical tools. Finally, we put forward the case for educators to take the opportunities GAI presents as an educational tool and be experimental, creative, and courageous in their practice. We end with a discussion of our findings in relation to the TPACK framework in HCI.

Combinational creativity, a form of creativity involving the blending of familiar ideas, is pivotal in design innovation. While most research focuses on how combinational creativity in design is achieved through blending elements, this study focuses on the computational interpretation, specifically identifying the 'base' and 'additive' components that constitute a creative design. To achieve this goal, the authors propose a heuristic algorithm integrating computer vision and natural language processing technologies, and implement multiple approaches based on both discriminative and generative artificial intelligence architectures. A comprehensive evaluation was conducted on a dataset created for studying combinational creativity. Among the implementations of the proposed algorithm, the most effective approach demonstrated a high accuracy in interpretation, achieving 87.5% for identifying 'base' and 80% for 'additive'. We conduct a modular analysis and an ablation experiment to assess the performance of each part in our implementations. Additionally, the study includes an analysis of error cases and bottleneck issues, providing critical insights into the limitations and challenges inherent in the computational interpretation of creative designs.

The trustworthiness of AI applications has been the subject of recent research and is also addressed in the EU's recently adopted AI Regulation. The currently emerging foundation models in the field of text, speech and image processing offer completely new possibilities for developing AI applications. This whitepaper shows how the trustworthiness of an AI application developed with foundation models can be evaluated and ensured. For this purpose, the application-specific, risk-based approach for testing and ensuring the trustworthiness of AI applications, as developed in the 'AI Assessment Catalog - Guideline for Trustworthy Artificial Intelligence' by Fraunhofer IAIS, is transferred to the context of foundation models. Special consideration is given to the fact that specific risks of foundation models can have an impact on the AI application and must also be taken into account when checking trustworthiness. Chapter 1 of the white paper explains the fundamental relationship between foundation models and AI applications based on them in terms of trustworthiness. Chapter 2 provides an introduction to the technical construction of foundation models and Chapter 3 shows how AI applications can be developed based on them. Chapter 4 provides an overview of the resulting risks regarding trustworthiness. Chapter 5 shows which requirements for AI applications and foundation models are to be expected according to the draft of the European Union's AI Regulation and Chapter 6 finally shows the system and procedure for meeting trustworthiness requirements.

Evaluating open-ended written examination responses from students is an essential yet time-intensive task for educators, requiring a high degree of effort, consistency, and precision. Recent developments in Large Language Models (LLMs) present a promising opportunity to balance the need for thorough evaluation with efficient use of educators' time. In our study, we explore the effectiveness of LLMs ChatGPT-3.5, ChatGPT-4, Claude-3, and Mistral-Large in assessing university students' open-ended answers to questions made about reference material they have studied. Each model was instructed to evaluate 54 answers repeatedly under two conditions: 10 times (10-shot) with a temperature setting of 0.0 and 10 times with a temperature of 0.5, expecting a total of 1,080 evaluations per model and 4,320 evaluations across all models. The RAG (Retrieval Augmented Generation) framework was used as the framework to make the LLMs to process the evaluation of the answers. As of spring 2024, our analysis revealed notable variations in consistency and the grading outcomes provided by studied LLMs. There is a need to comprehend strengths and weaknesses of LLMs in educational settings for evaluating open-ended written responses. Further comparative research is essential to determine the accuracy and cost-effectiveness of using LLMs for educational assessments.

OpenAI is partnering with another publisher as it moves towards a licensed approach to training materials. Dotdash Meredith, the owner of brands like People and Better Homes & Gardens, will license its content for OpenAI to train ChatGPT while the publisher will use the AI company's models to boost its in-house ad-targeting tool. As part of the arrangement, ChatGPT will display content and links attributed to Dotdash Meredith's publications. It also provides OpenAI with fully licensed training material from trusted publications. That's a welcome change after the company got in hot water for allegedly using content for training purposes without permission.

OpenAI is fighting lawsuits from artists, writers, and publishers who allege it inappropriately used their work to train the algorithms behind ChatGPT and other AI systems. On Tuesday the company announced a tool apparently designed to appease creatives and rights holders, by granting them some control over how OpenAI uses their work. The company says it will launch a tool in 2025 called Media Manager that allows content creators to opt out their work from the company's AI development. In a blog post, OpenAI described the tool as a way to allow "creators and content owners to tell us what they own" and specify "how they want their works to be included or excluded from machine learning research and training." OpenAI said that it is working with "creators, content owners, and regulators" to develop the tool and intends it to "set an industry standard."

This story originally appeared on WIRED Italia, and has been translated from Italian. The Met Gala is undoubtedly one of the most anticipated events of the year, but this time the music and entertainment celebrities who graced its red carpet had some competition for the public's attention: generative AI deepfakes. In a post published on X this morning--and now counting nearly 15 million views--Katy Perry is pictured wearing a stunning dress decorated with three-dimensional floral appliqués, which descends to the ground transforming into incredibly realistic-looking moss. But the image is far from real, as a Community Note attached to the post makes clear. Not surprisingly, a few minutes after this first photo of the singer at the Met Gala was shared, another one immediately arrived showing her wearing a bronze-colored corset and a gorgeous floral skirt, in perfect Xena style.

OpenAI is testing a new tool to identify images generated by AI and touts a 98 percent success rate. That rate, however, declines when the image has been modified.

Since late 2022, generative AI has taken the world by storm, with widespread use of tools including ChatGPT, Gemini, and Claude. Generative AI and large language model (LLM) applications are transforming how individuals find and access data and knowledge. However, the intricate relationship between open data and generative AI, and the vast potential it holds for driving innovation in this field remain underexplored areas. This white paper seeks to unpack the relationship between open data and generative AI and explore possible components of a new Fourth Wave of Open Data: Is open data becoming AI ready? Is open data moving towards a data commons approach? Is generative AI making open data more conversational? Will generative AI improve open data quality and provenance? Towards this end, we provide a new Spectrum of Scenarios framework. This framework outlines a range of scenarios in which open data and generative AI could intersect and what is required from a data quality and provenance perspective to make open data ready for those specific scenarios. These scenarios include: pertaining, adaptation, inference and insight generation, data augmentation, and open-ended exploration. Through this process, we found that in order for data holders to embrace generative AI to improve open data access and develop greater insights from open data, they first must make progress around five key areas: enhance transparency and documentation, uphold quality and integrity, promote interoperability and standards, improve accessibility and useability, and address ethical considerations.

Generative AI as a metacognitive agent: A comparative mixed-method study with human participants on ICF-mimicking exam performance Jelena Pavlović University of Belgrade, Faculty of Philosophy & Koučing centar Resarch Lab Jugoslav Krstić, Luka Mitrović, Đorđe Babić, Adrijana Milosavljević, Milena Nikolić, Tijana Karaklić & Tijana Mitrović Koučing centar Research Lab Abstract This study investigates the metacognitive capabilities of Large Language Models (LLMs) relative to human metacognition in the context of the International Coaching Federation (ICF)-mimicking exam, a situational judgment test related to coaching competencies. Using a mixed-method approach, we assessed the metacognitive performance--including sensitivity, accuracy in probabilistic predictions, and bias--of human participants and five advanced LLMs: GPT-4, Claude-3-Opus 3, Mistral Large, Llama 3, and Gemini 1.5 Pro. The results indicate that LLMs outperformed humans across all metacognitive metrics, particularly in terms of reduced overconfidence, compared to humans. However, both LLMs and humans showed less adaptability in ambiguous scenarios, adhering closely to predefined decision frameworks. The study suggests that Generative AI can effectively engage in human-like metacognitive processing without conscious awareness. Implications of the study are discussed in relation to development of AI simulators that scaffold cognitive and metacognitive aspects of mastering coaching competencies. More broadly, implications of these results are discussed in relation to development of metacognitive modules that lead towards more autonomous and intuitive AI systems. Keywords: Generative AI, metacognition, metacognitive agents, ICF exam Introduction Metacognition, the ability to understand and regulate one's cognitive processes, is a fundamental aspect of human learning, decision making and problem solving. Traditionally viewed as a conscious process, metacognition involves activities such as planning, monitoring, and evaluating one's performance during cognitive tasks. However, recent studies suggest that certain metacognitive processes can occur without conscious awareness, challenging the traditional boundaries of how metacognition is understood and measured Kentridge and Heywood (2000). In the field of generative artificial intelligence, particularly in Large Language Models (LLMs), metacognitive-like processes may manifest as algorithms adapt, learn, and optimize performance. This raises intriguing questions about the nature of metacognition in non-conscious entities and its comparison to human metacognitive processes. The present study aims to explore these questions by comparing the metacognitive processes of human participants and LLMs within the context of the International Coaching Federation (ICF) exam performance.