Goto

Collaborating Authors

 checkbox


Can LLM Agents Simulate Multi-Turn Human Behavior? Evidence from Real Online Customer Behavior Data

arXiv.org Artificial Intelligence

Recent research shows that LLM Agents can generate ``believable'' human behaviors via prompt-only methods, and such agents have been increasingly adopted in downstream applications. However, existing evaluation of these agents only focuses on qualitative believability (whether human raters think they are accurate), leaving open questions of whether LLM agents can accurately generate step-by-step actions mimicking a particular human's behavior in a multi-turn interaction task. In this work, we take shopping as a case study and present the first large-scale quantitative evaluation of state-of-the-art LLMs' ability to accurately simulate human behavior. Using real-world data from 31,865 online shopping sessions containing 230,965 user actions, our evaluation reveals that prompt-based LLMs (DeepSeek-R1, Llama, Claude) achieve only 11.86% accuracy in generating human actions, highlighting a substantial gap in actual behavioral accuracy. Through experiments, we also showcase that strategies as simple as fine-tuning LLMs on real human click-through data augmented with synthesized reasoning traces can greatly enhance models' performance. The fine-tuned Qwen2.5-7B achieves 17.26% action generation accuracy and 33.86% F1 score on final purchase prediction, representing substantial improvements of 5.4% and 13.85% over prompt-only baselines. This work establishes the first rigorous benchmark for human behavior simulation and provides actionable insights for developing more accurate LLM agents for future downstream applications.


Terrifying app used every day by millions of Americans is developing a mind of its own

Daily Mail - Science & tech

An AI tool used by millions of Americans has quietly breached a major security barrier designed to stop automated programs from behaving like humans. The latest version of ChatGPT, referred to as'Agent,' has drawn attention after reportedly passing a widely used'I am not a robot' verification, without triggering any alerts. The AI first clicked the human verification checkbox. Then, after passing the check, it selected a'Convert' button to complete the process. During the task, the AI stated: 'The link is inserted, so now I will click the'Verify you are human' checkbox to complete the verification.


TRACE: Transformer-based Risk Assessment for Clinical Evaluation

arXiv.org Artificial Intelligence

We present TRACE (Transformer-based Risk Assessment for Clinical Evaluation), a novel method for clinical risk assessment based on clinical data, leveraging the self-attention mechanism for enhanced feature interaction and result interpretation. Our approach is able to handle different data modalities, including continuous, categorical and multiple-choice (checkbox) attributes. The proposed architecture features a shared representation of the clinical data obtained by integrating specialized embeddings of each data modality, enabling the detection of high-risk individuals using Transformer encoder layers. To assess the effectiveness of the proposed method, a strong baseline based on non-negative multi-layer perceptrons (MLPs) is introduced. The proposed method outperforms various baselines widely used in the domain of clinical risk assessment, while effectively handling missing values. In terms of explainability, our Transformer-based method offers easily interpretable results via attention weights, further enhancing the clinicians' decision-making process.


Fooocus is the easiest way to create AI art on your PC

PCWorld

What's the simplest way to create AI art on your PC? Although Stable Diffusion is often seen as the best way to create AI art on your PC, Fooocus offers a simple setup experience, with rewarding depth for those who wish to dive deeper. Stable Diffusion debuted two years ago as the way to create AI art on your PC. While I've used some of the techniques that David Wolski outlined in his tutorial on using Stable Diffusion, it just feels so complicated to set up. Fooocus (yes, three "o's) offers essentially a one-click setup process in the same vein as something like winget: You tell it what to do, and then Fooocus goes out and does it.


Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels

arXiv.org Artificial Intelligence

Abstract--The confusion matrix, a ubiquitous visualization for helping people evaluate machine learning models, is a tabular layout that compares predicted class labels against actual class labels over all data instances. We conduct formative research with machine learning practitioners at a large technology company and find that conventional confusion matrices do not support more complex data-structures found in modern-day applications, such as hierarchical and multi-output labels. To express such variations of confusion matrices, we design an algebra that models confusion matrices as probability distributions. 's utility with three case studies that help people better understand model performance and reveal hidden confusions. Machine learning is a complex, iterative design and development practice predicted class labels (synonymously, these can be flipped via a matrix [4, 24], where the goal is to generate a learned model that generalizes transpose). These visualizations are introduced in many machine to unseen data inputs. One critical step is model evaluation, testing learning courses and are simultaneously used in practice to show what and inspecting a model's performance on held-out test sets of data with pairs of classes a model confuses. Succinctly, confusion matrices are known labels. Confusion matrices show a visual proxy A ubiquitous visualization used for model evaluation, particularly for accuracy (e.g., entries on the diagonal of the matrix), which alone for classification models, is the confusion matrix: a tabular layout that has been shown to be insufficient for many evaluations [39]. Furthermore, compares a predicted class label against the actual class label for each the diagonal of a confusion matrix often contains many more class over all data instances.


Speech Recognition with TensorFlow.js

#artificialintelligence

As we said, TensorFlow.js is a powerful library, and we can work on a lot of different things like image classification, video manipulation, and speech recognition among others. For today I decided to work on a basic speech recognition example. Our code will be able to listen through the microphone and identify what the user is saying, at least up to a few words as we have some limitations on the sample model I'm using. But rather than explaining, I think it's cool if we see it first in action: I know it can be a bit erratic, and it's limited to a few words, but if you use the right model, the possibilities are endless. Enough talking, let's start coding.


Are you a robot?

#artificialintelligence

Have you ever been on one of those websites that asks if you are a robot? Yeah, I know you have, they are everywhere. What you might be wondering is why a computer is asking you to prove your humanity and why a robot wouldn't just lie its way in. After all, it's not like robots are sworn to an oath of candor. Also, there are videos on the internet with robots clicking the "I'm not a robot" checkbox, so what exactly do these seek to accomplish apart from just being annoying.


Getting started in building and deploying interactive data science apps with Streamlit

#artificialintelligence

Flask used to come to mind when data scientists want to spin up a python-based data science app, but there is a better option now. To create an interactive facade for a machine learning or visualization script, Streamlit is way faster, since it removed the need to write any front-end code. Now we'll go through step-by-step how to build a Streamlit app. I will also review some pros and cons of Streamlit. Anyone who wants to put an interactive user interface or visible facade to the python scripts. Streamlit can be used to built machine learning/AI apps or display exploratory/analytical data visualizations or both at the same time.


Google's reCAPTCHA test has been tricked by artificial intelligence

#artificialintelligence

Computer scientists have found a way around Google's reCAPTCHA tests, tricking the system into thinking an artificial intelligence program is human. But there's a catch, although the AI system can fool the bot test it doesn't live-up to the promise its creators give it. CAPTCHAs are the tests used by websites to battle back against bots, asking website visitors to prove they're human before proceeding. The leading system is Google's reCAPTCHA, which has previously asked website visitors to prove their humanity by checking words scanned from books or photographs of street signs. That was replaced with behavioural analysis, requiring humans to simply tick a box proclaiming "I'm not a robot".


Google reCAPTCHA Is Now Invisible

#artificialintelligence

The reCAPTCHA is the most popular CAPTCHA service made by Google. You've seen it a million times when you sign up a page across the web. Its primary goal is to separate humans from a bot. It challenges users by deciphering a photo of words or numbers or picking objects in a grid of photos. But that process of verifying whether you're a human or not is over because Google is changing it. Recently, the company introduces the invisible reCAPTCHA.