Usability testing is a fundamental research method that user experience (UX) researchers use to evaluate and iterate their new designs. But what about evaluating and iterating the usability testing study design itself? Recent advances in Large Language Model-simulated Agent (LLM Agent) research inspired us to design UXAgent to support UX researchers in evaluating and iterating their study design before they conduct the real human-subject study. Our system features a Persona Generator module, an LLM Agent module, and a Universal Browser Connector module to automatically generate thousands of simulated users and to interactively test the target website. The system also provides a Result Viewer Interface so that the UX researchers can easily review and analyze the generated qualitative (e.g., agents' post-study surveys) and quantitative data (e.g., agents' interaction logs), or even interview agents directly. Through a heuristic evaluation with 16 UX researchers, participants praised the innovation of our system but also expressed concerns about the future of LLM Agent usage in UX studies.

Usability testing is a fundamental yet challenging (e.g., inflexible to iterate the study design flaws and hard to recruit study participants) research method for user experience (UX) researchers to evaluate a web design. Recent advances in Large Language Model-simulated Agent (LLM-Agent) research inspired us to design UXAgent to support UX researchers in evaluating and reiterating their usability testing study design before they conduct the real human subject study. Our system features an LLM-Agent module and a universal browser connector module so that UX researchers can automatically generate thousands of simulated users to test the target website. The results are shown in qualitative (e.g., interviewing how an agent thinks ), quantitative (e.g., # of actions), and video recording formats for UX researchers to analyze. Through a heuristic user evaluation with five UX researchers, participants praised the innovation of our system but also expressed concerns about the future of LLM Agent-assisted UX study.

The rise of AI is creating a lot of buzz in almost every modern sector. While it remains unclear what we can expect from AI for designers, there have been recent developments that signify that something huge is going to happen. We may see some groundbreaking development in the way we handle interactions. Digital humans are becoming more present on the internet, and may revolutionize how we interact with the world around us. Our creativity can also expect a huge boost from AI.

I am a "hollow vessel waiting to be filled with insights" [5]. I had one simple goal but no objective to achieve: attempt to study BlenderBot 3, Meta's "improved" artificial intelligence (AI) chatbot [7][9], inside its personalized context by interviewing it. I have no intention of learning about its pain points. None of the elements of body language come to play since such an endeavor is impossible. Having worked with many large language models (LLM) in my time, I have personally seen how they could potentially turn into raucous ideological donnybrooks [11].

It's a question that Diana Akrong found herself asking last year. Diana is a UX researcher based in Accra, Ghana, and the founding member of Google's Accra UX team. Across the world, her manager Dr. Courtney Heldreth, was equally interested in answering this question. Courtney is a social psychologist and a staff UX researcher based in Seattle, and both women work as part of Google's People Artificial Intelligence Research (PAIR) group. "Looking back on history, we can see how the industrial revolution played a significant role in creating global inequality," she says.

AI is freeing up time, creativity, and human capital, essentially letting people work more like humans and less like robots.