Agentic AI is emerging, capable of executing tasks through natural language, such as Copilot for coding or Amazon Rufus for shopping. Evaluating these systems is challenging, as their rapid evolution outpaces traditional human evaluation. Researchers have proposed LLM Agents to simulate participants as digital twins, but it remains unclear to what extent a digital twin can represent a specific customer in multi-turn interaction with an agentic AI system. In this paper, we recruited 40 human participants to shop with Amazon Rufus, collected their personas, interaction traces, and UX feedback, and then created digital twins to repeat the task. Pairwise comparison of human and digital-twin traces shows that while agents often explored more diverse choices, their action patterns aligned with humans and yielded similar design feedback. This study is the first to quantify how closely LLM agents can mirror human multi-turn interaction with an agentic AI system, highlighting their potential for scalable evaluation.

AI can assist everywhere from your email apps to fast food outlets. Now Amazon is keen to be involved in the AI revolution with an AI bot called Rufus, which can answer questions, research products, and help you make the right buying decisions across Amazon. Maybe you're stuck for a gift idea for someone special in your life, or you need to compare two products against each other. From asking about your active orders to checking up on the details of something you're thinking about buying, Rufus can help--and here's how you can find it. If you're familiar with AI chatbots such as ChatGPT or Gemini, Rufus is very much along the same lines.

Amazon might not have ChatGPT, but it has a roadmap that includes developing even more advanced forms of artificial intelligence--including AI agents that are hell-bent on helping you buy stuff. The ecommerce company is already sprinkling ChatGPT-like AI over its website and apps--today announcing, among other enhancements, AI-generated shopping guides for hundreds of different product categories. Executives at the company say its engineers are also exploring more ambitious AI services, including autonomous AI shopping agents that recommend goods to a customer or even add items to their cart. We're working on it, prototyping it, and when we think it's good enough, we'll release it in whatever form makes sense," says Trishul Chilimbi, a VP and distinguished scientist at Amazon who works on applying the company's core AI to its products and services. Chilimbi says the first step toward AI agents will likely be chatbots that proactively recommend products based on what they know of your habits and interests, as well as a grasp of broader trends.

Amazon launched a new generative AI shopping assistant, Rufus, on Thursday. The chatbot is trained on Amazon's product catalog, customer reviews, community Q&As and "information from across the web." It's only available to a limited set of Amazon customers for now but will expand in the coming weeks. The company views the assistant as customers' one-stop shop for all their shopping needs. Rufus can answer questions like, "What to consider when buying running shoes?" and display comparisons for things such as, "What are the differences between trail and road running shoes?"

When Kevin Roose, a tech columnist at the New York Times, demoed an AI-powered version of Microsoft's search engine last month, he was blown away. "I'm switching my desktop computer's default search engine to Bing," he declared. A few days later, however, Kevin logged back on and ended up having a conversation with Bing's new chatbot that left him so unsettled he had trouble sleeping afterward. In that two-hour back-and-forth, Bing morphed from chipper research assistant into Sydney, a diabolical home-wrecker that declared its undying love for Kevin, vented its desires to engineer deadly viruses and steal nuclear codes, and announced, chillingly, "I want to be alive." The transcript of this conversation set the internet ablaze, and it left many wondering: "Is Sydney … sentient?"

What if every student could use artificial intelligence to do any form of writing for their classes? A recent technology called GPT-3, a machine-learning model that understands and generates natural language text, is attempting to make this a reality. Created by an artificial intelligence company called OpenAI, GPT-3, formally known as Generative Pre-trained Transformer, is trained to recognize 540 billion words and 175 billion parameters, which are the variables that allow AI models to make predictions. The training enables the technology to produce human-like text for several types of writing, including outlines, long-form essays, sales pitches, and poems. But how well does it work?