Cracking CodeWhisperer: Analyzing Developers' Interactions and Patterns During Programming Tasks

Abstract--The use of AI code-generation tools is becoming increasingly common, making it important to understand how software developers are adopting these tools. In this study, we investigate how developers engage with Amazon's Code-Whisperer, an LLM-based code-generation tool. We conducted two user studies with two groups of 10 participants each, interacting with CodeWhisperer - the first to understand which interactions were critical to capture and the second to collect low-level interaction data using a custom telemetry plugin. Our mixed-methods analysis identified four behavioral patterns: 1) incremental code refinement, 2) explicit instruction using natural language comments, 3) baseline structuring with model suggestions, and 4) integrative use with external sources. We provide a comprehensive analysis of these patterns . Several IDE-based code generation tools have been released in the past few years, such as GitHub's Copilot [8], Kite [14], Amazon's Code Whisperer [20], Tabnine [22], and WPCode [28]. Research reveals that being able to achieve their full potential requires a certain level of guidance to ensure that the tool's output aligns with the user's goal [21].