Prototyping AI applications is notoriously difficult. While large language model (LLM) prompting has dramatically lowered the barriers to AI prototyping, designers are still prototyping AI functionality and UI separately. We investigate how coupling prompt and UI design affects designers' workflows. Grounding this research, we developed PromptInfuser, a Figma plugin that enables users to create semi-functional mockups, by connecting UI elements to the inputs and outputs of prompts. In a study with 14 designers, we compare PromptInfuser to designers' current AI-prototyping workflow. PromptInfuser was perceived to be significantly more useful for communicating product ideas, more capable of producing prototypes that realistically represent the envisioned artifact, more efficient for prototyping, and more helpful for anticipating UI issues and technical constraints. PromptInfuser encouraged iteration over prompt and UI together, which helped designers identify UI and prompt incompatibilities and reflect upon their total solution. Together, these findings inform future systems for prototyping AI applications.

Edward Boyden: Humans and machines have been merging for thousands of years. Right now I'm wearing shoes, I have a microphone on my jacket, we all probably used our phones at least once today… And we communicate with the augmentation of all sorts of amplification and even translation technologies: You can speak into a machine, and it'll translate the words you're saying in nearly real time. So I think what might be different in the years to come is a matter of degree, not a matter of kind. One concept that I think is emerging is what I like to call the brain coprocessor, a device that intimately interacts with the brain. Imagine that you could have a technology that could replace lost memories or augment decision making or boost attention or cognition.