Exploring Mobile Touch Interaction with Large Language Models
Zindulka, Tim, Sekowski, Jannek, Lehmann, Florian, Buschek, Daniel
–arXiv.org Artificial Intelligence
Interacting with Large Language Models (LLMs) for text editing on mobile devices currently requires users to break out of their writing environment and switch to a conversational AI interface. In this paper, we propose to control the LLM via touch gestures performed directly on the text. We first chart a design space that covers fundamental touch input and text transformations. In this space, we then concretely explore two control mappings: spread-to-generate and pinch-to-shorten, with visual feedback loops. We evaluate this concept in a user study (N=14) that compares three feedback designs: no visualisation, text length indicator, and length + word indicator. The results demonstrate that touch-based control of LLMs is both feasible and user-friendly, with the length + word indicator proving most effective for managing text generation. This work lays the foundation for further research into gesture-based interaction with LLMs on touch devices.
arXiv.org Artificial Intelligence
Feb-11-2025
- Country:
- Asia
- Japan > Honshū
- Kantō > Kanagawa Prefecture > Yokohama (0.05)
- Taiwan > Taiwan Province
- Taipei (0.04)
- Japan > Honshū
- Europe
- Austria > Vienna (0.14)
- Denmark > Capital Region
- Copenhagen (0.04)
- Finland > Uusimaa
- Helsinki (0.04)
- Germany
- Bavaria > Upper Franconia
- Bayreuth (0.05)
- Hamburg (0.04)
- Hesse > Darmstadt Region
- Darmstadt (0.04)
- Bavaria > Upper Franconia
- Netherlands > South Holland
- The Hague (0.04)
- North America
- Canada
- Ontario > Toronto (0.04)
- Quebec > Capitale-Nationale Region
- Quebec City (0.04)
- Québec (0.04)
- United States
- California > Santa Clara County
- San Jose (0.04)
- Colorado > Denver County
- Denver (0.04)
- Hawaii > Honolulu County
- Honolulu (0.04)
- Louisiana > Orleans Parish
- New Orleans (0.04)
- Massachusetts > Suffolk County
- Boston (0.04)
- New York > New York County
- New York City (0.06)
- California > Santa Clara County
- Canada
- Oceania > Australia
- New South Wales > Sydney (0.04)
- Tasmania (0.04)
- Asia
- Genre:
- Questionnaire & Opinion Survey (1.00)
- Research Report > New Finding (1.00)
- Industry:
- Health & Medicine > Consumer Health (0.93)
- Technology: