Bi-Fact: A Bidirectional Factorization-based Evaluation of Intent Extraction from UI Trajectories

Caduri, Sapir, Efros, Anatoly, Kahlon, Noam, Cohen, Danielle, Halpern, Yoni, Dagan, Ido

arXiv.org Artificial Intelligence 

Evaluating intent extraction from GUIs demands accurate, fine-grained metrics. This paper introduces Bi-Fact, a novel method that decomposes intents into atomic facts and performs bidirectional comparisons to assess precision and recall. Experiments demonstrate Bi-Fact's superior correlation with human judgments compared to existing metrics, establishing a more robust evaluation framework for UI-driven intent understanding.