Spot-On: A Mixed Reality Interface for Multi-Robot Cooperation

Engelbracht, Tim, Lukovic, Petar, Behrens, Tjark, Lascheit, Kai, Zurbrügg, René, Pollefeys, Marc, Blum, Hermann, Bauer, Zuria

arXiv.org Artificial Intelligence 

Recent progress in mixed reality (MR) and robotics is enabling increasingly sophisticated forms of human-robot collaboration. Building on these developments, we introduce a novel MR framework that allows multiple quadruped robots to operate in semantically diverse environments via a MR interface. Our system supports collaborative tasks involving drawers, swing doors, and higher-level infrastructure such as light switches. A comprehensive user study verifies both the design and usability of our app, with participants giving a "good" or "very good" rating in almost all cases. Overall, our approach provides an effective and intuitive framework for MR-based multi-robot collaboration in complex, real-world scenarios.