Interactive Generation of Laparoscopic Videos with Diffusion Models

Surgical simulations offer a significant advantage by eliminating the need for patient involvement in skills practice, providing trainees with essential technical lessons before performing procedures on humans [24]. However, current computer-based simulations have lots of drawbacks, such as unrealistic visual appearance, lacking variability, and complex creation procedures taking into account the varying anatomical properties, all of which lead to diminishing the quality of surgical training. Therefore, AI-generated surgical simulations promise significant advancements in medical education since the underlying machine-learning models can learn the anatomical and visual characteristics of surgeries as well as their interactions with surgical tools from real-world data. Similar to recent works on image-guided surgery by Ramalhinho et al. [18] and Schneider et al. [23], our work focuses on laparoscopic surgery. We propose an approach for generating realistic laparoscopic videos conditioned on both text prompts and surgical tool positions. This lays the groundwork for a dynamic and interactive surgical training platform that mimics real-world scenarios. With this approach, we achieve state-of-the-art realism with an FID score of 33.43 and a pixel-wise F1 score of 0.72 for the control of tool positions. Moreover, we successfully generate coherent videos of single surgical actions.