Multi-objective Cross-task Learning via Goal-conditioned GPT-based Decision Transformers for Surgical Robot Task Automation

Surgical robot task automation has been increasingly Furthermore, the introduction of task-specific rewards and studied for its potential to improve surgical efficiency and the loss of cross-task pretraining create varying internal augment robot intelligence. Recent advancements have witnessed dynamics across tasks, resulting in technical challenges in research on learning-based methods [1]-[5] to promote developing a unified framework for reasoning and decisionmaking automation of surgical robots. Still, current performances within the goal-reaching paradigm in surgical tasks. of the latest methods are impeded in long-horizon To leverage the advanced GPT-based decision-making goal-conditioned tasks, where a sequence of actions and substeps frameworks for improving surgical robot task automation, are required until reaching an ultimate goal. Previous we propose the goal-conditioned decision transformer that algorithms with reinforcement learning [6] and Markov decision embedds goal and time-to-goal as future indicators. Besides, process only predict actions from the current state while we formulate multiple training objectives: action prediction, overlooking information from historical sequential states and dynamics prediction, time-to-goal prediction, and sequence actions. This lacks temporal reasoning capability over actions reconstruction in our cross-task pretraining process, which and affects learning of the inherent sequential dynamics fosters a comprehensive representation of the temporal dynamics which is useful to the final success of a complex task. Despite inherent in goal-conditioned tasks and encourages some works [7], [8] combining task-specific strategies to the model to incorporate diverse temporal reasoning factors.