Towards Biosignals-Free Autonomous Prosthetic Hand Control via Imitation Learning

-- Limb loss affects millions globally, impairing physical function and reducing quality of life. Most traditional surface electromyographic (sEMG) and semi-autonomous methods require users to generate myoelec-tric signals for each control, imposing physically and mentally taxing demands. This study aims to develop a fully autonomous control system that enables a prosthetic hand to automatically grasp and release objects of various shapes using only a camera attached to the wrist. By placing the hand near an object, the system will automatically execute grasping actions with a proper grip force in response to the hand's movements and the environment. To release the object being grasped, just naturally place the object close to the table and the system will automatically open the hand. Such a system would provide individuals with limb loss with a very easy-to-use prosthetic control interface and greatly reduce mental effort while using. To achieve this goal, we developed a teleoperation system to collect human demonstration data for training the prosthetic hand control model using imitation learning, which mimics the prosthetic hand actions from human. Through training the model using only a few objects' data from one single participant, we have shown that the imitation learning algorithm can achieve high success rates, generalizing to more individuals and unseen This work has been submitted to the IEEE for possible publication. This work was supported in part by the Government of Canada's New Frontiers in Research Fund (NFRF, Grant No NFRFE-2022-00407) and Natural Sciences and Engineering Research Council of Canada's Research T ools and Instruments (NSERC RTI, Grant No RTI-2022-00688). This work involved human subjects or animals in its research. Approval of all ethical and experimental procedures and protocols was granted by the Memorial University Interdisciplinary Committee on Ethics in Human Research (20210316-SC). Kaijie Shi, Wanglong Lu are with Department of Computer Science, Memorial University of Newfoundland, St. John's, NL A1B 3X5, Canada, and also with College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou, 325000, China. Hanli Zhao is with College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou, 325000, China. Vinicius Prado da Fonseca is with Department of Computer Science, Memorial University of Newfoundland, St. John's, NL A1B 3X5, Canada. Ting Zou is with Department of Mechanical and Mechatronics Engineering, Memorial University of Newfoundland, St. John's, NL A1B 3X5, Canada.