Performance Assessment of Physiotherapy and Rehabilitation Exercises with Deep Learning

Abstract

Physiotherapy and rehabilitation process is critical for patients in postoperative recovery or the treatment of a wide variety of musculoskeletal disorders. However, providing access to a clinician for each rehabilitation session is a heavy burden and high cost to individuals. Also, it is very important to remotely check whether the exercises are performed correctly, especially during periods of lock-down due to the current pandemic, to provide motivation in the rehabilitation progress of the patients and to ensure that the recommended exercises contribute to the treatment. In this study, deep learning-based performance assessment of rehabilitation exercises has been proposed by using RGB videos obtained with low cost off-the-shelf cameras instead of high-cost, hard-to-reach depth cameras or wearable contact sensors. The proposed deep learning (DL) network models, PtConvNet, PtHybNet and PtBiLSTM, utilize three dimensional (3D) skeletal joint positions of patients extracted from exercise videos. Performance scores given by the physiotherapists have been used as the ground-truth in the training of the framework. We showed that the performance estimates of the learning models reliably follow the actual values and that the DL models confirm the ability to evaluate rehabilitation exercises.