Virtual and Augmented Reality-Based Assistive Interfaces for Upper-limb Prosthesis Control and Rehabilitation

Functional upper-limb prosthetic training can improve users performance in controlling prostheses and has been incorporated into occupational therapy for individuals in need. In recent years, virtual reality (VR) and augmented reality (AR) technologies have been shown to be promising avenues to improve the convenience of rehabilitative prosthesis training systems. However, it is uncertain if the comprehensive efficacy and effectiveness of VR or AR assistive tools are adequate compared to conventional prosthetic tools and if not, whether enhancements can be made through incorporation of other technical paradigms. This work first presents a mixed reality system we developed for prosthesis control and training. Five able-bodied subjects are involved to perform three-dimensional object manipulation tasks in analogous AR and VR environments. Multiple evaluation metrics are applied to assess subjects performances within the two paradigms. Based on the comparative analysis, we find that VR-based environment promotes more efficient motion along with higher task completion rate and path efficiency while AR paradigm allows subjects to perform motor tasks with shorter time consumed. Another study is conducted to evaluate the efficiency and feasibility of AR-facilitated prosthesis control system compared to that in real-world and if any technical additions can be applied to improve the AR-based system. Three able-bodied subjects were engaged in the experiment to perform object manipulation tasks in a) physical environment, b) AR-without-bypass environment, and c) AR-with-bypass environment. Based on the results obtained from the assessment, we conclude that while our AR-based system modestly lags behind the effectiveness of physical systems, the study conducted using a bypass prosthesis suggests that AR system has the potential to improve the efficacy of prosthesis control.