Meng-Fen Tsai, Rosalie H. Wang, José Zariffa
Abstract
Hand function is a central determinant of independence after stroke. Measuring hand use in the home environment is necessary to evaluate the impact of new interventions, and calls for novel wearable technologies. Egocentric video can capture hand-object interactions in context, as well as show how more-affected hands are used during bilateral tasks (for stabilization or manipulation).
Introduction
Upper limb function is a determinant of quality of life after stroke [1]. More than 65% of stroke survivors have remaining upper limb impairments six months after stroke [2,3]. Hemiplegia or hemiparesis is a common motor deficit after stroke that causes the more-affected limbs to experience difficulties isolating or executing movements. Novel interventions for upper limb function are required to improve independence in activities of daily living (ADLs). Prior to translating a new intervention into practice, its ultimate impact on the daily life of stroke survivors should be quantified through appropriate outcome measures. The upper limb function measured in a clinical setting is not always demonstrated in daily life [4–9]. According to the International Classification of Functioning, Disability and Health (ICF) from the World Health Organization, measuring function in a hospital and in the community corresponds to two different domains of function—capacity and performance, respectively [10].
Materials and method
Stroke survivors were invited to participate in the study, which was approved by the Research Ethics Board of the University Health Network. Informed consent from participants and their caregivers (if involved) were obtained before enrollment into the study. The inclusion criteria for study participants were the following: 1) at least six months post-stroke; 2) self-reported difficulty in daily life due to an impairment of the more-affected hand; 3) impaired but not absent hand function, defined as a total ARAT score above 10 [50]; 4) Montreal Cognitive Assessment (MoCA) above 21, to avoid potential cognitive difficulties [51]; 5) no subluxation or significant pain when using their upper limbs; 6) no other neuromusculoskeletal disease affecting upper limb movements other than stroke.
Results
Twenty-one stroke survivors, 15 males and 6 females, completed the study. The upper limb impairment levels of participants spanned across mild, moderate, and severe, according to the total FMA-UE score defined in [59]. The demographic information is provided in Table 1. Some examples of Home and HomeLab instances are shown in Figs 2 and 3.
Discussion
This study used automated analysis of egocentric videos to analyze for the first time the hand function of stroke survivors in their home environment. Detecting hand-object interactions was demonstrated to be feasible, corroborating earlier findings in a laboratory environment [25]. The Hand Object Detector had the highest macro and micro average MCCs and F1-scores for the hand-object interaction detection. The detection of hand roles was found to be more challenging using analogous methods, and will warrant additional investigations with other approaches.
Conclusions
Using automated analysis of egocentric videos to detect the hand-object interactions of stroke survivors at home is feasible. This study therefore provides a novel tool to evaluate independent hand use at home after stroke. Performance of the classifiers could be further improved in the future by conducting more training specific to impaired hand postures, such as the closed hand shapes that are associated with spasticity after stroke. Automatically identifying the role of the more-affected hand in bimanual interactions was found to be a more challenging task. Our results here provide a baseline for this novel problem. Possible avenues to improve on these results will include a greater focus on recognizing finger movements, such as by using pose estimation methods.
Citation: Tsai M-F, Wang RH, Zariffa J (2023) Recognizing hand use and hand role at home after stroke from egocentric video. PLOS Digit Health 2(10): e0000361. https://doi.org/10.1371/journal.pdig.0000361
Editor: Alexander Wong, University of Waterloo, CANADA
Received: August 2, 2022; Accepted: August 31, 2023; Published: October 11, 2023
Copyright: © 2023 Tsai et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data can be shared upon reasonable request and completion of a data sharing agreement. These access restrictions are required by our institutional policies, due to data containing sensitive information (video recordings inside participants' homes). Data access requests can be directed to the University Health Network's Data Access Committee ([email protected]), with the corresponding author CCed on the request.
Funding: JZ and RW are funded by the Heart and Stroke Foundation of Canada (https://www.heartandstroke.ca/) [Grant number: G-18-0020952]. MT is funded by the Healthcare Robotics (HeRo) NSERC Collaborative Research and Training Experience (CREATE) program at the University of Toronto. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000361#sec018
