iDrink (instrumented Drinking Assessment)

Running 2022 – 2025
Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt.

Core Project Collaborators

  • Rehabilitation Engineering Laboratory ETH 
  • ZHAW School of Health Sciences, Institute of Occupational Therapy, Winterthur, Switzerland 
  • ZHAW School of Management and Law, Institute of Business Information Technology, Winterthur, Switzerland 
  • USZ 
  • Cereneo 
  • University College London 

Core Project Funders

  • DIZH (Digitalisierungsinitiative Züricher Hochschulen) 

Neurological diseases are a significant burden for society. For instance, every fourth person over the age of 65 will statistically suffer from a stroke in their lifetime. In over 40% of the cases, upper limb movement impairments that impact quality of life persist, despite rehabilitation. With societies aging, incidence and prevalence of stroke are anticipated to increase, emphasizing the importance of maximizing rehabilitation strategies across the continuum of care. To improve recovery outcomes, researchers are developing new drugs, rehabilitation technologies, and therapies. However, evaluating improvements in movement capability after an intervention poses a challenge. Traditional clinical assessments, mainly reliant on therapists’ visual evaluations, are subjective and mainly score task completion. Recent Stroke expert conventions (ISRRA: https://strokerecoveryalliance.com) agree that quantifying movement quality would be a desirable outcome to increase sensitivity and specificity of movement measurements. Instrumented assessments could bridge this gap by providing objective measurements, scoring task completion, and quantifying movement quality – all at the same time.

To ensure that novel assessment tools are both implementable in clinical and home settings and scalable, it is crucial that they are low-cost and user-friendly. Two attributes that have long evaded the motion assessment tools that are traditionally used in research. In this project, we focus on affordable and accessible technologies for measuring movement, including wearable movement sensors (IMUs – Inertial Movement Units), single and multiple webcams with computer vision-based pose estimation, and RGB-D (depth) cameras.

Initially, we concentrate on stroke patients performing a drinking task, as the measures for movement quality in this scenario are well-established. Subsequently, we aim to explore data-driven methods to quantify movement quality independently of the task and study population. This approach will facilitate the quantification of upper limb movements across various conditions and tasks, offering a means to assess movement quality throughout the continuum of care.

A cost-efficient instrumented Drinking Task to quantify Movement Quality

To validate our novel assessment tools, we record stroke patients performing multiple repetitions of a drinking task in a movement laboratory. We simultaneously capture data using Optical Motion Capture, Inertial Movement Units (IMUs), a Kinect camera (RGB-D), and up to 10 commercial off-the-shelf webcams. For each sensor system, we calculate the kinematics and established movement quality measures. These are then compared to the data obtained from Optical Motion Capture, which serves as our ground truth.

We assess the measurement uncertainty for each movement quality measure of every system through the Limits of Agreement with Optical Motion Capture. This statistical method helps us determine how much the measurements of each system vary from the ground truth. Furthermore, we compare the measurement uncertainty of the movement quality measures from each system against the Minimal Clinically Important Difference (MCID). The MCID represents the smallest change in a treatment outcome that an individual patient would identify as important, approximating the minimal change of interest in our context.

This methodical approach allows us to rigorously evaluate the effectiveness and reliability of various low-cost sensor systems in quantifying movement quality, specifically within the context of a drinking task performed by stroke patients.

IMUs
We reconstruct the upper limb movement kinematics based on 5 IMUs (left/right arm, wrist and chest). We use orientation of the sensors to estimate joint angles and a kinematic forward model to calculate hand-velocity. We moreover calculate smoothness of the movement based on a single wrist sensor. We test different orientation estimation algorithms, one with the use of magnetometer (9D) and one without magnetometer (6D). We took an ad-hoc sample size of 15 stroke patients performing multiple repetitions of the drinking task and compared the IMU results to those of Optical Motion Capture clusters attached to the IMUs.

Webcams
We developed two distinct measurement tools using webcams: one involving recordings from a single camera, and the other, simultaneous recordings from multiple cameras for Markerless Motion Capture. For both, we arrange 10 cameras in a semi-circle around the patient, with half at shoulder level and the other half at ceiling height.
For the single camera setup, we calculate the kinematics and movement quality measures for each camera individually. This approach allows us to assess how the recording perspective influences the kinematic outcomes and movement quality measures. In the case of multi-camera recordings, we explore over 20 combinations of camera configurations. This includes varying the recording angles and the number of cameras used, ranging from 2 to 10. We then compare the results from these configurations to those obtained from Optical Motion Capture, to evaluate their effectiveness and accuracy.

Quantification of Movement Quality
Quantifying movement quality based on measurements traditionally involves using established movement quality measures, a process that requires significant research, including longitudinal studies, which limits its applicability across different populations and tasks. To address these limitations, we explore innovative techniques for quantifying movement quality.

One technique involves training artificial intelligence (AI) algorithms using expert scorings, such as those from physiotherapists, alongside sensor data (IMUs, webcams, etc.). In collaboration with physiotherapists and data scientists from ZHAW, physiotherapists evaluate movement quality based on video recordings, while data scientists develop classifiers. These classifiers use video or IMU data to predict the expert scorings. We assess various networks trained on different sensor inputs for their prediction accuracy for compensation patterns, but also for scoring task completion (www.frontiersin.org/journals/physiology/articles/10.3389/fphys.2022.877563/full)

A second approach, not reliant on labeled data sets, compares the kinematics of impaired movements against abled-bodied references. This method quantifies the differences in movements by analyzing kinematics, such as joint angles and hand velocity profiles. We extend our dataset beyond the drinking task to include additional upper limb movement tasks. The effectiveness of these novel metrics is then evaluated by correlating them with clinical scores and established movement quality measures related to the drinking task.

A cost-efficient instrumented Drinking Task to quantify Movement Quality

IMUs
Our findings revealed strong agreement between the IMUs and Optical Motion Capture, with 12 out of 15 measures demonstrating clinical applicability, evidenced by Limits of Agreement (LoA) below the Minimum Clinically Important Differences (MCID) for each measure. These results are promising, suggesting the clinical applicability of IMUs in quantifying movement quality for mildly and moderately impaired stroke patients performing the drinking task. Results have been published here: https://www.frontiersin.org/research-topics/52957/wearable-sensors-for-the-measurement-of-physiological-signals-what-about-their-measurement-uncertainty

Webcams
Initial findings from the single camera setup indicate that the recording angle significantly affects the accuracy of kinematics and movement quality measurements, primarily due to occlusions of body parts. However, recordings from unobstructed perspectives (e.g., from the left side to track the left elbow angle) exhibit only minor discrepancies compared to a multi-camera setup. (Mention WCNR?)

For the multi-camera configuration, preliminary outcomes suggest that the arrangement of five cameras at a lower level around the participant yields the most accurate results. A setup involving two cameras positioned at a 45-degree angle in front of the participant also delivers commendable performance, albeit slightly less optimal. Notably, the five-camera setup reached a measurement uncertainty that either falls below or is marginally above the Minimum Clinically Important Difference (MCID) for seven of the nine movement quality measures tested. This evaluation was based on a sample of eight stroke patients with mild to moderate impairments.

These preliminary findings were presented and received the accolade of best poster presentation at Rehabweek 2023 in Singapore, showcasing the potential of multi-camera configurations in accurately assessing movement quality in stroke rehabilitation (www.linkedin.com/feed/update/urn:li:activity:7113463443512258560).

Quantification of Movement Quality
Initial outcomes from our collaborative effort with ZHAW indicate that a straightforward classifier is capable of perfectly differentiating between the affected and unaffected sides of individual stroke patients. This level of accuracy was achieved when the classifier was trained on a specific subset of the patient’s task repetitions. The next phase of our research involves evaluating the classifier’s ability to generalize across different sets of sensor data.

In a separate line of inquiry, our work with UCL on comparing the kinematic data of stroke patients to that of able-bodied individuals undertaking a letter box task has yielded supportive evidence for our hypothesis. We found that stroke patients who show improvement in their Fugl-Meyer and ARAT scores over time tend to exhibit movement patterns that increasingly resemble those of the able-bodied population. This alignment was observed when clustering movement quality measures and condensing their dimensionality to a two-dimensional (2D) space, affirming the potential of kinematic comparisons in quantifying movement quality advancements in stroke rehabilitation.

Associated Projects

  • iARAT 
  • iGait 
  • Upper Limb Assistance for Targeted Rehabilitation 

Lake Lucerne Institute AG
Rubistrasse 9
6354 Vitznau