SINTEF
Research project 2019
"The training effect of the facial muscles can be measured using computer vision"
SINTEF Ålesund, together with Letsip AS, has carried out a pilot project in the period January – March 2019, where we wanted to investigate whether it is possible to use computer vision to record positions of muscles in the face.
The vision behind the research project "The training effect of the facial muscles can be measured using computer vision", is an innovative and user-friendly digital tool that makes it easy to train and to monitor and measure progress for everyone. Getting started early with training after a stroke can be crucial for good rehabilitation. Being able to detect inactive muscles or spasticity, and measure the effect during activity can be an important parameter for the best possible training program. Understanding the value of your own training on a digital platform can be motivating and provide knowledge before, after and during training. The project investigated the technical basis for being able to offer a holistic approach to training and stimulation in the future, which can further increase the flow of information and understanding between patients and practitioners.
Letsip develops products for training facial muscles. The training products are suitable for children and adults to increase or decrease muscle tone and to build muscles in the cheeks, lips and tongue. The training programs that have been developed are adapted for habilitation and rehabilitation to avoid skewed development, and for training or maintenance of muscles. Good muscle function in the cheeks, lips and tongue is important to maintain the basic prerequisites for good speech, facial expressions, oral hygiene, drooling, eating or drinking. The products are CE approved and patented.
SINTEF Ålesund helps SMEs in the region with their challenges in digitization through research. SINTEF Ålesund's mission is to contribute regionally with its cutting-edge expertise in machine learning, data analysis and simulation.
The method chosen in the project was to record video before and after the end of the training period. The test subjects followed a training program for six weeks based on a selected training program from Letsip. When using off-the-shelf methods within computer vision used in mobile phones, the position and movements of muscles were to be recognized and whether a difference could be measured between before and after the training period based on "facial landmarks" (Figure 1).
The central question was whether it is possible with an automatic measurement to detect progress in training on your own using this method.
Figure 1
The training programs used in the project are based on training principles for all striated muscles. SINTEF had an idea that computer vision could be used to prove the effectiveness of training, but was unsure whether the method would be suitable and accurate enough for the purpose.
The results
After six weeks of training, the method managed to reveal the rehabilitation progress quite clearly in some exercises.
The data knew:
- better muscle control
- better mobility
- better endurance
It turned out that computer vision was suitable for extracting a lot of information and provided clear and systematic measurement of muscle movements. The data also revealed the type of exercise and whether the exercise was performed correctly or not. (Figure 2)
Figure 2
The project also highlighted some challenges. Based on the data, it was not possible to tell the difference between a "real" smile before and after training. The computer vision method itself also gave somewhat limited results with beards and glasses, or with unusual exercises and movements. Both situations need specially adapted computer vision methods, which requires further work and research.
The preliminary project between Letsip and SINTEF Ålesund was financed through the Research Council's program for research-based innovation in the regions (FORREGION). This type of support is a very good choice for companies to get started with research and innovation.
Contact:
- Monica Berstad Mæland, CEO Letsip AS, 480 09 500,
- Severin Sadjina, researcher at SINTEF Ålesund, 47 25 34 62, sadjina@sintef.no
- Øyvind Herse, Regional Research Fund Mid-Norway Møre and Romsdal, 91 36 48 86, herse@vrimr.no