Readspeaker Menü

Irritationsfreies und emotionssensitives Trainingssystem (IRESTRA)

Oliver Mothes, Martin Thümmel and Joachim Denzler


The interaction between humans and machines is an ongoing research field in many scientific disciplines. To develop an optimal strategy for machines to interact with humans the interactions between humans can be used to detect signals for the mood of the interaction partner and react in an adequate way. Especially elderly people can benefit from an easy and intuitive handling of technical devices. The aim of the project is to develop an irritation free and non invasive emotion sensitive training device for elderly people and Facial Paresis patients. Older persons can use this system for memory training (e.g. person recognition), whereas paresis patients can do their daily facial training sessions with a direct feedback from the machine and motivation if necessary. Within this project these tasks are tackled by combining medical, psychological and neurological knowledge with machine learning and computer vision techniques both in 2D and 3D in coorperation with the Fraunhofer Institute for Applied Optics and Precision Engineering IOF, the Jena University Hospital (namely the Institute of Psychosocial Medicine and Psychological Therapy, the Department of Neurology and the Department of Otolaryngology) and the Department for General Psychology from the Friedrich Schiller University Jena.

Facial Paresis Index Prediction

  Framework for Facial Paresis Index Prediction


Luise Modersohn and Joachim Denzler. Facial Paresis Index Prediction by Exploiting Active Appearance Models for Compact Discriminative Features. International Conference on Computervision Theory and Application (VISAPP) 2016. 271-278. [pdf] [bib]
Abstract: In the field of otorhinolaryngology, the dysfunction of the facial nerve is a common disease which results in a paresis of usually one half of the patients face. The grade of paralysis is measured by physicians with rating scales, e.g. the Stennert Index or the House-Brackmann scale. In this work, we propose a method to analyse and predict the severity of facial paresis on the basis of single images. We combine feature extraction methods based on a generative approach (Active Appearance Models) with a fast non-linear classifier (Random Decision Forests) in order to predict the patients grade of facial paresis. In our proposed framework, we make use of highly discriminative features based on the fitting parameters of the Active Appearance Model, Action Units and Landmark distances. We show in our experiments that it is possible to correctly predict the grade of facial paresis in many cases, although the visual appearance is strongly varying. The presented method creates new opportunities to objectively document the patients progress in therapy.