Readspeaker menu

X-ray Analysis of Animal Locomotion

Oliver Mothes, Daniel Haase, Manuel Amthor, and Joachim Denzler


Motivation

The detailed understanding of animal locomotion plays an important role in many fields of research, e.g., biology, motion science, and robotics. In order to analyze the locomotor system in vivo, high-speed X-ray acquisition is applied where the retrieval of anatomical landmarks is of main interest. To this day, the evaluation of these sequences is a very time-consuming task, since human experts have to manually annotate anatomical landmarks in single images. Therefore, an automation of this task at a minimum level of user interaction is of urgent need. .
In this project computer vision principles combining with machine learning methods tackle this task of automation.


Tracking-by-Detection

 

Tracking-by-Detection

[Mothes17:ALTOLP]

Oliver Mothes and Joachim Denzler.Anatomical Landmark Tracking by One-shot Learned Priors for Augmented Active Appearance Models. International Conference on Computer Vision Theory and Applications (VISAPP). 2017. (accepted for publication)

 

X-ray Animal Skeleton Tracking

 

[Amthor14:RPS]

Manuel Amthor, Daniel Haase and Joachim Denzler.Robust Pictorial Structures for X-ray Animal Skeleton Tracking. International Conference on Computer Vision Theory and Applications (VISAPP). 2014.
Short summary: The detailed understanding of animals in locomotion is a relevant field of research in biology, biomechanics and robotics. To examine the locomotor system of birds in vivo and in a surgically non-invasive manner, high-speed X-ray acquisition is the state of the art. For a biological evaluation, it is crucial to locate relevant anatomical structures of the locomotor system. There is an urgent need for automating this task, as vast amounts of data exist and a manual annotation is extremely time-consuming. We present a biologically motivated skeleton model tracking framework based on a pictorial structure approach which is extended by robust sub-template matching. This combination makes it possible to deal with severe self-occlusions and challenging ambiguities. As opposed to model-driven methods which require a substantial amount of labeled training samples, our approach is entirely data-driven and can easily handle unseen cases. Thus, it is well suited for large scale biological applications at a minimum of manual interaction. We validate the performance of our approach based on 24 real-world X-ray locomotion datasets, and achieve results which are comparable to established methods while clearly outperforming more general approaches.
 

Large-scale Evaluation

  CoM"

[Haase13:AAO]

D. Haase and Andrada, Emanuel and John Alexander Nyakatura and B. M. Kilbourne and J. Denzler. Automated Approximation of Center of Mass Position in X-ray Sequences of Animal Locomotion. Journal of Biomechanics. 2013.
Short summary: A crucial aspect of comparative biomechanical research is the center of mass (CoM) estimation in animal locomotion scenarios. Important applications include the parameter estimation of locomotion models, the discrimination of gaits, or the calculation of mechanical work during locomotion. Several methods exist to approximate the CoM position, e.g. force-plate-based approaches, kinematic approaches, or the reaction board method. However, they all share the drawback of not being suitable for large scale studies, as detailed initial conditions from kinematics are required (force-plates), manual interaction is necessary (kinematic approach), or only static settings can be analyzed (reaction board). For the increasingly popular case of X-ray-based animal locomotion analysis, we present an alternative approach for CoM estimation which overcomes these shortcomings. The main idea is to only use the recorded X-ray images, and to map each pixel to the mass of matter it represents. As a consequence, our approach is surgically noninvasive, independent of animal species and locomotion characteristics, and neither requires prior knowledge nor any kind of user interaction. To assess the quality of our approach, we conducted a comparison to highly accurate reaction board experiments for lapwing and rat cadavers, and achieved an average accuracy of 2.6 mm (less than 2% of the animal body length). We additionally verified the practical applicability of the algorithm by comparison to a previously published CoM study which is based on the kinematic method, yielding comparable results.

An implementation of the algorithm described in this paper is available for download:

The code is free for use and comes in two versions: one for C++/OpenCV, and one for MATLAB (Octave works as well). The download includes one lapwing sequence as example data.

Download (version 0.1.0, 4.8 MB)


Animal Locomotion Analysis using Augmented Active Appearance Models

  Augmented Active Appearance Models

[Haase13:2A3]

Daniel Haase and Joachim Denzler. 2D and 3D Analysis of Animal Locomotion from biplanar X-ray Videos using Augmented Active Appearance Models. EURASIP Journal on Image and Video Procesing. 2013.
Short summary: For many fundamental problems and applications in biomechanics, biology, and robotics, an in-depth understanding of animal locomotion is essential. To analyze the locomotion of animals, high-speed X-ray videos are recorded, in which anatomical landmarks of the locomotor system are of main interest and must be located. To date, several thousand sequences have been recorded, which makes a manual annotation of all landmarks practically impossible. Therefore, an automatization of X-ray landmark tracking in locomotion scenarios is worthwhile. However, tracking all landmarks of interest is a very challenging task, as severe self-occlusions of the animal and low contrast are present in the images due to the X-ray modality. For this reason, existing approaches are currently only applicable for very specific subsets of anatomical landmarks. In contrast, our goal is to present a holistic approach which models all anatomical landmarks in one consistent, probabilistic framework. While active appearance models (AAMs) provide a reasonable global modeling framework, they yield poor fitting results when applied on the full set of landmarks. In this paper, we propose to augment the AAM fitting process by imposing constraints from various sources. We derive a general probabilistic fitting approach and show how results of subset AAMs, local tracking, anatomical knowledge, and epipolar constraints can be included. The evaluation of our approach is based on 32 real-world datasets of five bird species which contain 175,942 ground-truth landmark positions provided by human experts. We show that our method clearly outperforms standard AAM fitting and provides reasonable tracking results for all landmark types. In addition, we show that the tracking accuracy of our approach is even sufficient to provide reliable three-dimensional landmark estimates for calibrated datasets.
 

Online Tracking

  Subtemplate Matching

[Amthor12:FAR]

Manuel Amthor and Daniel Haase and Joachim Denzler.r. Fast and Robust Landmark Tracking in X-ray Locomotion Sequences Containing Severe Occlusions. International Workshop on Vision, Modelling, and Visualization (VMV). 2012.
Short summary:Recent advances in the understanding of animal locomotion have proven it to be a key element of many fields in biology, motion science, and robotics. For the analysis of walking animals, high-speed x-ray videography is employed. For a biological evaluation of these x-ray sequences, anatomical landmarks have to be located in each frame. However, due to the motion of the animals, severe occlusions complicate this task and standard tracking methods can not be applied. We present a robust tracking approach which is based on the idea of dividing a template into sub-templates to overcome occlusions. The difference to other sub-template approaches is that we allow soft decisions for the fusion of the single hypotheses, which greatly benefits tracking stability. Also, we show how anatomical knowledge can be included into the tracking process to further improve the performance. Experiments on real datasets show that our method achieves results superior to those of existing robust approaches.
 

Multi-view Active Appearance Models

  Multiview Active Appearance Models

[Haase11:MVA]

Daniel Haase, John A. Nyakatura and Joachim Denzler. Multi-view Active Appearance Models for the X-ray Based Analysis of Avian Bipedal Locomotion. German Association for Pattern Recognition DAGM. 2011.
Short summary: Many fields of research in biology, motion science and robotics depend on the understanding of animal locomotion. Therefore, numerous experiments are performed using high-speed biplanar x-ray acquisition systems which record sequences of walking animals. Until now, the evalu- ation of these sequences is a very time-consuming task, as human experts have to manually annotate anatomical landmarks in the images. There- fore, an automation of this task at a minimum level of user interaction is worthwhile. However, many difficulties in the data-such as x-ray occlu- sions or anatomical ambiguities-drastically complicate this problem and require the use of global models. Active Appearance Models (AAMs) are known to be capable of dealing with occlusions, but have problems with ambiguities. We therefore analyze the application of multi-view AAMs in the scenario stated above and show that they can effectively han- dle uncertainties which can not be dealt with using single-view models. Furthermore, preliminary studies on the tracking performance of human experts indicate that the errors of multi-view AAMs are in the same order of magnitude as in the case of manual tracking.

Comparative Evaluation of Human and Active Appearance Model

  Comparison

[Haase11:CEO]

Daniel Haase, John A. Nyakatura and Joachim Denzler. Comparative Evaluation of Human and Active Apperance Model based Tracking Performance of Anatomical Landmarks in Locomotion Analysis. Pattern Recognition and Image Analysis. 2011.
Short summary: The detailed understanding of animal locomotion is an important part of biology, motion science and robotics. To analyze the motion, high-speed x-ray sequences of walking an- imals are recorded. The biological evaluation is based on anatomical key points in the images, and the goal is to find these landmarks automatically. Unfortunately, low contrast and occlusions in the images drastically complicate this task. As recently shown, Active Appearance Models (AAMs) can be successfully applied to this problem. However, ob- taining reliable quantitative results is a tedious task, as the human error is unknown. In this work, we present the results of a large scale study which allows us to quantify both the tracking performance of humans as well as AAMs. Furthermore, we show that the AAM-based approach provides results which are comparable to those of human experts.

Anatomical Landmark Tracking

  AAM

[Haase11:ALT]

Daniel Haase and Joachim Denzler. Anatomical Landmark Tracking for the Analysis of Animal Locomotion in X-ray Videos UsingActive Appearance Models. Scandinavian Conference on Image Analysis. 2011.
Short summary:X-ray videography is one of the most important techniques for the locomotion analysis of animals in biology, motion science and robotics. Unfortunately, the evaluation of vast amounts of acquired data is a tedious and time-consuming task. Until today, the anatomical land- marks of interest have to be located manually in hundreds of images for each image sequence. Therefore, an automatization of this task is highly desirable. The main difficulties for the automated tracking of these land- marks are the numerous occlusions due to the movement of the animal and the low contrast in the x-ray images. For this reason, standard track- ing approaches fail in this setting. To overcome this limitation, we analyze the application of Active Appearance Models for this task. Based on real data, we show that these models are capable of effectively dealing with occurring occlusions and low contrast and can provide sound tracking results.

Locations of Site Visitors