Constellation models revisited
The algorithm we developed is able to automatically distinguish between different types of birds, flowers, dog breeds, and in general very similar object categories. It is based on computer vision and machine learning techniques that learn the appearance of object categories from a given set of images together with their annotations. Very recent ideas from the deep learning area allow for estimating very complex visual models and boost the recognition performance up to 82% for a dataset with 200 different bird categories. Would you be able to distinguish them?
Motivation of the Method
Part models of object categories are essential for challenging recognition tasks, where differences in categories are subtle and only reflected in appearances of small parts of the object. We present an approach that is able to learn part models in a completely unsupervised manner, without part annotations and even without given bounding boxes during learning. The key idea is to find constellations of neural activation patterns computed using convolutional neural networks. In our experiments, we outperform existing approaches for fine-grained recognition on the CUB200-2011, Oxford PETS, and Oxford Flowers dataset in case no part or bounding box annotations are available and achieve state-of-the-art performance for the Stanford Dog dataset. We also show the benefits of neural constellation models as a data augmentation technique for fine-tuning. Furthermore, our paper unites the areas of generic and fine-grained classification, since our approach is suitable for both scenarios.
Neural Activation Constellations
[Simon15:NAC]Marcel Simon and Erik Rodner. Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks. International Conference on Computer Vision (ICCV). 2015. [pdf][bib][code][slides]
Results on CUB200-2011:
||Xiao et al. 
Results on NA birds:
||Horn et al. ||75.0%
Easy to use Matlab code for the approach: https://github.com/cvjena/part_constellation_models
Fine-tuned models that were used in the paper: https://drive.google.com/file/d/0B6VgjAr4t_oTQXN2Y3VYaEMwVDA/view?usp=sharing
Further work on fine-grained recognition can be found on the fine-grained project page. There are more methods and projects to be discovered. The current project is funded by the DFG. The computer vision group of Prof. Joachim Denzler is currently also working on the application of methods in the area of biodiversity monitoring. Information about a preliminary work can be found here.
Marcel Simon and Erik Rodner.
Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks.
In: International Conference on Computer Vision (ICCV), 2015. [pdf][bib][code]
T. Xiao, Y. Xu, K. Yang, J. Zhang, Y. Peng, and Z. Zhang.
The application of two-level attention models in deep convolutional neural network for fine-grained image classification.
In CVPR, 2015.
G. Van Horn, S. Branson, R. Farrell, S. Haber, J. Barry, P. Ipeirotis, P. Perona, and S. Belongie.
Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection.
In CVPR, pages 595-604, 2015.