Readspeaker menu

Understanding Deep Learning

alt alt

Project Description

Deep neural networks are tremendously successful in many applications, but end-to-end trained networks often result in hard to understand black-box classifiers or predictors. In this Project, we face this problem. We develop methods, to determine whether a feature is relevant to the decision of a deep neural network. Further, we develop methods to influence which features will be considered by a neural network during training. This can help to learn more robust, trustworthy and fair deep neural networks.

To utilize the power of deep learning in other fields of research, strong predictions are often not enough. Together with the Climate Informatics Group, we developed methods for unsupervised dimensionality reduction in climate data.

In this project, we employ Causal Inference as a method to model supervised learning processes. This allows us to determine which feature is relevant to a deep neural network.

Understanding wich Features are Relevant to a Deep Neural Network

One research direction in this project is to determine which feature is relevant to a deep neural network. To determine which feature is considered by a deep neural network is can help us understand problems better and can help us to understand which situations will be challenging for a trained deep neural network. Especially important is this knowledge in safety- and security-critical tasks like autonomous driving or medical tasks and in situations of algorithmic fairness.

We presented a solution based on Causal Inference. To this end, we frame supervised learning as a structural causal model. In this framework, we can determine whether a feature is used by the deep neural network using a conditional independence test. We test the independence between the feature of interest and the prediction of the deep neural network given the ground truth label for the prediction. This method has the main advantage that it can handle features that are not regions in the input but abstract features of the input, such as symmetry or relative positioning.

In our publications, we demonstrated these methods for toy examples which clearly highlight its advantages and also for real-live datasets. We demonstrated that it can be used to compare classifiers without data, that it can verify classifiers in medical tasks, and that it can help us understand the behavior of trained classifiers.



Determining the Relevance of Features for Deep Neural Networks.
Christian Reimers and Jakob Runge and Joachim Denzler.
European Conference on Computer Vision (ECCV). 2020. (accepted)
[bibtex] [abstract]


Using Causal Inference to Globally Understand Black Box Predictors Beyond Saliency Maps.
Christian Reimers and Jakob Runge and Joachim Denzler.
International Workshop on Climate Informatics (CI). 2019.
[bibtex] [pdf] [abstract]

Unsupervised Representation learning for Earth System Science

The world climate is an enormous nonlinear dynamic system. Many large scale effects are heavily influenced by small scale effects and vice versa. Therefore, it is not possible to model effects locally. Global models, however, consist of such a high number of variables that modeling with classical methods is hard. This, the sheer endless amounts of data, and the fact that conducting controlled experiments where certain variables are kept constant and others are varied is not possible on a large scale make climate informatics nowadays a data science and machine learning the best choice for many climate informatics problems. Because deep neural networks are often most efficient if learned end-to-end and we lack profound knowledge on the learning process and the learned concepts, deep neural networks are often applied as a black-box approach which leads to a number of problems:

  • The uncertainty is hard to control. Due to the net being dependent on the data it is a probabilistic approach and any classification or prediction made by a neural net is only a probabilistic statement. The uncertainty of these Statements is, however, unclear and can only be estimated numerically by testing on more data. If these quantitative results are not in reach, we need to provide at least reasoning behind the neural net decisions to enable human experts to provide at least qualitative uncertainty values.
  • Since climate science aims to influence policy and public opinion, findings must be comprehensible for people not working in climate science and not familiar with machine learning methods. To achieve this we need the net to provide qualitative reasoning for its decisions.
  • Models in climate informatics have a high complexity but despite the huge amounts of data available, the amount of labeled data is small. Therefore, for climate science applications, it is needed to get a better understanding of the learning process and the learned concepts to design lean network topologies that are able to learn new concepts with limited amounts of training data.
  • Another approach to solving problems without a sufficient amount of labeled data is to transfer a pre-trained neural network from a different domain. This is a good choice for many tasks in remote sensing. Without sufficient insight into neural networks however it is not possible to predict whether such a transition will be successful. The only possibility is to test numerically, which again requires a sufficient amount of data.
  • The goal of climate science is not only to make predictions but to develop insights. The vast utilization of deep neural networks in climate science hence necessitates a better understanding of such networks.

To solve any of these problems we need a better understanding of deep learning as well as methods of visualizing reasons for a neural net decision.



Deep Learning--an Opportunity and a Challenge for Geo-and Astrophysics.
Reimers, Christian and Requena-Mesa, Christian. (2020)


SupernoVAE: VAE based Kernel-PCA for Analysis of Spatio-Temporal Earth Data.
Xavier-Andoni Tibau and Christian Requena-Mesa and Christian Reimers and Joachim Denzler and Veronika Eyring and Markus Reichstein and Jakob Runge.
International Workshop on Climate Informatics (CI). Pages 73-76. 2018.
[bibtex] [web] [abstract]