Readspeaker menu

Understanding Deep Learning

alt alt

Project Description

Deep neural networks are tremendously successful in many applications, but end-to-end trained networks often result in hard to understand black-box classifiers or predictors. In this Project, we face this Problem. We develop methods, to determine whether a feature is relevant towards the decision of a deep neural network. Further, we develop methods to influence which features will be considered by a neural network during training. This can help to learn more robust, trustworthy and fair deep neural networks.

To utelize the power of deep learning in other filds of research, strong predictions are often not enough. Together with the Climate Informatics Group, we develope methods for unsupervised dimensionalty reduction in climate data.

In this project we employ Causal Inference as a method to model supervised learning processes. To

Understanding wich Features are Relevant to a Deep Neural Network

One research direction in this project is to determine wich feature is relevant to a deep neural network. To determin wich feature is considered by a deep neural network is can help us understad problems better and can help us to understand wich situations will be challanging for a trained deep neural networks. Especially important is this knowledge in safty and security critical tasks like autonomus driving or medical tasks and in situations of algorithmic fairness.

We presented a solution based on Causal Inference. To this end, we frame supervised learning as a structural causal model. In this framework, we can determine whether a feature is used by the deep neural network using a conditional independence test. We test the independence between the feature of interest and the prediction of the deep neural network given the ground truth label for the prediction. This method has the main advantage that it can handle features that are not regions in the input but abstract features of the input, such as symmetry or relative positioning.

In our publications we demonstrated, this methods for toy examples wich clearly highlight its advantages and also for real-live datasets. We demonstrated that it can be used to compare classifiers without data, that it can verify classifiers in medical tasks and that it can help us understand the behavior of trained classifiers.

Publications

2020

Determining the Relevance of Features for Deep Neural Networks.
Christian Reimers and Jakob Runge and Joachim Denzler.
European Conference on Computer Vision (ECCV). 2020. (accepted)
[bibtex] [abstract]

2019

Using Causal Inference to Globally Understand Black Box Predictors Beyond Saliency Maps.
Christian Reimers and Jakob Runge and Joachim Denzler.
International Workshop on Climate Informatics (CI). 2019.
[bibtex] [pdf] [abstract]

Unsupervised Representation learning for Earth System Science

The world climate is an enormous nonlinear dynamic system. Many large scale effects are heavily influenced by small scale effects and vice versa. Therefore, it is not possible to model effects locally. Global models, however, consist of such a high number of variables that modeling with classical methods is hard. This, the sheer endless amounts of data and the fact that conducting controlled experiments where certain variables are kept constant and others are varied is not possible on a large scale make climate informatics nowadays a data science and machine learning the best choice for many climate informatics problems. Because deep neural networks are often most efficient if learned end-to-end and we lack profound knowledge on the learning process and the learned concepts, deep neural networks are often applied as a black-box approach which leads to a number of problems:

  • The uncertainty is hard to control. Due to the net being dependent on the data it is a probabilistic approach and any classification or prediction made by a neural net is only a probabilistic statement. The uncertainty of these Statements is, however, unclear and can only be estimated numerically by testing on more data. If this quantitative results are not in reach, we need to provide at least reasoning behind the neural net decisions to enable human experts to provide at least qualitative uncertainty values.

    Since climate science aims to influence policy and public opinion, findings must be comprehensible for people not working in climate science and not familiar with machine learning methods. To achieve this we need the net to provide qualitative reasoning for its decisions.

  • Models in climate informatics have a high complexity but despite the huge amounts of data available, the amount of labeled data is small. Therefor, for climate science applications, it is needed to get a better understanding of the learning process and the learned concepts to design lean network topologies that are able to learn new concepts with limited amounts of training data.

  • Another approach to solve problems without a sufficient amount of labeled data is to transfer a pretrained neural network from a different domain. This is a good choice for many tasks in remote sensing. Without sufficient insight into neural networks however it is not possible to predict whether such a transition will be successful. The only possibility is to test numerically, which again requires a sufficient amount of data.

  • The goal of climate science is not only to make predictions but to develop insights. The vast utilization of deep neural networks in climate science hence necessitates a better understanding of such networks.

To solve any of this problems we need a better understanding of deep learning as well as methods of visualizing reasons for a neural net decision.

Publications

2020

Deep Learning--an Opportunity and a Challenge for Geo-and Astrophysics.
Reimers, Christian and Requena-Mesa, Christian. (2020)
[bibtex]

2018

SupernoVAE: VAE based Kernel-PCA for Analysis of Spatio-Temporal Earth Data.
Xavier-Andoni Tibau and Christian Requena-Mesa and Christian Reimers and Joachim Denzler and Veronika Eyring and Markus Reichstein and Jakob Runge.
International Workshop on Climate Informatics (CI). Pages 73-76. 2018.
[bibtex] [web] [abstract]