Skip to content
Trung Trinh: Tackling covariate shift with node-based Bayesian neural network episode artwork

Jan 9, 2023 28 min

Trung Trinh: Tackling covariate shift with node-based Bayesian neural network

from Finnish Center for Artificial Intelligence FCAI Hosted by DASER

Abstract: Bayesian neural networks (BNNs) promise improved generalization under covariate shift by providing principled probabilistic representations of epistemic uncertainty. However, weight-based BNNs often struggle with high computational complexity of large-scale architectures and datasets. Node-based BNNs have recently been introduced as scalable alternatives, which induce epistemic uncertainty by multiplying each hidden node with latent random variables, while learning a point-estimate of the weights. In this paper, we interpret these latent noise variables as implicit representations of simple and domain-agnostic data perturbations during training, producing BNNs that perform well under covariate shift due to input corruptions. We observe that the diversity of the implicit corruptions depends on the entropy of the latent variables, and propose a straightforward approach to increase the entropy of these variables during training. We evaluate the method on out-of-distribution image classification benchmarks, and show improved uncertainty estimation of node-based BNNs under covariate shift due to input perturbations. As a side effect, the method also provides robustness against noisy training labels. Speakers: Trung Trinh is currently a Ph.D. student in the Probabilistic Machine Learning group at Aalto University under the guidance of Samuel Kaski and Markus Heinonen. His research focuses on using Bayesian methods to quantify predictive uncertainty of deep learning models.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Trung Trinh: Tackling covariate shift with node-based Bayesian neural network

0:00 28:38

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Finnish Center for Artificial Intelligence FCAI?

This episode is 28 minutes long.

When was this Finnish Center for Artificial Intelligence FCAI episode published?

This episode was published on January 9, 2023.

Can I download this Finnish Center for Artificial Intelligence FCAI episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.