papers AI Learner
The Github is limit! Click to go to the new site.

Estimating Risk and Uncertainty in Deep Reinforcement Learning

2019-05-23
William R. Clements, Benoît-Marie Robaglia, Bastien Van Delft, Reda Bahi Slaoui, Sébastien Toth

Abstract

This paper demonstrates a novel method for separately estimating aleatoric risk and epistemic uncertainty in deep reinforcement learning. Aleatoric risk, which arises from inherently stochastic environments or agents, must be accounted for in the design of risk-sensitive algorithms. Epistemic uncertainty, which stems from limited data, is important both for risk-sensitivity and to efficiently explore an environment. We first present a Bayesian framework for learning the return distribution in reinforcement learning, which provides theoretical foundations for quantifying both types of uncertainty. Based on this framework, we show that the disagreement between only two neural networks is sufficient to produce a low-variance estimate of the epistemic uncertainty on the return distribution, thus providing a simple and computationally cheap uncertainty metric. We demonstrate experiments that illustrate our method and some applications.

Abstract (translated by Google)
URL

http://arxiv.org/abs/1905.09638

PDF

http://arxiv.org/pdf/1905.09638


Similar Posts

Comments