Using Variance as a Risk Metric in Reinforcement Learning and Markov Decision Processes

What is it about?

The tradeoff between risk and reward is well-known. Choosing the right metric for risk is critical for making good decisions. The Markov Decision Process (MDP) is an important problem within the domain of machine learning, in particular Reinforcement Learning (RL), where this tradeoff becomes critical. This paper presents a new RL algorithm in which variance of the rewards is used as a risk metric. The new algorithm is shown to work successfully on problems where the classical exponential utility risk metric breaks down.

Featured Image

Why is it important?

This work shows that the new algorithm works successfully on problem instances where the well-studied Exponential Utility (EU) risk metric breaks down. This is explained with a numerical example; a theoretical explanation of why the computer overflows when the EU framework is utilized is also provided.