Parameter estimation for stationary Markovian systems based on partially observable information

What is it about?

This article provides a comprehensive review of the parameter estimation problem for stationary Markovian systems based on partially observable information. First, we introduce the problem and its relevant background. Subsequently, we recall several classical methods, including the maximum likelihood method based on state aggregation, the Bayesian method, and the sufficient statistic method, among others. Following this, we highlight recent advances in the Markov chain inversion approach, including the specific techniques, key findings, numerical algorithms, and illustrative examples applicable to discrete and continuous time Markov chains in both reversible and stationary irreversible scenarios. Finally, we provide a detailed comparison between these methods and explore future prospects of parameter estimation for Markovian systems operated in random environments as well as Markov processes with continuous state spaces based on partial observations.

Featured Image

Photo by Luke Jones on Unsplash

Photo by Luke Jones on Unsplash

Why is it important?

In real-world applications, the generator matrix of Markovian systems can only be obtained using observable data, which might be from partial states, thereby questioning whether the partially available information be used to identify the dynamics of the system and how such partially observable data be used to identify the generator matrix.

Perspectives