Single Depth Image Super-resolution

What is it about?

Although there is an increasing interest in employing the depth data in computer vision applications, the spatial resolution of depth maps is still limited compared with typical visible light images. In this paper, a novel method is proposed to synthetically improve the spatial resolution of a single depth image. It integrates the higher order terms into the Markov random field (MRF) formulation of example based methods in order to improve the representational power of those methods. The inference is performed by approximately minimizing the higher or der multi label MRF energies. In addition, to improve the efficiency of the inference algorithm, a hierarchical scheme on the number of MRF states is proposed. First, a large number of states are used to obtain an initial labelling by solving the minimization problem of inference for only the first order energies. Then, the problem is solved for the higher order energies in a smaller number of states. Performance comparisons show that proposed method improves the results of first order approaches that are based on simple four connected MRF graph structure, both qualitatively and quantitatively.

Featured Image

Why is it important?

Although MRFs with first order energies are efficient (especially from learning and inferring aspects), the representational power of those MRFs is usually limited, in contrast with higher order MRFs. This is due to the fact that higher order cliques can model more complex interactions and therefore can better reflect the natural statistics. However, until recently, there was a lack of efficient algorithms for optimizing MRF energies with higher order interactions. Fortunately, several promising work were introduced, which transform a general higher order MRF with binary labels into a first order one with the same minima. Combining the new transformation with fusion move algorithms, higher order multi label MRF energies can be approximately minimized by quadratic pseudo-Boolean optimization (QPBO) algorithms . Thanks to these recent advances, one can integrate higher order energies to the image super resolution formulation in order to enhance the representational power of existing example based approaches that are based on first order energies. To the best of our knowledge, this paper is the first exploration of integrating higher order energies to the MRF formulation of example based super resolution approaches.

Perspectives