Using Semi-discrete Decomposition for Topic Identification

What is it about?

In the area of information retrieval, the dimension of document vectors plays an important role. We may need to find a few words or concepts, which characterize the document based on its contents, to overcome the problem of the "curse of dimensionality", which makes indexing of high-dimensional data problematic. To do so, we earlier proposed a Wordnet and Wordnet+LSI (Latent Semantic Indexing) based model for dimension reduction. While LSI concepts contain identifiable terms in top-level concepts, we show in this paper that semi-discrete decomposition provides mostly smaller list of terms and we need to cope only with ternary weights. With this size of term list, the identification of document's topic becomes much more feasible.

Featured Image