TagLDA: Bringing document structure knowledge into topic models

@techreport{citeulike:828875, abstract = {Latent Dirichlet Allocation models a document by a mixture of topics, where each topic itself is typically modeled by a unigram word distribution. Documents however often have known structures, and the same topic can exhibit different word distributions under different parts of the structure. We extend latent Dirichlet allocation model by replacing the unigram word distributions with a factored representation conditioned on both the topic and the structure. In the resultant model each topic is equivalent to a set of unigrams, reflecting the structure a word is in. The proposed model is more flexible in modeling the corpus. The factored representation prevents combinatorial explosion and leads to efficient parameterization. We derive the variational optimization algorithm for the new model. The model shows improved perplexity on text and image data, but no significant accuracy improvement when used for classification.}, author = {Zhu, Xiaojin and Blei, David and Lafferty, John }, citeulike-article-id = {828875}, institution = {University of Wisconsin, Madison}, keywords = {bayesian\_network, lda}, number = {TR-1553}, posted-at = {2006-09-05 17:24:01}, priority = {0}, title = {TagLDA: Bringing document structure knowledge into topic models}, year = {2006} }

TY - RPRT ID - citeulike:828875 L3 - citeulike-article-id:828875 TI - TagLDA: Bringing document structure knowledge into topic models IS - TR-1553 N2 - Latent Dirichlet Allocation models a document by a mixture of topics, where each topic itself is typically modeled by a unigram word distribution. Documents however often have known structures, and the same topic can exhibit different word distributions under different parts of the structure. We extend latent Dirichlet allocation model by replacing the unigram word distributions with a factored representation conditioned on both the topic and the structure. In the resultant model each topic is equivalent to a set of unigrams, reflecting the structure a word is in. The proposed model is more flexible in modeling the corpus. The factored representation prevents combinatorial explosion and leads to efficient parameterization. We derive the variational optimization algorithm for the new model. The model shows improved perplexity on text and image data, but no significant accuracy improvement when used for classification. KW - bayesian_network KW - lda AU - Zhu, Xiaojin AU - Blei, David AU - Lafferty, John PY - 2006/// ER -