Poisson-Process Topic Model for Integrating Knowledge from Pre-trained Language Models
整合预训练语言模型知识的泊松过程主题模型
Topic modeling is traditionally applied to word counts without accounting for the context in which words appear. Recent advances in large language models (LLMs) offer contextualized word embeddings that capture deeper semantic meaning and relationships. We aim to leverage such embeddings to improve topic modeling.We use a pre-trained LLM to convert each document into a sequence of word embeddings. This sequence is then modeled as a Poisson point process, with its intensity…