arXiv 2505.15045
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
By Siyue Zhang, Yilun Zhao, et al.
Published 2025-05-21
Citation lineage
Review the prior work and downstream research connected to this paper.
Large language model (LLM)-based embedding models, benefiting from large scale pre-training and post-training, have begun to surpass BERT and T5-based models on general-purpose text embedding tasks such as document retrieval. However, a fundamental limitation of LLM embeddings lies in the unidirectional attention used during autoregressive pre-training, which misaligns with the bidirectional nature of text embedding…