arXiv 2006.10413
Are Pretrained Language Models Symbolic Reasoners Over Knowledge?
By Nora Kassner, Benno Krojer, et al.
Published 2020-06-18
Wiki summary
Explore the paper's summary, context, and related research on Papiers.
How can pretrained language models (PLMs) learn factual knowledge from the training set? We investigate the two most important mechanisms: reasoning and memorization. Prior work has attempted to quantify the number of facts PLMs learn, but we present, using synthetic data, the first study that investigates the causal relation between facts present in training and facts learned by the PLM. For reasoning, we show that…