arXiv 2507.15821
Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks
By Hope Schroeder, Deb Roy, et al.
Published 2025-07-21
Mindmap
Browse the paper's core ideas, clusters, and relationships in a structured outline.
LLM use in annotation is becoming widespread, and given LLMs' overall promising performance and speed, simply "reviewing" LLM annotations in interpretive tasks can be tempting. In subjective annotation tasks with multiple plausible answers, reviewing LLM outputs can change the label distribution, impacting both the evaluation of LLM performance, and analysis using these labels in a social science task downstream. We…