arXiv 2305.13455
Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents
By Kranti Chalamalasetti, Jana Götze, et al.
Published 2023-05-22
Citation lineage
Review the prior work and downstream research connected to this paper.
Recent work has proposed a methodology for the systematic evaluation of "Situated Language Understanding Agents"-agents that operate in rich linguistic and non-linguistic contexts-through testing them in carefully constructed interactive settings. Other recent work has argued that Large Language Models (LLMs), if suitably set up, can be understood as (simulators of) such agents. A connection suggests itself, which t…