arXiv 2305.13455
Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents
By Kranti Chalamalasetti, Jana Götze, et al.
Published 2023-05-22
Mindmap
Browse the paper's core ideas, clusters, and relationships in a structured outline.
Recent work has proposed a methodology for the systematic evaluation of "Situated Language Understanding Agents"-agents that operate in rich linguistic and non-linguistic contexts-through testing them in carefully constructed interactive settings. Other recent work has argued that Large Language Models (LLMs), if suitably set up, can be understood as (simulators of) such agents. A connection suggests itself, which t…