arXiv 2508.13526
MATA (māta): Mindful Assessment of the Telugu Abilities of Large Language Models
By Chalamalasetti Kranti and Sowmya Vajjala
Published 2025-08-19
Discussion
Read the public discussion and references gathered around this paper.
In this paper, we introduce MATA, a novel evaluation dataset to assess the ability of Large Language Models (LLMs) in Telugu language, comprising 729 carefully curated multiple-choice and open-ended questions that span diverse linguistic dimensions. We evaluate 11 open-weight and closed-source LLMs on our dataset and present a fine-grained analysis of their performance. Further, we empirically show how LLMs rely on…