arXiv 2403.02419
Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems
By Lingjiao Chen, Jared Quincy Davis, et al.
Published 2024-03-04
Discussion
Read the public discussion and references gathered around this paper.
Many recent state-of-the-art results in language tasks were achieved using compound systems that perform multiple Language Model (LM) calls and aggregate their responses. However, there is little understanding of how the number of LM calls - e.g., when asking the LM to answer each question multiple times and taking a majority vote - affects such a compound system's performance. In this paper, we initiate the study o…