arXiv 2306.01708
TIES-Merging: Resolving Interference When Merging Models
By Prateek Yadav, Derek Tam, et al.
Published 2023-06-02
Citation lineage
Review the prior work and downstream research connected to this paper.
Transfer learning - i.e., further fine-tuning a pre-trained model on a downstream task - can confer significant advantages, including improved downstream performance, faster convergence, and better sample efficiency. These advantages have led to a proliferation of task-specific fine-tuned models, which typically can only perform a single task and do not benefit from one another. Recently, model merging techniques ha…