papersSEP 12 04:00 UTC
Study examines catastrophic forgetting in skill retrieval for LLM agents
A new arXiv paper studies how synthetic data affects the ability of LLM agents to select the right external skill from large repositories. The authors describe a deployed skill router covering 34,396 skills and run a large-scale evaluation of retrieval under limited data conditions. The findings point to catastrophic forgetting as a risk when synthetic data is used for training these routers.