Balanced k-shot sampling causes exact degeneracy in discriminant analysis on LLM embeddings
A new arXiv paper proves that balanced k-shot sampling, which draws exactly k labeled examples per class, induces an exact and provable degeneracy in a family of small-sample discriminant estimators. The result concerns kernelized linear discriminant methods applied to LLM embeddings, where the within-class scatter operator breaks down under equal per-class sample counts. The finding carries practical consequences for few-shot classification pipelines that rely on embeddings from large language models.