papersTODAY 04:00 UTC
Paper Shows TF-IDF and BM25 Are Exact KL Divergences
A new arXiv preprint argues that the two most common query-document scoring methods, TF-IDF and BM25, both correspond exactly to Kullback-Leibler divergences. This gives the widely used retrieval heuristics a probabilistic justification and places them inside a single statistical framework. The author frames the work as filling a long-standing gap in how these ranking functions are derived.