RiLM paper proposes geodesic decoding to cut output-layer cost in sub-million-parameter language models

A new arXiv paper introduces RiLM, a method for training language models with under one million parameters, targeting use cases such as edge deployment, domain adaptation, and reproducible research. The authors note that a small two-layer LSTM or Transformer with 128-dimensional embeddings loses about a third of its capacity to the output matrix, and RiLM addresses this overhead through a geodesic decoding approach. The work appeared as a new submission in cs.CL with a cross-listing in cs.AI.