A new arXiv paper describes FrogNano, a 4-billion-parameter agent built to handle software engineering work even on limited hardware. The model is post-trained solely with reinforcement learning across roughly 1,500 SWE environments generated through online task synthesis rather than relying on fixed training data.
A new arXiv paper argues that knowledge distillation breaks down when the student model is much smaller than its teacher, because imitating the teacher's logits locks the student into its sharpest output modes and harms generalization. The authors propose letting the large teacher guide the small student through prompts during reinforcement-learning fine-tuning instead of through gradient-based distillation. The work appears in the computational linguistics category on arXiv.
A new arXiv paper takes a cognitive-simulation approach to episodic and semantic memory by feeding text from a web-crawled individual text corpus into small language models. The authors compare retrieval-augmented generation against DoRA fine-tuning for encoding personal knowledge, evaluating performance on multiple-choice question answering.
A new arXiv paper examines how physical-state inputs shape a 0.8B hybrid language model adapted for robotic manipulation with only 6.2M trainable parameters. The researchers train six conditions on three LIBERO-Spatial tasks and assess robustness across three seeds and 540 held-out rollouts. The results provide training controls and diagnostic measures for geometry conditioning in small embodied models.
A new arXiv paper introduces RiLM, a method for training language models with under one million parameters, targeting use cases such as edge deployment, domain adaptation, and reproducible research. The authors note that a small two-layer LSTM or Transformer with 128-dimensional embeddings loses about a third of its capacity to the output matrix, and RiLM addresses this overhead through a geodesic decoding approach. The work appeared as a new submission in cs.CL with a cross-listing in cs.AI.