#rubrics

3 curated events

papersTODAY 04:00 UTC

SkillLift: Learning Dense Rubrics from Sparse Oracles for Agent Skill Evolution

A new arXiv paper introduces SkillLift, a method for improving the reusable procedural prompts that LLM agents keep as persistent skills, which lets them adapt without retraining model weights. Rather than rewriting skill text directly from execution feedback, the approach derives dense scoring rubrics from limited, costly oracle evaluations to make skill evolution more efficient. The work targets lower evaluation cost during agent skill self-improvement.

agents/llm-agents/rubrics/arxiv

papersSEP 10 04:00 UTC

Study Reveals Position Bias in Rubric-Based LLM-as-a-Judge Evaluations

A new arXiv paper examines large language models acting as evaluators under rubric-based protocols, a setting that has received less attention than pointwise and pairwise comparison methods. The authors find that the ordering of responses systematically influences the judge's verdicts, exposing position bias in this evaluation setup. The results suggest that pipelines relying on LLM judges may need safeguards or reordering strategies to produce reliable assessments.

evaluation/llm-judges/position-bias/rubrics

papersSEP 10 04:00 UTC

Evidence-Grounded Text Evaluation with LLM Judges Aims to Make Rubric Scoring Reliable

A research paper on arXiv introduces a method for scoring text against evaluation rubrics using large language models, addressing how black-box judge models can apply identical criteria in inconsistent ways. The approach ties each score to concrete evidence drawn from the evaluated text, making the reasoning behind judgments easier to audit and reproduce. The work is cross-listed under arXiv categories for artificial intelligence, computational linguistics, and machine learning.

evaluation/llm-judges/rubrics/auditability