#efficiency

40 curated events

papersTODAY 04:00 UTC

Grouped Value Attention Cuts KV Cache Cost via On-Demand Key Reconstruction

A new arXiv paper proposes Grouped Value Attention, a method that lowers the memory and read-bandwidth burden of the key-value cache during Transformer decoding. Unlike grouped-query attention, which still stores separate keys and values per shared head, the approach reconstructs keys on demand from stored values. This targets the KV cache, which the authors identify as a main bottleneck as sequence lengths grow.

kv-cache/efficiency/attention/inference

papersTODAY 04:00 UTC

UniRank Proposes Unified Rank Allocation for Low-Rank LLM Compression

A new arXiv paper introduces UniRank, a method for deciding how much rank to assign to each weight matrix when compressing large language models via low-rank decomposition. The authors argue that uniform or hand-tuned allocation schemes overlook differences in module importance. The work is released as a revised submission on arXiv.

llm-compression/low-rank/arxiv/efficiency

papersTODAY 04:00 UTC

arXiv Paper Proposes 'Post-Reasoning' Method to Improve Non-Thinking LLMs

A revised arXiv preprint introduces a technique called post-reasoning that aims to raise the quality of large language model outputs produced without intermediate reasoning traces. The work targets the cost and latency that come from generating long chains of thought at inference time, arguing that many practical tasks do not need them. The authors present the approach as an add-on that improves non-reasoning models without extra expense.

reasoning/inference/efficiency/llm

papersTODAY 04:00 UTC

Paper Proposes Slimmer Action Backbones for Diffusion-Based Robot Policies

A new arXiv paper argues that the action-generation backbones in Vision-Language-Action models are far larger than the task requires, since robot actions carry much less information than image pixels. The authors introduce a freeze-share-shrink strategy to cut parameters in diffusion and flow-matching policies while preserving performance. The work targets more efficient manipulation models for robotics.

robotics/diffusion-models/efficiency/vision-language-action

papersTODAY 04:00 UTC

Minibatch persistency revisited: trade-offs in steps, energy and data use

A new arXiv paper re-examines minibatch persistency, a technique where the same batch is reused for K consecutive optimizer steps rather than sampling fresh data each time. The authors quantify what batch reuse costs in terms of optimization steps and energy consumption, and what it saves in data throughput. The work revisits a long-standing objection to the method, which was folded into data echoing in 2019.

training/efficiency/batch-reuse/arxiv

papersTODAY 04:00 UTC

OpWeave: Operator-Level Disaggregation for Heterogeneous LLM Serving

A new arXiv paper introduces OpWeave, a system that breaks LLM inference into finer-grained operators rather than coarse stages, extending recent work that separates attention from FFN or MoE execution during decoding. The authors argue this operator-level disaggregation improves how workloads are matched to heterogeneous hardware during serving.

llm-serving/inference/efficiency/arxiv

papersTODAY 04:00 UTC

Stopping and restarting strategy speeds up multi-turn agentic on-policy distillation

A new arXiv paper addresses the high cost of on-policy distillation, which relies on expensive autoregressive rollouts by the student model and scales poorly when tasks span multiple turns. The authors propose deciding when to halt a rollout and where to resume it, aiming to cut the compute spent on generating student trajectories. The method targets more efficient transfer of capabilities from large teacher models to smaller students in agentic settings.

distillation/agents/efficiency/training

papersTODAY 04:00 UTC

Paper Proposes Reusing Rejected Tokens to Speed Up Speculative Decoding

A new arXiv paper introduces a technique called carryover drafting that reuses the model states computed for rejected draft tokens during speculative decoding. Because verification already produces representations for both accepted and rejected tokens, the method puts the discarded ones to further use instead of throwing them away. The goal is to improve the efficiency of LLM inference, where speculative decoding lets one target forward pass accept multiple drafted tokens at once.

speculative-decoding/llm-inference/efficiency/arxiv

papersTODAY 04:00 UTC

Question-Guided Token Pruning Proposed as Privacy Defense for Vision-Language Models

A new arXiv paper proposes pruning visual tokens based on the question being asked, rather than sending all visual features to the server in split-learning setups. The approach targets vision-language question answering in federated, split, and U-shaped split learning, where raw data stays local but transmitted representations can still leak information. The authors frame selective transmission as a way to reduce both privacy exposure and bandwidth use.

privacy/vision-language-models/federated-learning/efficiency

papersTODAY 04:00 UTC

arXiv Paper Proposes Dynamic Semantic Extraction Framework for LLM Inference

A new arXiv preprint introduces a Dynamic Semantic Extraction and Inference (DSEI) framework aimed at reducing the memory overhead that comes with token-level inference in large language models. The approach compresses semantics dynamically so that inference can partly happen in latent space rather than relying solely on token-by-token processing. The authors present it as a step toward more computationally efficient LLM deployment.

llm-inference/efficiency/latent-space/arxiv

papersTODAY 04:00 UTC

Adaptive Phase-Switching Method Targets Communication Costs in Federated LoRA Tuning

A new arXiv paper proposes adaptively switching between training phases to reduce the communication overhead of federated fine-tuning with low-rank adaptation. The authors argue that existing accounting methods for federated LoRA protocols overlook asymmetric transit costs between clients and the server. Their approach aims to make the dominant per-round communication expense more efficient while keeping trainable parameters small on each client.

federated-learning/lora/fine-tuning/efficiency

papersTODAY 04:00 UTC

Adaptive Context Management Method Targets Memory Limits in On-Device AI Agents

A revised arXiv paper proposes adaptive context management to reduce the memory burden of running AI agents locally on devices. The authors note that agent workloads inflate context size through large static tool schemas and long interaction histories, which strains the limited memory of phones and similar hardware. The work aims to make personalized, low-latency on-device assistance more practical under those constraints.

agents/on-device/context-management/efficiency

papersTODAY 04:00 UTC

Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation

A new arXiv paper presents Qwen-Image-Flash, a revised training approach for few-step distillation in visual generative models. The method aims to cut inference costs and enable real-time image generation without sacrificing output quality. It targets efficient deployment of generative foundation models in production settings.

image-generation/distillation/efficiency/research

papersTODAY 04:00 UTC

arXiv Paper Proposes Application Behavior Modeling to Cut Token Costs for Web Agents

A new arXiv preprint introduces a method for making web agents more token-efficient by modeling how web applications behave. The approach aims to reduce the cost of processing tokens, which the authors note is rising quickly alongside investment in agentic infrastructure. The paper is titled "Token Efficient Task Execution via Application Behavior Modeling for Web Agents."

agents/web-agents/efficiency/research

papersTODAY 04:00 UTC

Paper proposes setting inference depth per deployment from its expected questions

A new arXiv paper argues that although a transformer model is trained to handle any prompt, each real-world deployment only receives a narrow slice of queries, such as delivery complaints for a support assistant or Python for a coding tool. Yet every deployment currently pays the same inference compute cost. The authors propose choosing inference depth based on the questions a given deployment actually asks, rather than applying one uniform setting across all uses.

inference/efficiency/llm/research

papersTODAY 04:00 UTC

arXiv paper explores mixture-of-experts LLMs as efficient retrievers

A new arXiv preprint examines whether decoder-only language models built with mixture-of-experts layers can serve as effective first-stage retrievers. The authors note that retrieval quality typically rises with backbone size, but dense models force every query and document through all parameters. The work targets strong retrieval performance while limiting the computational cost per query.

retrieval/mixture-of-experts/llm/efficiency

papersTODAY 04:00 UTC

arXiv Paper Casts LLM Prefix Sharing as a Sorting Problem

A new arXiv paper argues that when prompts are built from reusable components such as retrieved passages, tool definitions, or few-shot examples, the order in which those pieces are arranged determines how much KV cache computation can be reused. Since current serving systems only reuse cache on exact prefix matches, the work recasts the ordering decision as a sorting problem. No specific benchmark results are described in the provided abstract.

kv-cache/llm-serving/efficiency/research

papersTODAY 04:00 UTC

AgentKV: Phase-Aware KV Eviction for Agentic LLMs

A new arXiv paper introduces AgentKV, a KV-cache eviction scheme designed for agentic serving, where token usage and decode-time bandwidth demands far exceed those of ordinary chatbot workloads. Existing eviction methods typically score cached keys against queries taken from the most recent tokens; AgentKV instead adapts this scoring to the distinct phases of agent execution. The approach targets both cache capacity pressure and decode throughput in long-running agent pipelines.

kv-cache/agents/llm-inference/efficiency

papersTODAY 04:00 UTC

Pull Router Cuts Token Costs in Long LLM Conversations via Lazy Memory Loading

A new arXiv paper proposes Pull, a session router that keeps conversation history in an addressable store instead of injecting all prior turns into the context window. Rather than summarizing or truncating old dialogue, the system materializes only the historical state needed at each step, aiming to avoid the quadratic token growth of full-context prompting without permanently losing information. The approach targets multi-hundred-turn, stateful LLM sessions.

research/context-window/memory/efficiency

papersTODAY 04:00 UTC

Paper Proposes LLM Routing to Optimize Text-to-SQL Generation

A new arXiv preprint describes a method that routes text-to-SQL requests between different large language models rather than relying on a single model. The approach aims to keep accuracy on complex database queries while cutting the latency and cost that come from always using the most capable model. The authors frame the work as a way to balance performance against efficiency in natural-language database access.

text-to-sql/llm-routing/arxiv/efficiency

papersTODAY 04:00 UTC

Geometric Flow Method Improves Graph Coarsening for GNN Pooling

A new arXiv paper proposes using geometric flow techniques to enhance graph coarsening, a pooling step used in graph convolutional networks to cut computational cost. The approach aims to reduce the expense of graph pooling operations that mirror pooling in standard convolutional networks. The work is a preprint and reports on method design rather than deployed results.

graph-neural-networks/arxiv/efficiency/research

papersTODAY 04:00 UTC

AdaVSkip Method Skips Visual Tokens Across Layers to Speed Up Multimodal LLM Inference

A new arXiv paper introduces AdaVSkip, a technique that reduces the number of visual tokens processed at each transformer layer to lower the cost of multimodal large language model inference. Rather than only compressing tokens along the sequence dimension, the approach adapts skipping decisions per layer. The work targets efficiency gains without retraining the underlying model.

multimodal/efficiency/inference/research

papersTODAY 04:00 UTC

EI-DDLGN: Encrypted Inference with Differentiable Logic Gate Networks under TFHE

A new arXiv preprint introduces EI-DDLGN, a framework for privacy-preserving deep learning inference built on Torus Fully Homomorphic Encryption (TFHE). The authors note that most existing TFHE-compatible neural network designs rely on arithmetic neurons, and their approach instead uses deep differentiable logic gate networks to improve efficiency. The work targets outsourced inference scenarios where sensitive input data must stay encrypted.

privacy/homomorphic-encryption/research/efficiency

papersTODAY 04:00 UTC

URCHIN: A Horizontal Spiking Language Model for Data-Constrained Pretraining

Researchers introduce URCHIN, a spiking neural language model designed for pretraining on small, developmentally plausible text corpora, as targeted by the BabyLM challenge. The work argues that most existing language models ignore the biological properties of the neural circuitry that underlies human language acquisition. It tests how much language a model can learn from child-scale data instead of internet-scale datasets.

spiking-neural-networks/language-models/babylm/efficiency

papersTODAY 04:00 UTC

LayerRoute Combines Adaptive Layer-Skipping With LoRA for Efficient LLM Inference

A new arXiv paper introduces LayerRoute, a parameter-efficient technique that skips selected transformer layers at inference time to cut compute costs. Each layer uses a hard-gated routing decision trained with a straight-through estimator, and the model is jointly fine-tuned with LoRA so that output quality is largely retained. The authors position the approach as a way to speed up large language model inference without sacrificing accuracy.

research/efficiency/lora/llm-inference

papersTODAY 04:00 UTC

Lightning Weave: Capability Composition for More Efficient Reasoning Models

A new arXiv paper introduces Lightning Weave, a method aimed at pushing the accuracy-efficiency frontier of reasoning models. The authors argue that accuracy and inference efficiency often pull toward different reasoning behaviors, making joint improvement difficult. Their approach relies on composing capabilities rather than optimizing the two objectives independently.

reasoning/efficiency/arxiv/llm

papersTODAY 04:00 UTC

MoARa Technique Targets Faster Low-Rank LLM Pre-training

A new arXiv paper introduces MoARa, a method for low-rank gradient projection aimed at cutting the memory used by optimizer states during large language model pre-training. The authors argue that existing approaches still need too many steps and too much wall-clock time to reach a given quality level, and trace this to two design choices they address with module-aware rank allocation and structure-preserving decomposition. The work is a research preprint and has not yet been peer reviewed.

llm/pretraining/efficiency/research

papersTODAY 04:00 UTC

Coresets for Softmax Attention: Improved Bounds and Efficient Constructions

A new arXiv paper studies query-oblivious coresets for softmax attention heads, defined as subsets of key-value pairs that approximate the full attention output within an error bound for every query in a ball. The authors build on earlier work by Liberty, Andoni and Kleiner on unweighted coresets to derive tighter bounds and faster construction methods. The goal is to reduce the compute and memory needed when evaluating attention over long key-value sets.

attention/efficiency/coresets/research

papersTODAY 04:00 UTC

arXiv Paper Proposes Training-Free Lexical Prompt Compression for LLMs

A new arXiv preprint describes a deterministic, training-free pipeline for shortening the prompts given to large language models by compressing their lexical content. The authors report a Pareto analysis of the trade-offs between compression and task performance across eleven task categories. The work targets the growing cost and context limits caused by long prompts in techniques like chain-of-thought and in-context learning.

prompt-compression/llm/efficiency/arxiv

papersTODAY 04:00 UTC

MarKey: Marginal Utility Guided Greedy Keyframe Selection for Long Video Understanding

Researchers propose MarKey, a method that picks keyframes for long-video question answering using a greedy strategy driven by marginal utility rather than dense encoding or uniform sampling. The approach targets multimodal large language models, where encoding every frame is costly and even sampling can overlook brief but important moments. The paper is an arXiv preprint and appears under a cross-listing.

video-understanding/multimodal/keyframes/efficiency

papersTODAY 04:00 UTC

Paper Derives Exact Finite Attention Responses from RoPE Derivatives

A research paper presents a method for computing exact local responses to attention interventions in transformer models. Using the derivative of rotary position embeddings, candidate edits can be scored from a cached baseline plus a single backward pass, avoiding full recomputation. The approach aims to make attention-level analysis and editing more computationally efficient.

research/transformers/attention/efficiency

papersTODAY 04:00 UTC

arXiv Paper Proposes Self-Orchestrating LLMs to Cut Inference Latency

A new arXiv preprint introduces a method for having language models coordinate their own computation by exploiting semantic dependencies between generated tokens. The authors argue that standard autoregressive decoding is slow and leaves GPUs underused when batch sizes are small, and that their approach improves inference efficiency. The work is currently a research preprint and has not been peer reviewed or released as a product.

research/inference/efficiency/llm

papersTODAY 04:00 UTC

NeuroFlex Enables Element-Level Co-Execution of ANNs and SNNs for Sparse Inference

A new arXiv paper proposes NeuroFlex, a scheme that lets artificial and spiking neural networks run together at the level of individual elements rather than whole layers or tiles. The authors argue this finer granularity avoids the idle hardware and wasted energy that hybrid accelerators suffer when workload traits change inside a layer, and that it does so without losing accuracy. The work targets sparse inference efficiency on specialized DNN accelerators.

spiking-neural-networks/efficiency/hardware/inference

papersTODAY 04:00 UTC

SpectralShift Method Extends Context Window of Gated DeltaNet Models

A new arXiv paper proposes SpectralShift, a technique that reparameterizes the spectral properties of gated DeltaNet layers to stretch their usable context window. The authors note that linear attention layers are increasingly used in place of softmax attention for long-context work, but existing extension methods typically rely on continued pretraining without altering the layer internals. Their approach instead modifies the layers themselves, aiming to make context extension more effective.

long-context/linear-attention/arxiv/efficiency

papersTODAY 04:00 UTC

T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning

A new arXiv preprint introduces T-LoopFormer, a looped Transformer variant that varies how many times each token passes through a shared parameter block rather than applying a fixed depth uniformly. The approach uses dynamic routing to decide token-level elastic depth, aiming to support latent reasoning while keeping parameter counts low. The announcement is an abstract-only submission, so reported results and comparisons are not yet detailed.

research/transformers/reasoning/efficiency

papersTODAY 04:00 UTC

ZAPS Method Aims to Speed Up Neural Architecture Search with Zero-Cost Proxies

A new arXiv preprint introduces ZAPS, a technique for selecting zero-cost proxies that estimate how well a neural network architecture will perform before any training takes place. Because evaluating candidates normally requires full training runs, such proxies could make architecture search far cheaper. The paper focuses on how to identify which proxy to use for a given search task.

neural-architecture-search/efficiency/research/zero-cost-proxies

papersTODAY 04:00 UTC

arXiv Paper Examines Quantization Trade-offs for Machine Translation Inference

A newly revised arXiv preprint analyzes how quantization affects large language models deployed for machine translation in server settings. The work weighs reduced memory use and faster inference against the quality loss that compression can introduce. It aims to help practitioners pick quantization settings that balance latency and translation accuracy.

quantization/machine-translation/inference/efficiency

papersTODAY 04:00 UTC

SparseTalk sparsifies 3D Gaussian language fields for 3D visual QA

Researchers propose SparseTalk, a method that compresses the semantic features stored in 3D Gaussian language fields used for 3D visual question answering. Dense scene representations can require tens of thousands of embeddings, driving up storage and memory costs. The approach aims to cut that redundancy while preserving performance on 3D VQA tasks.

3d-gaussian/efficiency/vqa/research

papersTODAY 04:00 UTC

Audit Compares ConFlayers and SWIFT for Periodic-Step Layer Skipping in LLM Inference

A three-seed, rigor-matched study audits two periodic-step, search-based layer-skipping methods, ConFlayers and SWIFT, which choose which transformer layers to run for a given input. The work also examines trained routing approaches as an alternative to search-based selection. The authors frame it as a reproducibility-focused comparison of efficiency techniques for LLM inference.

llm-inference/layer-skipping/efficiency/benchmark

papersTODAY 04:00 UTC

S-CEReBrO Architecture Targets Memory Limits in Continuous EEG Monitoring

A new arXiv paper introduces S-CEReBrO, an approach aimed at overcoming the memory constraints that arise when applying Transformer-based foundation models to long-running EEG recordings. Global attention scales poorly with signal length, which limits how far such models can be used for continuous brain-monitoring data. The work proposes a redesign intended to make long-sequence EEG analysis more tractable while retaining the generalization benefits of large pretrained models.

eeg/foundation-models/transformers/efficiency