vision-language models

topic42 events

papersTODAY 04:00 UTC

Fine-Tuning Vision-Language Models with Listener Gaze for Referring Expressions

Researchers propose using recordings of how listeners' eyes move while they interpret a speaker's description as a training signal for vision-language models. By converting these gaze scanpaths into incremental feedback, the models learn to produce referring expressions that are more pragmatically suited to the listener. The work is presented as an arXiv preprint in the computation and language category.

vision-language/gaze/fine-tuning/referring-expressions

papersTODAY 04:00 UTC

arXiv Paper Revisits Correctness Measures for Uncertainty Estimation in Clinical VLMs

A new preprint examines how correctness is defined when vision-language models are used to make clinical predictions from medical images and electronic health records. The authors argue that current uncertainty estimation methods may be evaluated in ways that do not reflect whether a prediction is actually reliable. The work targets safer deployment by improving how unreliable outputs are detected.

uncertainty-estimation/medical-ai/vision-language/research

papersTODAY 04:00 UTC

Study Uses Activation Patching to Trace How VLMs Read Bar Chart Values

A new arXiv paper examines how vision-language models arrive at exact values when reading vertical bar charts. The authors apply counterfactual activation patching to trace where and how chart evidence is combined across space and depth in the network. The work argues that correct answers alone do not reveal the underlying mechanisms models use.

vision-language/interpretability/benchmarks/research

papersTODAY 04:00 UTC

E2A-Bench Tests Whether Financial Chart VLMs Turn Evidence Into Reliable Actions

A new arXiv paper introduces E2A-Bench, a benchmark aimed at measuring how reliably financial vision-language models convert chart evidence into action recommendations. The authors argue that existing hallucination tests focus on whether individual claims are supported, rather than whether the underlying evidence actually drives the recommended action. The benchmark is designed to close that gap in evaluating financial chart reasoning.

benchmark/vision-language/finance/evaluation

papersTODAY 04:00 UTC

Question-Guided Token Pruning Proposed as Privacy Defense for Vision-Language Models

A new arXiv paper proposes pruning visual tokens based on the question being asked, rather than sending all visual features to the server in split-learning setups. The approach targets vision-language question answering in federated, split, and U-shaped split learning, where raw data stays local but transmitted representations can still leak information. The authors frame selective transmission as a way to reduce both privacy exposure and bandwidth use.

privacy/vision-language-models/federated-learning/efficiency

papersTODAY 04:00 UTC

Multi-Agent Vision-Language Framework Turns Product Images into Textual Reviews

Researchers present a multi-agent vision-language system that helps generate written product reviews grounded in user-uploaded images and videos from e-commerce platforms. The approach is intended to make use of visual feedback showing item quality, defects, packaging, and real-world use. The work is published as an arXiv preprint.

agents/vision-language/e-commerce/multimodal

papersTODAY 04:00 UTC

Audit Finds Persona Prompts Bias Vision-Language Model Affordance Reports

A revised arXiv paper re-examines an earlier seven-prompt study on how vision-language models describe objects and their possible uses under different persona prompts. The authors argue that low overlap between responses alone does not prove an affordance effect, so they add matched-question controls to test whether the differences hold up. The work is a methodological audit aimed at tightening how such prompt-sensitivity claims are evaluated.

vision-language/evaluation/prompting/research

papersTODAY 04:00 UTC

arXiv paper examines robustness, cost and governance trade-offs in VLM document extraction

A new arXiv preprint argues that evaluations of vision-language models for extracting structured fields from business documents focus too heavily on accuracy against clean benchmarks. The authors propose assessing approaches along additional dimensions such as robustness, cost, and governance considerations, aiming to help practitioners pick a method suited to a given task complexity. No specific model or tool is released with the work.

vision-language-models/document-extraction/benchmarks/governance

papersTODAY 04:00 UTC

TestHallVQA benchmark probes document-level reasoning in vision-language models

A new arXiv paper introduces TestHallVQA, a benchmark built from scientific exam material for evaluating large vision-language models on visual question answering over long, multi-page documents. The authors argue that current planar VQA benchmarks tend to test isolated skills rather than document-level reasoning amid redundant context. The benchmark is intended to expose where such models fail when relevant information is buried in lengthy inputs.

benchmarks/vision-language/document-understanding/evaluation

papersTODAY 04:00 UTC

Probing Method Aims to Improve PEFT Layer Selection in Vision-Language Models

A new arXiv paper proposes a probing technique that analyzes weight statistics and perturbation robustness before fine-tuning to decide which layers of a vision encoder should be adapted. The authors argue this pre-fine-tuning approach can yield more stable improvements while training fewer parameters in large vision-language models. The work targets parameter-efficient fine-tuning, where only a small subset of weights is updated.

peft/vision-language/fine-tuning/research

papersTODAY 04:00 UTC

EventVL Uses Multimodal LLMs to Interpret Event Camera Streams

Researchers present EventVL, a multimodal large language model designed to interpret event-based camera data rather than relying on CLIP-style encoders. The work targets explicit understanding of event streams, a sensing modality where most prior vision-language approaches have focused only on conventional perception tasks. The paper is a revised cross-listing on arXiv.

multimodal/event-cameras/vision-language/research

papersTODAY 04:00 UTC

Survey Reviews Causes, Corrections and Evaluation of Hallucination in Multimodal AI Models

A revised arXiv paper surveys research on hallucination in multimodal foundation models, focusing on large vision-language models. It organizes the literature around why these errors occur, methods proposed to reduce them, and how they are measured. The authors frame the work as a structured overview connecting causes, corrections and evaluation practices.

hallucination/multimodal/vision-language/evaluation

papersTODAY 04:00 UTC

Robusto-2 Benchmark Tests Vision-Language Models for Self-Driving in Lima and New York

A new arXiv paper introduces Robusto-2, a benchmark evaluating both humans and vision-language models on autonomous driving tasks in Lima, Peru and New York City. The work targets how well multi-modal systems generalize when deployed in unfamiliar, out-of-distribution urban environments. It is a cross-listed replacement submission on arXiv cs.AI.

benchmark/autonomous-driving/vision-language-models/generalization

papersTODAY 04:00 UTC

Paper Details Option-Aware Retrieval and VLM Tuning for Offline Medical VQA

An arXiv paper describes a submission to the MedReason 2026 challenge that handles both multiple-choice and open-ended medical visual question answering with fully offline, containerized inference. The authors report that retrieval for multiple-choice questions needs to compare candidate options in a specific way, and they also adapt a vision-language model to the clinical task. The work is listed under both cs.AI and cs.CL.

medical-ai/vlm/retrieval/benchmark

papersTODAY 04:00 UTC

arXiv Paper Proposes Human-Grounded Calibration for Long-Text Image-Text Matching

A new arXiv preprint addresses the difficulty of judging whether lengthy descriptive text actually matches an image, a task relevant to vision-language systems. The authors note that raw similarity scores from dual-encoder models are hard to interpret and propose calibrating them against human judgments. The work targets more reliable long-text image-text congruence scoring.

vision-language/benchmarks/evaluation/arxiv

papersTODAY 04:00 UTC

NoteVQA Benchmark Targets Everyday Visual Questions From Human Communities

Researchers introduce NoteVQA, a benchmark that evaluates vision-language models on visual questions drawn from real human communities rather than pre-defined task categories. The work argues that current benchmarks focus on narrow capabilities such as multi-hop retrieval and miss the variety of questions users ask in daily life, including consumer AI search. It is published as an arXiv preprint.

benchmarks/vision-language/evaluation/vlm

papersTODAY 04:00 UTC

SportD benchmark tests whether vision-language models can make strategic soccer decisions

A new arXiv paper introduces SportD, a dataset that evaluates vision-language models on strategic decision-making using soccer as a testbed where action outcomes can be scored numerically. The work asks whether models that can describe a scene are also able to choose effective actions within it. This is a replacement version of the preprint.

vision-language-models/benchmarks/agents/arxiv

papersTODAY 04:00 UTC

VLM-Driven Parsing Method Aims to Improve Compositional SVG Generation

A new arXiv paper addresses the difficulty of producing structured, editable SVG graphics with vision-language models. Current approaches tend to output flat sets of paths with little semantic organization, which limits editability. The proposed method uses hierarchical semantic parsing driven by a VLM to make generation more compositional.

svg/vision-language-models/generation/arxiv

papersTODAY 04:00 UTC

Study probes whether vision-language models truly capture temporal structure

A new arXiv paper reframes temporal grounding as an anomaly-detection problem in order to test whether vision-language models actually represent time ordering in video and image sequences. The authors report that strong results on existing video benchmarks do not necessarily show that these models rely on temporal structure rather than shortcuts. They propose this setup as a way to measure temporal consistency more directly.

vision-language/evaluation/video/benchmarks

papersTODAY 04:00 UTC

MARCUS: Agentic Multimodal Vision-Language Model for Cardiac Diagnosis

Researchers present MARCUS, a vision-language system designed to assist with cardiac diagnosis and management. Unlike earlier medical AI models that handle only one type of input and cannot interact with users, it is described as agentic and multimodal. The work targets cardiovascular disease, the leading cause of death worldwide, where interpreting complex cardiac tests remains a bottleneck.

agents/multimodal/medical-ai/vision-language

papersTODAY 04:00 UTC

DiffAdapterVLA: Planner-Integrated Backbone for Driving VLMs

A new arXiv paper introduces DiffAdapterVLA, a method that folds continuous trajectory planning directly into the backbone of a pretrained driving vision-language model. The authors argue that while driving VLMs absorb rich visual, route, language and driving context, their training objectives stay disconnected from continuous planning, so prior work tends to bolt planning on separately. The approach targets native continuous trajectory generation rather than a detached planning head.

autonomous-driving/vlm/diffusion/arxiv

papersTODAY 04:00 UTC

GroundBench benchmark aims to pinpoint where vision-language models fail on affordance tasks

A new arXiv paper introduces GroundBench, described as a factorized, counterfactual benchmark for identifying the specific points at which vision-language models break down on affordance tasks. The work cites a companion evaluation in which explicitly naming the target part in a manipulation prompt improved action accuracy by 0.32 to 0.63 across eight vision-language models, and no model exceeded a constant baseline before that part was named. The benchmark is intended to isolate these failures rather than report only aggregate scores.

benchmarks/vision-language-models/evaluation/robotics

papersTODAY 04:00 UTC

Study Splits VLM Affordance Errors Into Part Grounding and Action Knowledge

A new arXiv paper argues that overall accuracy scores hide which stage of affordance prediction vision-language models actually fail at. The authors break the task into locating the relevant object part and knowing what action to apply, then test models on each separately. Their results indicate the bottleneck lies in part grounding rather than action knowledge.

research/vision-language/benchmarks/robotics

papersTODAY 04:00 UTC

arXiv paper studies image-question dependence in VLM test-time reinforcement learning

A new arXiv preprint examines how test-time reinforcement learning adapts vision-language models to unlabeled target data, noting that results depend heavily on the quality of self-generated training signals. The authors argue that consensus-based learning signals are inherently limited and propose exploiting dependence between images and their questions to improve reliability. The work is categorized under machine learning and has not yet been peer reviewed.

vision-language/reinforcement-learning/research/test-time-adaptation

papersTODAY 04:00 UTC

Paper studies how distractors affect test-time scaling in reasoning VLMs

A new arXiv preprint examines whether irrelevant information, known as distractors, changes how vision-language models behave when allowed to spend more compute at inference time. Prior work on text-only models found that such distractors can worsen inverse scaling, where reasoning degrades as test-time compute grows. The authors extend that question to multimodal settings, where models must handle both images and text. The submission is a cross-listed replacement in cs.AI and cs.LG.

vision-language/reasoning/test-time-scaling/arxiv

papersTODAY 04:00 UTC

Image-swap audit tests whether medical vision-language models rely on scans

A new arXiv paper introduces ModaLens, an audit method that swaps paired images to check how much a medical vision-language model actually depends on the image versus the accompanying radiology report. Because a report may already contain the answer to a clinical question, the technique measures how image sensitivity shifts when report text is available. The authors report results for MedGemma-27B among other models.

medical-ai/vision-language/evaluation/arxiv

papersTODAY 04:00 UTC

Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Models

A new arXiv paper proposes using transcoders, an alternative to sparse autoencoders, to study how vision-language models turn image inputs into text. The authors argue that sparse autoencoders decompose static representations and miss the flow of information, while transcoders can follow that transformation more directly. The method is used to locate where visual grounding happens and where hallucinated content originates in these models.

interpretability/vision-language/hallucination/research

modelsSEP 12 05:56 UTC

DeepSeek releases v4.1-Flash, a 763B-parameter encoder-decoder model with vision

DeepSeek has introduced v4.1-Flash, a large-scale model with 763B total parameters built on a new causal encoder-decoder design that also handles vision input. Commentary from Latent Space and Sebastian Raschka argues the release is significant enough that it should have been branded as a new major version rather than a point update. Details on training, availability and licensing were not included in the report.

deepseek/model-release/vision/encoder-decoder

papersSEP 12 04:00 UTC

arXiv Paper Tests How Vision-Language Models Choose Physical Experiments

A new arXiv preprint examines whether vision-language models can pick which physical measurement experiment to run next when given an image from an earlier trial. The setup asks a model to reason from one observation, such as how far a block slid, to predict the outcome of a new trial. The work probes how well these models transfer physical understanding to fresh experimental choices.

vision-language/reasoning/benchmark/arxiv

papersSEP 12 04:00 UTC

arXiv paper proposes using vision-language models to automate classification error analysis

A new arXiv preprint describes a method that applies vision-language models to verification and validation of classification systems. The approach aims to replace the slow manual review of misclassified samples by automatically surfacing systematic failure patterns under realistic conditions. The authors frame the work as moving evaluation beyond standard benchmarks toward conditions closer to deployment.

vision-language-models/benchmarks/validation/research

papersSEP 12 04:00 UTC

TeleOCR Paper Targets Document Parsing for Both Digital and Camera-Captured Files

A new arXiv paper introduces TeleOCR, a method for converting unstructured documents into structured, machine-readable output. The work focuses on the gap between digitally born PDFs and photos of physical documents, a distinction that existing vision-language model approaches often handle unevenly. It falls within ongoing research into applying VLMs to document understanding tasks.

document-parsing/vision-language-models/ocr/research

papersSEP 12 04:00 UTC

Study Evaluates Edge-Deployable Vision-Language Models for Species ID

A new arXiv paper argues that species identification from camera traps should be assessed using small vision-language models that can run locally on edge hardware, rather than frontier-scale systems. The authors note that field deployments often have weak or no network connectivity, which makes compact, on-device models the realistic option to study. The work positions this evaluation setting as the practically relevant benchmark for the task.

vision-language/edge-ai/benchmarks/wildlife

papersSEP 12 04:00 UTC

Soft Prompting Approach Adapts Vision-Language Models with Few Shots

A new arXiv paper tackles few-shot object detection with vision-language models in domains that differ from their training data, such as aerial, industrial, and medical imagery, where only ten labeled images are available. Rather than optimizing discrete text prompts, the authors propose a soft prompting method for adaptation. The work is a cross-listing on arXiv cs.AI.

vision-language/few-shot/prompting/object-detection

papersSEP 10 04:00 UTC

Show-Harness: arXiv framework lets VLM agents act as robot controllers

Researchers behind the Show-Harness project propose an embodied harness that lets general-purpose vision-language models operate robots directly. The approach aims to bridge the gap between the broad world knowledge of VLMs and the practical challenge of turning that intelligence into robot control.

vlm/robotics/embodied-ai/research

papersSEP 10 04:00 UTC

Study Quantifies Text Branch Sensitivity in Medical Vision-Language Segmentation

Researchers on arXiv examine whether clinical text inputs genuinely drive pixel-level predictions in pretrained vision-language models for medical image segmentation. They propose an evidence-decoupling approach to characterize the sensitivity of the text branch, aiming to clarify the real contribution of textual information to segmentation outputs.

medical-ai/vision-language/segmentation/interpretability

papersSEP 10 04:00 UTC

On-Policy Distillation Proposed for Vision-Language Model Adaptation on Low-Quality Data

A new arXiv paper introduces an on-policy distillation approach for adapting compact vision-language models from a larger task-trained teacher. Rather than relying solely on teacher predictions as training targets, the method lets the student learn from its own outputs, which the authors report makes it especially effective when multimodal training data is noisy or low quality.

distillation/vision-language/multimodal/efficiency

papersSEP 10 04:00 UTC

Two-token features and small-large VLM ensembles for hallucination detection at SHROOM-Visions 2026

Researchers present their system for the SHROOM-Visions 2026 shared task, which targets character-level detection of hallucinations in vision-language model outputs. The method fine-tunes a 4-billion-parameter VLM as a per-token classifier that reads a two-token feature from its own hidden states, then combines it with larger models in an ensemble.

hallucination/vlm/benchmark/ensembles

papersSEP 10 04:00 UTC

LogiScope-VQA: A Benchmark for Vision-Language Models on Warehouse Hazard Detection

Researchers have released LogiScope-VQA, a benchmark that evaluates whether large multimodal models can perceive, understand, and reason about safety hazards in industrial warehouse environments at a level comparable to human experts. The work addresses the lack of domain-specific evaluation data for deploying such models in logistics settings. The paper appears on arXiv with cross-listings in artificial intelligence and computational linguistics.

benchmarks/vision-language/safety/logistics

papersSEP 10 04:00 UTC

LightNav-0 Paper Applies VLM Spatial Reasoning to Generalist Embodied Navigation

A new arXiv paper presents LightNav-0, an approach that leverages the spatial knowledge already encoded in vision-language models to guide embodied navigation. The research targets agents that can convert varied goals and visual observations into actions across different tasks, environments, and robot embodiments.

embodied-ai/vlm/navigation/spatial-reasoning

papersSEP 10 04:00 UTC

ViSR-KGC: Vision-Language Model Approach to Multimodal Knowledge Graph Completion

Researchers have introduced ViSR-KGC, a method that uses vision-language models to reason over visual subgraphs when filling in missing entities or relations in knowledge graphs. The approach extends knowledge graph completion to multimodal settings by combining textual graph structure with information from entity-associated images. The work is detailed in a revised preprint posted on arXiv.

knowledge-graphs/multimodal/vlm/research