#multimodal

40 curated events

papersTODAY 04:00 UTC

PIVOT Method Uses Physics Cues to Detect AI-Generated Audio-Video

A new arXiv preprint proposes PIVOT, a detection approach that checks whether AI-generated audio and video follow real-world physical behavior rather than relying on visual artifacts. The authors argue that as generative models improve, common artifact-based detectors become less reliable, so physical consistency offers a more durable signal. The abstract is truncated, so full details on the method and evaluation are not available here.

deepfakes/detection/research/multimodal

papersTODAY 04:00 UTC

UniCAR-RL targets fine-grained perception failures in multimodal math reasoning

A new arXiv paper introduces UniCAR-RL, a reinforcement learning approach aimed at improving how multimodal large language models handle math problems involving diagrams and figures. The authors argue that weak fine-grained visual perception leads models to hallucinate details early, which then causes errors to compound through the rest of the reasoning chain. The method is framed as improving perception before deeper reasoning steps are attempted.

multimodal/reinforcement-learning/visual-reasoning/mllm

papersTODAY 04:00 UTC

FriendBench Benchmark Tests Whether AI Can Tell Friends From Strangers

Researchers introduced FriendBench, a benchmark that evaluates how well humans and multimodal large language models can judge whether two people in a short video clip are already acquainted or meeting for the first time. The task uses 20-second recordings of ice-breaker conversations, where cues come from behavior and body language rather than spoken content alone. The work aims to measure social perception abilities that go beyond text-based reasoning.

benchmark/multimodal/social-intelligence/research

papersTODAY 04:00 UTC

Paper Proposes Exploration-Guided Prompt Scaffolding for Multimodal RL Post-Training

A new arXiv paper argues that training prompts in online reinforcement learning vary widely in how useful they are to the current policy, with some already solved and others too hard to give a dependable learning signal. The authors propose an exploration-guided prompt scaffolding method that selects or structures prompts for multimodal reinforcement post-training so rollouts are better spent. The work appears in both the cs.AI and cs.LG listings as arXiv:2609.15051v1.

reinforcement-learning/multimodal/training-data/arxiv

papersTODAY 04:00 UTC

AURA: Unified Multimodal Framework for Conversational Music Editing

Researchers present AURA, a multimodal framework that lets users edit music through a back-and-forth conversation rather than one-off commands. Existing instruction-guided editors handle each request in isolation, which makes it hard to iteratively refine a track. AURA is designed to keep track of prior edits so a session can build progressively toward a desired result.

music/multimodal/audio/research

papersTODAY 04:00 UTC

TwinICL Benchmark Tests Multimodal In-Context Learning With Paired Counterfactuals

Researchers released TwinICL, a procedurally generated benchmark that provides matched text and image versions of the same tasks, allowing direct comparison of in-context learning across modalities. The paired counterfactual design is intended to isolate how much a model's few-shot performance depends on the input format rather than the task itself. The work appears on arXiv under cs.LG.

benchmark/multimodal/in-context-learning/research

papersTODAY 04:00 UTC

Language-Guided Multimodal Foundation Model Targets Brain Signal Analysis

Researchers posted a preprint describing a multimodal foundation model that uses language guidance to handle brain signal analysis without task-specific retraining. The approach is presented as supporting zero-shot and multi-task settings, addressing limited generalization in existing end-to-end and pre-trained models. The work appears on arXiv under cs.AI and cs.LG.

arxiv/foundation-models/multimodal/neuroscience

papersTODAY 04:00 UTC

VANGUARD Team Proposes Biometric and Demographic Conditioning for Multimodal Sexism Detection

A research team called VANGUARD submitted a framework for the EXIST 2026 Task 2 challenge that treats sexism detection as a subjective task shaped by individual perspective. The approach combines multimodal analysis with psychological and demographic conditioning, including biometric signals, to better model how different people judge the same content. The work appears in two arXiv cross-listings under cs.AI and cs.LG.

multimodal/sexism-detection/arxiv/bias

papersTODAY 04:00 UTC

Study uses voice activity projection to test visual turn-taking cues in face-to-face dialogue

Researchers examined whether visual information from face-to-face interaction improves prediction of conversational turn-taking, since most dialogue systems rely only on audio. The work applies voice activity projection to combine verbal and non-verbal signals. Results suggest visual cues can complement audio-based models of when speakers take turns.

research/conversation/multimodal/speech

papersTODAY 04:00 UTC

arXiv Paper Proposes Multimodal Method for Outside-the-Vehicle Landmark Referencing

A new arXiv preprint addresses Outside-the-Vehicle Referencing (OVR), the task of letting occupants query physical landmarks outside a car. The authors note that ego-motion and referential ambiguity make this difficult in autonomous vehicles and XR headsets. The abstract introduces a multimodal resolution approach, though the full method details are not given in the preview.

arxiv/autonomous-vehicles/multimodal/xr

papersTODAY 04:00 UTC

TimeThink: Method Aims to Improve Compositional Reasoning in Time-Series LLMs

A new arXiv paper introduces TimeThink, a technique intended to help time-series multimodal large language models reason more compositionally. The authors note that such models often struggle to capture dynamic temporal patterns when answering questions. The work focuses on eliciting stronger reasoning behavior from these models rather than treating forecasting as pure pattern matching.

research/time-series/reasoning/multimodal

papersTODAY 04:00 UTC

Func-R1: Method Aims to Improve Mathematical Function Reasoning in Multimodal LLMs

A new arXiv paper introduces Func-R1, an approach aimed at strengthening mathematical function reasoning in multimodal large language models. The work targets the challenge of combining visual perception with symbolic logic when solving math problems from images. The abstract frames deliberate mathematical reasoning in visual settings as an indicator of advanced multimodal model capability.

multimodal/reasoning/math/llm

papersTODAY 04:00 UTC

MedTRACE: tool-augmented multimodal agents for evidence-grounded clinical decisions

A new arXiv paper introduces MedTRACE, an agent framework that combines tools with multimodal clinical reasoning instead of mapping electronic health records, medical images, and physiological signals straight to diagnoses. The work targets evidence-grounded decision-making so that outputs can be traced back to the underlying patient data. It is a research contribution and has not been described as a deployed clinical product.

agents/multimodal/healthcare/research

papersTODAY 04:00 UTC

ParaBridge Ties Paralinguistic Cues to Dialogue Behavior in Speech Language Models

A new arXiv paper introduces ParaBridge, a method aimed at connecting the paralinguistic information in speech — such as vocal tone, speaker traits, or background noise — with the responses a spoken dialogue system produces. The authors note that while existing speech language models can detect such cues, they often fail to let that perception shape their replies, and the work targets closing that gap.

speech/dialogue/research/multimodal

papersTODAY 04:00 UTC

arXiv Paper Asks Where Language Belongs in Multimodal Models

A new arXiv paper examines the role of language in multimodal systems, noting that language models use text as input, output, and an increasingly internal representation. The author argues that whether language should hold all of these positions depends on what language does to the system that relies on it, drawing on evidence from human perception and cognition.

multimodal/language-models/research/cognition

papersTODAY 04:00 UTC

Multimodal Foundation Model Pretrained for Lunar Remote Sensing

Researchers introduce a multimodal, multiresolution foundation model trained from scratch for lunar remote sensing. It was pretrained on SomBench, a geographically partitioned dataset of roughly two million co-registered tile bundles covering 11 sensor modalities at two spatial resolutions of 1 m/pixel. The work targets general-purpose representation learning for planetary surface analysis.

remote-sensing/multimodal/foundation-models/space

papersTODAY 04:00 UTC

Retrieval-Grounded Reasoning Approach Proposed for Universal Multimodal Embeddings

A new arXiv paper introduces a method that grounds chain-of-thought reasoning in retrieved evidence to improve universal multimodal embeddings, which aim to represent text, images and other modalities in one shared space. The authors argue that reasoning steps should be tied to retrieval so that only relevant information shapes the final embedding. The work targets a single model that can handle a range of cross-modal retrieval tasks.

multimodal/retrieval/embeddings/reasoning

papersTODAY 04:00 UTC

Multi-Agent Vision-Language Framework Turns Product Images into Textual Reviews

Researchers present a multi-agent vision-language system that helps generate written product reviews grounded in user-uploaded images and videos from e-commerce platforms. The approach is intended to make use of visual feedback showing item quality, defects, packaging, and real-world use. The work is published as an arXiv preprint.

agents/vision-language/e-commerce/multimodal

papersTODAY 04:00 UTC

arXiv paper reports in-context learning emerges similarly across modalities

A new arXiv preprint examines few-shot in-context learning, the ability of a model to pick up abstract patterns from examples in its prompt and apply them to new inputs. The authors note this behavior has been studied mainly in large language models trained on next-token prediction, and report that it arises in a convergent way across different modalities. The announcement provides only the abstract, so methodological details are not yet available.

in-context-learning/multimodal/research/arxiv

papersTODAY 04:00 UTC

Paper examines how streaming omni-modal models decide what to answer and when

A new arXiv paper studies streaming omni-modal systems that process video chunks alongside synchronized audio and must choose what to respond to and at which moment. The authors note that visual cues can support an interpretation before a spoken utterance or sound event has finished, which complicates response timing. The abstract is truncated, but references a memo mechanism tied to when an interpretation is formed.

omni-modal/streaming/multimodal/arxiv

papersTODAY 04:00 UTC

Generative AI Framework Proposed for Low-Latency Multi-Camera Person Re-Identification

Researchers propose a multimodal framework that combines generative AI with person re-identification to improve matching across multiple cameras. The approach targets common failure cases such as viewpoint changes, varying lighting, occlusion, cluttered backgrounds, and low-resolution footage while aiming for low latency. The work is presented in an arXiv preprint.

person-reid/multimodal/surveillance/arxiv

papersTODAY 04:00 UTC

Fisher-Guided Adaptive Multimodal Fusion Proposed for Vulnerability Detection

A new arXiv preprint treats software vulnerability detection as a binary classification task and proposes a fusion approach guided by Fisher information to combine natural code sequence representations with other modalities. The method aims to adaptively weight each modality's contribution rather than fusing them uniformly. The work targets improved accuracy in flagging code snippets that contain security defects.

vulnerability-detection/multimodal/code-security/arxiv

papersTODAY 04:00 UTC

Paper studies switching between language and symbolic forms for spatial reasoning

A revised arXiv preprint examines how reasoning improves when models move between natural language and symbolic representations such as grids or sketches. The authors argue that human problem-solving is multimodal, with people offloading difficult steps into diagrams to expose structure and reduce errors. The work proposes treating this modality shift as a mechanism AI systems can adopt for spatial tasks.

reasoning/multimodal/arxiv/spatial-reasoning

papersTODAY 04:00 UTC

Sensory Precision Inference Proposed for Multimodal Arbitration in Agents

A new arXiv preprint introduces a method for autonomous agents to weigh sensory modalities against each other when inputs are noisy, incomplete, or contradictory. The approach infers how reliable each stream is and uses that estimate to arbitrate between modalities rather than assuming all sensors are equally trustworthy. The work targets robustness in real-world environments where sensory data quality varies over time.

agents/multimodal/uncertainty/arxiv

papersTODAY 04:00 UTC

MarKey: Marginal Utility Guided Greedy Keyframe Selection for Long Video Understanding

Researchers propose MarKey, a method that picks keyframes for long-video question answering using a greedy strategy driven by marginal utility rather than dense encoding or uniform sampling. The approach targets multimodal large language models, where encoding every frame is costly and even sampling can overlook brief but important moments. The paper is an arXiv preprint and appears under a cross-listing.

video-understanding/multimodal/keyframes/efficiency

papersTODAY 04:00 UTC

Graph-Guided Mixture of Experts Model Predicts Tumor Survival from Multi-Modal Data

A new arXiv preprint introduces a graph-guided mixture-of-experts approach for predicting tumor survival outcomes using multiple data modalities. The method aims to combine heterogeneous clinical inputs within a single predictive framework. The paper appears in the cs.LG category as a new submission.

medical-ai/multimodal/mixture-of-experts/arxiv

papersTODAY 04:00 UTC

ChartAnno Benchmark Tests Multimodal LLMs on Chart Annotation Generation

A new research benchmark called ChartAnno evaluates how well multimodal large language models can generate annotations for charts, a task that helps explain data and highlight key findings in visualizations. The work examines whether these models can automate annotation authoring, which is normally done by hand. It is presented as an arXiv paper revision.

benchmarks/multimodal/visualization/mllm

papersTODAY 04:00 UTC

Audio language models track speakers via text backbone attention, study finds

A new study examines how audio language models attribute speech to the correct speaker, finding accuracy of only 6 to 16 percent on a six-speaker task, below random guessing. The authors show that speaker tracking relies on attention heads in the model's text backbone, and that altering a subset of those heads shifts which speaker the model retrieves.

audio/attention/multimodal/research

papersSEP 11 04:00 UTC

arXiv paper proposes brain-network priors for multimodal language models

A preprint introduces the "Platonic brain bridge hypothesis," arguing that large models handling video, audio and text together tend to converge on representations resembling those found in the human brain, with the relationship working in both directions. The authors suggest that brain-like alignment could shift from being only a way to measure models toward an actual design principle for building them. The revised version mentions the hypothesis as an architectural prior, while the earlier cross-listed version frames it around so-called omni models.

research/multimodal/neuroscience/arxiv

papersTODAY 04:00 UTC

arXiv Paper Benchmarks Model-Agnostic Keyframe Selection for Long Video MLLMs

A new arXiv preprint evaluates keyframe selection techniques that can be plugged into existing multimodal large language models without modifying them. The work targets the constraint that MLLMs cannot ingest every frame of a long video due to visual-token and compute limits, and compares the main families of approaches proposed to address this. The study positions keyframe selection as a model-agnostic add-on for improving long-video understanding.

video-understanding/multimodal/benchmark/research

papersTODAY 04:00 UTC

Study Measures Generation Gap Between Speech-Only and Speech-Text Models

A new arXiv paper proposes a way to quantify the quality gap between speech-only language models and those that also handle text. The authors note that the gap is hard to measure because speech and text systems are usually trained on different data and judged with different metrics. The method aims to put the two modalities on a comparable footing.

speech/language-models/evaluation/multimodal

papersTODAY 04:00 UTC

Valley3: Omni Multimodal LLM Targets Global E-commerce Tasks

Researchers introduce Valley3, a multimodal large language model designed for e-commerce applications across different markets. The model handles text, images, video, and audio within a single framework, aiming to combine understanding and reasoning abilities across those modalities. The paper is a revised arXiv preprint, with an updated version posted to the cs.AI category.

multimodal/e-commerce/llm/arxiv

papersTODAY 04:00 UTC

Paper Proposes EventGraph and EventField Pipeline for Interpretable Temporal Video Reasoning

A new arXiv preprint describes a video reasoning approach that pairs a discrete event graph with a continuous event field, plus a human-readable glyph view, so intermediate reasoning steps can be inspected. The authors evaluate the pipeline on a curated EPIC-KITCHENS subset containing 10 videos and 50 questions about temporal relationships. The work sits in the interpretability and video-language research space rather than announcing a product or model release.

video-reasoning/interpretability/multimodal/benchmarks

papersTODAY 04:00 UTC

CapGeo-Bench separates visual perception from geometric reasoning in multimodal models

A new arXiv paper introduces CapGeo-Bench, a benchmark designed to evaluate geometric understanding in multimodal large language models while distinguishing failures in visual perception from failures in reasoning. The authors note that even strong closed models such as GPT-o3 continue to lag on geometry problems despite success on purely textual math tasks. The benchmark aims to give a clearer picture of where these systems break down.

benchmarks/multimodal/reasoning/geometry

papersTODAY 04:00 UTC

Unified benchmark targets multimodal time series forecasting with heterogeneous context

Researchers present a new benchmark for time series forecasting that moves beyond purely numeric data to include contextual signals such as text and other modalities. They argue existing multimodal benchmarks are limited in both the volume of data and the range of context they cover. The work is posted as an arXiv preprint in cs.AI and cs.LG.

benchmark/time-series/multimodal/research

papersTODAY 04:00 UTC

Study Audits Misalignment in Multi-Modal World Models

A new arXiv paper examines world models, systems that predict what happens next from current conditions, and how they behave when generating several modalities such as visual simulations at once. The authors propose an auditing approach to detect misalignment across these outputs, arguing that a single model can encode conflicting physical accounts. The work frames such inconsistency as a safety concern for multi-modal generation.

world-models/safety/multimodal/alignment

papersTODAY 04:00 UTC

Closed-Form Occlusal Geometry Proposed for Orthodontic Report Generation

This arXiv paper notes that intraoral scan datasets such as Bite2Text arrive already aligned in occlusion, which means key occlusal measurements can be calculated directly rather than inferred by a learned captioning model. The authors argue for deriving these quantities in closed form as a basis for automatically generating orthodontic reports. The work falls under computation and language research and has not been peer reviewed.

research/medical-ai/multimodal/arxiv

papersTODAY 04:00 UTC

Open-UniMo Framework Unifies Motion-Language Understanding and Generation

A new arXiv paper introduces Open-UniMo, a framework that aims to combine human motion generation with motion understanding in a single model for open-world settings. The authors note that most existing motion-language models treat motion as a secondary modality attached to language, which limits how well they generalize. The work targets embodied AI systems that need to both produce and interpret human actions.

motion-generation/embodied-ai/multimodal/arxiv

papersTODAY 04:00 UTC

ReH-FUSE: Reliability-Aware Fusion of Experts for Multimodal Emotion Recognition

A new arXiv paper introduces ReH-FUSE, a method for multimodal emotion recognition in conversation that accounts for how much each evidence source can be trusted in a given instance. The approach hierarchically combines expert predictions so that lexical, vocal, and other cues are weighted according to their reliability rather than treated as equally informative. It targets the problem that different modalities may dominate depending on the conversational context.

multimodal/emotion-recognition/fusion/arxiv

papersTODAY 04:00 UTC

SyRHM combines symbolic reasoning and associative retrieval for zero-shot harmful meme detection

Researchers propose SyRHM, a method for detecting harmful memes without task-specific training data. It targets implicit harm that comes from mismatches between image and text or from cultural stereotypes, which tripped up earlier multimodal detectors. The approach adds symbolic-language reasoning alongside associative retrieval to improve zero-shot performance.

multimodal/safety/content-moderation/zero-shot