Misleading chart question answering (MQA) addresses a critical vulnerability in automated data analysis systems. Current Vision-Language Models (VLMs) fail catastrophically when confronted with deceptive visualizations designed to manipulate perception through axis manipulation, cherry-picking, or disproportionate encoding, scoring below 50% accuracy on specialized benchmarks.

The task requires answering multiple-choice questions about chart images that contain visual deceptions. Given a chart image $I$ and natural language question $Q$ with candidate options $O = {a_1, a_2, …, a_n}$, the system must predict the correct answer $a^*$. The challenge lies in resolving conflicts between visual trends $V_{trend}$ and actual numerical relationships $D_{rel}$, where misleading charts incorporate a manipulation function $f_{deceptive}(D) \rightarrow I$ that creates trap answers:

\[a_{trap} = \text{Inference}(V_{trend}) \neq a^* = \text{Inference}(D_{rel})\]

Success is measured through three metrics: overall Accuracy (Acc), Wrong due to Misleader (WM) tracking errors from visual traps, and Wrong due to Other factors (WO) capturing non-deception-related failures.

The evaluation uses Misleading ChartQA (305 test samples), Curated Deceptive Chart Collection (110 expert-validated samples), and Mixed Standard and Misleading Benchmark (244 balanced samples) to test both robustness against deception and absence of over-skepticism on benign data.

ChartCynics employs an agentic dual-path framework that decouples visual perception from numerical verification:

1. Diagnostic Vision Path: Uses strategic ROI cropping with nemotron-graphic-elements-v1 to extract high-resolution crops of critical chart components (title, legend, x-axis, y-axis). A Diagnostic Agent performs blind structural analysis without access to questions/options to identify anomalies, generating an Action Directive.

2. OCR-Driven Data Path: Utilizes LlamaParse (GPT-4o) to extract numerical literals and serialize charts into structured Markdown format, bypassing deceptive visual encodings to recover underlying data relationships.

3. Agentic Summarizer: Implements Detective Chain-of-Thought (D-CoT) reasoning through a 5-step process: perception audit, numerical anchoring, deception mapping, sufficiency check, and adversarial trap rejection.

   The core technical contribution is the conflict arbitration mechanism that weighs evidence hierarchically rather than simply discarding one modality. The objective becomes:

   \[a^* = \arg\max_{a \in O} P(a | P_v(I), P_d(I), Q, T)\]

   where $P_v$ and $P_d$ represent visual and data path outputs, and $T$ is the misleading taxonomy used for expert guidance.

Two-stage optimization pipeline:

1. Oracle-Informed SFT: Supervised fine-tuning on 5,238 reasoning chains distilled from Qwen3-VL-32B teacher model using Misleading ChartQA training set (2,619 samples). Teacher model uses ground-truth CSV data but generates reasoning chains based only on visible chart elements to bridge epistemic gap between structured data and visual pixels.

2. Deception-Aware GRPO: Group Relative Policy Optimization with group size G=8. Multi-objective reward function with coefficients: factual grounding (w_fact=0.20), semantic contradiction (w_contra=0.25), logical consistency (w_logic=0.20), format enforcement (w_fmt=0.10). Asymmetric reward shaping applies -2.0 penalty for selecting trap answers and +1.0 reward for correct answers.

   Training conducted on 4× NVIDIA A800 (80GB) GPUs using Qwen3-VL-8B backbone. Optimizer details, learning rates, and wall-clock times not reported.

Prior work:

• MATCHA (2023): Enhanced visual language pretraining with math reasoning and chart derendering for standard ChartQA
• DEPLOT (2023): One-shot visual reasoning by plot-to-table translation using OCR linearization
• Misleading ChartQA (2025): Benchmark showing VLMs score below 50% on deceptive visualizations

Delta: This paper introduces dual-path architecture that synergistically combines diagnostic visual analysis with OCR extraction, rather than choosing one modality over another. The key innovations are:

1. decoupled agentic workflow preventing confirmation bias through blind structural analysis
2. detective chain-of-thought reasoning for conflict resolution, and

3. deception-aware GRPO alignment specifically targeting visual trap avoidance.

   Applied-specific assessment:

   • Architectural idea is a reasonable engineering solution combining existing components (ROI extraction, OCR, structured reasoning)
   • Benchmark gains are substantial (~29% absolute improvement over backbone) and validated across multiple datasets
   • Comparisons appear fair, testing on same models with consistent evaluation protocols
   • Gains likely depend on specialized training data and dual-path processing overhead

   Verdict: SIGNIFICANT — The dual-path conflict resolution approach addresses a real problem with substantial empirical gains, though the core insight of combining visual structure detection with OCR verification is somewhat incremental.

1. Misleading ChartQA (MC): ChartCynics achieves 74.43% accuracy vs. Qwen3-VL-8B baseline 45.57% (+28.86% improvement). Outperforms Gemini-3.1-Pro (70.49%).

2. Curated Deceptive Chart Collection (CDCC): ChartCynics reaches 64.55% accuracy vs. Qwen3-VL-8B baseline 35.45% (+29.1% improvement).

3. Mixed Standard and Misleading Benchmark (MSMB): Training-free ChartCynics achieves 81.15% overall vs. ChartMoE 60.25%. On standard charts: 94.26% vs. ChartMoE 88.52%. On misleading charts: 68.03% vs. ChartMoE 31.97%.

4. Wrong due to Misleader (WM) reduction: ChartCynics reduces WM from 40.00% to 11.15% on MC dataset, demonstrating effective trap avoidance.

   Results consistently show large improvements across different base models (GPT-o4-mini, Gemini-2.5-Flash) and datasets, indicating robust performance gains.

1. Model size: 8B parameters (Qwen3-VL-8B backbone)

2. Training compute: 4× NVIDIA A800 (80GB) GPUs for SFT and GRPO stages, specific GPU hours not reported

3. Inference speed/latency: Not reported, but dual-path processing with ROI extraction and OCR parsing likely adds significant latency overhead

4. Memory footprint: Not reported

5. Deployment practicality: Moderate - requires additional components (nemotron-graphic-elements-v1 for ROI extraction, LlamaParse for OCR) and specialized training data, but achieves strong performance with relatively small 8B model

1. Real-world data evaluation: Uses expert-validated deceptive visualizations from established HCI studies in the Curated Deceptive Chart Collection, demonstrating effectiveness beyond synthetic benchmarks.

2. Deployment considerations: Framework tested on both proprietary (GPT-4o-mini, Gemini) and open-source models, showing broad applicability across different systems.

3. Production integration: No specific deployment results or production integration discussed.

4. Robustness validation: Mixed benchmark testing confirms the system doesn’t suffer from over-skepticism on benign data, actually improving standard chart comprehension (94.26% vs. 88.52% for ChartMoE).

1. ENGINEERING: Dependency on external components (ROI detector, OCR parser) that could introduce failure points or computational overhead

2. ENGINEERING: Requires specialized training data with expert-annotated deception types and reasoning chains, limiting scalability to new deception patterns

3. EVALUATION: Limited evaluation on chart types beyond standard statistical visualizations (bar, line, scatter plots)

4. ENGINEERING: Two-stage optimization pipeline adds training complexity compared to end-to-end approaches

5. FUNDAMENTAL: May struggle with novel deception types not covered in the misleading taxonomy used for training

   Failure modes:

6. OCR parsing errors could misalign numerical data with visual elements
7. Complex multi-panel or interactive visualizations might overwhelm the ROI-based structural analysis

Real-world context: Quantum software development relies on complex programming frameworks like Qiskit that expose intricate abstractions and evolve rapidly. As quantum computing matures, there’s increasing need for automated code generation assistants to help developers navigate these specialized APIs and quantum programming patterns.

Task definition: The paper studies quantum code generation using the Qiskit-HumanEval benchmark. Input: natural language task descriptions with function signatures. Output: executable Qiskit code that passes unit tests. The benchmark contains 151 programming tasks derived from HumanEval, adapted for quantum programming with Qiskit APIs. Tasks range from basic quantum circuit construction to advanced quantum algorithm implementations.

Evaluation criteria: Success measured using pass@1 metric - fraction of tasks where a single generated solution passes all unit tests. Tasks are executed in sandboxed Python environments with Qiskit installed.

Dataset: Qiskit-HumanEval benchmark with 151 tasks categorized as basic (majority), intermediate, and advanced difficulty levels. Tasks require generating functionally correct Qiskit code from natural language specifications.

1. Parameter-specialized baseline: Fine-tuned Granite-20B model trained on curated Qiskit corpora (from Dupuis et al. 2024)

2. General-purpose LLMs: OpenAI GPT models (4o, 4.1, 5, o3 variants), Anthropic Claude models (Opus 4.6, Sonnet variants, Haiku variants), Google Gemini models (Pro 3, Flash variants)

3. Retrieval-Augmented Generation (RAG): Dense retrieval using FAISS with text-embedding-3-large embeddings, retrieving from combined Qiskit documentation + source code corpus with k=4 chunks

4. Agent-based inference: Iterative generate-execute-repair loop where models receive Python error messages as feedback and generate revised solutions, bounded by 1-5 repair attempts

   Core contribution: Demonstrates that inference-time system-level specialization (RAG + agents) can match/exceed parameter-level fine-tuning performance without domain-specific training.

1. Parameter-specialized baseline: Granite-20B fine-tuned on curated Qiskit corpora (training details from Dupuis et al. - not fully detailed in this paper)

2. General-purpose models: No additional training - evaluated out-of-the-box via commercial APIs

3. RAG setup: Built retrieval corpora from Qiskit 2.0.1 documentation and Qiskit 2.2.0 source code, indexed using text-embedding-3-large embeddings

4. Agent configuration: Up to 5 iterative repair attempts with 10-minute per-attempt timeout

   Training compute, optimizer details, and other specifics: Not reported for the main models (commercial API access). Original Granite fine-tuning details referenced but not reproduced.

Step 1 — Prior work:

• Dupuis et al. (2024) “Qiskit Code Assistant” showed that fine-tuning Granite-20B on Qiskit corpora achieved 46.53% pass@1 vs 20.79% for base model
• General RAG and agent-based inference approaches (Lewis et al. 2021, Yao et al. 2023) established inference-time augmentation techniques
• HumanEval (Chen et al. 2021) provided the base code generation benchmark framework

Step 2 — Delta: This paper applies modern general-purpose LLMs (GPT-5, Claude Opus 4.6, etc.) with RAG and execution-feedback agents to quantum code generation, achieving up to 85.4% pass@1 vs 46.5% baseline.

Step 3 — Applied-specific assessment:

• Architectural novelty: INCREMENTAL - standard RAG + execution feedback, not novel architectures
• Benchmark gains: Substantial (85.4% vs 46.5% baseline, >20% over zero-shot) but limited to one benchmark
• Fair comparisons: Somewhat questionable - inference-time methods use much more compute than single-pass fine-tuned baseline
• Scale dependence: Results rely on frontier commercial models with substantial compute

Verdict: INCREMENTAL — Solid engineering study showing inference-time augmentation can replace fine-tuning, but uses well-established techniques without architectural novelty.

1. Qiskit-HumanEval pass@1: Parameter-specialized baseline 46.5%, best general-purpose models 60-65% zero-shot, up to 85.4% (Claude Opus 4.6) with 5-step agents - improvement of 38.9 absolute percentage points over baseline

2. Execution time: Agent-based inference 2-10x slower than zero-shot, with Claude models showing better time scaling than OpenAI/Gemini models

3. Difficulty-tiered performance: Advanced tasks remain challenging across all models (~20-40% success rate vs >80% on basic tasks)

4. RAG effectiveness: Mixed results, modest gains for OpenAI models, neutral/negative for Claude and Gemini models

5. Agent iteration scaling: Performance generally improves from 1 to 5 repair attempts across all model families

   Results show consistent improvements from inference-time augmentation, but with significant computational overhead trade-offs.

1. Model size: Parameter-specialized baseline ~20B parameters, commercial models likely 70B-405B+ range (not disclosed)

2. Training compute: Not reported for commercial models, Granite fine-tuning details referenced but not reproduced

3. Inference speed: Zero-shot ~400-4000s total benchmark time, RAG similar, agent-based 2-10x slower (up to 12,000s for some configurations)

4. Memory footprint: Not reported

5. Deployment practicality: Commercial API dependency limits deployment flexibility; agent-based approaches require execution environments and substantial API cost increases due to multiple generation cycles

1. Benchmark-only evaluation: All results on Qiskit-HumanEval synthetic benchmark tasks, no real quantum application deployments reported

2. Production considerations: Paper discusses maintenance advantages of inference-time approaches for rapidly evolving quantum SDKs, but no actual production deployments

3. Cost analysis: Execution time reported but not API costs - agent-based approaches likely 2-5x more expensive due to multiple model calls

4. Quantum hardware integration: Not addressed - purely code generation without quantum backend execution

   Limited real-world validation beyond benchmark performance.

1. EVALUATION: Benchmark exposure risk - Qiskit-HumanEval is public and models may have seen similar code during pretraining

2. EVALUATION: Unfair compute comparison - agent methods use substantially more inference compute than fine-tuned baseline

3. EVALUATION: Single benchmark evaluation limits generalization claims across quantum frameworks

4. FUNDAMENTAL: Advanced tasks remain challenging (20-40% success) even with best methods

5. ENGINEERING: Commercial API dependency creates deployment constraints and cost unpredictability

6. ENGINEERING: Agent execution requires sandboxed environments and timeout handling for non-terminating code

   Failure modes: Models can generate syntactically correct but semantically meaningless quantum code; iterative repair can lead to increasingly complex but incorrect solutions.

LanteRn addresses visual reasoning limitations in large multimodal models (LMMs) that predominantly rely on verbalizing visual content into text, losing fine-grained spatial information crucial for perception-heavy tasks.

The task is visual question answering with structured reasoning. The input consists of an image I and question Q, producing a hybrid reasoning trajectory τ = [s1, s2, …, sT] where each state st is either a discrete token from vocabulary V or a continuous latent vector zt ∈ R^d. The model generates interleaved text and K-token latent blocks using control tokens <

lvr_start

>, <

lvr_sep

>, and <

lvr_end

>.

Success is measured by final answer accuracy on three benchmarks: VisCoT (visual chain-of-thought reasoning), V* (visual search in real-world scenarios), and Blink (core visual perception where textual priors are insufficient). The evaluation covers object localization, spatial relationships, and fine-grained visual understanding.

The paper doesn’t introduce a new dataset but constructs synthetic training data from Visual-CoT by prompting Qwen3-VL-235B-Thinking to generate structured reasoning traces with bounding box supervision.

1. Base architecture: Qwen2.5-VL-3B-Instruct with extended vocabulary including control tokens <

   lvr_start

   >, <

   lvr_sep

   >, <

   lvr_end

   >

2. Hybrid reasoning mechanism: Model operates in two modes - text mode (standard autoregressive) and visual latent mode where hidden states bypass language head for K consecutive steps

3. Target latent representation extraction: Uses frozen vision encoder to extract features Fb for bounding box regions b, then applies average pooling:

   \[F_b = \text{VisionEncoder}(I, b)\] \[Z_{\text{target}} = \text{Pool}(F_b) \in \mathbb{R}^{K \times d}\]

4. Multi-objective training loss combining text generation and latent alignment:

   \[L_{\text{LanteRn}} = L_{\text{text}} + \gamma L_{\text{latent}}\]

5. Latent alignment loss using MSE between generated and target latent sequences:

   \[L_{\text{latent}} = \frac{1}{K} \sum_{i=1}^{K} ||h_{gen}^{(i)} - z_{target}^{(i)}||_2^2\]

   The core technical contribution is enabling continuous latent visual “thoughts” interleaved with discrete text tokens, allowing reasoning in visual feature space rather than forcing verbalization.

1. Stage 1 - Supervised Fine-Tuning: Custom dataset from Visual-CoT with bounding box supervision. AdamW optimizer, learning rate 1×10^-5, cosine schedule with 0.05 warmup ratio. Frozen vision encoder, latent loss weight γ=0.1. Variants trained with K∈{4,8,16,32} latent tokens.

2. Stage 2 - Reinforcement Learning: VIRL-39k dataset without bounding box supervision. Group Relative Policy Optimization (GRPO) with learning rate 5×10^-6, warmup ratio 0.03, KL regularization β=0.1. Sampling with temperature T=0.6, top-p=0.85, G=4 rollouts per prompt. Combined accuracy reward (weight 1.0) and format reward (weight 1.0). Latent state replay mechanism to stabilize training.

   Training time and specific hardware details not reported. Uses TRL library with custom GRPOTrainer extension for hybrid action spaces.

Prior work:

• Latent Visual Reasoning (Li et al., 2025): Conditions final answers on latent visual tokens but primarily appends them for downstream decoding
• Machine Mental Imagery (Yang et al., 2025b): Introduces latent image representations but lacks iterative refinement
• Tool-based methods (Yang et al., 2023; Surís et al., 2023): Use external vision modules but require hand-designed tools

Delta: LanteRn formulates latent visual reasoning as an interleaved process alternating between text and latent tokens, enabling iterative refinement of internal visual representations rather than one-shot latent conditioning.

Applied-specific assessment:

• Architectural novelty: The interleaved text-latent mechanism is a reasonable extension of prior latent reasoning work, not fundamentally novel
• Benchmark gains: Mixed results across benchmarks, with some improvements (BlinkOL: 0.48→0.54) but also degradations on certain tasks
• Fair comparisons: Limited by using 3B model vs. 7B baselines in prior work, though authors acknowledge this limitation
• Scale dependence: Gains appear modest and may not hold without the two-stage training recipe

Verdict: INCREMENTAL — Solid engineering contribution that extends latent visual reasoning to interleaved sequences, but the core idea is an expected evolution of existing approaches with mixed empirical gains.

1. VisCoT: Accuracy improved from 0.66 (base) to 0.83 (LanteRn-RL-8), previous SOTA not reported, +0.17 improvement
2. V* Direct Attribution: 0.75 (base) to 0.76 (LanteRn-RL-8), minimal +0.01 improvement
3. V* Relative Position: 0.63 (base) to 0.67 (LanteRn-RL-8), +0.04 improvement
4. V* overall: 0.70 (base) to 0.71 (LanteRn-RL-8), +0.01 improvement
5. Blink Object Localization: 0.48 (base) to 0.54 (LanteRn-RL-8), +0.06 improvement

6. Blink Relative Position: 0.81 (base) to 0.81 (LanteRn-RL-8), no improvement

   Results are mixed across benchmarks with modest improvements. Some tasks show degradation during SFT stage before RL recovery. Performance doesn’t scale monotonically with latent size K, suggesting optimization challenges.

1. Model size: 3B parameters (Qwen2.5-VL-3B backbone)
2. Training compute: Not reported (GPU hours, hardware unspecified)
3. Inference speed/latency: Not reported
4. Memory footprint: Additional K×d latent vectors per reasoning step, where d is hidden dimension
5. Deployment practicality: Reasonable given 3B scale, but requires specialized inference code for hybrid text-latent generation and control token handling. Latent state replay mechanism adds implementation complexity.

1. Evaluation limited to curated benchmarks (VisCoT, V*, Blink) with no real-world deployment results reported
2. No hardware experiments, robotics applications, or production integration discussed
3. No sim-to-real transfer analysis provided
4. Training data derived from synthetic annotations using Qwen3-VL-235B-Thinking, raising questions about generalization to truly novel visual scenarios
5. Fixed latent size K may not adapt well to varying complexity of real-world visual reasoning tasks

1. FUNDAMENTAL: Fixed latent size K doesn’t adapt to task complexity, leading to performance degradation with larger K values
2. ENGINEERING: Training stability requires latent state replay mechanism and careful hyperparameter tuning of multi-objective loss
3. EVALUATION: Limited to curated benchmarks without real-world validation or analysis of latent representation quality
4. FUNDAMENTAL: Supervision strategy relies on frozen vision encoder features, potentially limiting latent representations to encoder’s capabilities

5. ENGINEERING: Two-stage training pipeline adds complexity compared to end-to-end approaches

   Failure modes:

   • Performance degradation on relational reasoning tasks during SFT stage
   • Potential collapse to purely textual reasoning without format reward enforcement

The task addresses ultra-high-resolution image generation at extreme aspect ratios (EAR), particularly for panoramic images and traditional scroll paintings. Current text-to-image diffusion models trained on conventional dimensions suffer from catastrophic structural failures when generating images with extreme aspect ratios (e.g., 8:1) due to lack of robust spatial priors, resulting in object repetition and spatial fragmentation.

The task takes text prompts as input and generates ultra-high-resolution images with extreme aspect ratios up to 32K resolution. The approach reformulates EAR synthesis as a sequential video generation process, mapping spatial expansion of a massive canvas to temporal evolution of video frames. The objective is to minimize the Flow Matching loss:

\[\mathcal{L}_{\text{FM}} = \mathbb{E}_{z_0, z_1, \tau} \left[\left\|v_\theta (z_\tau , \tau , c, \mathcal{R})-(z_1 - z_0)\right\|_2^2\right]\]

Success is measured using patch-based FID and KID scores, CLIP alignment, Intra Style Loss for visual continuity, and Global Structural Diversity (GSD) metrics using LPIPS and DINOv2 features to detect object repetition across panoramas.

The authors contribute a curated dataset of 3,000 high-resolution multi-ratio images including 2,000 natural landscapes (6:1+ ratios) and 1,000 traditional Chinese scroll paintings (6:1 ratio).

1. Base architecture: Wan2.1-T2V-1.3B video diffusion model used as foundation

2. Scanning Positional Encoding (ScanPE): Re-engineers 3D RoPE by distributing global coordinates across video frames, with global anchor position defined as:

   \[\mathbf{O}_t = \sum_{k=1}^{t-1} \delta \cdot \mathbf{d}_k + \mathbf{P}_{init}\]

3. Global coordinate projection from frame-centric to unified system:

   \[\mathbf{P}_g (t, \mathbf{p}_{loc}) = \mathbf{p}_{loc} + \mathbf{O}_t\]

4. Trajectory-aware rotational embedding:

   \[\mathcal{R}(t, H_g, W_g) = \text{Concat} \left[ \mathbf{\Theta}(t \theta_j), \mathbf{\Theta}(H_g \theta_j), \mathbf{\Theta}(W_g \theta_j) \right]\]
5. Scrolling Super-Resolution (ScrollSR): Leverages video super-resolution diffusion priors (FlashVSR) for frame-by-frame enhancement
6. Trajectory Anchored Partitioning (TAP): Zero-shot spatial alignment strategy for seamless 3D VAE decoding

7. Median Consensus Selection (MCS) for frame fusion:

   \[\bar{I}_t = \arg \min_i \left| f_{t,i} - \operatorname{Median}\{ f_{t,k} \}_{k=1}^N \right|\]

   The core contribution is reformulating spatial EAR synthesis as temporal video generation to leverage video models’ inherent temporal consistency as global structural constraint.

1. Base model: Initialize with pre-trained Wan2.1-T2V-1.3B video diffusion model
2. Fine-tuning stage: Train on curated 3K multi-ratio dataset using conditional Flow Matching objective
3. Hardware: 2x A100 GPUs with total batch size of 4
4. Optimizer: AdamW with learning rate 1×10^-5
5. Training duration: 10,000 iterations
6. Inference: Base frames generated at reduced resolution, then processed by modified FlashVSR on single A100 (80GB) for 32K upscaling
7. Data filtering: Curated dataset of 2,000 natural landscapes (6:1+ ratios) + 1,000 traditional Chinese scroll paintings (6:1 ratio)
8. No additional training reported for ScrollSR component - leverages pre-trained video super-resolution priors

Prior work:

1. SyncDiffusion (2023) - partitions target space into overlapping patches for EAR generation but suffers from fragmented compositions
2. ScaleCrafter & DyPE (2023-2024) - manipulate internal representations via dilated convolutions and position embedding interpolation for high-resolution synthesis

3. MultiDiffusion & Tiled Diffusion - use tiling-based approaches for large image generation

   Delta: This paper reformulates EAR synthesis from static image generation to temporal video generation, introducing ScanPE for flexible coordinate distribution across video frames and ScrollSR for video super-resolution-based scaling.

   Applied-specific assessment:

   • The architectural idea of mapping spatial coordinates to temporal frames is genuinely novel and non-obvious
   • Benchmark gains are substantial: FID improves from 241-334 to 215, with clear elimination of object repetition artifacts
   • Comparisons appear fair using same evaluation protocols, though baselines use different underlying models
   • The approach requires pre-trained video diffusion models, making gains somewhat dependent on that foundation
   • User study shows 74-92% preference over baselines across multiple dimensions

   Verdict: SIGNIFICANT — The spatial-to-temporal reformulation is a genuinely clever architectural insight that effectively leverages video priors to solve a fundamental limitation of image diffusion models, with clear practical benefits demonstrated across multiple metrics.

1. FID (↓): ScrollScape 214.7 vs best baseline 241.2 (Tiled Diffusion), improvement of 26.5 points
2. CLIP Score (↑): ScrollScape 30.0 vs best baseline 29.7 (MultiDiffusion), marginal improvement
3. KID (↓): ScrollScape 2.0×10^-2 vs best baseline 3.0×10^-2 (Tiled Diffusion), 33% improvement
4. Style Loss (↓): ScrollScape 4.0×10^-3 vs best baseline 4.5×10^-3 (Tiled Diffusion), modest improvement
5. Global Structural Diversity LPIPS (↑): ScrollScape 0.674 vs best baseline 0.658 (MultiDiffusion), indicating better diversity
6. Global Structural Diversity DINOv2 (↓): ScrollScape 0.670 vs best baseline 0.682 (DyPE), indicating less repetition

7. User study: 74-92% preference over baselines across structural coherence, content richness, and image quality

   Results show consistent improvements across metrics, with particularly strong gains in eliminating object repetition (GSD metrics) and overall image quality (FID/KID). The method addresses fundamental failure modes of existing approaches.

1. Model size: Based on Wan2.1-T2V-1.3B (1.3 billion parameters)
2. Training compute: 2x A100 GPUs for 10,000 iterations, wall-clock time not reported
3. Inference speed/latency: Two-stage process - base generation then ScrollSR upsampling on single A100 (80GB), specific timing not reported
4. Memory footprint: Single A100 (80GB) required for 32K resolution generation via ScrollSR
5. Deployment practicality: Requires significant GPU resources (A100-class) for 32K generation, but approach is more memory-efficient than direct pixel-space generation due to latent-space processing. Sequential video generation may be slower than single-shot image generation but enables much higher resolution outputs.

1. Evaluation on curated dataset only - no reported deployment in production systems
2. Qualitative results shown across diverse domains: natural landscapes, traditional scroll paintings, microscopic textures, ice crystals
3. No hardware experiments beyond standard GPU evaluation reported
4. No sim-to-real transfer discussed as method focuses on static image generation
5. Generated samples appear suitable for real-world applications like digital art, panoramic photography, and traditional scroll creation
6. Method designed for creative/artistic applications rather than safety-critical deployment scenarios

1. ENGINEERING - Requires pre-trained video diffusion models as foundation, limiting independence
2. ENGINEERING - Two-stage inference process (base generation + super-resolution) increases computational cost
3. ENGINEERING - Memory requirements still substantial (A100 80GB) for 32K generation
4. EVALUATION - Training dataset relatively small (3K images) may limit generalization
5. EVALUATION - Evaluation primarily on 8:1 aspect ratios, unclear performance on more extreme ratios

6. FUNDAMENTAL - Sequential generation approach may be inherently slower than single-shot methods

   Failure modes:

   • May struggle with aspect ratios significantly beyond training distribution (>8:1)
   • Sequential approach could accumulate errors across long panoramic sequences leading to drift

Theory of Mind (ToM) evaluation for multimodal large language models (MLLMs) focuses on assessing their ability to infer mental states from visual information, addressing a critical gap in human-AI interaction capabilities. While most ToM evaluations use text-based inputs, real-world scenarios require understanding intentions, beliefs, and goals from visual cues alone. This is challenging because MLLMs suffer from hallucinations and over-reliance on linguistic priors when reasoning about mental states.

The task takes egocentric video sequences as input (24 frames) paired with multiple-choice questions about agents’ mental states. The formal objective is to maximize accuracy on three ToM reasoning subtasks:

\[P(\text{correct choice}|\text{video}, \text{question}) \rightarrow \max\]

Success is measured using Top-1 accuracy on three specific ToM tasks: goal inference (“What will C most likely do next?”), belief reasoning (inferring what agents know/believe), and action inference (predicting future behaviors).

The paper evaluates on EgoToM, a real-world egocentric video dataset with carefully curated question-answer pairs across diverse social interaction scenarios, providing ecological validity compared to simulated environments.

1. Base MLLMs: LLaVA-Next-Video-7B and Qwen2.5-VL-7B for video understanding and generation tasks

2. Internal representation extraction: Decompose visual ToM reasoning into visual attention and belief representation components by extracting activations from attention heads

   \[T_{l+1} = T_l + \sum_{h=1}^{H} Attn^h_l(P_l^h T_l) \cdot W^o_l\]

3. Visual attention enhancement: Generate adversarial examples using ℓ∞-bounded PGD attack (ε=16/255, 300 iterations) to create positive/negative sample pairs, compute activation offset vectors:

   \[\{\delta_{V,l}^h\} = \frac{1}{S}\sum_{i=1}^{S} (X_{V,i,l}^{pos,h} - X_{V,i,l}^{neg,h})\]

4. ToM reasoning guidance: Use correct answers as positive samples and incorrect answers as negatives, employ clustering-based approach with encoder networks to separate semantic spaces

5. Intervention: Apply learned intervention vectors to top-K sensitive attention heads during inference:

   \[T_{l+1} = T_l + \sum_{h=1}^{H} (Attn^h_l(P_l^h T_l) + \alpha \times \Delta) \cdot W^o_l\]

   The core technical contribution is the joint optimization of visual attention and ToM-specific reasoning through targeted attention head interventions.

1. Probing stage: Train lightweight logistic regression classifiers on 30% calibration split using cross-entropy loss to identify task-sensitive attention heads, freeze MLLM backbone

2. Encoder training: Train separate encoder networks (two linear layers with GELU, 128→256→128 dimensions) for each attention head using Adam optimizer, learning rate 1×10^-3, optimize clustering-based separation loss

3. Intervention vector computation: Extract activation offsets from positive/negative sample pairs, compute intervention directions as combination of visual and reasoning components

4. Hardware: Not reported for training compute, calibration takes 0.2 hours for probing + 1 hour for encoder training

5. Data: Use EgoToM dataset 30%/70% train/test split, no additional data augmentation or filtering reported

6. Inference: Apply interventions at zero temperature, zero-shot setting with intervention strength α=1.0 and top-K=64 heads

Prior work:

1. GridToM (2025): Introduced intervention methods for ToM in MLLMs using binary logistic regression classifiers to derive intervention directions, limited to simulated grid-world environments
2. Neural Theory-of-Mind (TOMI) dataset (2022): Evaluated LLM ToM abilities on text-based tasks, showed instability in mental state reasoning

3. MMToM-QA (2024): Created multimodal ToM benchmark but relied heavily on textual input alongside visual information

   Delta: This paper extends intervention methods to real-world egocentric video scenarios, introduces clustering-based encoder networks for fine-grained reasoning failure correction, and operates purely on visual input without textual annotations.

   Applied-specific assessment:

   • Architectural novelty: The clustering-based encoder approach for handling heterogeneous negative samples is a reasonable extension, but the core attention intervention technique builds directly on GridToM
   • Benchmark gains: Improvements are meaningful (13.0% on Goal task for LLaVA-Next-Video) but the baseline performance is quite poor (24.0% on Actions), suggesting potential for larger gains
   • Fair comparisons: Uses same evaluation protocol as EgoToM benchmark, though limited to two open-source models
   • Generalizability concerns: Method requires calibration data and shows mixed transfer results (60.5% vs 66.3% when transferring to MMToM-QA)

   Verdict: INCREMENTAL — Solid application of known intervention techniques to a new visual ToM setting, but the core methodology builds incrementally on GridToM with reasonable engineering improvements rather than fundamental algorithmic advances.

1. EgoToM Goal Inference: LLaVA-Next-Video improves from 61.5% to 74.5% (+13.0%), Qwen2.5-VL from 86.9% to 88.9% (+2.0%) vs human baseline 88%

2. EgoToM Belief Reasoning: LLaVA-Next-Video improves from 38.9% to 45.3% (+6.4%), Qwen2.5-VL from 35.6% to 42.0% (+6.4%) vs human baseline 72%

3. EgoToM Action Inference: LLaVA-Next-Video improves from 24.0% to 29.7% (+5.7%), Qwen2.5-VL from 31.1% to 37.6% (+6.5%) vs human baseline 78%

4. MMToM-QA transfer: Qwen2.5-VL achieves 66.3% vs 60.5% when transferring learned vectors from EgoToM (+5.8%)

5. Open-ended generation (TruthfulQA metrics): VisionToM improves truthfulness scores, e.g., LLaVA-Next-Video Goal task from 8.5% to 27.3% for True metric

   Results show consistent but modest improvements, with particularly large gaps remaining on Belief and Action tasks. The method shows mixed generalization across models and datasets.

1. Model size: 7B parameters for main experiments (LLaVA-Next-Video, Qwen2.5-VL), also tested 72B parameter Qwen2.5-VL

2. Training compute: Calibration stage takes 0.2 hours for probing + 1 hour for encoder training, specific GPU hours and hardware not reported

3. Inference speed/latency: Not reported, but method adds minimal overhead as intervention vectors are precomputed and applied during forward pass

4. Memory footprint: Minimal additional memory required as method only stores learned intervention vectors, backbone remains frozen

5. Deployment practicality: High - lightweight approach requiring only one-time calibration, compatible with existing MLLM inference pipelines without fine-tuning

1. Dataset evaluation: Uses EgoToM benchmark with real-world egocentric videos from natural social interactions, providing ecological validity compared to simulated environments

2. Video-only setting: Method operates solely on visual input without textual annotations, more realistic for embodied AI applications

3. Cross-dataset transfer: Shows limited but positive transfer from EgoToM to MMToM-QA (60.5% vs baseline 38.2%), indicating some generalization

4. Production considerations: Method requires calibration data and manual hyperparameter tuning (K=64 heads, α=1.0), potentially limiting deployment flexibility

5. Hardware deployment: Tested on standard MLLM architectures, should be compatible with existing video understanding systems

1. FUNDAMENTAL: Method requires calibration data from target task domain, limiting zero-shot applicability across different ToM scenarios

2. ENGINEERING: Performance gains are modest, especially on challenging Belief (6.4%) and Action (5.7%) tasks, suggesting need for more sophisticated intervention strategies

3. EVALUATION: Limited to two open-source models, missing comparison with latest closed-source models like GPT-4o beyond baseline reporting

4. FUNDAMENTAL: Cross-dataset transfer shows degraded performance (66.3% vs 60.5%), indicating learned representations may be dataset-specific

5. ENGINEERING: Hyperparameter sensitivity not thoroughly analyzed - only reports single configuration (K=64, α=1.0)

   Failure modes: 1) Method fails when visual attention patterns differ significantly between calibration and test scenarios, 2) Clustering-based approach may struggle with novel types of reasoning failures not seen during calibration