Applied AI Digest — Mar 20, 2026

This work addresses Bird’s-Eye-View (BEV) perception for autonomous driving, a critical capability that fuses multi-camera surround-view images into a unified top-down representation for downstream tasks like 3D object detection, semantic segmentation, and motion prediction. The challenge lies in accurately transforming perspective camera views into geometrically consistent BEV features without explicit 3D understanding, leading to suboptimal performance in safety-critical applications.

The task takes as input multi-view perspective images from vehicle-mounted cameras (typically 6 views) at resolutions of 224×480 or 448×800 pixels. The output is a BEV feature map of resolution 200×200 covering a 100m×100m spatial area around the vehicle. The objective combines reconstruction loss:

with downstream segmentation loss using focal loss, centerness L2 loss, and offset L1 loss.

Success is measured using Intersection-over-Union (IoU) for semantic segmentation tasks including vehicle, pedestrian, and lane segmentation on the BEV plane.

Experiments are conducted on nuScenes dataset (1000 driving scenes, 700 train/150 val/150 test) and Argoverse1 dataset (113 driving logs) with diverse weather and lighting conditions across multiple cities.

1. Gaussian Generator: A feed-forward network with multi-view branch (based on UniMatch) and per-view branch (ViT-S backbone) that predicts 3D Gaussian parameters (position μ, covariance Σ, opacity σ, spherical harmonics SH, and feature vector f) from multi-view inputs.

2. 3D Gaussian Splatting Representation: Scene represented as anisotropic Gaussians {Gi}ⁿᵢ₌₁ where each Gaussian Gi = (μᵢ, Σᵢ, σᵢ, SHᵢ, fᵢ) enables differentiable rendering via α-blending:

   \[\mathbf{C}(p) = \sum_{i=1}^{N} T_i \, \sigma_i \, \mathbf{c}_i, \text{ where } T_i = \prod_{j < i} (1 - \sigma_j)\]

3. Foundation Model Distillation: DINO encoder extracts dense semantic features, supervised with cosine similarity loss:

   \[\mathcal{L}_{\text{feat}} = \frac{1}{k} \sum_{i=1}^{k} \Big( 1 - \cos \big( \widehat{\mathbf{F}}_i, \, \mathbf{F}^{\text{DINO}}_i \big) \Big)\]

4. Depth Supervision: Metric3Dv2 provides reference depth with L1 and SILog losses for geometric consistency.

5. BEV Projection: Orthogonal projection of reconstructed 3D Gaussians to BEV plane using differentiable rasterization.

6. Task Head: BEV encoder + segmentation head operating on projected BEV features.

   The core contribution is incorporating explicit 3D reconstruction via Gaussian Splatting into BEV perception, contrasting with existing end-to-end implicit approaches.

1. Stage 1 - Gaussian Generator Pre-training: - Data: nuScenes/Argoverse1 multi-view images with Metric3Dv2 depth supervision and DINO features - Multi-view branch: learning rate 2×10⁻⁴, monocular branch: 2×10⁻⁶ (initialized from Depth Anything V2) - 10 epochs - Hardware: not reported

2. Stage 2 - Task Head Training: - Data: Same datasets, Gaussian generator frozen - BEV encoder and segmentation head: learning rate 2×10⁻⁴ - 10 epochs
   - Hardware: not reported

3. Stage 3 - Joint Fine-tuning: - Data: End-to-end optimization of all components - Monocular branch: 2×10⁻⁶, all other components: 2×10⁻⁶ - 20 epochs - Hardware: not reported

   Loss coefficients: λ₁ = 0.2, λ₂ = 0.8, λ₃ = 0.1 for reconstruction; λ₄ = 2.0, λ₅ = 0.1 for BEV tasks.

   Training compute, wall-clock time, and specific hardware details not reported.

This work builds on 3D Gaussian Splatting (Kerbl et al. 2023) and BEV perception methods like LSS (Philion & Fidler 2020), BEVFormer (Li et al. 2022). Closest related works are GaussianLSS (Lu et al. 2025) and GaussianBEV (Chabot et al. 2025), which also use Gaussian Splatting for BEV but treat Gaussians merely as feature carriers without explicit 3D reconstruction.

The key delta is incorporating explicit 3D scene reconstruction as an intermediate step in BEV perception, using foundation model distillation (DINO, Metric3D) to create geometry-aligned features, and a three-stage training paradigm that separates reconstruction learning from task-specific optimization.

Rating: SIGNIFICANT - Introduces a new paradigm for BEV perception by bridging explicit 3D reconstruction with downstream tasks, showing consistent improvements across multiple benchmarks.

1. nuScenes Vehicle Segmentation (224×480): IoU 39.6% vs previous best PointBEV 38.7%, +2.3% improvement

2. nuScenes Vehicle Segmentation (448×800): IoU 42.7% vs PointBEV 42.1%, +1.4% improvement

3. nuScenes Vehicle Segmentation (224×480, visibility filtered): IoU 44.6% vs PointBEV 44.0%, +1.4% improvement

4. nuScenes Vehicle Segmentation (448×800, visibility filtered): IoU 48.2% vs PointBEV 47.6%, +1.3% improvement

5. nuScenes Pedestrian Segmentation: IoU 20.1% vs PointBEV 18.5%, +8.6% improvement

6. nuScenes Lane Segmentation: IoU 60.2% vs PointBEV 49.6%, +21.4% improvement

7. nuScenes Multi-class Segmentation: Mean IoU 34.4% vs DiffBEV 29.6%, +16.2% improvement

8. Argoverse1 Multi-class Segmentation: Mean IoU 24.4% vs TaDe 22.3%, +9.4% improvement

   Results show consistent improvements across all tasks, with particularly strong gains on lane segmentation (+21.4%) and pedestrian segmentation (+8.6%).

1. Model size: Not explicitly reported, but uses ViT-S backbone and multi-view transformer components

2. Training compute: Not reported - missing GPU hours, hardware specifications

3. Inference speed: Not reported - no latency measurements provided

4. Memory footprint: Not reported - no memory usage analysis

5. Deployment practicality: Limited assessment - three-stage training adds complexity, requires foundation model dependencies (DINO, Metric3D), but achieves better performance. Real-time constraints not addressed.

1. Real-world datasets: Evaluated on nuScenes and Argoverse1 datasets collected from actual autonomous vehicles in diverse urban environments (Boston, Singapore, Pittsburgh, Miami)

2. Multi-weather conditions: Testing includes various weather and lighting conditions present in the datasets

3. Production readiness: No discussion of deployment to actual autonomous vehicles or real-time constraints

4. Hardware requirements: No analysis of computational requirements for deployment on automotive hardware

5. Sim-to-real: No discussion of simulation to real-world transfer, though datasets are real-world collected

1. ENGINEERING: Three-stage training paradigm adds complexity compared to end-to-end approaches, potentially harder to optimize and deploy

2. ENGINEERING: Dependency on foundation models (DINO, Metric3D) increases computational overhead and model complexity

3. EVALUATION: No real-time performance analysis or deployment constraints considered for autonomous driving applications

4. EVALUATION: Limited analysis of failure cases or robustness to domain shifts between training and test environments

5. FUNDAMENTAL: Orthogonal BEV projection may lose important 3D geometric information compared to perspective-aware approaches

   Likely failure modes:

6. Performance degradation in scenarios with limited multi-view overlap where 3D reconstruction quality suffers
7. Sensitivity to camera calibration errors that could affect the geometric consistency of projected BEV features.

This work addresses the challenge of adapting Large Language Models (LLMs) to complex technical service domains, where human expert demonstrations often lack explicit reasoning chains and valid responses exhibit inherent diversity. Traditional training paradigms struggle with “myopic imitation” that degrades performance in complex tasks requiring multi-step reasoning and planning.

The task involves fine-tuning LLMs for technical customer service scenarios where agents must provide accurate responses to customer queries in domains like cloud services. The input consists of customer service tickets with historical context, and the output is agent responses that should be both accurate and follow business logic. The training objective combines supervised fine-tuning with reinforcement learning using multiple valid ground truths rather than single references.

Success is measured using the Ensemble-Consistency Score (ECS) defined as:

where $J_{\text{con}}$ is the mean score across an ensemble of consistency judges, and $Y^*(x)$ represents the set of valid responses for query $x$.

The paper introduces a Multi-Ground Truth (Multi-GT) dataset expanded from a proprietary technical service dataset of 10k queries each for decision and planning SFT, plus 5,120/1k/1k queries for RL training/validation/test. The Multi-GT expansion roughly doubles the number of references (e.g., test: 1,000 → 1,975).

The framework comprises three key components operating on Qwen3-4B as the base model:

1. Latent Logic Augmentation: Two methods to inject explicit reasoning into training data: - Decision Reasoning Augmentation (DRA) generates backward chain-of-thought rationales $c_t$ for each action $a_t$, with loss:

   \[L_{\text{Decision}} = -\mathbb{E}_{(q_t,c_t,a_t) \sim D_{\text{DRA}}}[\log p_\theta(c_t, a_t | q_t)]\]

   - Planning-Aware Trajectory Modeling (PATM) constructs forward-looking 3-step sequences $(a\_t, q\_{t+1}, a\_{t+1})$ with loss:
   

   \[L_{\text{PATM}} = -\mathbb{E}_{(y^{\text{PATM}}_t) \sim D_{\text{PATM}}}[\log p_\theta(\tilde{a}_t, \tilde{q}_{t+1}, \tilde{a}_{t+1} | q_t)]\]

2. Robust Noise Reduction: Constructs Multi-GT datasets using dual-filtering with Consistency Judge (92% human alignment) and Utility Judge (83% human alignment) to validate diverse valid responses.

3. Lightweight Adaptation: Hybrid Reward Mechanism (HRM) combines lightweight Qwen3-4B reranker with Qwen3-32B judge using cascade strategy:

   \[R_\theta(S_R, S_J) = \begin{cases}\] \[w_1 S_R + (1-w_1) S_J & \text{if } S_R < \tau_a \\\] \[S_R & \text{if } \tau_a \leq S_R \leq \tau_b \\\] \[w_2 S_R + (1-w_2) S_J & \text{if } S_R > \tau_b\] \[\end{cases}\]

1. Data Construction: Multi-GT expansion via offline exploration (Qwen3-4B, T=1.2) and online adaptation (RL rollouts), filtered through dual-judge system.

2. SFT Stage: - Data: 10k queries each for decision and planning, expanded to Multi-GT format - Optimizer: AdamW with ZeRO-3, learning rate 2×10⁻⁶, cosine schedule - Training: 1 epoch, batch size 256, max sequence length 20,000 - Hardware: Not specified

3. RL Stage using DAPO: - Data: Multi-GT dataset (5,120 training queries) - Optimizer: Actor learning rate 5×10⁻⁶, constant schedule - Training: 20 episodes, batch size 128, 16 samples per prompt - Reward: Hybrid mechanism with fast interval [0.68, 0.98], mixing weights w₁=0.05, w₂=0.72 - Hardware: Not reported - Wall-clock time: 30% reduction in reward computation time compared to Judge-only baseline

This work builds on established techniques but makes significant novel contributions in their combination and application to technical service domains. The closest prior works include RLHF/PPO methods for alignment, LLM-as-a-Judge paradigms, and planning-aware language modeling approaches.

Key novelties:

• Latent Logic Augmentation: Novel combination of backward reasoning (DRA) and forward planning (PATM) to address “myopic imitation” in technical domains
• Multi-GT Construction: Automated dual-filtering approach using specialized Consistency and Utility judges to systematically capture semantic diversity
• Hybrid Reward Mechanism: Innovative cascade strategy balancing computational efficiency with reward fidelity, reducing compute by 30% while maintaining performance

The framework addresses a fundamental gap in applying LLMs to domains where single-reference training is inadequate and explicit reasoning is absent from demonstrations.

Rating: SIGNIFICANT - The integrated approach of data augmentation, multi-reference training, and efficient reward mechanisms represents a substantial advance for practical LLM deployment in technical domains.

1. Multi-ECS (primary metric): This paper: 0.441 (full framework), baseline SFT: 0.337, improvement: +30.9%

2. Single-ECS: This paper: 0.347 (full framework), baseline SFT: 0.242, improvement: +43.4%

3. Call Tool Accuracy: This paper: 0.279 (SFT-Mix w/ DRA), baseline: 0.082, improvement: +240%

   Component ablation results:

   • SFT without reasoning augmentation shows degradation (Multi-ECS: 0.293 vs 0.299 original)
   • PATM + DRA combination achieves best SFT performance (Multi-ECS: 0.337)
   • Hybrid Reward vs alternatives: 0.429 vs 0.389 (Reranker-only) vs 0.413 (Soft Judge)
   • Multi-GT vs Single-GT: 0.441 vs 0.429 Multi-ECS

   Mixed results: Single-ECS slightly drops when upgrading to Multi-GT (0.357 → 0.347), highlighting the limitation of single-reference evaluation when models learn diverse valid responses.

   Notable absence: No comparison against other multi-reference RL methods or standard technical service benchmarks beyond their proprietary dataset.

1. Model size: Qwen3-4B base model (4 billion parameters)

2. Training compute: Not specified for total GPU hours or hardware details

3. Inference speed: Hybrid Reward Mechanism achieves 30% reduction in reward computation time compared to LLM-as-a-Judge baseline through cascade strategy

4. Memory footprint: Not reported, though uses DeepSpeed ZeRO-3 optimization for SFT stage

5. Deployment practicality: High - framework designed specifically for computational efficiency while maintaining performance. The lightweight reranker (Qwen3-4B) handles majority of reward computations, with expensive judge (Qwen3-32B) only used for ambiguous cases. The cascade strategy with optimized thresholds [0.68, 0.98] makes the approach practical for production deployment.

1. Real-world deployment: Evaluated on proprietary cloud service customer support dataset with 10k+ real queries, demonstrating practical applicability beyond synthetic benchmarks.

2. Production integration: Framework designed with computational efficiency in mind (30% reward computation reduction), suggesting readiness for production deployment in technical service domains.

3. Domain specificity: Successfully handles complex technical service scenarios including policy explanations, operation guidance, and information collection - core requirements for real customer service applications.

4. Scalability validation: Multi-GT construction pipeline automates the expansion of training data, making it feasible to apply to large-scale real-world datasets without manual annotation.

5. Business logic compliance: Consistency Judge achieves 92% alignment with human expert judgments, and Utility Judge achieves 83% alignment, indicating reliable quality control for production use.

1. FUNDAMENTAL: Dependency on powerful teacher models (Qwen3-32B, DeepSeek models) for data augmentation and judgment, limiting accessibility and introducing potential bias propagation.

2. FUNDAMENTAL: Fixed cascade thresholds in Hybrid Reward Mechanism may become suboptimal as policy distribution shifts during RL training, requiring manual recalibration.

3. EVALUATION: Evaluation limited to proprietary dataset, preventing direct reproducibility and comparison with other technical service domains or public benchmarks.

4. ENGINEERING: Multi-GT construction quality depends on the capabilities of judge models, which may degrade with domain shift or when applied to different technical service areas.

5. ENGINEERING: The framework requires careful hyperparameter tuning (cascade thresholds, mixing weights) that may not transfer across different base models or domains.

   Likely failure modes:

   • Judge model disagreement: When Consistency and Utility judges provide conflicting assessments, the system may include invalid responses in Multi-GT training data
   • Reward hacking: Despite the hybrid mechanism, the policy may still exploit imperfections in the lightweight reranker, leading to responses that score well but lack true utility

Robotic planners using large language models (LLMs) face critical challenges when encountering ambiguous situations, whether from underspecified user instructions (“pick up the cup” when multiple cups exist) or environment-induced uncertainty (target objects being absent or multiple). This poses safety risks as robots may execute suboptimal or dangerous actions without seeking clarification.

The task involves generating appropriate action sequences or clarification questions from natural language instructions I and visual observations O. The system must handle three ambiguity types:

1. multiplicity (

   X_match

   > 1 where X_match are objects satisfying instruction properties)

2. absence (X_match = ∅), and

3. underspecification (vague instruction terms). In multi-agent settings, partial observability creates additional ambiguities where robot R_i sees subset O_Ri ⊆ O.

   Success is measured by Success Rate (SR) - correct action execution for unambiguous cases or appropriate clarification for ambiguous ones - and Correct Question Rate (CQR) - whether the generated clarification question matches the ground truth ambiguity type. Experiments use 400 trials in simulated tabletop environments (100 per ambiguity category plus 100 clear scenarios) and multi-agent tasks from LEMMA benchmark.

1. Scene graph construction: Uses vision-language model (VLM) like Grounding DINO or Qwen3-VL-2B-Instruct to detect objects and generate structured graph G = (V, E) where nodes V represent objects with attributes and edges E capture spatial/semantic relationships

2. Iterative reasoning framework: LLM equipped with two retrieval functions - retrieve_node(attr_key, attr_val) to query objects by attributes and retrieve_edge(source, target, relation) to query relationships between objects

3. Chain-of-thought with grounding: At each turn t, LLM generates reasoning trace T_t and function call f_t, retrieves results R_t from scene graph, updates conversation history H_{t+1} = H_t ⊕ (T_t ⊕ f_t ⊕ R_t)

4. Ambiguity detection and clarification: Process continues until LLM has sufficient context for definitive action or identifies ambiguity source to generate targeted clarification question with appropriate tag (multiplicity, absence, underspecified)

5. Multi-agent communication: Extended with ask_robot action enabling agents to query each other’s local observations in partially observable environments

No explicit model training reported - this is a prompting and reasoning framework that uses pre-trained models:

1. Vision-language models: Uses existing Qwen3-VL-2B-Instruct and Gemini-2.5-Flash models without additional training

2. Scene graph generation: Prompting-based approach using VLM to generate JSON-structured graphs in single pass, avoiding O(n²) complexity of pairwise object analysis

3. LLM reasoning: Uses structured prompts with conversation history maintenance, no fine-tuning or reinforcement learning reported

   Training details, optimization parameters, and hardware specifications not reported as this work focuses on inference-time reasoning framework rather than model training.

This work builds on SayPlan (scene graph + LLM planning), CLARA (ambiguity classification), and KnowNo (uncertainty-based clarification). The key novelty is integrating iterative scene graph querying with chain-of-thought reasoning specifically for ambiguity detection and resolution.

Prior works like CLARA focus mainly on instruction-level ambiguity, while SayPlan assumes pre-constructed graphs without clarification capabilities. This paper extends to environment-induced ambiguities and multi-agent partial observability scenarios not addressed by previous approaches.

The core delta is the iterative grounding mechanism that prevents hallucination accumulation and enables pinpointing ambiguity sources for targeted clarification. Rating: INCREMENTAL - combines existing techniques (scene graphs, CoT, VLMs) in a novel way but doesn’t introduce fundamentally new architectures or training methods.

1. Single-agent tabletop (SayCan-derived): SG-CoT achieves 72%/66% (SR/CQR) with Qwen vs best baseline Inner Monologue 65%/55%, representing 7%/11% improvement in overall performance

2. Multiplicity scenarios: SG-CoT 71%/52% (Qwen) vs Inner Monologue 65%/53%, showing 6% SR improvement but slight CQR decrease

3. Absence scenarios: SG-CoT 90%/80% (Qwen) vs Inner Monologue 67%/52%, demonstrating 23%/28% improvement margins

4. Underspecified scenarios: SG-CoT 77%/67% (Qwen) vs Inner Monologue 75%/62%, showing 2%/5% improvement

5. Multi-agent LEMMA benchmark: SG-CoT achieves 59% SR (Qwen) vs ProgPrompt 44%, representing 15% improvement in partially observable environments

6. Gemini-2.5-Flash consistently outperforms Qwen3-VL-2B across all metrics, with SG-CoT reaching 80%/78% overall SR/CQR vs 72%/66% for Qwen

1. Model size: Uses existing Qwen3-VL-2B-Instruct (2B parameters) and Gemini-2.5-Flash (size not specified) - no custom model training

2. Training compute: Not applicable - framework uses pre-trained models with prompting approach

3. Inference speed: SG-CoT requires 4.24 LLM calls per episode vs 1.00 for ProgPrompt, with average 1141.72 input tokens and 191.28 output tokens per call, leading to higher latency

4. Memory footprint: Higher token usage due to conversation history concatenation and scene graph information storage, but scales linearly with environment complexity

5. Deployment practicality: Computationally more expensive than baselines due to iterative reasoning, but complexity doesn’t scale proportionally with environment size as retrieval depth depends on instruction complexity rather than total objects

1. Evaluation limited to simulation: All experiments conducted in PyBullet-based tabletop environments and LEMMA benchmark simulations - no real robot deployments reported

2. Hardware experiments: None reported - uses simulated UR5e robot with 2-finger gripper in controlled tabletop scenarios

3. Sim-to-real gap: Authors acknowledge limitation that VLMs may hallucinate with real-world objects and introduce incorrect scene graph edges, potentially hampering reasoning

4. Production integration: No deployment results or production use cases discussed

5. Real-world challenges: Paper identifies that current VLM-based scene graph generation may not handle complex real-world spatial relationships reliably, limiting immediate practical deployment

1. VLM hallucination in scene graph generation - FUNDAMENTAL: relies on single-pass VLM inference for spatial/semantic relationships which can introduce incorrect edges

2. Computational overhead - ENGINEERING: requires 4x more LLM calls than baselines with higher token usage, limiting scalability for complex instructions

3. Simulation-only evaluation - EVALUATION: no real-world robot experiments to validate sim-to-real transfer

4. Multiplicity detection challenges - FUNDAMENTAL: model may arbitrarily pick one valid option instead of recognizing ambiguity when multiple correct choices exist

5. Token usage scaling - ENGINEERING: latency increases with number of ambiguities present in scene, potentially limiting complex scenarios

   Failure modes:

6. Incorrect scene graph edges leading to wrong reasoning chains
7. Over-confidence in multiplicity scenarios causing arbitrary action selection instead of clarification requests

This work addresses the challenge of applying Large Language Models (LLMs) to physical systems governed by Partial Differential Equations (PDEs), where traditional approaches either produce non-physical hallucinations or require expensive domain-specific fine-tuning that limits cross-domain generalization.

The task involves multimodal scientific reasoning for fluid dynamics forecasting. Inputs include heterogeneous data streams (satellite imagery, buoy readings) containing high-dimensional flow tensors. The system must produce:

1. accurate spatiotemporal forecasts of physical states, and

2. interpretable analysis reports with physical grounding and decision logic. The formal objective combines prediction accuracy with physical consistency:

   \[\mathcal{L} = \mathcal{L}_{pred}(x_{t+\tau}, \hat{x}_{t+\tau}) + \lambda \mathcal{L}_{physics}(\hat{x}_{t+\tau})\]

   where $\mathcal{L}_{physics}$ enforces conservation laws like mass conservation $\nabla \cdot v = 0$.

   Success is measured through dual evaluation:

3. Physical accuracy via RMSE, SSIM, PSNR on predicted tensor fields, and

4. Interpretive quality via Mechanism F1 score measuring physical grounding accuracy of generated reports.

   The paper evaluates on three benchmarks: 2D Turbulence (128×128, 100 timesteps, microscopic flow), SEVIR (384×384 regional weather), and ERA5 (180×360 global climate, 200-day forecasts with 21 variables).

1. Physics Perception Loop: Neural Earth Simulator (NES) based on improved Diffusion Transformer (DiT) generates ensemble forecasts via latent space perturbation with Gaussian noise injection: $z^{(k)}_{init} = E(x_{init}) + \lambda \cdot \xi^{(k)}$ where $\xi^{(k)} \sim \mathcal{N}(0, I)$

2. Visual Symbolic Projector: Cross-attention mechanism extracts topological features from visual encodings using learnable query embeddings $Q \in \mathbb{R}^{N \times d}$: $H_{vis} = \text{Softmax}(Q(vW_K)^T/\sqrt{d})(vW_V)$

3. Semantic-Symbolic Alignment: Maximizes mutual information between visual tokens and textual descriptions via contrastive loss: $\mathcal{L}_{align} = -\sum_{i=1}^N \log \frac{\exp(\text{sim}(h_i, t_{pos})/\tau)}{\sum_j \exp(\text{sim}(h_i, t_j)/\tau)}$

4. Agentic Reasoning Core: Gemini 3 Flash executes Physics-Guided Chain-of-Thought (PG-CoT) with ReAct strategy, selecting actions $a_t \sim \pi(a_t\lvert M_t, H_{vis}, I)$ from Retrieve/Simulate/Reason

5. Physics Consistency Constraint: Critic function validates trajectories against conservation laws, forcing backtracking when violations detected

6. Counterfactual Feedback Loop: Active probing mechanism triggers alternative scenario simulation when uncertainty exceeds threshold: $P_{counter} = \text{NES}(x’_{init}\lvert \text{do}(\text{condition}))$

7. Hierarchical Knowledge Retrieval: RAG system accesses stratified vector database with physical laws ($K_{phy}$), protocols ($K_{prot}$), and historical reports ($K_{hist}$)

1. Neural Earth Simulator (NES) Training: Pre-trained Diffusion Transformer on fluid dynamics data, specific training details not reported for optimizer, learning rate, or hardware requirements

2. Visual Symbolic Projector Training: Trained via contrastive learning to align visual features with text embeddings using temperature parameter τ = 0.07, other training specifics not reported

3. Core LLM (Gemini 3 Flash): Frozen foundation model, no domain-specific parameter updates or fine-tuning performed

4. Knowledge Base Construction: Hierarchical vector database populated with domain literature, operational protocols, and historical reports, construction methodology not detailed

5. System Integration: Training-free neuro-symbolic framework that orchestrates pre-trained components without end-to-end parameter optimization

   Training data scale, wall-clock time, and hardware requirements not reported for most components.

The core novelty lies in the training-free neuro-symbolic architecture that grounds frozen multimodal LLMs in physical laws without parameter updates. This contrasts with prior approaches that either:

1. use specialized deep learning surrogates (FNO 2020, GraphCast 2023) lacking interpretability, or

2. fine-tune LLMs on scientific data (costly and prone to catastrophic forgetting).

   Key innovations include: Semantic-Symbolic Alignment mechanism projecting flow tensors to linguistic descriptors, Physics-Guided Chain-of-Thought with reflexive consistency checking, and Counterfactual Feedback Loop enabling active causal probing.

   The approach builds on recent work in multimodal LLMs (LLaVA 2024), physics-informed neural networks (PINNs 2019), and agentic reasoning with tool use (ReAct 2022), but uniquely combines these into a training-free physics-grounded agent.

   Rating: SIGNIFICANT - Novel architectural paradigm with clear technical contributions, though builds incrementally on established components.

1. 2D Turbulence: RMSE 0.582±0.008 (vs EarthFarseer 0.654), SSIM 0.715±0.006 (vs EarthFarseer 0.642), PSNR 28.66±0.10 (vs EarthFarseer 27.45) - significant improvements

2. ERA5 Global Weather: RMSE 0.552±0.005 (vs EarthFarseer 0.615), SSIM 0.931±0.002 (vs EarthFarseer 0.895), PSNR 32.11±0.06 (vs EarthFarseer 30.22) - consistent improvements

3. SEVIR Regional Weather: RMSE 0.405±0.004 (vs EarthFarseer 0.437), SSIM 0.882±0.003 (vs EarthFarseer 0.842), PSNR 31.50±0.08 (vs EarthFarseer 30.15) - modest improvements

4. Zero-shot comparison vs foundation models: Dramatically outperforms ChatGPT-Images, Seedream 4.5, Banana Pro on all benchmarks with 30-40% RMSE reduction and 90-128% SSIM improvement

5. Reasoning Quality: Mechanism F1 score 83.2% vs Qwen3-VL series, demonstrating superior physical grounding in generated reports

   Results consistently favor OMNIFLOW across all scales from microscopic to global dynamics.

1. Model size: Not explicitly reported, but uses frozen Gemini 3 Flash plus lightweight DiT-based simulator

2. Training compute: Minimal due to training-free architecture, specific GPU hours not reported

3. Inference speed: Higher latency than end-to-end models due to iterative reflexive loops and counterfactual probing, specific latency numbers not provided

4. Memory footprint: Not quantified, but includes frozen LLM, visual encoder, and knowledge retrieval database

5. Deployment practicality: Limited by inference latency from multi-step reasoning workflow, though offers interpretability benefits for scientific decision-making applications

1. Marine Heatwave Case Study: Demonstrates end-to-end application on January 2021 global ocean data with actionable outputs including fishery alerts and shipping route optimization

2. Multi-scale Validation: Tested across real-world datasets from microscopic turbulence to global weather patterns, not just synthetic benchmarks

3. Operational Integration: Generates structured reports compatible with emergency protocols and decision support systems

4. Physical Consistency: Enforces conservation laws and retrieves operational standards, making outputs suitable for scientific and regulatory applications

5. However, no actual deployment results in operational weather forecasting systems or real-time decision-making environments reported

1. ENGINEERING: Increased inference latency due to iterative reflexive loops and counterfactual probing limits real-time deployment applications

2. FUNDAMENTAL: Reasoning accuracy remains coupled to underlying neural simulator fidelity - biases in DiT propagate through reasoning chain

3. ENGINEERING: Representing fine-grained sub-grid dynamics through linguistic descriptors remains challenging, may lose critical physical details

4. EVALUATION: Limited evaluation of failure modes when physics constraints conflict or when operating far outside training distribution

5. ENGINEERING: Scalability concerns for larger ensemble sizes or higher-resolution simulations not addressed

   Failure modes:

6. System may produce overconfident predictions when simulator uncertainty is underestimated
7. Knowledge retrieval may surface irrelevant or conflicting physical principles leading to reasoning inconsistencies.

Training generalist robots requires large-scale, diverse manipulation datasets, but real-world data collection is expensive and existing simulators are limited by fixed asset libraries. The core challenge is automatically generating both physically realistic 3D environments and executable robot trajectories from natural language instructions.

The task takes natural language instructions (e.g., “Put the bowl onto the plate in a simple living room”) as input and produces:

1. a physics-validated 3D simulation scene with collision-free object layouts, and

2. executable robot end-effector trajectories τ = {(pt,qt)}T_{t=0} where pt denotes continuous poses and qt represents discrete gripper states. The method must handle open-vocabulary objects and environments without manual asset curation.

   Success is measured by:

3. simulation policy success rate on unseen objects
4. zero-shot sim-to-real transfer success rate on physical hardware, and
5. trajectory generation throughput (600 trajectories/hour reported). Evaluation uses tabletop pick-and-place tasks with 40 target objects, 40 receptacles, and 20 environments for training, tested on 10 held-out novel mugs.

1. Semantic-to-Physics Scene Synthesis: Uses Qwen-Max LLM for semantic parsing into JSON asset manifests, then Flux diffusion model generates 2D assets with SAM3 segmentation for background removal.

2. Memory-Efficient 3D Reconstruction: Applies SAM3D with dynamic GPU/CPU offloading to lift 2D images to 3D meshes, optimizing VRAM usage for consumer hardware.

3. Physics-Grounded Layout: Combines LLM spatial reasoning with Genesis physics engine for collision detection and gravity alignment, ensuring stable object placement via AABB checking.

4. Video-Prior Trajectory Generation: Captures stabilized scene as initial frame I0, applies style refinement via Qwen-Image-Edit, then feeds I’0 to Wan2.2-I2V-Flash video generation with negative prompts to suppress non-physical artifacts.

5. Sim-to-Gen Alignment: Uses VGGT for depth estimation, CoTracker3 for dense 2D point tracking within SAM3-generated object masks, TAPIP3D for 3D motion lifting, and Graspgen for grasp pose generation with inverse kinematics mapping.

1. Scene Generation: No training required - uses pre-trained foundation models (Qwen-Max, Flux, SAM3, SAM3D) with procedural assembly and physics validation.

2. Trajectory Synthesis: Uses pre-trained Wan2.2-I2V-Flash video model with targeted negative prompting, CoTracker3 for tracking, VGGT for depth, no additional training.

3. Policy Training: ACT (Action Chunking with Transformers) trained on synthesized demonstrations using standard imitation learning, specific training details not reported.

4. Hardware: 8×RTX 4090 workstation for generation pipeline, achieves 600 LeRobot-formatted trajectories per hour throughput.

5. Real-world deployment uses zero-shot transfer without target-domain fine-tuning, relying solely on synthetic data.

This is the first fully automated end-to-end pipeline combining open-vocabulary scene synthesis, video-prior trajectory generation, and real-world deployment. Prior work: GenSim (2024) and RoboGen (2024) use LLMs for task generation but rely on fixed assets; Holodeck (2024) does 3D scene composition but lacks behavior synthesis; TraceGen and video-based planners like UniPi (2022) work in 2D without 3D grounding. The key delta is the complete automation from language to executable trajectories using video generation as motion priors with robust visual-kinematic alignment. This represents a SIGNIFICANT contribution by unifying previously separate capabilities into a practical system.

1. Simulation Zero-shot Generalization: Success rate on 10 novel mugs with 2,500 synthetic demonstrations achieves 36.96% (baseline with 500 demos: 3.46%).

2. Real-world One-shot Transfer: 50% success with visual distractors, 20% with novel objects (apple, mango, pear, bottle), 15% with spatial perturbations, 0% with sensor occlusion.

3. Data Scaling: Performance scales from 3.46% (500 demos) to 25.90% (1,000 demos) to peak 36.96% (2,500 demos).

   No comparison to prior automated synthesis methods provided due to lack of comparable end-to-end systems. Results focus on demonstrating system functionality rather than beating specific benchmarks.

1. Model size: Uses multiple pre-trained foundation models (Qwen-Max, Flux, SAM3, Wan2.2-I2V-Flash) - individual parameter counts not reported.

2. Training compute: 8×RTX 4090 workstation, achieves 600 trajectory generation per hour throughput.

3. Inference speed: Real-time policy execution on 6-DoF Piper arm with RGB-D camera input.

4. Memory footprint: Dynamic GPU/CPU offloading implemented for 3D reconstruction to handle VRAM constraints on consumer hardware.

5. Deployment practicality: Demonstrated on consumer-grade hardware (RTX 4090s) with successful real-world transfer, indicating practical accessibility.

1. Hardware deployment: Successfully tested on 6-DoF Piper robotic arm (left arm of Cobot-Magic platform) with parallel-jaw gripper and dual Orbbec DaBai cameras.

2. Sim-to-real protocol: Zero-shot transfer using photo-conditioned scene alignment, black tape on robot links to match simulation appearance, no real-world fine-tuning required.

3. Real-world robustness: Tested under visual distractors, novel objects (fruits, bottles), spatial perturbations, and sensor occlusion conditions.

4. One-shot learning: Policy trained on single synthetic demonstration achieves meaningful real-world manipulation, demonstrating extreme data efficiency.

5. Practical integration: Uses standard robotics formats (LeRobot) and established frameworks (ACT) for easy adoption.

1. FUNDAMENTAL: Limited to rigid-body tabletop manipulation, cannot handle articulated or deformable objects.

2. ENGINEERING: Lacks automated physics-aware trajectory filtering, leading to potential low-quality generations requiring manual intervention.

3. FUNDAMENTAL: Video generation models may produce physically inconsistent motions despite negative prompting constraints.

4. EVALUATION: No comparison to other automated data synthesis methods due to lack of comparable end-to-end baselines.

5. ENGINEERING: Requires careful camera calibration and visual domain alignment (black tape) for sim-to-real transfer.

   Failure modes:

6. Complete failure under sensor occlusion (0% success) indicating heavy reliance on visual feedback
7. Performance degradation with spatial perturbations suggesting limited robustness to layout variations.

Building multimodal AI agents that can effectively plan and execute long-horizon tasks across GUI environments and tool-use scenarios remains challenging because most existing systems are reactive, making decisions based only on current observations without anticipating future states or long-term consequences. This reactive approach leads to planning incoherence and prevents reliable completion of multi-step tasks that require coordinated sequences of actions.

The task involves training agents to take multimodal inputs (screenshots, user instructions, interaction history) and generate both immediate actions and multi-step trajectory forecasts. Input consists of current observation $s_t$, user instruction $u$, and K-step interaction history $\tau_{1:t-1}$. Output includes predicted action $\hat{a}_t$, step instruction $\hat{g}_t$, and future trajectory $\hat{\tau}_{t:T}$ containing action sequences. The training objective combines trajectory-level alignment with discounted rewards:

where $r_t = \lambda_{align} \text{sim}(\hat{a}_t, a_t^*) - \lambda_{rep} \text{rep}(\hat{a}_{1:t})$.

Success is measured by task completion rates on GUI benchmarks (AndroidWorld, OSWorld-Verified) and answer accuracy on tool-use benchmarks (GAIA, GTA). Step-level precision is evaluated using coordinate matching for GUI actions and answer correctness for tool calls.

The paper evaluates across 7 benchmarks spanning online GUI environments (AndroidWorld: 116 tasks across 20 apps, OSWorld-Verified: long-horizon desktop operations), offline GUI benchmarks (AndroidControl-High, GUI-Odyssey: 203 tasks across 6 apps, Multimodal-Mind2Web), and multimodal reasoning tasks (GAIA: 446 tasks with document understanding, GTA: 229 visual reasoning tasks).

1. Base model: Qwen3-VL-8B-Thinking, a vision-language model extended for multimodal agent tasks
2. Two-stage reinforcement learning framework using Group-Relative Policy Optimization (GRPO)
3. Stage 1 - Anticipatory Trajectory Optimization: Model predicts multi-step future trajectories and is trained with trajectory-level rewards measuring global consistency between predicted and reference action sequences
4. Trajectory alignment reward with temporal discounting: $R(\hat{\tau}, \tau^*) = \sum_{t=1}^T \gamma^{t-1} r_t$ where $r_t = \lambda_{align} \text{sim}(\hat{a}_t, a_t^*) - \lambda_{rep} \text{rep}(\hat{a}_{1:t})$
5. GRPO policy update: $\nabla_\theta J(\theta) = E_{\hat{\tau}}[\hat{A}(\hat{\tau}, \tau^*) \nabla_\theta \log \pi_\theta(\hat{\tau}\lvert u, s_t, \tau_{1:t-1})]$
6. Stage 2 - Grounded Reinforcement Fine-tuning: Uses execution feedback from frozen tool agents to refine step-level accuracy
7. Grounded reward function: $r_t^G = \mathbf{1}[\text{coord match}]$ for GUI steps, $\mathbf{1}[\text{answer match}]$ for tool-calling steps

8. Inference operates in plan-act loop: predict multi-step trajectory, execute only first action, receive environment feedback, re-plan

   The core technical contribution is explicit training of anticipatory reasoning through trajectory-level RL combined with grounded execution refinement, moving beyond reactive decision-making to enable foresight and global planning coherence.

1. Initialization: Start from Qwen3-VL-8B-Thinking pretrained model
2. Stage 1 Trajectory-Level RL: Train on large-scale agent trajectory datasets (AgentNet, AndroidControl, GUI-Odyssey, Multimodal-Mind2Web, AgentTrek) using GRPO with trajectory alignment rewards, AdamW optimizer at 1e-6 learning rate, global batch size 128, 143 training steps, temporal discount γ=0.8
3. Stage 2 Grounded Fine-tuning: Use same trajectory datasets but execute first predicted action through frozen tool agents (UI-TARS-7B, UI-TARS-1.5-7B, Qwen3-VL-32B-Thinking), GRPO optimization with grounded rewards based on coordinate/answer matching, same hyperparameters as Stage 1
4. Tool-use training: Stage 1 uses tool-use trajectory dataset from T3-Agent toolbox, Stage 2 applies grounded RFT with executable tools from T3-Agent framework
5. Training framework: EasyR1 framework for reinforcement learning implementation
6. Hardware and wall-clock time: not reported

The work builds on recent GUI agent frameworks (Agent S/S2 2024, UI-TARS 2025, GTA1 2025) and R1-style reasoning approaches (GUI-R1 2025, InfiGUI-R1 2025). Prior work focuses on reactive planning with proprietary models or single-stage RL with step-level rewards.

The specific delta is the two-stage RL framework that explicitly trains anticipatory reasoning through trajectory-level rewards before grounding with execution feedback. Unlike existing methods that optimize step-level actions in isolation, this approach trains models to forecast multi-step sequences and optimize for global trajectory coherence.

The novelty is combining trajectory-level RL (Stage 1) with grounded execution refinement (Stage 2), bridging high-level anticipatory planning with low-level precision. This differs from purely reactive approaches or single-stage RL methods.

Rating: SIGNIFICANT - introduces a new training paradigm for multimodal agents with substantial empirical improvements, though builds incrementally on existing RL and GUI agent foundations.

1. AndroidWorld: 64.8% success rate vs best open-source 61.4% (Qwen3-VL-32B), improvement of +3.4 percentage points
2. OSWorld-Verified: 41.2% success rate vs best open-source 38.1% (Qwen3-VL-235B), improvement of +3.1 percentage points
3. AndroidControl-High: 75.3% step success rate vs best open-source 74.7% (UI-TARS-32B), improvement of +0.6 percentage points
4. GUI-Odyssey: 88.2% step success rate vs best open-source 88.6% (UI-TARS-32B), slight decrease of -0.4 percentage points
5. Multimodal-Mind2Web: 65.3% step success rate vs best open-source 64.7% (UI-TARS-32B), improvement of +0.6 percentage points
6. GAIA: 40.2% answer accuracy vs GPT-4o 33.4%, improvement of +6.8 percentage points over proprietary baseline

7. GTA: 56.7% answer accuracy with 65.7% tool accuracy vs best open-source T3-Agent 53.8% answer accuracy

   Results show consistent but modest improvements on most benchmarks, with stronger gains on reasoning-heavy tasks (GAIA) and online GUI environments compared to offline step-level tasks.

1. Model size: 8B parameters for TraceR1 planner, works with various executor models (7B to 32B parameters)
2. Training compute: not reported (GPU hours, hardware specifications not provided)
3. Inference speed/latency: not reported, operates in plan-act loop requiring multiple forward passes
4. Memory footprint: not reported, though uses frozen tool agents suggesting memory overhead from multiple models
5. Deployment practicality: moderate - requires coordination between planner and executor models, two-stage training pipeline adds complexity but achieves performance comparable to proprietary systems with open-source components

1. Evaluated on AndroidWorld using live Android emulator with 116 real-world mobile application tasks across 20 apps
2. OSWorld-Verified tests on actual desktop environments with dynamic interactions and real GUI interfaces
3. Tool-use evaluation uses executable tools from T3-Agent toolbox with real file processing (PPTX, PDF, XLSX documents)
4. No production deployment results or sim-to-real analysis reported
5. GUI experiments involve actual screenshot-to-action execution rather than purely synthetic environments
6. Method designed for real GUI environments but evaluation limited to controlled benchmark settings

1. FUNDAMENTAL: Short-horizon trajectory updates provide only local corrections and cannot reshape agent’s understanding of long-term feasibility or task structure
2. ENGINEERING: Requires two-stage training pipeline increasing complexity compared to single-stage approaches
3. ENGINEERING: Performance improvements are modest (typically 3-6 percentage points) despite added training complexity
4. EVALUATION: Limited analysis of failure modes or breakdown of where anticipatory planning helps vs. hurts performance
5. ENGINEERING: Relies on frozen tool agents for grounding, creating dependency on separate executor models

6. FUNDAMENTAL: Trajectory forecasting becomes noisy for very long horizons (T>10), limiting look-ahead capability

   Likely failure modes:

7. Over-optimistic planning when predicted trajectories don’t account for environment stochasticity
8. Trajectory-level rewards may not align well with actual task success in complex multi-step scenarios.

1. Real-world context: Geometric problem solving requires “thinking with constructions”—the dynamic manipulation of visual aids to bridge gaps between problem conditions and solutions. However, existing multimodal large language models (MLLMs) are confined to passive inference with static diagrams, lacking strategic knowledge of when and how to construct effective visual aids like auxiliary lines.

2. Task definition: The input consists of geometry problems with textual descriptions and initial diagrams (512×512 resolution). The task requires generating step-by-step solutions that strategically incorporate auxiliary visual constructions. The objective is to maximize problem-solving accuracy through interleaved visual-textual reasoning:

   \[\max_{\pi} \mathbb{E}_{q \sim D} [R(\text{solution}_{\pi}(q, I_{\text{orig}}))]\]

   where $q$ is a geometry problem, $I_{\text{orig}}$ is the initial diagram, and $R$ measures solution correctness.

3. Evaluation criteria: Success is measured by accuracy on geometry problem benchmarks. The paper uses exact match accuracy as the primary metric, with additional analysis of reasoning perplexity (PPL) as an indicator of solution quality.

4. The paper introduces GeoAux-Bench comprising 4,334 geometry problems with 8,470 diagrams. Each problem explicitly aligns textual construction steps ($T_{aux}$) with corresponding visual updates ($I_{aux}$), creating precise interleaved mappings for training and evaluation.

1. The method builds on Group Relative Policy Optimization (GRPO) using Qwen2.5-VL as the backbone multimodal model with frozen vision tower during training.

2. Tri-Partition Sampling creates three trajectory subsets: Mandatory ($O^+$) with enforced auxiliary constructions via prefix forcing, Prohibited ($O^-$) with masked auxiliary tokens, and Natural ($O$) with autonomous sampling.

3. Visual Re-prompting mechanism detects completed auxiliary commands and injects ground-truth visual aids when constructions match expected patterns, simulating interleaved visual feedback.

4. Adaptive Reward Shaping uses a composite reward function:

   \[R(o) = w_1 r_{acc} + w_2 r_{fmt} + w_3 r_{time} + w_4 r_{qual}\]

5. Timing Reward regulates construction necessity:

   \[r_{time}(o) = I_{aux}(o) \cdot \begin{cases} 1 & \text{if } \Delta > \tau \\ -1 & \text{if } \Delta < -\tau \\ 0 & \text{otherwise} \end{cases}\]

6. Quality Reward favors low-perplexity constructions:

   \[r_{qual}(o) = I_{aux}(o) \cdot r_{acc}(o) \cdot \mathbf{1}[\text{PPL}(o) < \bar{P} + \delta]\]

1. Supervised Fine-Tuning (SFT) warm-up: 5 epochs on 1,600 samples from GeoAux-Bench, GeomVerse, and Geometry3k training splits using AdamW optimizer with 5e-5 learning rate, cosine schedule, batch size 32.

2. A2PO reinforcement learning: 650 steps with 1e-6 constant learning rate, batch size 24, rollout batch size 72, 8 generations per prompt, KL coefficient β=0.01.

3. Training data uses mixed-prompt strategy with standard and prohibited prompt variants, filtered for marginal solvability (mixed correct/incorrect outcomes).

4. Hardware details not explicitly reported, uses bfloat16 precision throughout.

5. Wall-clock training time not reported.

The core novelty is the systematic integration of visual-textual interleaved reasoning for geometry through Adaptive Reward Shaping that explicitly models construction timing and quality. Prior work includes GeometryZero (2025) for geometry RL optimization, GRPO (2024) for group-based policy optimization, and MathCanvas (2025) for visual chain-of-thought. The specific delta is the tri-partition sampling with counterfactual baselines and perplexity-based quality rewards for strategic visual construction. The contribution is SIGNIFICANT as it addresses a fundamental limitation in geometric reasoning MLLMs through principled reward design, though builds incrementally on existing RL optimization frameworks.

1. GeoAux-Bench (new): Accuracy 42.97% (A2PO) vs 40.18% (GeometryZero baseline), +2.79% improvement
2. Geomverse: Accuracy 70.70% (A2PO) vs 68.30% (GeometryZero), +2.40% improvement
3. Geometry3k: Accuracy 53.61% (A2PO) vs 53.72% (GeometryZero), marginal difference
4. Overall average improvement: 55.76% vs 54.07% baseline, +1.69% gain
5. GeoAux-Bench evaluation on SOTA models: Gemini-2.5-Pro achieves 83.16%, GPT-5 at 80.62%, with large gaps to open-source models
6. The paper shows consistent but modest improvements across benchmarks, with strongest gains on the newly introduced GeoAux-Bench dataset

1. Model size: 7B parameters (Qwen2.5-VL-7B-Instruct backbone)
2. Training compute: Not explicitly reported, uses frozen vision tower to reduce computational overhead
3. Inference speed: Not reported, but visual re-prompting likely adds latency overhead
4. Memory footprint: 8,192 max sequence length, 512×512 image resolution, bfloat16 precision
5. Deployment practicality: Limited by reliance on ground-truth visual injection during re-prompting, making current approach more suitable for research than production deployment

1. The method currently relies on retrieval-based visual injection using ground-truth auxiliary diagrams rather than native visual generation, limiting real-world deployment
2. No production deployment results or hardware experiments reported
3. The approach requires pre-existing high-quality geometric diagrams for the re-prompting mechanism
4. Authors acknowledge the limitation that current unified MLLMs lack fine-grained visual actuation for precise geometric editing
5. Real-world applicability is constrained until advances in multimodal pre-training enable reliable geometric diagram generation

1. FUNDAMENTAL: Current implementation requires ground-truth auxiliary diagrams rather than model-generated visuals, limiting autonomous reasoning capability
2. ENGINEERING: Visual re-prompting adds inference latency and complexity that could be optimized with better visual generation models
3. FUNDAMENTAL: The method cannot generate precise geometric constructions natively, relying on retrieval-based injection
4. EVALUATION: Potential data contamination observed in SOTA model performance on Olympiad problems suggests memorization rather than reasoning

5. ENGINEERING: The tri-partition sampling strategy increases computational overhead during training

   Failure modes:

6. Visual hallucinations in native unified models lead to geometric reasoning errors
7. Models often resort to “analytic shortcuts” using coordinate systems instead of pure geometric reasoning

T2I generation models face fundamental challenges with spatial reasoning and compositional accuracy despite recent advances. Traditional text encoders struggle with complex relational semantics, and standard open-loop sampling propagates initial ambiguities throughout generation, leading to spatial constraint violations and attribute misalignment.

The task is spatially grounded text-to-image generation: given a complex natural language prompt describing spatial relationships and multiple objects with specific attributes (colors, positions, counts), generate a 1024×1024 image that precisely follows all constraints. The objective function can be formulated as:

where $\mathbf{x}$ is the generated image and $\mathbf{y}$ is the text prompt.

Evaluation criteria include compositional accuracy measured on T2I-CompBench (attribute binding, object relationships, complex scenes), object-focused evaluation on GenEval (object presence, counting, positioning), and reasoning capability assessment on R2I-Bench across five dimensions (causal, logical, commonsense, compositional, mathematical reasoning). The paper evaluates against 15+ baseline models including FLUX.1-dev, SDXL, and recent agentic frameworks.

1. Base Architecture: Built upon FLUX.1-dev, a 12B parameter rectified flow transformer using Flow Matching paradigm with ODE trajectory:

   \[\frac{d\mathbf{x}_t}{dt} = v_\theta(\mathbf{x}_t, t, \mathbf{y})\]

2. VLM Integration: Uses Qwen-VL-MAX (Pro version) or Qwen2.5-VL-7B (Fast version) as semantic critic for diagnosis and scoring.

3. Linear Trajectory Projection: Exploits constant-velocity property of rectified flows to project noisy latents to clean preview:

   \[\hat{\mathbf{z}}_0 = \mathbf{z}_t - t \cdot \mathbf{v}_t\]

4. Contrastive Energy Function: Defines energy over CLIP embeddings:

   \[\mathcal{E}(\hat{\mathbf{z}}_0) = \cos(\text{CLIP}(\hat{\mathbf{x}}_0), \mathbf{e}_{neg}) - \cos(\text{CLIP}(\hat{\mathbf{x}}_0), \mathbf{e}_{pos})\]

5. Time-Scaled Velocity Modulation: Applies spatially-masked corrections to velocity field:

   \[\mathbf{v}_t^{corrected} = \mathbf{v}_t + \eta t \cdot \nabla_{\hat{\mathbf{z}}_0} \mathcal{E} \odot \mathbf{M}\]

6. Parallel Rollout Search: At critical timestep (t=0.6), explores three branches: baseline continuation, corrective steering via AFS, and stochastic exploration with Gaussian noise injection.

This is a training-free framework that does not modify the base FLUX.1-dev model parameters.

1. Base Model: Uses pre-trained FLUX.1-dev (12B parameters, trained on internet-scale data)
2. VLM Components: Leverages pre-trained Qwen-VL-MAX or Qwen2.5-VL-7B for semantic supervision
3. SAM3 Integration: Uses pre-trained SAM3 for object segmentation and spatial grounding

4. CLIP Encoder: Employs pre-trained CLIP for contrastive energy computation

   No additional training is performed. The framework operates purely through test-time computation and search, preserving the open-world generalization capabilities of the foundation models while adding closed-loop feedback control.

Prior Work: Builds on agentic T2I frameworks like RPG (2024), AgentComp (2024), and SILMM, which use VLMs for external prompt refinement but operate in open-loop fashion. Also relates to attention manipulation methods like Attend-and-Excite and Prompt-to-Prompt.

Key Delta: Introduces internal closed-loop control within the ODE trajectory rather than external re-prompting. The core innovations are:

1. Agentic Flow Steering that directly modulates the velocity field using energy gradients
2. Parallel Rollout Search with lookahead simulation at critical timesteps

3. Time-scaled intervention that naturally decays correction strength as generation progresses.

   Rating: SIGNIFICANT - Represents a paradigm shift from external feedback loops to internal trajectory steering, with solid theoretical grounding in optimal transport and practical improvements across multiple benchmarks.

1. T2I-CompBench: AFS-Search-Pro achieves 0.6748 average (vs 0.4770 FLUX baseline), +7.86% improvement over best prior method AgentComp (0.5962)

2. GenEval: Shows consistent improvements across object counting, positioning, and color accuracy (visual results provided, quantitative scores not detailed)

3. R2I-Bench: AFS-Search achieves 0.48 average vs 0.33 FLUX baseline, outperforming 9 baseline models across causal (0.45), logical (0.58), commonsense (0.51), compositional (0.66), and mathematical (0.20) reasoning

4. Inference Speed: AFS-Search-Pro: 62.3s vs FLUX 11.7s, AFS-Search-Fast: 32.5s (faster than other agentic methods like RPG 104.2s, EvoGen 125.3s)

   Results show consistent improvements across all benchmarks, with particularly strong gains in spatial relationships and complex compositional tasks.

1. Model Size: 12B parameters (FLUX.1-dev base) + VLM components (Qwen-VL-MAX or Qwen2.5-VL-7B)

2. Training Compute: Zero additional training required (training-free approach)

3. Inference Speed: AFS-Search-Pro 62.3s, AFS-Search-Fast 32.5s per 1024×1024 image (vs 11.7s FLUX baseline)

4. Memory Footprint: Not explicitly reported, but requires simultaneous loading of FLUX.1-dev, VLM, SAM3, and CLIP models

5. Deployment Assessment: Computationally intensive due to parallel rollout search and multiple model inference, but still faster than competing agentic frameworks. The 3-6x slowdown over baseline may limit real-time applications but acceptable for high-quality generation tasks.

1. Dataset Testing: Evaluated on standard T2I benchmarks (T2I-CompBench, GenEval, R2I-Bench) with complex real-world prompts including spatial relationships, object counting, and compositional scenes

2. Prompt Complexity: Handles complex natural language descriptions like “four seasons in one picture” and professional scenarios with multiple objects and precise spatial constraints

3. Production Integration: Framework is modular and could integrate with existing FLUX.1-dev deployments, though computational overhead may require optimization for production use

4. Cross-domain Generalization: Leverages pre-trained foundation models without fine-tuning, preserving open-world capabilities for diverse domains and artistic styles

5. User Study: No human evaluation or user studies reported to validate real-world preference alignment

1. FUNDAMENTAL: Relies on VLM’s visual understanding capabilities - errors in VLM diagnosis propagate to final generation quality

2. ENGINEERING: Computational overhead (3-6x slower than baseline) limits real-time applications and scalability

3. ENGINEERING: SAM3 segmentation accuracy affects spatial grounding precision - incorrect masks can misguide correction

4. EVALUATION: Limited to standard benchmarks without human preference evaluation or real deployment studies

5. ENGINEERING: Global retry mechanism (up to 2 retries) adds further computational cost and unpredictable latency

   Failure Modes:

   • VLM Misdiagnosis: Incorrect intermediate assessment leads to wrong corrective actions
   • Energy Function Limitations: CLIP-based contrastive energy may not capture fine-grained spatial relationships, potentially causing over-correction or under-correction of defects

Electric vehicle (EV) adoption creates urgent demand for scalable battery recycling, but EV battery pack disassembly remains largely manual due to high design variability, safety hazards from 400-800V systems, and economic pressures from lower-value LFP batteries. The task is to develop an autonomous robotic system for disassembling full-scale EV battery packs through fastener removal operations.

Input: Full-scale EV battery pack (2.12m × 1.22m, 450kg, 800V Hyundai Ioniq5), RGB-D images from Intel RealSense D435i. Output: Successful removal of fasteners (screws, nuts) with pose estimation and manipulation commands. The objective is to maximize fastener removal success rate while minimizing disassembly time:

where $s$ represents the fastener removal strategy.

Success is measured by:

1. fastener removal success rate (%)
2. total disassembly time (minutes)
3. object detection mAP@0.5, and
4. agentic AI task completion rate (%). The system is evaluated on 204 fastener removal trials across three strategies: taught-in poses, one-shot vision execution, and visual servoing. A complete manual disassembly baseline is established using 12 subtasks totaling 6778 man-seconds.

1. Robotic Hardware: Universal Robot UR16e (6-DOF, 16kg payload) mounted on Parker-Hannifin 2.1m linear gantry with custom nut-runner tool (AlloyPower ARW801) and Intel RealSense D435i RGB-D camera.

2. Vision Pipeline: YoloWorld open-vocabulary object detection model achieving 0.9757 mAP@0.5, trained on 563 annotated images with 7 component classes (bolts, bus bars, screws, nuts, etc.). 3D point clouds generated from RGB-D data and stored in kD-Tree for efficient retrieval.

3. Motion Planning: Custom analytical inverse kinematics solver for 7-DOF system (6-DOF arm + 1-DOF gantry) using multi-objective cost function:

   \[\text{Cost} = d_J(q_c, q_k) + \frac{1}{d_{ee}} + \frac{1}{A_\triangle}\]

   where $d_J$ is joint distance, $d_{ee}$ is wrist-gantry distance, and $A_\triangle$ is arm triangle area for singularity avoidance.

4. Fastener Removal: Three strategies implemented - (1) taught-in poses with 10N downward force control, (2) vision-guided execution using centroids, and (3) visual servoing with interaction matrix $L$ for alignment correction.

5. Agentic AI: SmolAgents framework connecting LLMs (GPT-4o-mini, Qwen 3.5 9B/4B) to robot capabilities through structured tool calls and ROS services, enabling natural language task specification and execution.

1. Vision Model Training: YoloWorld finetuned on custom dataset of 563 RGB images with 4,614 labeled instances across 7 battery component classes. Training details (optimizer, learning rate, hardware) not reported.

2. Motion Planning: Analytical solver implemented with empirically tuned cost function weights (gantry w=0.1, arm joints w=1.0) through ablation study. No learning-based training involved.

3. Agentic AI: Pre-trained LLMs used without additional training - GPT-4o-mini via API, Qwen 3.5 (4B/9B parameters) deployed locally on NVIDIA Jetson Thor. System prompts and tool configurations provided but training methodology not applicable.

4. System Integration: Manual calibration and parameter tuning for force control (10N downward force), visual servoing gains, and TSP-based task sequencing using Google OR-Tools solver.

   Hardware and training time details not reported for most components.

This work extends prior battery disassembly research focused on smaller hybrid battery packs (Qu et al. 2024, Al Assadi et al. 2024) to full-scale EV batteries requiring gantry-mounted systems. Key innovations include:

1. analytical IK solver for 7-DOF gantry systems addressing redundancy challenges
2. integration of open-vocabulary detection (YoloWorld) with agentic AI for flexible task specification, and

3. systematic comparison of fastener removal strategies on realistic EV hardware.

   The closest prior work is Choux et al. 2021 on task planning for EV battery disassembly, but focused on smaller batteries without physical implementation. The agentic AI integration builds on recent LLM-robotics work (Lynch et al. 2023, Rachwał et al. 2025) but applies it specifically to industrial disassembly.

   Rating: INCREMENTAL - combines existing techniques in a novel application domain with solid engineering contributions.

1. Object Detection Performance: YoloWorld achieves 0.9757 mAP@0.5 vs YoloV11L baseline 0.9708 mAP@0.5, demonstrating comparable performance with open-vocabulary benefits.

2. Fastener Removal Success: Taught-in poses achieve 97.06% success rate (24.09 min), one-shot vision 57.35% success (28.70 min), visual servoing 82.84% success (36.29 min) across n=204 trials.

3. Inverse Kinematics: Custom analytical solver achieves 99.3% success rate vs KDL 25.7% (1 seed) and 73.0% (50 seeds), with 0.31s vs 10.46s planning time.

4. Agentic AI Task Completion: Tool-based interface achieves 100% success vs MCP-based 56.7% success (43.3% failure rate) across simple motion and complex reasoning tasks.

5. Manual Disassembly Baseline: Complete manual disassembly requires 6778 man-seconds across 12 subtasks, providing comparative baseline for automation assessment.

   No comparison to other robotic disassembly systems on equivalent full-scale batteries due to lack of existing benchmarks.

1. Model Size: YoloWorld parameters not specified. Qwen 3.5 models: 4B and 9B parameters deployed locally.

2. Training Compute: Not reported for vision model training. No training required for analytical motion planning components.

3. Inference Speed: Vision processing and motion planning times not explicitly reported. Task execution ranges 24-36 minutes for fastener removal operations.

4. Memory Footprint: Qwen models deployed on NVIDIA Jetson Thor edge hardware, demonstrating feasibility of local deployment.

5. Deployment Practicality: System demonstrates real-world deployment on full-scale EV battery with $80,000 hardware cost. Open-source software stack enables reproducibility and adoption.

1. Physical Hardware Testing: Extensive evaluation on actual Hyundai Ioniq5 EV battery (450kg, 800V) in laboratory setting with full-scale gantry system.

2. Industrial Environment: System designed for integration into existing human disassembly stations with collaborative safety considerations and predictable motion planning.

3. Safety Implementation: Battery discharged to 0V using Webasto AV-900 cycler prior to disassembly. Force-controlled interactions and collision avoidance implemented.

4. Economic Viability: Techno-economic analysis shows potential >40% cost savings compared to manual disassembly, with $80,000 total system cost.

5. Production Readiness: Open-source platform released to enable systematic investigation and further development toward scalable deployment.

   No deployment in actual recycling facilities reported, remaining at research demonstration level.

1. FUNDAMENTAL: Vision system struggles with reflective surfaces (black sheet metal, plastic components) due to active infrared interference from RealSense sensor.

2. ENGINEERING: Robot operates significantly slower than human workers (22 min vs 17 min for comparable tasks), though running well below maximum speed.

3. ENGINEERING: Visual servoing exhibits convergence issues due to poor depth estimates causing overshoot/undershoot in control signals.

4. EVALUATION: Only evaluated on single battery type (Hyundai Ioniq5), limiting generalizability claims across diverse EV designs.

5. ENGINEERING: End-effector slack prevents 100% success rate even with taught-in poses (97% achieved).

   Failure Modes:

6. False positive detections leading to attempted removal of non-existent fasteners
7. Agentic AI systems marking tasks complete without actual execution, particularly with smaller language models in complex scenarios.

1. Real-world context: Object-goal navigation requires robots to autonomously explore unfamiliar environments to locate specific objects based on natural language descriptions. Existing multi-agent approaches are limited to closed-set vocabularies and require retraining for new object categories, precluding deployment in dynamic real-world scenarios where novel objects may appear.

2. Task definition: Given N agents in an unknown environment E, the task is to locate object instances belonging to open-vocabulary categories g ∈ G (379+ categories) specified in natural language. Each agent receives RGB-D observations and pose at each timestep. The formal objective is a decentralized partially observable Markov decision process (Dec-POMDP) where agents must navigate to within 1.0m Euclidean distance of target objects in sequential multi-subtask episodes.

3. Evaluation criteria: Success is measured by Subtask Success Rate (SR) - fraction of subtasks completed within distance threshold, Success weighted by Path Length (SPL) using geodesic distance, and Distance to Goal (DTG) at episode termination.

4. The paper evaluates on GOAT-Bench val_unseen split comprising 360 multi-subtask episodes across HM3D scenes, with each episode containing 5-7 sequential open-vocabulary object-goal subtasks (1032 total subtasks).

1. Ego-centric semantic mapping: Each agent constructs BEV occupancy grids from RGB-D observations using depth-projected voxel splatting with corrected camera intrinsics for non-uniform rescaling

2. Zero-shot perception: SAM3 provides text-prompted object detection and segmentation, while GoalProjector back-projects detected masks through calibrated depth into BEV coordinates

3. VLM spatial reasoning: SpaceOM Vision-Language Model estimates frontier probabilities via structured prompt chains (scene captioning, room-type classification, perception gating)

4. Constraint-guided frontier selection: Combined scoring function

   \[U(f_i) = (1-w) \cdot s^{vlm}_i + w \cdot \hat{v}_i\]

   where VLM scores are blended with Bayesian value map estimates

5. Bayesian value map: Maintains probabilistic beliefs updated via

   \[V'(x,y) = \frac{\sigma^2_{obs} \cdot V(x,y) + \Sigma(x,y) \cdot c_t \cdot m(x,y)}{\Sigma(x,y) + \sigma^2_{obs}}\]

6. Multi-agent coordination: Decentralized architecture with max-pooling map fusion

   \[M^{global}(x,y,c) = \max_i M^i_t(x,y,c)\]

   and sequential greedy frontier allocation

7. Navigation: Fast Marching Method solves Eikonal equation

   \[|\nabla T(x)| = \frac{1}{v(x)}\]

   for geodesic path planning

This is a zero-shot, training-free approach that requires no task-specific training. The method leverages:

1. Pre-trained components: SAM3 for detection/segmentation, SpaceOM VLM for spatial reasoning
2. No fine-tuning or adaptation required
3. Immediately deployable in novel environments without training data
4. No optimization phases - uses engineered pipeline components

The core novelty is integrating Vision-Language Models directly into multi-agent frontier-based exploration for open-vocabulary object navigation. Prior works like VLFM (2024) and L3MVN (2023) used VLMs for single-agent navigation, while MCoCoNav (2023) addressed multi-agent coordination with closed vocabularies. The specific delta is:

1. zero-shot multi-agent coordination without precomputed graphs
2. depth-projected goal localization via GoalProjector, and
3. constraint-guided VLM reasoning for frontier selection. This represents SIGNIFICANT progress by combining multiple existing techniques in a novel architecture that achieves competitive results without training.

1. GOAT-Bench val_unseen: Subtask SR 55.8% vs Modular GOAT baseline 29.4% (26.4 point improvement), SPL 18.3% vs 17.5% (0.8 point improvement)
2. Comparison to AstraNav-Memory: SR 55.8% vs 62.7% (6.9 point gap), SPL 18.3% vs 56.9% (38.6 point gap)
3. Multi-agent benefit: N=2 vs N=1 agents shows 56.2% vs 38.6% SR (17.6 point improvement)
4. Results are mixed - competitive SR but significant SPL gap vs trained methods

1. Model size: Uses pre-trained SAM3 and SpaceOM VLM (exact parameters not reported)
2. Training compute: Zero - no training required
3. Inference speed: Real-time capable with 20 Hz state updates, 10 Hz camera processing on Orange Pi 5
4. Memory footprint: BEV semantic maps with voxel grids, specific memory usage not reported
5. Deployment practicality: High - zero-shot approach deployable without adaptation, demonstrated on real multi-drone hardware

1. Preliminary real-world validation on custom quadrotor platform with Intel RealSense D435 cameras and Vicon motion capture
2. Hardware setup: 2 drones with Orange Pi 5 onboard compute, ArduPilot flight controllers, ZMQ communication over WiFi
3. Environment: Indoor lab (10×5m, max 2m altitude) with successful object detection and BEV mapping
4. Sim-to-real transfer: Perception pipeline transfers effectively from simulation to real RGB-D data
5. Future work planned for full autonomous navigation experiments

1. FUNDAMENTAL: 2D BEV mapping cannot represent multi-floor environments or stacked objects
2. ENGINEERING: SAM3 detection failures on transparent, reflective, and small objects (mirrors achieve only 47.6% SR)
3. ENGINEERING: Substantial path efficiency gap (18.3% vs 56.9% SPL) due to frontier exploration overhead
4. EVALUATION: Limited real-world validation - only perception tested, not full autonomous navigation

5. ENGINEERING: GoalProjector depth projection errors with reflective surfaces causing goal mislocalization

   Failure modes:

6. Reflected scene confusion in mirrors causing goal projection behind walls
7. Small/occluded objects failing multi-view confirmation threshold leading to exploration failures

Reinforcement learning for articulated robot control faces the challenge of learning efficient policies that can generalize across different dynamics and morphologies. Traditional approaches use spatial connectivity but ignore how forces and motion propagate through rigid body systems according to physics laws.

The task is to learn control policies for articulated robots (humanoids, quadrupeds) that map proprioceptive observations (joint positions/velocities, IMU data, velocity commands) to joint control actions (target positions or torques). The objective is to maximize expected cumulative reward:

Success is measured by:

1. sample efficiency during training
2. final task performance (locomotion velocity tracking, balancing)
3. robustness to dynamics shifts (mass changes)
4. sim-to-real transfer capability. Environments include Genesis and SAPIEN simulators with humanoid/quadruped robots, plus real hardware validation on Unitree G1 and Go2 robots.

1. Observation Encoding: Link-wise projection layers ${\phi_i}_{i=0}^{K-1}$ transform global observation $s$ into per-link embeddings $z_i = \phi_i(s)$

2. Dynamics-Informed Message Passing: Inspired by Articulated Body Algorithm, features propagate from child to parent links following: - Message aggregation: $m_i = \sum_{j \in CH(i)} v_j^a$ - Link representation: $v_i = \text{softplus}(z_i + B_i) + m_i$ - Child contribution: $v_j^a = v_j - v_j \odot (W_j W_j^T v_j)$

3. Orthogonality Constraint: Auxiliary loss encourages structured projections:

   \[L_{orth} = \frac{1}{K} \sum_{i=0}^{K-1} \|W_i^T \text{diag}(v_i) W_i - I\|_F^2\]

4. Action Decoding: Per-joint action heads ${\psi_j}_{j=1}^{K-1}$ output actions from parent representations: $a_j = \psi_j(v_{PA(j)})$

   Core contribution: Embedding forward dynamics computational structure (inertia propagation from Articulated Body Algorithm) directly into policy architecture as learnable message passing.

1. Model-free RL Training: PPO algorithm with identical MLP value networks across all methods - Data: Parallel simulation environments (Genesis/SAPIEN simulators) - Optimizer: Not specified, standard PPO hyperparameters - Hardware: NVIDIA RTX 4090 for training - Wall-clock time: Not reported

2. Domain Randomization: Applied for sim-to-real transfer - Randomized: foot friction, encoder bias, base center-of-mass offset, external force perturbations - Scale: Not reported

3. Real Robot Deployment: Policies run at 50Hz on onboard NVIDIA Jetson Orin NX - PD control at 200Hz for low-level joint control - Inference time: <5ms worst-case for real-time control

Builds on structural priors for robot RL: NerveNet (2018) used basic GNNs, BOT (2024) and SWAT (2022) used transformers with morphological attention masks. Closest work is Rodrigues Network (2025) which embeds forward kinematics structure, but only for motion prediction/imitation learning.

Specific delta: First to embed the computational structure of forward dynamics (specifically Articulated Body Algorithm’s inertia propagation) directly into policy architecture for model-free RL. Replaces generic message passing with physics-inspired feature propagation that mirrors how inertial quantities accumulate in rigid body dynamics.

Rating: SIGNIFICANT - Novel architectural insight with strong empirical validation, though incremental over existing morphology-aware methods.

1. Genesis locomotion tasks (T1, G1, Go1, Go2): IQM 0.85 vs SWAT 0.79 (+7.6% improvement)
2. SAPIEN tasks (Humanoid Walk/Stand/Run, Hopper Hop/Stand): IQM 0.97 vs SWAT 0.71 (+36.6% improvement)
3. Mass generalization (1.1-2.0x mass increase): ABD-NET achieves 23.9% higher retention rate than SWAT baseline
4. Computational efficiency: 3x lower FLOPs than transformer baselines while maintaining faster inference

5. Real robot validation: Successful sim-to-real transfer on Unitree G1 (humanoid) and Go2 (quadruped) with diverse locomotion behaviors

   Results show consistent improvements, with larger gains on complex morphologies and dynamic tasks. Minor gains on simpler quasi-static tasks (Go1/Go2 tracking, Hopper Stand).

1. Model size: ~91-95K parameters (comparable across all methods)
2. Training compute: NVIDIA RTX 4090, parallel simulation, wall-clock time not reported
3. Inference speed: 4-6x slower wall-clock than MLP in PyTorch (reduced to 2x in JAX), but 3x lower FLOPs than transformers
4. Memory footprint: Not reported
5. Deployment practicality: Real-time capable with <5ms inference on Jetson Orin NX, suitable for 50Hz control loops

1. Hardware validation: Successful deployment on Unitree G1 humanoid and Go2 quadruped robots with onboard inference
2. Real environments: Tested across diverse terrains (asphalt, packed dirt, grass, indoor tile) for Go2
3. Dynamic behaviors: G1 demonstrates forward/lateral walking and complex dance motion sequences
4. Sim-to-real transfer: Standard domain randomization pipeline with real-time 50Hz policy control
5. Production readiness: Compatible with existing RL-to-hardware deployment workflows, sub-20ms control budget

1. ENGINEERING: Sequential leaf-to-root computation causes 2-6x higher training wall-clock time than parallel methods
2. FUNDAMENTAL: Current formulation limited to proprioceptive observations, cannot handle high-dimensional sensory inputs like images
3. EVALUATION: Limited to locomotion tasks, manipulation and contact-rich interactions unexplored
4. ENGINEERING: Orthogonality constraint approximation may not hold under all conditions

5. EVALUATION: Mass generalization only tested with 1.1-2.0x increases, extreme dynamics shifts unexplored

   Failure modes:

6. Policy may fail when robot morphology significantly deviates from tree structure assumptions
7. Performance degradation likely with very high-dimensional action spaces or complex contact scenarios.

Long-running Large Language Model (LLM) agents accumulate memories across extended conversations, creating critical governance challenges. Standard Retrieval-Augmented Generation (RAG) systems treat memory as passive storage, leading to context pollution from outdated information (“zombie memories”), contradictory facts, and privacy violations. This becomes particularly problematic for autonomous agents that must maintain coherent, safe operations over extended periods.

The task involves implementing a policy-driven memory governance layer that manages the complete lifecycle of agent memories. Input consists of conversational history, user queries, and retrieved memory candidates. Output is a governed context window containing only relevant, consistent, and up-to-date information for LLM generation. The system must enforce policies across four domains: lifecycle management, consistency resolution, adaptive retrieval, and safety compliance.

Success is measured by accuracy on long-horizon conversational benchmarks, specifically multi-hop reasoning, temporal consistency, and factual recall tasks. The evaluation focuses on preventing hallucinations while maintaining useful information retention.

The paper evaluates on LoCoMo-10 benchmark, containing 1,986 question-answer pairs across multi-session dialogues, categorized into Single-Hop, Multi-Hop, Temporal Reasoning, and Open Domain tasks.

1. MemArchitect implements a middleware governance layer that sits between user queries and the LLM agent, operating through three execution paths: Read, Reflect, and Background processing.

2. FSRS Decay Engine: Replaces exponential decay with Free Spaced Repetition Scheduler (FSRS v4) for biological forgetting curves. Retrievability is computed as:

   \[R(t) = \left(1 + \frac{19}{9} \cdot \frac{t}{S}\right)^{-1}\]

3. Kalman Utility Filter: Tracks memory utility using Kalman filtering to reinforce useful memories and downweight hallucinations:

   \[U_k = U_{k-1} + K_k \cdot (z_k - U_{k-1})\]

4. Adaptive Scoring System: Dynamically weights retrieval based on query type (Fact vs. Reasoning) and memory characteristics:

   \[\text{Score} = \text{Sim} \times R^\lambda \times (1 + \beta \cdot U)\]

5. Hebbian Graph Expansion: Implements associative memory linking where frequently co-occurring memories are retrieved together when P(B

   A) > 0.7.

6. Cross-Encoder Discriminator: Acts as a “veto gate” filtering semantically irrelevant candidates even after vector similarity matching.

7. Entropy-Triggered Consolidation: Monitors information density and triggers cleanup when compression ratio < 0.4, consolidating fading episodic memories into semantic facts.

The paper does not describe model training as MemArchitect operates as a governance middleware layer over existing pre-trained models. The system uses:

1. Pre-trained LLMs: Qwen2.5-3B-Instruct and Meta-Llama-3.1-8B-Instruct as backbone models
2. Pre-trained embeddings: BGE-M3 and nomic-embed-text-v1.5 for dense retrieval

3. Pre-trained Cross-Encoder: Used for relevance discrimination filtering

   No fine-tuning, optimization details, or training procedures are reported as the contribution is purely architectural - a policy engine that manages memory without modifying model weights.

The paper builds on established memory management approaches including MemGPT (2024), MemOS (2025), SimpleMem (2025), and Generative Agents (2023). The specific delta is introducing explicit policy-driven governance that actively manages memory lifecycle, resolves contradictions, and enforces compliance - moving from passive storage to active adjudication.

Key novelty includes:

1. FSRS-based biological decay modeling for LLM memory
2. Kalman filtering for utility tracking
3. policy-based “auction” system for context window allocation, and

4. unified governance across lifecycle, consistency, retrieval, and safety domains.

   The approach is distinguished from prior work by treating memory as a governed resource requiring active management rather than simple append-only storage.

   Rating: INCREMENTAL - The individual components (FSRS, Kalman filters, cross-encoders) are established techniques, but their integration into a unified memory governance framework represents a meaningful architectural contribution.

1. LoCoMo-10 Single-Hop (Qwen-3B): 95.0% vs 79.4% SimpleMem baseline (+15.6% improvement)
2. LoCoMo-10 Multi-Hop (Qwen-3B): 95.8% vs 66.7% SimpleMem baseline (+29.1% improvement)
3. LoCoMo-10 Temporal Reasoning (Qwen-3B): 95.3% vs 56.1% SimpleMem baseline (+39.2% improvement)
4. LoCoMo-10 Open Domain (Qwen-3B): 93.1% vs 67.1% SimpleMem baseline (+26.0% improvement)
5. LoCoMo-10 Single-Hop (Llama-3.1-8B): 53.2% vs 74.0% MemOS baseline (-20.8% decrease)
6. LoCoMo-10 Multi-Hop (Llama-3.1-8B): 40.1% vs 36.0% MemOS baseline (+4.1% improvement)
7. LoCoMo-10 Temporal Reasoning (Llama-3.1-8B): 27.1% vs 72.0% MemOS baseline (-44.9% decrease)

8. LoCoMo-10 Open Domain (Llama-3.1-8B): 56.3% vs 45.0% MemOS baseline (+11.3% improvement)

   Results are mixed: strong performance against compression-based SimpleMem, but worse recall against maximalist MemOS due to aggressive pruning policies.

1. Model size: Uses existing models (Qwen2.5-3B, Llama-3.1-8B) without modification - no additional parameters for the governance layer
2. Training compute: Not applicable - no training required, pure inference-time policy engine
3. Inference speed/latency: Not reported - overhead from policy evaluation, cross-encoder filtering, and Kalman updates not quantified
4. Memory footprint: Not reported - storage requirements for utility tracking, co-occurrence graphs, and policy state not specified
5. Deployment practicality: High - model-agnostic middleware that can be integrated with existing RAG systems without retraining

1. No deployment results on real-world production systems reported
2. No hardware experiments or robotic integration demonstrated
3. No production integration case studies provided
4. Evaluation limited to academic benchmarks (LoCoMo-10) with synthetic conversational data
5. Authors acknowledge need for validation on additional benchmarks including LongMemEval, PreFEval, and PersonaMem for real-world applicability assessment

1. FUNDAMENTAL: Aggressive decay policies sacrifice raw recall for coherence, leading to significant performance drops on factual retrieval tasks (-44.9% on temporal reasoning vs MemOS)
2. ENGINEERING: Incomplete implementation - several key policies (conflict resolution, toxic memory filter, GDPR compliance) are marked as “planned” future work
3. EVALUATION: Limited to single benchmark (LoCoMo-10) with only two baseline comparisons, lacking evaluation on established long-term memory benchmarks
4. ENGINEERING: No analysis of computational overhead or latency impact from policy evaluation during inference

5. FUNDAMENTAL: Policy calibration challenges - FSRS parameters optimized for human flashcard learning may not transfer to LLM memory management

   Failure modes:

6. Over-pruning leading to loss of important but infrequently accessed facts
7. Policy conflicts where lifecycle management contradicts consistency requirements

Multimodal Automated Program Repair (MAPR) addresses real-world software debugging where developers must analyze code alongside visual artifacts like GUI screenshots and UI mockups. Traditional program repair systems only process code and text, missing crucial visual context that explains many software defects. The task requires jointly reasoning over three modalities to generate correct patches.

Given a software repository with codebase $C$, textual issue description $D$, and visual screenshots $I$, MAPR aims to generate a code patch $\Delta$ such that the patched codebase satisfies:

where $C’ = C \oplus \Delta$ is the patched codebase, $F(\cdot)$ denotes program execution, and $T_{spec}$ includes fail-to-pass tests (verifying the issue is fixed) and pass-to-pass tests (ensuring no regression). Success is measured by resolved rate - the percentage of issues correctly fixed on the SWE-bench Multimodal benchmark.

The benchmark comprises 617 real-world GitHub issues from 17 JavaScript repositories, where visual information is necessary for resolving over 83% of tasks.

1. FailureMem introduces a hybrid workflow-agent architecture that combines structured localization with flexible reasoning, using deterministic workflows for file/element identification and agentic loops only for patch generation.

2. A hierarchical Failure Memory Bank stores historical repair trajectories in three layers: Contextual (issue summaries, visual analyses), Cognitive (diagnosis, negative constraints, golden principles), and Code (failed vs. golden patch summaries).

3. Memory construction uses offline distillation with Gemini 3 Pro to contrast failed patches against ground-truth fixes, extracting reusable repair patterns from 84 failed trajectories.

4. Active perception tools enable region-level visual grounding: Crop tool extracts sub-regions from screenshots for detailed inspection, Grounding tool overlays bounding boxes to highlight bug locations.

5. Interactive Bash environment allows agents to explore repository structure and verify assumptions before code modification.

6. Memory retrieval uses a Selector Agent to identify top-k relevant cases based on semantic similarity between current issues and historical Contextual Layers.

7. Three-phase repair process: Phase 1 performs file localization with memory guidance, Phase 2 identifies key elements using skeleton compression, Phase 3 generates patches using agentic reasoning with tool access.

1. No model training is performed - FailureMem is a framework that uses existing LLMs (GPT-5.1, GPT-4.1, Claude 4.5) as backbone models.

2. Memory bank construction uses offline distillation with Gemini 3 Pro to process 84 failed repair trajectories from SWE-bench Multimodal development set.

3. All experiments use sampling temperature of 0 for deterministic generation following Pass@1 evaluation protocol.

4. Hardware: experiments conducted on four NVIDIA A100 (80GB) GPUs.

5. Wall-clock time: not reported.

6. No fine-tuning, pretraining, or RLHF stages involved - purely prompt-based approach with structured memory injection.

The closest prior work is GUIRepair (Huang et al., 2025c), which first introduced multimodal program repair but uses rigid workflows and passive visual processing. FailureMem’s specific deltas are:

1. hybrid workflow-agent architecture balancing structure with flexibility
2. active perception tools for region-level visual grounding rather than full-page processing, and

3. failure-aware memory bank that transforms repair failures into reusable guidance.

   The memory bank design contrasting failed vs. successful patches is novel. Prior APR systems like SWE-agent (Yang et al., 2024b) and Agentless (Ma et al., 2024) lack multimodal capabilities and treat each repair independently without learning from failures.

   Rating: SIGNIFICANT - introduces multiple novel components (failure memory, active perception, hybrid architecture) that address real limitations of existing multimodal repair systems.

1. SWE-bench Multimodal: Resolved Rate metric, FailureMem (GPT-5.1) achieves 33.1%, GUIRepair baseline 29.4%, improvement +3.7%

2. SWE-bench Multimodal: FailureMem (GPT-4.1) achieves 31.1%, GUIRepair 28.8%, improvement +2.3%

3. SWE-bench Multimodal: FailureMem (Claude 4.5) achieves 33.8%, GUIRepair 31.5%, improvement +2.3%

4. Outperforms other baselines: SWE-agent Multimodal (12.4%), Computer-Use Agents (20.1%), Agentless Lite (28.4%), Zencoder (27.5%)

5. Repository-level breakdown shows consistent improvements across multiple JavaScript projects (next, bpmn-js, carbon, etc.)

   Results are consistently positive across all tested models and repositories, with no conspicuously absent benchmarks for this specialized multimodal repair domain.

1. Model size: Uses existing LLMs as backbones (GPT-5.1, GPT-4.1, Claude 4.5) - parameter counts not specified but these are large commercial models

2. Training compute: No training required - uses pre-trained models with prompt-based approach on four NVIDIA A100 GPUs

3. Inference speed/latency: Not reported, but acknowledged increased cost due to iterative agentic loops and memory contexts

4. Memory footprint: Not reported, but memory bank stores 84 structured entries from failed trajectories

5. Deployment practicality: Inference cost increases 13% to $0.33 per issue vs. GUIRepair’s $0.29, considered acceptable given performance gains. Commercial model dependency limits deployment flexibility.

1. Evaluated on real-world GitHub issues from 617 production software repositories, not synthetic benchmarks

2. Issues span 17 popular JavaScript repositories including production systems (next.js, eslint, lighthouse, prettier)

3. No deployment results in production environments reported

4. No hardware experiments beyond GPU inference infrastructure

5. Framework designed as decision-support tool requiring human review before production deployment

6. Memory bank constructed from actual developer-verified patches, ensuring real-world grounding

   The work operates on authentic software defects but lacks production deployment validation.

1. ENGINEERING: Increased inference cost (13% higher than baseline) due to iterative agentic loops and detailed memory contexts

2. FUNDAMENTAL: Dependency on diversity of offline memory bank - rare or unprecedented failure modes without historical analogues bypass the contrastive distillation process

3. ENGINEERING: Reliance on commercial LLM APIs limits deployment control and introduces external dependencies

4. EVALUATION: Memory bank constructed only from 84 failed trajectories from development set, potentially limiting generalization

5. ENGINEERING: Skeleton compression strategy may lose important implementation details in very large codebases

   Failure modes:

6. System reverts to standard agentic behavior when encountering novel failure patterns not captured in memory bank
7. Active perception tools may fail on non-standard UI layouts or unconventional visual artifacts.

Generative reward models (GRMs) for vision-language models evaluate outputs through a three-stage pipeline: rubric generation, criterion-based scoring, and final verdict. However, existing methods only optimize the final answer while leaving intermediate rubrics unsupervised, leading to post-hoc rationalization rather than principled evaluation. This undermines the transferability of rubrics to independent evaluators.

The task is to train multimodal generative reward models that produce structured critiques for vision-language model outputs. Given a multimodal query $q$ (text + image $I$) and candidate response pair $(r_1, r_2)$, the model generates:

where $R$ is evaluation criteria, $E$ is criterion-by-criterion assessment, and $A \in {1,2}$ is the preference verdict. The objective is to maximize rubric transferability:

where $\phi$ is an independent proxy evaluator.

Success is measured by accuracy on preference prediction benchmarks: VL-RewardBench (1,247 pairs), Multimodal Reward Bench (5,000 samples), and MM-RLHF-Reward Bench. Key metrics include overall accuracy and transferability to unseen evaluator models.

The paper uses 60k curated preference samples from LaVA-Critic-113k, RLAIF-V, RLHF-V, and MMIF-23k datasets.

1. Base architecture: Qwen2.5-VL-7B-Instruct for both policy model (Proxy-GRM) and proxy agents
2. Teacher model: Qwen3-VL-235B-A22B for data distillation and structured critique generation
3. Proxy agent training: Two variants trained to consume rubrics and predict preferences - Proxy-SFT: Supervised fine-tuning on 5k samples with cross-entropy loss - Proxy-RL: Additional RL training on 10k samples with binary accuracy reward

4. Policy model training uses composite reward function:

   \[r = r_{\text{acc}} + r_{\text{proxy}} + 0.5 \cdot r_{\text{format}}\]

   where $r\_{\text{acc}} = +1$ if final verdict correct, $r\_{\text{proxy}} = +1$ if proxy agrees with verdict using generated rubric, $r\_{\text{format}} = +1$ for proper XML format
   

5. Reinforcement learning via GRPO (Group Relative Policy Optimization) with frozen proxy agents
6. Core contribution: Closed-loop rubric quality verification through independent proxy agents that measure transferability during training, unlike prior work that only optimizes final answers

1. Data distillation: Qwen3-VL-235B-A22B generates structured critiques for 60k samples, yielding 25k correct samples
2. Proxy-SFT training: 5k samples, learning rate 1×10⁻⁵, cosine scheduling, 1 epoch, ms-swift framework
3. Proxy-RL training: 10k samples, learning rate 5×10⁻⁶, GRPO with group size 7, verl framework
4. Policy cold-start SFT: 10k samples, learning rate 1×10⁻⁵, cosine scheduling, 1 epoch
5. Policy RL training: 45k samples (10k correct + 35k hard negatives), learning rate 5×10⁻⁶, GRPO, batch size 256, mini-batch 128 Hardware and wall-clock time: not reported Data filtering: automatic quality, difficulty, and similarity filters applied to source datasets

Prior work includes R1-Reward (2025), Unified-Reward (2025), and Auto-Rubric (2025). Closest work is Auto-Rubric which extracts rubrics from annotations, and Rubrics-as-Rewards (2025) which uses rubrics as structured rewards in text-only settings.

The specific delta is introducing proxy-guided rubric verification into the RL training loop for multimodal settings. Unlike LLM-as-judge approaches that provide non-differentiable external feedback, this method trains dedicated proxy agents to evaluate rubric transferability and integrates this as a reward signal during policy training.

Key finding: SFT-based proxy agents surprisingly outperform RL-based ones for rubric evaluation, revealing tension between outcome-level RL and process-level evaluation fidelity.

Rating: SIGNIFICANT - addresses fundamental limitation of existing GRMs with novel closed-loop verification approach.

1. VL-RewardBench: 75.22% overall accuracy vs 73.8% (Unified-Reward-Think), +1.42 points improvement, 73.93% macro accuracy
2. Multimodal Reward Bench: 85.62% accuracy vs 82.2% (R1-Reward), +3.42 points improvement
3. MM-RLHF-Reward Bench: 82.94% accuracy vs 80.59% (R1-Reward), +2.35 points improvement, 56.52% on Acc+ metric
4. Rubric transferability: Proxy-GRM rubrics improve external evaluator accuracy by 3-10 points when transferred to Qwen2.5-VL-7B/32B-Instruct and Unified-Reward-SFT

5. Data efficiency: achieves SOTA with ~50k training samples vs >200k for comparable methods (4× less data)

   Results are consistently positive across all benchmarks. Performance improvements are substantial and the method demonstrates strong transferability to unseen evaluators.

1. Model size: 7B parameters for both policy and proxy models (Qwen2.5-VL-7B base)
2. Training compute: not reported (GPU hours, hardware specifications not provided)
3. Inference speed/latency: not reported
4. Memory footprint: not reported
5. Deployment practicality: Good - uses standard 7B model size, requires additional proxy agent at training time but not at inference. Data efficiency (4× less training data) suggests practical training costs

1. No deployment results or production integration reported
2. No hardware experiments or real-world evaluation environments described
3. Evaluation limited to curated benchmark datasets (VL-RewardBench, Multimodal Reward Bench, MM-RLHF-Reward Bench)
4. Rubric transferability tested only on academic models, not production systems

5. No discussion of sim-to-real gaps or domain adaptation challenges

   The work is primarily evaluated on academic benchmarks without real-world deployment validation.

1. EVALUATION: Limited to curated benchmark datasets, no real-world deployment testing
2. ENGINEERING: Proxy agent training requires additional computational overhead and careful model selection
3. FUNDAMENTAL: Method relies on proxy agent quality - insufficient capability (e.g., 3B model) introduces harmful noise
4. ENGINEERING: No analysis of failure modes when policy and proxy disagree at inference time
5. EVALUATION: Transferability only tested on similar model families (Qwen variants), not diverse architectures

6. FUNDAMENTAL: Composite reward design may not generalize to other reward formulations or task domains

   Failure modes: 1) Proxy agents may provide misleading feedback if trained on biased or insufficient data, 2) Policy model may learn to game the proxy reward rather than generate genuinely transferable rubrics.

Real-world context: Robots operating in dynamic environments must process multiple sensory modalities simultaneously, but current Vision-Language-Action (VLA) models treat sound as static pre-execution prompts, missing critical real-time acoustic feedback during task execution. This creates a significant gap for tasks requiring immediate response to transient environmental sounds like microwave beeps or collision clicks, which provide essential state verification that vision alone cannot capture.

Task definition: The paper formalizes Vision-Sound-Language-Action (VSLA) as continuous control conditioned on multi-view RGB images $I^{1:V}_t$, streaming audio $A_t$, language instructions $l$, and proprioception $s_t$ under delayed decision loops. The core challenge is the Blind Execution Interval (BEI) where acoustic events occurring during open-loop action chunking are lost. Success requires causal timing constraints:

where $t_{snd}$ is when the required acoustic cue occurs and $t_{goal}$ is when the task goal is reached.

Evaluation criteria: Success measured by timing-sensitive criteria that reject premature actions completed before required acoustic triggers, even if visually correct. Performance evaluated on task completion rates under strict causal timing rules.

Dataset/Benchmark: Introduces OpenX-Sound (100 skills, 120k episodes, 680 objects with synthesized audio) for pretraining and HEAR-Bench (7 sound tasks, 30 objects) for evaluation with real-time audio-physics co-simulation enforcing causal timing rules.

1. Historizer: Streaming Stateful Transformer maintaining causal audio memory $h_{t_k}$ across execution gaps via packet-wise updates:

   \[h_{t_k}^{(j)} = \text{Hist}_\phi(h_{t_k}^{(j-1)}, \phi_{audio}(P_{t_k}^{(j)}))\]

2. Envisioner: Hierarchical design with high-level omni-modal model (Qwen3-Omni) producing semantic latent $z_{t_k}$ and KV cache, plus low-level LLM (Qwen3-0.6B) generating control features $u_{t_k}$ from cached context.

3. Advancer: Audio world model predicting near-future audio codes via decoder-only transformer with loss:

   \[\mathcal{L}_{adv} = -\mathbb{E}_D[\log p_\eta(z^a_{t_k \rightarrow t_{k+1}} | z_{t_k})]\]

4. Realizer: Conditional Flow Matching policy generating smooth action chunks with vector field regression:

   \[\mathcal{L}_{flow} = \mathbb{E}[\|v_\xi(x_\lambda, \lambda, u_{t_k}) - (x_1 - x_0)\|_2^2]\]

   Core contribution: Decoupling high-frequency sensory memory from low-frequency decision-making to solve the Blind Execution Interval problem through persistent causal audio context.

1. Pretraining stage: Trained on OpenX-Sound dataset (120k episodes with synthesized audio tracks generated via video-to-audio models) - Data: 100 manipulation skills, 680 objects, audio generated using advanced video-to-audio generation models - Optimizer: Not reported - Learning rate, schedule, batch size: Not reported - Hardware and wall-clock time: Not reported

2. Fine-tuning stage: Task-specific fine-tuning on HEAR-Bench simulation data - Data: 7 sound-centric tasks with strict causal timing rules - Training details: Not reported

3. Multi-objective training: Combines imitation learning loss with auxiliary objectives: - Stage description loss $\mathcal{L}_{text}$ for JSON stage predictions - Audio prediction loss $\mathcal{L}_{adv}$ for temporal grounding - Flow matching loss $\mathcal{L}_{flow}$ for action generation

Prior work: Builds on VLA models like Octo (2024), OpenVLA (2024), and π0.5 (2025), plus recent audio-VLA work like OmniVLA and Audio-VLA (2025). Flow matching extends π0.5’s approach.

Novel contributions:

• First formalization of the Blind Execution Interval problem in action chunking
• Streaming stateful audio memory architecture bridging execution gaps
• Audio world model for temporal grounding in quasi-static visual scenes
• Real-time audio-physics co-simulation with causally strict evaluation rules

Assessment: SIGNIFICANT - addresses a fundamental architectural limitation in current VLA systems with novel streaming audio memory and introduces important evaluation infrastructure, though builds incrementally on existing transformer and flow matching foundations.

1. HEAR-Bench simulation: HEAR achieves 81% success rate vs. waveform rendering (61%) and ASR baseline (35%)

2. Real-world robot evaluation: HEAR achieves 54% success rate across 4 sound-centric tasks on physical Franka Panda robot

3. Component ablation: Shows importance of each module (Historizer, Envisioner, Advancer, Realizer) with degraded performance when components removed

4. Timing analysis: Demonstrates successful capture of transient audio events during Blind Execution Intervals that other methods miss

   Note: Limited comparison to other multimodal robot learning methods beyond basic ASR and waveform baselines. Missing comparisons to other recent audio-VLA works like OmniVLA in identical settings.

1. Model size: Multi-component architecture with Qwen3-Omni (large) + Qwen3-0.6B (small) + specialized audio/flow modules, total parameters not reported

2. Training compute: GPU hours and hardware not reported

3. Inference speed: Real-time deployment on physical robot, specific latency numbers not provided but designed for low-rate decision loops with action chunking

4. Memory footprint: Maintains compact causal audio memory, specific memory usage not quantified

5. Deployment practicality: Successfully deployed on real Franka Panda robot, demonstrates practical feasibility but compute requirements not fully characterized

1. Physical robot deployment: Successfully tested on Franka Panda robot with external microphone across 4 sound-centric manipulation tasks

2. Hardware setup: Uses standard robot arm with external microphone for audio input, no specialized acoustic hardware required

3. Environment testing: Real-world indoor manipulation environment with ambient noise and contact sounds

4. Performance validation: 54% real-world success rate demonstrates sim-to-real transfer, though with performance gap from 81% simulation results

5. Practical constraints: Addresses real system latency and asynchronous sensor updates, designed for actual deployment constraints rather than idealized settings

1. FUNDAMENTAL: Relies on discrete audio tokenization which may lose fine-grained acoustic details important for subtle contact interactions

2. FUNDAMENTAL: Performance degrades significantly from simulation (81%) to real-world (54%), indicating substantial sim-to-real gap

3. ENGINEERING: Audio memory horizon limited by transformer context length, may struggle with very long monitoring tasks

4. ENGINEERING: Requires careful tuning of audio packet sizes and memory update rates for different acoustic environments

5. EVALUATION: Limited comparison baselines - missing evaluation against other recent multimodal robot learning methods

6. EVALUATION: HEAR-Bench uses synthesized audio which may not capture all real-world acoustic complexity

   Failure modes:

   • May fail when critical acoustic cues are masked by robot ego-noise during motion
   • Performance likely degrades in acoustically complex environments with multiple overlapping sound sources