世界模型与评测 · World Models and Evaluation

近一年为何说反应式VLA已死世界模型当立

Reactive VLA vs. World Models

更新于

30 秒结论

先读这个判断

本备忘录检视 2025 年 7 月 15 日至 2026 年 7 月 17 日公开的具身智能论文。候选池含 1,547 篇论文,175 篇获得可读全文;本次综合使用其中 28 篇完成全文精读和主张支持审计的论文。证据集中包含支持、限制、条件性结果和明示缺口,而非只选“世界模型有效”的正例。

精读论文
28
正式证据事件
28
知识单元
EA-MODEL, EA-EVAL, EA-4D, EA-ALIGN
审计状态
当前结算 run 已通过仓库校验

v0 · 完整正文

研究综述

研究边界

本备忘录检视 2025 年 7 月 15 日至 2026 年 7 月 17 日公开的具身智能论文。候选池含 1,547 篇论文,175 篇获得可读全文;本次综合使用其中 28 篇完成全文精读和主张支持审计的论文。证据集中包含支持、限制、条件性结果和明示缺口,而非只选“世界模型有效”的正例。

这一范围能回答的是研究重心如何迁移,不能证明 VLA 会作为模型类别消失,也不能将预印本的局部结果外推为工业系统已经成熟。

中心判断

“VLA 已死”只在一个严格限定下成立:如果 VLA 指的是一个从当前图像和语言指令直接回归下一段动作、主要靠行为克隆训练、不显式检验动作后果的反应式策略,那么近一年的论文正在系统性地拆掉它的前提。它在跨本体动作语义、长时程归因、失败恢复、接触可观测性和真实闭环评测上都遇到结构性瓶颈。

但“世界模型当立”也不是用视频生成器替代策略模型。证据更支持一个融合架构:VLA 保留语义理解、任务分解和动作先验;动作条件世界模型负责预测后果、比较候选、识别失败和产生纠错信号;本体适配器与底层控制器负责把意图变成可执行运动。所谓范式转移,是从“动作预测就是全部”转向“策略、动态与控制分账而联合”。

一、VLA 的第一道死穴:动作 token 没有通用的物理语义

统一 token 形式容易制造一种错觉:只要不同机器人的动作都能编码,它们就可以直接混合学习。SPACE 指出,同一记录控制命令在不同控制器、机械本体、硬件个体和部署动态中可以产生不同运动。SA-VLA 则把问题收窄到离散动作 token 的解码端:一个固定 token 在不同机器人状态和接触情境下,未必对应同一连续控制量。

这不是编码器再大一点就会自动消失的问题。它要求模型把“语义上想做什么”、“世界会怎么变”和“这台机器人怎么动”分成不同接口。一项 UR5 真机实验也表明,离线 VLA 指标很好,仍可能因坐标系、模态时间对齐、图像预处理和动作语义不一致而无法稳定转成真实行为(UR5 实验)。

结论是:跨本体不应优先共享原始 command,而应优先共享任务阶段、空间状态变化、可达性和后果,再由本体适配器落地到具体控制器。

二、VLA 的第二道死穴:预测下一步,却不判断下一步会造成什么

反应式策略的根本缺口不是不会产生动作,而是它通常没有一个可查询的“假如执行这个动作,世界会变成什么”模块。在短时程、轨迹密集覆盖的模仿任务中,这个缺口可以被数据局部掩盖;在长任务、干扰和不可逆接触中,小偏差会沿动作块累积,而且策略不知道何时已经进入不可恢复状态。

近期 VLA 研究其实已在承认这一点。ERVLA 显示,有效的具身思维链必须落到末端执行器运动或图像空间轨迹等动作相关表示,把显式文本思维链当作自回归动作前缀反而会累积误差。Fast-ThinkAct 尝试把长程规划和失败自纠正压缩成隐变量推理;ReCoVLA 则将失败类型判断、恢复阶段和 reward 选择与底层 residual 纠错分开。这些工作不是在证明 VLA 没用,而是在把一个单一端到端函数拆回认知、动态与控制三层。

三、为什么世界模型正在成为新的中心层

世界模型的价值不在于多生成一段视频,而在于同时改变训练目标、规划接口和评测方式。

第一,它把监督从“专家当时做了什么”扩展到“动作使世界如何变化”。$\tau_0$-WM 将真实遥操、UMI 式交互、人类第一视角视频与 rollout/失败轨迹放入统一视频—动作模型,用模态监督掩码处理异构标注缺失。GEM-4D 通过几何特征蒸馏提升视频世界模型的操作相关性,而不增加推理时几何负担。3PoinTr 进一步说明,遮挡期间的物体点身份不应因暂时不可见而被整段丢掉。

第二,它使候选动作可以在执行前被比较。World Pilot 的核心动机正是:静态图文预训练无法为接触丰富操作提供充分信号,策略需要看到动作条件场景演化。Interactive World Simulator 表明,中等规模的机器人交互数据可以训练世界模型并生成策略训练示例,但前提是长时程交互一致性与 sim-real 排序相关性成立。

第三,它能把失败附近变成可学习的数据,而不是将失败 episode 整体丢弃。Hi-WM 强调失败高发状态周围的密集纠正轨迹;TACO 用触觉感知世界模型联合预测未来视频和力序列,再将真实失败附近状态转成带纠正动作的想象片段,用于 VLA 后训练。Dream-Tac 把动作块、未来视觉和未来触觉放入同一目标,就是因为纯视觉未来无法完整观测接触成败。

四、基准高分为什么不能挽救旧范式

近一年最关键的变化之一,是“什么算做对了”正在被重写。LIBERO-PRO 指出,LIBERO 标准协议中的训练与测试任务过度接近,固定布局和指令—动作映射的记忆可被误认为泛化。EgoVLA 提供了另一个冷静例子:人类第一视角视频预训练不会直接消除本体差距;在缺少机器人微调时,其仿真 humanoid 任务成功率为 0%。这不否定人类视频预训练的价值,但否定了“视觉语义规模自动等于物理可执行性”。

世界模型也面临同样的评测重构。MiraBench 把缺口概括为动作条件可靠性:不只看视觉保真,还要看物理约束、动作跟随忠实度和失败乐观偏差。WEAVER 进一步要求结果保真、长时一致与足够高的规划效率同时成立。SANTS 则直接提醒:后期去噪即使让视频更好看,也可能更不动作相关或更不符合物理。

因此,世界模型不是凭“能生成”就取得系统权力。它必须证明自己能正确排序候选动作、对不可执行未来拒识、对失败不过度乐观,并且在真实闭环中产生稳定增益。

五、范式转移的可操作形态

系统层应保留的能力应新增的验收门槛
语义与任务层VLA/VLM 的指令理解、子任务分解、开放词汇指令置换敏感性、阶段判断、失败类型准确率
动态与评估层动作条件世界模型的未来预演、候选排序、失败检测action fidelity、physics adherence、failure optimism、sim-real 排序相关性
动作接口层连续 action expert、本体 adapter、状态条件解码不可达率、坐标系/频率一致性、接触负载、时延
真实执行层底层控制、传感反馈、急停与恢复闭环成功、滑移/掉落/过力、恢复率、接管次数、连续运行时间

对数据工程的直接含义是,从只收成功示教转向收集干预、失败附近、恢复、接触、力/触觉和几何对应。SKIP 说明 approach、contact、grasp、release 等稀疏关键事件不能被普通抽帧删除;TacForeSight 表明,如果目标包含扰动恢复,数据就必须显式包含正常示教和 recovery interaction,不能指望模型从成功轨迹自行猜出如何重建接触。

条件、分歧与可证伪性

第一,世界模型还未取得普遍的规划可采信性。Pri4R 仍将 4D point-track 监督能否扩大到更大预训练、是否需要在推理时显式引入几何观测留作未解问题。可变形物、强遮挡、精细摩擦与长时接触仍是高风险区。

第二,本文不主张删除 VLA。如果未来一个没有预测性训练目标、没有外部动态评估器的纯反应式 VLA,能在未见环境、跨本体、长时程和接触干扰任务上,以相同计算与真机数据预算,在动作忠实度、失败恢复和真实闭环成功率上稳定追平或超过融合系统,那么“反应式 VLA 已进入死胡同”的判断就被证伪。

结论

近一年论文没有宣告 VLA 寿终正寝。它们做的事更具体:拒绝把视觉语义、连续物理、本体控制和失败恢复压成一个无差别的 next-action 函数。“世界模型当立”的实质,是为机器人引入能被验证的后果预演层,让系统在行动前可以比较,在失败时可以归因,在接触中可以利用非视觉信号。最可能胜出的不是“世界模型 vs. VLA”,而是“带世界模型的 VLA 系统 vs. 只会反应的 VLA”。

References

Accepted evidence

去重论文引用

  1. $\texttt{WEAVER}$, Better, Faster, Longer: An Effective World Model for Robotic ManipulationArnav Kumar Jain, Yilin Wu, Jesse Farebrother, Gokul Swamy, Andrea Bajcsy · 2026
  2. $τ_0$-WM: A Unified Video-Action World Model for Robotic ManipulationPengfei Zhou, Shengcong Chen, Di Chen, Jiaxu Wang, Rongjun Jin, Bingwen Zhu, Yike Pan, Songen Gu et al. · 2026
  3. 3PoinTr: 3D Point Tracks for Learning Manipulation from Unconstrained Human VideosAdam Hung, Bardienus Pieter Duisterhof, Jeffrey Ichnowski · 2026
  4. ComSim: Building Scalable Real-World Robot Data Generation via Compositional SimulationYiran Qin, Jiahua Ma, Li Kang, Wenzhan Li, Yihang Jiao, Xin Wen, Xiufeng Song, Heng Zhou et al. · 2026
  5. Developing Vision-Language-Action Model from Egocentric VideosTomoya Yoshida, Shuhei Kurita, Taichi Nishimura, Shinsuke Mori · 2025
  6. Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot ManipulationYunfan Lou, Yifan Ye, Yankai Fu, Jun Cen, Xiaowei Chi, Yaoxu Lyu, Peidong Jia, Sirui Han et al. · 2026
  7. EgoVLA: Learning Vision-Language-Action Models from Egocentric Human VideosRuihan Yang, Qinxi Yu, Yecheng Wu, Rui Yan, Borui Li, An-Chieh Cheng, Xueyan Zou, Yunhao Fang et al. · 2025
  8. Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent PlanningChi-Pin Huang, Yunze Man, Zhiding Yu, Min-Hung Chen, Jan Kautz, Yu-Chiang Frank Wang, Fu-En Yang · 2026
  9. GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic ManipulationZijian Zhang, Yuqing Jiang, Qian Cheng, Xiaofan Li, Si Liu, Ding Zhao, Ping Luo, Weitao Zhou et al. · 2026
  10. GEM-4D: Geometry-Enhanced Video World Models for Robot ManipulationKaichen Zhou, Yuzhen Chen, Fangneng Zhan, Hang Hua, Grace Chen, Xinhai Chang, Ao Qu, Yilun Du et al. · 2026
  11. Hi-WM: Human-in-the-World-Model for Scalable Robot Post-TrainingYaxuan Li, Zhongyi Zhou, Yefei Chen, Yanjiang Guo, Jiaming Liu, Shanghang Zhang, Jianyu Chen, Yichen Zhu · 2026
  12. Interactive World Simulator for Robot Policy Training and EvaluationYixuan Wang, Rhythm Syed, Fangyu Wu, Mengchao Zhang, Aykut Onol, Jose Barreiros, Hooshang Nayyeri, Tony Dear et al. · 2026
  13. Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied SimulationMutian Xu, Tianbao Zhang, Tianqi Liu, Zhaoxi Chen, Xiaoguang Han, Ziwei Liu · 2026
  14. LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond MemorizationXueyang Zhou, Yangming Xu, Guiyao Tie, Yongchao Chen, Guowen Zhang, Duanfeng Chu, Pan Zhou, Lichao Sun · 2025
  15. MiraBench: Evaluating Action-Conditioned Reliability in Robotic World ModelsTianzhuo Yang, Zihan Shen, Zirui Mi, Zhaoyi Zhang, Jiayi Zhou, Jiaming Ji, Juntao Dai, Jiawei Chen et al. · 2026
  16. Predictive Spatio-Temporal Scene Graphs for Semi-Static ScenesMiguel Saavedra-Ruiz, Charlie Gauthier, Kumaraditya Gupta · 2026
  17. Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D RepresentationJisoo Kim, Jungbin Cho, Sanghyeok Chu · 2026
  18. ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action PoliciesHaodi Hu, Chung-Ta Huang, Jing Liu, Ye Wang, Kei Suzuki, Matthew Brand, Toshiaki Koike-Akino · 2026
  19. Revisiting Embodied Chain-of-Thought for Generalizable Robot ManipulationNan Sun, Yuan Zhang, Yongkun Yang, Wentao Zhao, Peiyan Li, Jun Guo, Wenxuan Song, Pengxiang Ding et al. · 2026
  20. RoboDream: Compositional World Models for Scalable Robot Data SynthesisJunjie Ye, Rong Xue, Basile Van Hoorick, Runhao Li, Harshitha Rajaprakash, Pavel Tokmakov, Muhammad Zubair Irshad, Vitor Guizilini et al. · 2026
  21. SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performanceTengyue Jiang, Chunpu Xu, Jiayue Kang, Yao Mu · 2026
  22. SANTS: A State-Adaptive Scheduler for World Action ModelsSANTS authors · 2026
  23. SKIP: Sparse Keyframe Interpolation Paradigm for Efficient Embodied World ModelsZiheng He, Yixiang Chen, Ning Yang, Zhanqian Wu, Qisen Ma, Yuan Xu, Jiabing Yang, Peiyan Li et al. · 2026
  24. SPACE: Enabling Learning from Cross-Robot Data Toward Generalist PoliciesHaeone Lee · 2026
  25. TacForeSight: Force-Guided Tactile World Model for Contact-Rich ManipulationYujie Zang, Yuhang Zheng, Xian Nie, Yupeng Zheng, Shuai Tian, Songen Gu, Chen Gao, Zining Wang et al. · 2026
  26. TACO: TActile World Model as a Self-COrrector forScalable VLA Post-TrainingShengbang Liu, Yueru Jia, Yuyang Yan, Jiaming Liu, Xinran Zhang, Qiuxuan Feng, Yandong Guo, Shiji Zhou et al. · 2026
  27. Vision-Language-Action Models: Experimental Insights from a Real-World UR5 PlatformMathilde Hochedel, Marc Lalonde · 2026
  28. World Pilot: Steering Vision-Language-Action Models with World-Action PriorsZefu Lin, Rongxu Cui, Junjia Xu, Xiaojuan Jin, Wenling Li, Lue Fan, Zhaoxiang Zhang · 2026
完整证据附录(28 条正式证据事件)
  • Time range: 2025-07-15..2026-07-17
  • Events: 28
  • 每个事件一节,标题即锚点;trace-map 中的 event 链接跳转到这里。

EA-WMDATA-READ-0007

  • Claim: A robot world model can be trained from a moderate-sized robot interaction dataset and then used to generate policy-training demonstrations, but its value depends on interaction-consistent long-horizon rollouts and sim-real correlation.
  • Stance: support | Confidence: direct
  • Paper: 2603.08546 Interactive World Simulator for Robot Policy Training and Evaluation
  • Locator: IV-C Data Generation for Policy Training
  • Evidence: The paper builds an Interactive World Simulator from a moderate-sized robot interaction dataset, reports world-model-generated policy data comparable to the same amount of real-world data, and evaluates sim-real performance correlation.
  • Quote: “Notably, policies trained on 100% world simulator data perform comparably to those trained on an equivalent volume of real-robot expert data. This suggests that our simulator can generate data with quality similar to that of real-world demonstrations.”
  • Authors: yixuan-wang; rhythm-syed; fangyu-wu; et al.

EA-WMDATA-READ-0009

  • Claim: World-model training and post-training data should include dense corrective trajectories around failure-prone states, not only successful demonstrations.
  • Stance: support | Confidence: direct
  • Paper: 2604.21741 Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training
  • Locator: 3.5 Collecting Corrective Trajectories for Post-Training
  • Evidence: Hi-WM rolls policies inside a world model, lets humans intervene when rollouts become incorrect or failure-prone, caches and branches failure states, and adds corrective trajectories back into the training set for post-training.
  • Quote: “Within Hi-WM, post-training data are collected in a closed-loop inside the interactive world model. The pre-trained policy first runs in the world model from the current observation and generates a rollout. When the rollout enters unfamiliar or failure-prone states, a human operator intervenes through the hardware-agnostic interface and provides corrective actions. The world model then continues the rollout from the current state using these human actions. Once the rollout has been guided back t”
  • Authors: yaxuan-li; zhongyi-zhou; yefei-chen; et al.

EA-WMDATA-READ-0008

  • Claim: Robot videos can be converted into richer world-model supervision by pairing RGB with depth and pseudo 3D scene-flow targets, training models to represent current 3D structure and short-horizon future evolution rather than only behavior-cloning actions.
  • Stance: support | Confidence: direct
  • Paper: 2605.20752 GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic Manipulation
  • Locator: 3.4 GaussianDream Training and Efficient Inference
  • Evidence: GaussianDream trains current Gaussian reconstruction and future Gaussian prediction heads with RGB rendering, depth, and pseudo 3D scene-flow supervision, then retains only a compact prefix for control at inference.
  • Quote: “GaussianDream follows an asymmetric strategy: dense Gaussian reconstruction and prediction supervise training, while only the compact prefix is retained for online control. Stage I: GaussianDream pretraining. We first train the reconstruction and prediction heads without action learning. For each demonstration sequence, RGB frames are paired with pseudo depth and pseudo 3D scene-flow targets constructed from adjacent frames. The GaussianDream objective combines current reconstruction and future”
  • Authors: zijian-zhang; yuqing-jiang; qian-cheng; et al.

EA-4D-READ-0013

  • Claim: Dream-Tac将动作块、未来视觉和未来触觉放进同一世界动作建模目标,以弥补单纯视觉未来对接触互动线索的不足。
  • Stance: support | Confidence: direct
  • Paper: 2606.08737 Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
  • Locator: 3.1. Problem Formulation
  • Evidence: 问题建模段先定义动作与视觉未来的联合分布,再明确把未来触觉纳入联合预测目标。
  • Quote: “Building on these two formulations, a world action model combines action prediction and future observation prediction into a unified framework. Specifically, it jointly models (3) or equivalently factorizes the joint distribution as (4) where future visual prediction provides predictive structure for action generation. However, in contact-rich manipulation, vision alone is often insufficient to capture physical interaction cues. To address this limitation, we introduce Dream-Tac, an enhanced wor”
  • Authors: yunfan-lou; yifan-ye; yankai-fu; et al.

EA-4D-READ-0012

  • Claim: 3PoinTr 保留含暂时遮挡点的轨迹,只屏蔽不可见的单个点—时间损失,因而能继续利用操作中任务关键的物体点监督。
  • Stance: conditional | Confidence: direct
  • Paper: 2603.08485 3PoinTr: 3D Point Tracks for Learning Manipulation from Unconstrained Human Videos
  • Locator: 4.3 Results: 3D Point Track Prediction
  • Evidence: 结果段对比了删除整条不可见轨迹的基线与仅屏蔽不可见 point-timestep 损失的 3PoinTr。
  • Quote: “The primary advantage of 3PoinTr is that it trains on data General Flow ignores. Real-world points are often temporarily occluded; General Flow removes any trajectory with invisible point-timestep pairs during preprocessing, whereas 3PoinTr retains all trajectories and masks losses for individual invisible point-timestep pairs. This provides additional supervision over task-critical object points that are temporarily occluded during manipulation. For example, in the Throw Away Paper task, every”
  • Authors: adam-hung; bardienus-pieter-duisterhof; jeffrey-ichnowski

EA-4D-READ-0011

  • Claim: 如果目标包含扰动恢复,数据集必须显式收集nominal demonstrations和recovery interaction data;否则模型很难学习接触丢失后的再建立。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.11184 TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
  • Locator: IV-B 2 Perturbation-Aware Evaluation
  • Evidence: TacForeSight在perturbation-aware设置中额外收集恢复示教,让外部扰动发生在执行中并要求示教者重新建立稳定接触;完整模型在三个扰动任务上报告90%、85%、85%平均完成/成功分数。
  • Quote: “Policies in this setting are trained using both nominal demonstrations and recovery interaction data.”
  • Authors: yujie-zang; yuhang-zheng; xian-nie; et al.

EA-EGO-2026-0001

  • Claim: EgoVLA 的人类第一视角预训练不能直接消除本体差距:没有机器人微调时,仿真 humanoid 的全部任务成功率为 0%。
  • Stance: limit | Confidence: direct
  • Paper: 2507.12440 EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos
  • Locator: 5.2 Humanoid Robot Evaluation
  • Evidence: 论文把失败归因于外观、感知和运动学错配,并用同一 benchmark 的零样本结果明确展示。
  • Quote: “zero-shot deployment on humanoid robots without fine-tuning on robot data results in 0% success across all tasks”
  • Authors: ruihan-yang; qinxi-yu; yecheng-wu; et al.

EA-EGO-2026-0003

  • Claim: 从 egocentric 视频恢复的 object trajectory 不是完整机器人动作:该路线无法获得 gripper state,只能把动作表示为 9D 物体位姿增量。
  • Stance: limit | Confidence: direct
  • Paper: 2509.21986 Developing Vision-Language-Action Model from Egocentric Videos
  • Locator: III-C Policy Training
  • Evidence: 策略训练段明确说明 gripper state 缺失,并以 object pose displacement 作为替代动作。
  • Quote: “Because gripper states cannot be obtained from Section III-B , each action is represented by a 9-dimensional vector”
  • Authors: tomoya-yoshida; shuhei-kurita; taichi-nishimura; et al.

EA-CONTAM-2026-0007

  • Claim: LIBERO 标准协议中训练与评测任务过度接近,会让记忆固定布局与动作映射的 VLA 获得过度乐观的泛化结论。
  • Stance: limit | Confidence: direct
  • Paper: 2510.03827 LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization
  • Locator: 5.2 Main Results
  • Evidence: LIBERO-PRO 在保持逻辑可执行的前提下改变物体位置与任务,标准设置中的高分模型在这些轻微改变下近乎崩溃。
  • Quote: “Despite achieving success rates above 90% on the standard LIBERO benchmark, models nearly collapse under changes to object positions or minor task modifications, even when constructed from training components.”
  • Authors: xueyang-zhou; yangming-xu; guiyao-tie; et al.

EA-WMEVAL-READ-0007

  • Claim: Kinema4D argues that robot-world interaction should be simulated as a 4D event: robot control is represented with kinematically correct 4D trajectories, while a generative model predicts environment reactions.
  • Stance: support | Confidence: direct
  • Paper: 2603.16669 Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulation
  • Locator: Abstract (full-text section)
  • Evidence: The method disentangles precise robot control from generative environmental reaction by driving a URDF robot through kinematics, projecting a 4D robot pointmap sequence, and jointly generating synchronized RGB/pointmap futures.
  • Quote: “Abstract Simulating robot-world interactions is a cornerstone of Embodied AI. Recently, a few works have shown promise in leveraging video generations to transcend the rigid visual/physical constraints of traditional simulators. However, they primarily operate in 2D space or are guided by static environmental cues, ignoring the fundamental reality that robot-world interactions are inherently 4D spatiotemporal events that require precise interactive modeling. To restore this 4D essence while ensu”
  • Authors: mutian-xu; tianbao-zhang; tianqi-liu; et al.

EA-WMEVAL-READ-0005

  • Claim: GEM-4D supports geometry-feature distillation as a way to make video world models more actionable for robot manipulation without adding inference-time cost.
  • Stance: support | Confidence: direct
  • Paper: 2605.22882 GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: The model distills 4D geometry foundation-model representations into a video backbone during training, discards the geometry branch at inference, and uses an inverse dynamics module to convert generated rollouts into executable trajectories; the paper reports real-world manipulation success improving from 61% to 81%.
  • Quote: “Abstract Video world models can generate realistic futures from a single instruction, but they often fail to track the same physical points consistently across time. As a result, the generated videos appear plausible, yet lack the physical grounding required for reliable action execution, such as robot manipulation. We present GEM-4D , a geometry-grounded video world model that resolves this limitation by injecting dense 4D correspondence supervision distilled from a pretrained geometry foundati”
  • Authors: kaichen-zhou; yuzhen-chen; fangneng-zhan; et al.

EA-WMEVAL-READ-0003

  • Claim: Training data for efficient embodied world-model rollouts must preserve sparse task-relevant events such as approach, contact, grasp, and release; generic frame dropping can remove the information downstream policies need.
  • Stance: support | Confidence: direct
  • Paper: 2606.00664 SKIP: Sparse Keyframe Interpolation Paradigm for Efficient Embodied World Models
  • Locator: Abstract (full-text section)
  • Evidence: SKIP argues that manipulation rollouts concentrate task-relevant information in sparse events, selects event-preserving keyframes through robot-aware multimodal fusion, and reports that generated videos can serve as policy-training data.
  • Quote: “Abstract Embodied world models have emerged as a promising paradigm in robotics by predicting how robot actions affect the surrounding scene. However, the rollout inference remains computationally expensive in pixel space, as long-horizon manipulation videos typically have to be generated frame by frame. This cost cannot be easily reduced by indiscriminately dropping frames, since downstream policies rely on complete preservation of sparse task-relevant events such as approach, contact, grasp, a”
  • Authors: ziheng-he; yixiang-chen; ning-yang; et al.

EA-WMEVAL-READ-0001

  • Claim: τ0-WM 使用真实机器人遥操作、UMI 式交互、人类第一视角视频与 rollout/失败轨迹的异构语料,并用按模态的监督掩码训练统一视频—动作世界模型。
  • Stance: support | Confidence: direct
  • Paper: 2606.01027 $τ_0$-WM: A Unified Video-Action World Model for Robotic Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 摘要直接报告了异构数据组成与 modality-specific supervision masks。
  • Quote: “Abstract Robotic manipulation requires models that generate executable actions while anticipating and evaluating their future consequences before physical execution. We present -World Model ( -WM), a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone, -WM provides two complementary interfaces. First, a video action model jointly predicts future visual lat”
  • Authors: pengfei-zhou; shengcong-chen; di-chen; et al.

EA-WMEVAL-READ-0010

  • Claim: WEAVER defines useful robot world models by three joint requirements: fidelity to real outcomes, temporal consistency over long horizons, and enough efficiency for evaluation, improvement, and planning.
  • Stance: support | Confidence: direct
  • Paper: 2606.13672 $\texttt{WEAVER}$, Better, Faster, Longer: An Effective World Model for Robotic Manipulation
  • Locator: 3 WEAVER : World Estimation Across Views for Embodied Reasoning
  • Evidence: The paper argues that manipulation world models must satisfy fidelity, consistency, and efficiency together, then designs a multi-view latent world model with reward/value prediction to support policy evaluation, synthetic policy improvement, and test-time planning.
  • Quote: “Figure 2 : WEAVER Architecture. Left: The world model encodes memory, history, and action sequences to image future rollouts in latent space. Middle: The latent verifier, equipped with reward and critic heads, selects samples with high advantage to steer the policy distribution. Right: Decoded generation corresponding to different outcomes of action sequences. We now describe the key ingredients in WEAVER : a robot world model designed to support policy evaluation, policy improvement, and test-t”
  • Authors: arnav-kumar-jain; yilin-wu; jesse-farebrother; et al.

EA-WMEVAL-READ-0014

  • Claim: A scalable world-model data pipeline can use a small amount of real-world data to align classical simulation with neural simulation, generating action-video pairs that preserve real-world consistency and broaden scenario coverage.
  • Stance: conditional | Confidence: direct
  • Paper: 2604.11386 ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation
  • Locator: Abstract (full-text section)
  • Evidence: ComSim proposes a real-sim-real data augmentation pipeline: collect a small real trajectory set, align classical simulation to the real platform, transform simulation videos into real-world representations, and generate large-scale action-video training datasets.
  • Quote: “Abstract Recent advancements in foundational models, such as large language models and world models, have greatly enhanced the capabilities of robotics, enabling robots to autonomously perform complex tasks. However, acquiring large-scale, high-quality training data for robotics remains a challenge, as it often requires substantial manual effort and is limited in its coverage of diverse real-world environments. To address this, we propose a novel hybrid approach called Compositional Simulation ,”
  • Authors: yiran-qin; jiahua-ma; li-kang; et al.

EA-WMEVAL-READ-0011

  • Claim: Synthetic world-model data for robot learning needs embodiment anchoring: rendered robot motion plus explicit scene and object priors can synthesize demonstrations in novel objects, scenes, and viewpoints while reducing teleoperation burden.
  • Stance: conditional | Confidence: direct
  • Paper: 2606.02577 RoboDream: Compositional World Models for Scalable Robot Data Synthesis
  • Locator: Abstract (full-text section)
  • Evidence: RoboDream anchors generation to rendered robot motion, conditions on scene/object priors, and introduces retrieval-and-rebirth plus prop-free teleoperation to generate demonstrations and reduce real data collection cost.
  • Quote: “Abstract Scaling robot learning requires large-scale, diverse demonstrations, yet real-world data collection via teleoperation remains prohibitively expensive and time-consuming. While video diffusion models offer a promising avenue for data scaling, existing generative approaches are often limited to superficial visual augmentation, or suffer from embodiment hallucinations that yield physically infeasible motions. We present a generalizable embodiment-centric world model that achieves scalable”
  • Authors: junjie-ye; rong-xue; basile-van-hoorick; et al.

EA-WMEVAL-READ-0008

  • Claim: PredictiveGraphs is bounded by simplified independence assumptions, perception ambiguity among similar objects, and possible LLM hallucinations during verification and planning.
  • Stance: limit | Confidence: direct
  • Paper: 2605.00121 Predictive Spatio-Temporal Scene Graphs for Semi-Static Scenes
  • Locator: VIII Limitations
  • Evidence: The limitations section says object-receptacle edges are modeled independently, indistinguishable objects are treated as interchangeable, and LLM hallucinations remain a risk for open-vocabulary verification and planning.
  • Quote: “While embedding open-vocabulary scene graphs with persistence estimators yields tempo-spatio-semantic capabilities, our approach is not free of limitations. First, object-receptacle edges are modelled independently. While joint distributions over multiple locations are possible [ 30 ] , inference costs grow with the number of observed locations per object, inducing a trade-off between expressivity and real-time performance. We believe that striking the right balance in this trade-off is a promis”
  • Authors: miguel-saavedra-ruiz; charlie-gauthier; kumaraditya-gupta

EA-WMEVAL-READ-0015

  • Claim: World-model training and post-training objectives should be tied to downstream action quality rather than intermediate video fidelity, because later denoising can become less action-relevant or physically unreliable.
  • Stance: limit | Confidence: direct
  • Paper: 2605.27947 SANTS: A State-Adaptive Scheduler for World Action Models
  • Locator: Abstract (full-text section)
  • Evidence: SANTS reports that fully denoised video is not always the best action condition, trains a scheduler with a path-level reward after action generation, and explicitly optimizes downstream action quality rather than video fidelity.
  • Quote: “Abstract World Action Models (WAMs) improve robot manipulation by using video-based future representations to condition action generation. In pixel-space WAMs, however, the best action condition is not necessarily the fully denoised video. Controlled denoising-depth scans show that video refinement can reduce action error up to a state-dependent point, after which the gain may saturate or even reverse when late predictions become less action-relevant or physically unreliable. This suggests that”
  • Authors: sants-authors

EA-WMEVAL-READ-0013

  • Claim: Static image-text pretraining is insufficient training signal for contact-rich manipulation; world-model data should expose action-conditioned scene evolution and contact dynamics.
  • Stance: limit | Confidence: direct
  • Paper: 2606.12403 World Pilot: Steering Vision-Language-Action Models with World-Action Priors
  • Locator: Abstract (full-text section)
  • Evidence: World Pilot argues that VLA semantic grounding from static image-text pairs cannot capture continuous contact-rich dynamics, and uses WAM-derived scene-evolution and trajectory priors to complement the policy.
  • Quote: “Abstract Vision-Language-Action (VLA) models inherit semantic grounding from large-scale pretraining and perform competently across in-distribution manipulation tasks. This grounding, however, is built on static image-text pairs, whereas manipulation is a continuous, contact-rich process whose dynamics this pretraining cannot capture. We present World Pilot, a VLA framework that augments the policy with priors from a World-Action Model (WAM), routed into the decision chain through two complement”
  • Authors: zefu-lin; rongxu-cui; junjia-xu; et al.

EA-WMEVAL-READ-0006

  • Claim: Pri4R leaves open whether 4D point-track supervision should be used at larger pretraining scale or with explicit test-time geometric inputs.
  • Stance: gap | Confidence: direct
  • Paper: 2603.01549 Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D Representation
  • Locator: VI Discussion, Limitations, and Future Work
  • Evidence: The conclusion says Pri4R was evaluated mainly as fine-tuning on demonstrations and small real-world rollouts, and suggests that pretraining-scale 3D point-track supervision or explicit test-time computation could further improve robustness.
  • Quote: “We presented Pri4R, a framework that enhances the world dynamics understanding of VLA models through privileged 4D representations. By supervising the model to predict 3D point tracks during training, we demonstrated that VLA backbones can develop a more physically-aware context, leading to improved control performance without any inference-time overhead. Our results across various benchmarks suggest that capturing the spatiotemporal evolution of a scene is a critical component for robust robot”
  • Authors: jisoo-kim; jungbin-cho; sanghyeok-chu

EA-WMEVAL-READ-0004

  • Claim: Robotic world-model evaluation should move beyond visual fidelity toward action-conditioned reliability, including physical adherence, action-following fidelity, and optimism-bias detection.
  • Stance: gap | Confidence: direct
  • Paper: 2605.29360 MiraBench: Evaluating Action-Conditioned Reliability in Robotic World Models
  • Locator: Abstract (full-text section)
  • Evidence: The paper frames existing evaluations as weak evidence for whether action-conditioned predictions are reliable, then defines MiraBench around physics adherence, action fidelity, and failure-case optimism bias.
  • Quote: “Abstract Action-conditioned world models are increasingly used as scalable simulators for robot learning, yet current evaluations provide limited evidence that their predictions are reliable under the actions they condition on. Existing benchmarks largely emphasize visual fidelity, leaving unclear whether predicted futures are physically plausible, faithful to commanded actions, and calibrated to failure when actions should not succeed. We introduce MiraBench , a hierarchical benchmark that defi”
  • Authors: tianzhuo-yang; zihan-shen; zirui-mi; et al.

EA-ALIGN-READ-0013

  • Claim: 长程规划、失败后自我纠正、少样本适应属于认知层能力;显式 CoT 能提升它们但推理延迟高,认知处理可以压缩为 latent 推理而保持规划与恢复能力。
  • Stance: support | Confidence: direct
  • Paper: 2601.09708 Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planning
  • Locator: 5 Conclusion
  • Evidence: 论文指出 VLA 靠动作监督擅长基本技能,但在长程规划、失败自我纠正、新场景适应上泛化差;Fast-ThinkAct 用 preference-guided 蒸馏把冗长文本推理压缩为紧凑 latent CoT,在保持 long-horizon planning、few-shot adaptation 和 failure recovery 的同时推理延迟最多降 89.3%。
  • Quote: “By distilling lengthy textual reasoning into compact latent representations via preference-guided distillation and visual trajectory alignment, our approach bridges high-level embodied reasoning with low-level action execution through reasoning-enhanced policy learning. Extensive experiments across diverse robotic manipulation and embodied reasoning benchmarks demonstrate that Fast-ThinkAct achieves strong performance with significantly reduced inference latency while enabling effective long-hor”
  • Authors: chi-pin-huang; yunze-man; zhiding-yu; et al.

EA-ALIGN-READ-0015

  • Claim: 失败恢复可以把认知层(failure mode/recovery stage 判断与 reward 选择)与控制层(residual 纠正)分开;VLM 的失败分类错误由此成为与感知误差独立的认知误差来源。
  • Stance: support | Confidence: direct
  • Paper: 2606.09630 ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action Policies
  • Locator: 1 Introduction
  • Evidence: ReCoVLA 用外部 VLM 只推断 failure type、recovery stage、active entities、confidence 和 reward mask,不直接生成动作;确定性 reward compiler 做实体 grounding 与 stage gates,residual policy 在冻结 VLA latents 上学纠正。Limitations 明确列出 VLM failure-classification mistakes 与 perception errors、sim-to-real mismatch 并列为失败来源。
  • Quote: “Instead, it produces a structured recovery descriptor containing the failure type, recovery stage, active entities, confidence, and reward mask.”
  • Authors: haodi-hu; chung-ta-huang; jing-liu; et al.

EA-ALIGN-READ-0006

  • Claim: ERVLA 的大规模实验表明,有效的具身 CoT 需要落到末端执行器运动或图像空间轨迹等动作相关表示,而将显式 CoT 作为自回归动作前缀会在推理时累积误差。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.03784 Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 摘要同时给出了动作相关 grounding 的有效性与 autoregressive action prefix 的 compounding-error 限制。
  • Quote: “Abstract Embodied chain-of-thought (CoT) aims to bridge linguistic reasoning with robotic control, yet its effective form and integration remain underexplored. In this paper, we revisit embodied CoT for robotic control at an unprecedented scale. We curate the largest embodied CoT corpus to date, comprising 978,743 trajectories, 226.3M samples, and 2592.5 hours of data. Through extensive experiments, we show that effective CoT must ground high-level semantic understaning in concrete linguistic ac”
  • Authors: nan-sun; yuan-zhang; yongkun-yang; et al.

EA-ALIGN-READ-0001

  • Claim: A recorded robot action is not a universal supervision signal: the same command can produce different motions across controllers, embodiments, hardware units, and deployment-time dynamics.
  • Stance: limit | Confidence: direct
  • Paper: 2606.24049 SPACE: Enabling Learning from Cross-Robot Data Toward Generalist Policies
  • Locator: 3.2 Inconsistency of Control Commands across Robots
  • Evidence: SPACE predicts Cartesian state deltas as a shared end-effector-space representation and uses an action adapter to convert them into robot-specific control commands, improving cross-robot and dynamics-shift robustness.
  • Quote: “Recent work has scaled robot learning by training policies on data from multiple embodiments [ 27 , 23 , 32 ] , often using the Cartesian delta action space [ 23 , 32 ] since it is less dependent on robot-specific kinematics and invariant to base-frame translation [ 18 , 14 ] . In practice, this is typically realized by predicting Cartesian delta control commands that are fed to the underlying robot controller [ 23 , 32 ] . Figure 2: Different robots (e.g., UR5 vs. Franka Research 3) require dif”
  • Authors: haeone-lee

EA-ALIGN-READ-0003

  • Claim: Discrete action tokenization is a compact interface for autoregressive VLA policies, but decoding fixed tokens back into continuous robot controls is a bottleneck when the same token must mean different controls under different robot states and contacts.
  • Stance: limit | Confidence: direct
  • Paper: 2606.30113 SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performance
  • Locator: Abstract (full-text section)
  • Evidence: SA-VLA conditions action-token decoding on proprioceptive state via adapters or cross-attention, reporting improved RoboTwin and zero-shot sim-to-real success over tokenizer baselines.
  • Quote: “Abstract Discrete action tokenization provides a compact interface for autoregressive VLA policies, but accurately recovering continuous robot actions from discrete codes remains challenging. Existing tokenizers typically map each discrete code to a fixed continuous action prototype, ignoring the robot’s current proprioceptive state. This limitation is particularly pronounced in manipulation, where the same action token may require different continuous controls under different joint configuratio”
  • Authors: tengyue-jiang; chunpu-xu; jiayue-kang; et al.

EA-ALIGN-READ-0004

  • Claim: Offline VLA indicators can fail to transfer to stable real-robot behavior when action semantics, coordinate frames, temporal modality alignment, image preprocessing, and dataset coverage are not controlled together.
  • Stance: limit | Confidence: direct
  • Paper: 2606.30456 Vision-Language-Action Models: Experimental Insights from a Real-World UR5 Platform
  • Locator: Abstract (full-text section)
  • Evidence: The UR5 study reports a gap between offline indicators and unstable closed-loop physical behavior, attributing it to data-model-control pipeline consistency rather than model capacity alone.
  • Quote: “Instead, it is strongly influenced by a combination of factors, including action semantics, coordinate frame conventions, temporal alignment between modalities, image preprocessing consistency, and dataset coverage and quality. These observations lead to a key interpretation: the successful deployment of VLA systems in real-world settings depends less on incremental improvements in model capacity and more on precise control of the entire data–model–control pipeline.”
  • Authors: mathilde-hochedel; marc-lalonde

EA-ALIGN-READ-0009

  • Claim: TACO 用触觉感知世界模型联合预测未来视频和力序列,再将真实失败附近状态转成带纠正动作的想象片段,用于接触丰富任务的 VLA 后训练。
  • Stance: limit | Confidence: direct
  • Paper: 2607.02840 TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training
  • Locator: 5 Conclusion and Limitations
  • Evidence: 结论的 Recognize–Imagine–Label 回路明确连接了真实失败、视频—力联合想象与纠正动作标注。
  • Quote: “We presented TACO, a tactile-aware world-model-driven framework for scalable VLA post-training in contact-rich manipulation. Following a Recognize–Imagine–Label loop, TACO converts real-world failures into imagined corrections without repeated human intervention: a tactile-aware world model jointly denoises future video and force sequences, while a unified progress-action model recognizes failure-adjacent states and labels imagined segments with corrective actions. To incorporate this supervisio”
  • Authors: shengbang-liu; yueru-jia; yuyang-yan; et al.

References

Interactive Wiki

进入交互阅读

静态页采用默认版本;v0、v0、v0可在同一交互 Wiki 中阅读,不建立相互竞争的索引页。

Method and limits

研究方法、证据边界与引用说明

本页由当前结算的证据层自动派生,不改写已结算 run。论文只有在完整非 OCR 全文、结构化论文笔记和 claim-support audit 通过后,才进入接受证据。

页面结论是对所列论文与研究窗口的综合,不等同于无限外推。引用本专题时,请同时保留本页 canonical 地址、访问日期,并在需要时回溯原论文。