VLA 与模型 · VLA and Foundation Models

近半年VLA在具身领域最大的技术突破

Recent Breakthroughs in VLA Models

更新于

30 秒结论

先读这个判断

本备忘录考察 2026 年 1 月 19 日至 7 月 19 日的具身 VLA 研究,问题不是“哪个模型的单项分数最高”,而是哪种技术变化最可能改写未来的系统形态。本轮纳入 27 篇通过完整全文阅读和主张核验的论文,覆盖 VLA、世界模型、4D/几何表征、动作接口、失败恢复与闭环评测。这些证据足以识别趋势和机制,但不足以声称已解决通用家庭机器人、多本体迁移或安全部署。

精读论文
27
正式证据事件
31
知识单元
EA-MODEL, EA-EVAL, EA-4D, EA-ALIGN
审计状态
当前结算 run 已通过仓库校验

v0 · 完整正文

研究综述

研究边界

本备忘录考察 2026 年 1 月 19 日至 7 月 19 日的具身 VLA 研究,问题不是“哪个模型的单项分数最高”,而是哪种技术变化最可能改写未来的系统形态。本轮纳入 27 篇通过完整全文阅读和主张核验的论文,覆盖 VLA、世界模型、4D/几何表征、动作接口、失败恢复与闭环评测。这些证据足以识别趋势和机制,但不足以声称已解决通用家庭机器人、多本体迁移或安全部署。

中心判断

近半年 VLA 最大的技术突破,不是更大的 VLM 骨干、更精细的 action tokenizer,也不是单纯把视频生成器接到策略前面;而是 VLA 开始从“看到当前画面就直接输出动作”,转向“提出动作、表示关键后果、校验可执行性、再闭环执行与纠错”。更准确地说,突破是一个“VLA 语义/动作先验 + 结构化或世界模型后果预演 + 本体控制器”的融合栈。

这一判断是跨论文推断,不是某篇论文的口号。它可被反驳:如果在相同数据、计算和真机试验预算下,反应式 VLA 在长时程、扰动恢复和未见场景中与带后果预演的系统无显著差异,或者世界模型无法提高真实结果排序和闭环成功率,那么这个范式转移就不成立。

突破一:世界模型不再追求“更长的视频”,而是预测“更少但更可执行的中间状态”

早期的直觉是:只要生成足够逼真的未来视频,机器人就能“先想后做”。问题是,像素中大量信息对动作没有价值,长视频还会积累几何和接触误差。近期方法的关键转变,是把未来压缩成任务进度真正需要的状态。

StructVLA 从夹爪开合和末端运动转折点中抽取稀疏“结构化帧”,先学习这些物理里程碑,再将其迁移到低层动作生成。其 LIBERO 平均成功率为 94.8%;在 Franka 实机长时程 tidy-up 中完成 8/10,对照 UniVLA 为 4/10。这个结果的价值不在于某个榜单提升,而在于它表明:后果表征一旦与运动阶段对齐,长历史反而能帮助策略,而不只是引入更多噪声。

H-WM 走了另一条路:高层用符号逻辑保持任务顺序,低层用潜在视觉子目标提供感知 grounding,再由 VLA 高频执行。在五个 5–7 步 LIBERO-LoHo 任务上,双层引导的平均成功率为 64.8%,比仅逻辑引导高 16.4 个百分点,也高于像素级生成子目标。两篇论文的共识是:一个好的“未来”不必是完整视频,它应当是可验证的进度、几何或逻辑状态。

与此相呼应,Pri4R 和 GEM-4D 把 4D 点轨迹或几何特征作为训练期的特权教师/蒸馏目标,试图保留物体身份、跨帧对应和动作相关动态,同时避免部署时为显式几何支付过高延迟。这进一步说明,本轮进展的实质是从“视觉上像未来”转向“在几何和动作上是未来”。

突破二:训练目标从模仿成功轨迹,转向预演失败与学习恢复

反应式 VLA 通常从成功示教中学习“这时候专家会怎么动”。这使它在离线评测上看起来不错,却不知道偏离示教走廊后如何回来。世界模型真正改变的是训练问题:不再只预测专家动作,而是在模型内部制造候选未来、找到容易失败的状态,并把纠正轨迹加回后训练。

Hi-WM 让人在世界模型 rollout 中介入,在错误或高风险状态处分支并采集纠正轨迹;ReCoVLA 把恢复拆成认知层的失败类型/恢复阶段判断,以及控制层的 residual 纠正。对接触任务,纯视觉未来仍然缺少力学信息:TACO 和 Dream-Tac 把未来视觉、力/触觉与动作联合建模,再用想象的失败附近片段做 VLA 后训练。

这类方法的判别标准不应是“生成的视频多逼真”,而是在相同真机数据和计算预算下,是否提高扰动后恢复率、减少人工接管、降低失败成本。如果没有真实恢复数据或目标机器人锚定,想象轨迹也可能只是漂亮的离线扩增。

突破三:评测从“未来看起来合理”转向“未来与动作是否同步”

世界模型一旦进入决策环,评测就不能再停留在视频清晰度。WEAVER 将可用世界模型概括为三个同时条件:对真实结果的保真、长时程一致和足够低的计算代价;MiraBench 进一步要求 action-following fidelity、物理遵循和失败过度乐观检测。换句话说,世界模型取得“规划权”之前,必须证明它对候选动作的排序与真实闭环结果相关。

最强的反证来自 BadWAM。在完整 LIBERO 闭环扫描中,它的黑盒动作攻击将高成功率 WAM 从 96.5% 降到 43.1%。更值得警惕的是,攻击可以让预测未来仍接近干净预测,同时把真正执行的 action chunk 推向失败。因此,“模型想象的画面没问题”不能作为安全通行证;系统必须检查想象、动作、控制器响应和真实观测之间的闭环同步。

为什么不能简化为“世界模型替代 VLA”

第一,VLA 仍然是把语言任务、视觉状态和动作先验连接起来的核心。世界模型擅长表示“做了什么会发生什么”,却不会自动解决本体、坐标系、控制频率和接触负载不一致。SPACE 表明同一条记录命令在不同控制器、硬件单元和部署动力学下可能对应不同运动。ERVLA 也显示,有价值的具身推理必须落到末端运动或图像空间轨迹;纯文本推理前缀可能只会增加延迟和误差累积。

第二,世界模型还没有通过广泛真机的 admissibility 验收。当前大部分强结果仍来自 LIBERO、SimplerEnv 或少量桌面夹爪任务。H-WM 要求任务能被符号化,StructVLA 的里程碑主要依赖夹爪转换和末端速度,BadWAM 证明了安全攻击面而不是真实攻击发生率。这些都不支持把单一世界模型直接当作上线裁决器。

对研发的可操作框架

系统层应该学什么最小验收问题
VLA/语义层任务、子目标、动作先验更换指令或任务阶段后,动作是否相应改变?
后果预演层关键进度、几何、接触与失败后果能否拒绝不可执行未来,并正确排序候选动作?
动作适配层共享状态变化到本体命令的映射坐标系、频率、延迟和接触负载改变时是否仍可执行?
底层控制/恢复层跟踪、力控、residual 纠正扰动后能否在安全范围内重建接触并回到任务轨迹?

真正有判别力的实验,应该在相同数据、动作候选、控制器和计算预算下,直接对比有无后果预演层;除了成功率,还应记录恢复率、过力/碰撞、人工接管、候选排序与真实结果的相关性、以及 rollout 延迟。

研究空白与下一步

第一,需要在统一预算下直接比较纯文本 CoT、潜在规划、结构化帧、显式 4D 和视频 rollout,而不是各自使用不同基础模型和数据。第二,需要从夹爪桌面操作扩展到灵巧手、可变形物、移动操作和长时运行,纳入力/触觉、不可逆接触窗口与失败恢复。第三,安全评测必须从单独的未来质量扩展到“未来—动作—控制—真实观测”同步性,并在真机上验证误报和恢复代价。

结论

如果必须用一句话概括:近半年 VLA 最大的技术突破,是它开始拥有“先表示可执行后果,再用闭环证据校正动作”的系统接口。这不意味着 VLA 被世界模型取代,而是意味着反应式 VLA 被降格为融合栈中的语义和动作先验模块。当前证据已足以说明这是最重要的方向变化;但只有当后果预演通过动作忠实、真实结果排序、延迟和安全同步性验收时,它才能被称为成熟的部署突破。

References

Accepted evidence

去重论文引用

  1. $\texttt{WEAVER}$, Better, Faster, Longer: An Effective World Model for Robotic ManipulationArnav Kumar Jain, Yilin Wu, Jesse Farebrother, Gokul Swamy, Andrea Bajcsy · 2026
  2. $τ_0$-WM: A Unified Video-Action World Model for Robotic ManipulationPengfei Zhou, Shengcong Chen, Di Chen, Jiaxu Wang, Rongjun Jin, Bingwen Zhu, Yike Pan, Songen Gu et al. · 2026
  3. 3PoinTr: 3D Point Tracks for Learning Manipulation from Unconstrained Human VideosAdam Hung, Bardienus Pieter Duisterhof, Jeffrey Ichnowski · 2026
  4. BadWAM: When World-Action Models Dream Right but Act WrongQi Li, Xingyi Yang, Xinchao Wang · 2026
  5. Beyond Dense Futures: World Models as Structured Planners for Robotic ManipulationMinghao Jin, Mozheng Liao, Mingfei Han, Zhihui Li, Xiaojun Chang · 2026
  6. ComSim: Building Scalable Real-World Robot Data Generation via Compositional SimulationYiran Qin, Jiahua Ma, Li Kang, Wenzhan Li, Yihang Jiao, Xin Wen, Xiufeng Song, Heng Zhou et al. · 2026
  7. Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot ManipulationYunfan Lou, Yifan Ye, Yankai Fu, Jun Cen, Xiaowei Chi, Yaoxu Lyu, Peidong Jia, Sirui Han et al. · 2026
  8. GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic ManipulationZijian Zhang, Yuqing Jiang, Qian Cheng, Xiaofan Li, Si Liu, Ding Zhao, Ping Luo, Weitao Zhou et al. · 2026
  9. GEM-4D: Geometry-Enhanced Video World Models for Robot ManipulationKaichen Zhou, Yuzhen Chen, Fangneng Zhan, Hang Hua, Grace Chen, Xinhai Chang, Ao Qu, Yilun Du et al. · 2026
  10. H-WM: Robotic Task and Motion Planning Guided by Hierarchical World ModelJinbang Huang, Wenyuan Chen, Zhiyuan Li, Oscar Pang, Xiao Hu, Lingfeng Zhang, Yuanzhao Hu, Zhanguang Zhang et al. · 2026
  11. Hi-WM: Human-in-the-World-Model for Scalable Robot Post-TrainingYaxuan Li, Zhongyi Zhou, Yefei Chen, Yanjiang Guo, Jiaming Liu, Shanghang Zhang, Jianyu Chen, Yichen Zhu · 2026
  12. Interactive World Simulator for Robot Policy Training and EvaluationYixuan Wang, Rhythm Syed, Fangyu Wu, Mengchao Zhang, Aykut Onol, Jose Barreiros, Hooshang Nayyeri, Tony Dear et al. · 2026
  13. Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied SimulationMutian Xu, Tianbao Zhang, Tianqi Liu, Zhaoxi Chen, Xiaoguang Han, Ziwei Liu · 2026
  14. MiraBench: Evaluating Action-Conditioned Reliability in Robotic World ModelsTianzhuo Yang, Zihan Shen, Zirui Mi, Zhaoyi Zhang, Jiayi Zhou, Jiaming Ji, Juntao Dai, Jiawei Chen et al. · 2026
  15. Predictive Spatio-Temporal Scene Graphs for Semi-Static ScenesMiguel Saavedra-Ruiz, Charlie Gauthier, Kumaraditya Gupta · 2026
  16. Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D RepresentationJisoo Kim, Jungbin Cho, Sanghyeok Chu · 2026
  17. ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action PoliciesHaodi Hu, Chung-Ta Huang, Jing Liu, Ye Wang, Kei Suzuki, Matthew Brand, Toshiaki Koike-Akino · 2026
  18. Revisiting Embodied Chain-of-Thought for Generalizable Robot ManipulationNan Sun, Yuan Zhang, Yongkun Yang, Wentao Zhao, Peiyan Li, Jun Guo, Wenxuan Song, Pengxiang Ding et al. · 2026
  19. RoboDream: Compositional World Models for Scalable Robot Data SynthesisJunjie Ye, Rong Xue, Basile Van Hoorick, Runhao Li, Harshitha Rajaprakash, Pavel Tokmakov, Muhammad Zubair Irshad, Vitor Guizilini et al. · 2026
  20. SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performanceTengyue Jiang, Chunpu Xu, Jiayue Kang, Yao Mu · 2026
  21. SANTS: A State-Adaptive Scheduler for World Action ModelsSANTS authors · 2026
  22. SKIP: Sparse Keyframe Interpolation Paradigm for Efficient Embodied World ModelsZiheng He, Yixiang Chen, Ning Yang, Zhanqian Wu, Qisen Ma, Yuan Xu, Jiabing Yang, Peiyan Li et al. · 2026
  23. SPACE: Enabling Learning from Cross-Robot Data Toward Generalist PoliciesHaeone Lee · 2026
  24. TacForeSight: Force-Guided Tactile World Model for Contact-Rich ManipulationYujie Zang, Yuhang Zheng, Xian Nie, Yupeng Zheng, Shuai Tian, Songen Gu, Chen Gao, Zining Wang et al. · 2026
  25. TACO: TActile World Model as a Self-COrrector forScalable VLA Post-TrainingShengbang Liu, Yueru Jia, Yuyang Yan, Jiaming Liu, Xinran Zhang, Qiuxuan Feng, Yandong Guo, Shiji Zhou et al. · 2026
  26. Vision-Language-Action Models: Experimental Insights from a Real-World UR5 PlatformMathilde Hochedel, Marc Lalonde · 2026
  27. World Pilot: Steering Vision-Language-Action Models with World-Action PriorsZefu Lin, Rongxu Cui, Junjia Xu, Xiaojuan Jin, Wenling Li, Lue Fan, Zhaoxiang Zhang · 2026
完整证据附录(31 条正式证据事件)
  • Time range: 2026-01-19..2026-07-19
  • Events: 31
  • 每个事件一节,标题即锚点;trace-map 中的 event 链接跳转到这里。

EA-WMDATA-READ-0007

  • Claim: A robot world model can be trained from a moderate-sized robot interaction dataset and then used to generate policy-training demonstrations, but its value depends on interaction-consistent long-horizon rollouts and sim-real correlation.
  • Stance: support | Confidence: direct
  • Paper: 2603.08546 Interactive World Simulator for Robot Policy Training and Evaluation
  • Locator: IV-C Data Generation for Policy Training
  • Evidence: The paper builds an Interactive World Simulator from a moderate-sized robot interaction dataset, reports world-model-generated policy data comparable to the same amount of real-world data, and evaluates sim-real performance correlation.
  • Quote: “Notably, policies trained on 100% world simulator data perform comparably to those trained on an equivalent volume of real-robot expert data. This suggests that our simulator can generate data with quality similar to that of real-world demonstrations.”
  • Authors: yixuan-wang; rhythm-syed; fangyu-wu; et al.

EA-WMDATA-READ-0009

  • Claim: World-model training and post-training data should include dense corrective trajectories around failure-prone states, not only successful demonstrations.
  • Stance: support | Confidence: direct
  • Paper: 2604.21741 Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training
  • Locator: 3.5 Collecting Corrective Trajectories for Post-Training
  • Evidence: Hi-WM rolls policies inside a world model, lets humans intervene when rollouts become incorrect or failure-prone, caches and branches failure states, and adds corrective trajectories back into the training set for post-training.
  • Quote: “Within Hi-WM, post-training data are collected in a closed-loop inside the interactive world model. The pre-trained policy first runs in the world model from the current observation and generates a rollout. When the rollout enters unfamiliar or failure-prone states, a human operator intervenes through the hardware-agnostic interface and provides corrective actions. The world model then continues the rollout from the current state using these human actions. Once the rollout has been guided back t”
  • Authors: yaxuan-li; zhongyi-zhou; yefei-chen; et al.

EA-WMDATA-READ-0008

  • Claim: Robot videos can be converted into richer world-model supervision by pairing RGB with depth and pseudo 3D scene-flow targets, training models to represent current 3D structure and short-horizon future evolution rather than only behavior-cloning actions.
  • Stance: support | Confidence: direct
  • Paper: 2605.20752 GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic Manipulation
  • Locator: 3.4 GaussianDream Training and Efficient Inference
  • Evidence: GaussianDream trains current Gaussian reconstruction and future Gaussian prediction heads with RGB rendering, depth, and pseudo 3D scene-flow supervision, then retains only a compact prefix for control at inference.
  • Quote: “GaussianDream follows an asymmetric strategy: dense Gaussian reconstruction and prediction supervise training, while only the compact prefix is retained for online control. Stage I: GaussianDream pretraining. We first train the reconstruction and prediction heads without action learning. For each demonstration sequence, RGB frames are paired with pseudo depth and pseudo 3D scene-flow targets constructed from adjacent frames. The GaussianDream objective combines current reconstruction and future”
  • Authors: zijian-zhang; yuqing-jiang; qian-cheng; et al.

EA-4D-READ-0013

  • Claim: Dream-Tac将动作块、未来视觉和未来触觉放进同一世界动作建模目标,以弥补单纯视觉未来对接触互动线索的不足。
  • Stance: support | Confidence: direct
  • Paper: 2606.08737 Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
  • Locator: 3.1. Problem Formulation
  • Evidence: 问题建模段先定义动作与视觉未来的联合分布,再明确把未来触觉纳入联合预测目标。
  • Quote: “Building on these two formulations, a world action model combines action prediction and future observation prediction into a unified framework. Specifically, it jointly models (3) or equivalently factorizes the joint distribution as (4) where future visual prediction provides predictive structure for action generation. However, in contact-rich manipulation, vision alone is often insufficient to capture physical interaction cues. To address this limitation, we introduce Dream-Tac, an enhanced wor”
  • Authors: yunfan-lou; yifan-ye; yankai-fu; et al.

EA-4D-READ-0012

  • Claim: 3PoinTr 保留含暂时遮挡点的轨迹,只屏蔽不可见的单个点—时间损失,因而能继续利用操作中任务关键的物体点监督。
  • Stance: conditional | Confidence: direct
  • Paper: 2603.08485 3PoinTr: 3D Point Tracks for Learning Manipulation from Unconstrained Human Videos
  • Locator: 4.3 Results: 3D Point Track Prediction
  • Evidence: 结果段对比了删除整条不可见轨迹的基线与仅屏蔽不可见 point-timestep 损失的 3PoinTr。
  • Quote: “The primary advantage of 3PoinTr is that it trains on data General Flow ignores. Real-world points are often temporarily occluded; General Flow removes any trajectory with invisible point-timestep pairs during preprocessing, whereas 3PoinTr retains all trajectories and masks losses for individual invisible point-timestep pairs. This provides additional supervision over task-critical object points that are temporarily occluded during manipulation. For example, in the Throw Away Paper task, every”
  • Authors: adam-hung; bardienus-pieter-duisterhof; jeffrey-ichnowski

EA-4D-READ-0011

  • Claim: 如果目标包含扰动恢复,数据集必须显式收集nominal demonstrations和recovery interaction data;否则模型很难学习接触丢失后的再建立。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.11184 TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
  • Locator: IV-B 2 Perturbation-Aware Evaluation
  • Evidence: TacForeSight在perturbation-aware设置中额外收集恢复示教,让外部扰动发生在执行中并要求示教者重新建立稳定接触;完整模型在三个扰动任务上报告90%、85%、85%平均完成/成功分数。
  • Quote: “Policies in this setting are trained using both nominal demonstrations and recovery interaction data.”
  • Authors: yujie-zang; yuhang-zheng; xian-nie; et al.

EA-WMEVAL-READ-0007

  • Claim: Kinema4D argues that robot-world interaction should be simulated as a 4D event: robot control is represented with kinematically correct 4D trajectories, while a generative model predicts environment reactions.
  • Stance: support | Confidence: direct
  • Paper: 2603.16669 Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulation
  • Locator: Abstract (full-text section)
  • Evidence: The method disentangles precise robot control from generative environmental reaction by driving a URDF robot through kinematics, projecting a 4D robot pointmap sequence, and jointly generating synchronized RGB/pointmap futures.
  • Quote: “Abstract Simulating robot-world interactions is a cornerstone of Embodied AI. Recently, a few works have shown promise in leveraging video generations to transcend the rigid visual/physical constraints of traditional simulators. However, they primarily operate in 2D space or are guided by static environmental cues, ignoring the fundamental reality that robot-world interactions are inherently 4D spatiotemporal events that require precise interactive modeling. To restore this 4D essence while ensu”
  • Authors: mutian-xu; tianbao-zhang; tianqi-liu; et al.

EA-WMEVAL-READ-0005

  • Claim: GEM-4D supports geometry-feature distillation as a way to make video world models more actionable for robot manipulation without adding inference-time cost.
  • Stance: support | Confidence: direct
  • Paper: 2605.22882 GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: The model distills 4D geometry foundation-model representations into a video backbone during training, discards the geometry branch at inference, and uses an inverse dynamics module to convert generated rollouts into executable trajectories; the paper reports real-world manipulation success improving from 61% to 81%.
  • Quote: “Abstract Video world models can generate realistic futures from a single instruction, but they often fail to track the same physical points consistently across time. As a result, the generated videos appear plausible, yet lack the physical grounding required for reliable action execution, such as robot manipulation. We present GEM-4D , a geometry-grounded video world model that resolves this limitation by injecting dense 4D correspondence supervision distilled from a pretrained geometry foundati”
  • Authors: kaichen-zhou; yuzhen-chen; fangneng-zhan; et al.

EA-WMEVAL-READ-0003

  • Claim: Training data for efficient embodied world-model rollouts must preserve sparse task-relevant events such as approach, contact, grasp, and release; generic frame dropping can remove the information downstream policies need.
  • Stance: support | Confidence: direct
  • Paper: 2606.00664 SKIP: Sparse Keyframe Interpolation Paradigm for Efficient Embodied World Models
  • Locator: Abstract (full-text section)
  • Evidence: SKIP argues that manipulation rollouts concentrate task-relevant information in sparse events, selects event-preserving keyframes through robot-aware multimodal fusion, and reports that generated videos can serve as policy-training data.
  • Quote: “Abstract Embodied world models have emerged as a promising paradigm in robotics by predicting how robot actions affect the surrounding scene. However, the rollout inference remains computationally expensive in pixel space, as long-horizon manipulation videos typically have to be generated frame by frame. This cost cannot be easily reduced by indiscriminately dropping frames, since downstream policies rely on complete preservation of sparse task-relevant events such as approach, contact, grasp, a”
  • Authors: ziheng-he; yixiang-chen; ning-yang; et al.

EA-WMEVAL-READ-0001

  • Claim: τ0-WM 使用真实机器人遥操作、UMI 式交互、人类第一视角视频与 rollout/失败轨迹的异构语料,并用按模态的监督掩码训练统一视频—动作世界模型。
  • Stance: support | Confidence: direct
  • Paper: 2606.01027 $τ_0$-WM: A Unified Video-Action World Model for Robotic Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 摘要直接报告了异构数据组成与 modality-specific supervision masks。
  • Quote: “Abstract Robotic manipulation requires models that generate executable actions while anticipating and evaluating their future consequences before physical execution. We present -World Model ( -WM), a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone, -WM provides two complementary interfaces. First, a video action model jointly predicts future visual lat”
  • Authors: pengfei-zhou; shengcong-chen; di-chen; et al.

EA-WMEVAL-READ-0010

  • Claim: WEAVER defines useful robot world models by three joint requirements: fidelity to real outcomes, temporal consistency over long horizons, and enough efficiency for evaluation, improvement, and planning.
  • Stance: support | Confidence: direct
  • Paper: 2606.13672 $\texttt{WEAVER}$, Better, Faster, Longer: An Effective World Model for Robotic Manipulation
  • Locator: 3 WEAVER : World Estimation Across Views for Embodied Reasoning
  • Evidence: The paper argues that manipulation world models must satisfy fidelity, consistency, and efficiency together, then designs a multi-view latent world model with reward/value prediction to support policy evaluation, synthetic policy improvement, and test-time planning.
  • Quote: “Figure 2 : WEAVER Architecture. Left: The world model encodes memory, history, and action sequences to image future rollouts in latent space. Middle: The latent verifier, equipped with reward and critic heads, selects samples with high advantage to steer the policy distribution. Right: Decoded generation corresponding to different outcomes of action sequences. We now describe the key ingredients in WEAVER : a robot world model designed to support policy evaluation, policy improvement, and test-t”
  • Authors: arnav-kumar-jain; yilin-wu; jesse-farebrother; et al.

EA-WMEVAL-READ-0014

  • Claim: A scalable world-model data pipeline can use a small amount of real-world data to align classical simulation with neural simulation, generating action-video pairs that preserve real-world consistency and broaden scenario coverage.
  • Stance: conditional | Confidence: direct
  • Paper: 2604.11386 ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation
  • Locator: Abstract (full-text section)
  • Evidence: ComSim proposes a real-sim-real data augmentation pipeline: collect a small real trajectory set, align classical simulation to the real platform, transform simulation videos into real-world representations, and generate large-scale action-video training datasets.
  • Quote: “Abstract Recent advancements in foundational models, such as large language models and world models, have greatly enhanced the capabilities of robotics, enabling robots to autonomously perform complex tasks. However, acquiring large-scale, high-quality training data for robotics remains a challenge, as it often requires substantial manual effort and is limited in its coverage of diverse real-world environments. To address this, we propose a novel hybrid approach called Compositional Simulation ,”
  • Authors: yiran-qin; jiahua-ma; li-kang; et al.

EA-WMEVAL-READ-0011

  • Claim: Synthetic world-model data for robot learning needs embodiment anchoring: rendered robot motion plus explicit scene and object priors can synthesize demonstrations in novel objects, scenes, and viewpoints while reducing teleoperation burden.
  • Stance: conditional | Confidence: direct
  • Paper: 2606.02577 RoboDream: Compositional World Models for Scalable Robot Data Synthesis
  • Locator: Abstract (full-text section)
  • Evidence: RoboDream anchors generation to rendered robot motion, conditions on scene/object priors, and introduces retrieval-and-rebirth plus prop-free teleoperation to generate demonstrations and reduce real data collection cost.
  • Quote: “Abstract Scaling robot learning requires large-scale, diverse demonstrations, yet real-world data collection via teleoperation remains prohibitively expensive and time-consuming. While video diffusion models offer a promising avenue for data scaling, existing generative approaches are often limited to superficial visual augmentation, or suffer from embodiment hallucinations that yield physically infeasible motions. We present a generalizable embodiment-centric world model that achieves scalable”
  • Authors: junjie-ye; rong-xue; basile-van-hoorick; et al.

EA-WMEVAL-READ-0008

  • Claim: PredictiveGraphs is bounded by simplified independence assumptions, perception ambiguity among similar objects, and possible LLM hallucinations during verification and planning.
  • Stance: limit | Confidence: direct
  • Paper: 2605.00121 Predictive Spatio-Temporal Scene Graphs for Semi-Static Scenes
  • Locator: VIII Limitations
  • Evidence: The limitations section says object-receptacle edges are modeled independently, indistinguishable objects are treated as interchangeable, and LLM hallucinations remain a risk for open-vocabulary verification and planning.
  • Quote: “While embedding open-vocabulary scene graphs with persistence estimators yields tempo-spatio-semantic capabilities, our approach is not free of limitations. First, object-receptacle edges are modelled independently. While joint distributions over multiple locations are possible [ 30 ] , inference costs grow with the number of observed locations per object, inducing a trade-off between expressivity and real-time performance. We believe that striking the right balance in this trade-off is a promis”
  • Authors: miguel-saavedra-ruiz; charlie-gauthier; kumaraditya-gupta

EA-WMEVAL-READ-0015

  • Claim: World-model training and post-training objectives should be tied to downstream action quality rather than intermediate video fidelity, because later denoising can become less action-relevant or physically unreliable.
  • Stance: limit | Confidence: direct
  • Paper: 2605.27947 SANTS: A State-Adaptive Scheduler for World Action Models
  • Locator: Abstract (full-text section)
  • Evidence: SANTS reports that fully denoised video is not always the best action condition, trains a scheduler with a path-level reward after action generation, and explicitly optimizes downstream action quality rather than video fidelity.
  • Quote: “Abstract World Action Models (WAMs) improve robot manipulation by using video-based future representations to condition action generation. In pixel-space WAMs, however, the best action condition is not necessarily the fully denoised video. Controlled denoising-depth scans show that video refinement can reduce action error up to a state-dependent point, after which the gain may saturate or even reverse when late predictions become less action-relevant or physically unreliable. This suggests that”
  • Authors: sants-authors

EA-WMEVAL-READ-0013

  • Claim: Static image-text pretraining is insufficient training signal for contact-rich manipulation; world-model data should expose action-conditioned scene evolution and contact dynamics.
  • Stance: limit | Confidence: direct
  • Paper: 2606.12403 World Pilot: Steering Vision-Language-Action Models with World-Action Priors
  • Locator: Abstract (full-text section)
  • Evidence: World Pilot argues that VLA semantic grounding from static image-text pairs cannot capture continuous contact-rich dynamics, and uses WAM-derived scene-evolution and trajectory priors to complement the policy.
  • Quote: “Abstract Vision-Language-Action (VLA) models inherit semantic grounding from large-scale pretraining and perform competently across in-distribution manipulation tasks. This grounding, however, is built on static image-text pairs, whereas manipulation is a continuous, contact-rich process whose dynamics this pretraining cannot capture. We present World Pilot, a VLA framework that augments the policy with priors from a World-Action Model (WAM), routed into the decision chain through two complement”
  • Authors: zefu-lin; rongxu-cui; junjia-xu; et al.

EA-WMEVAL-READ-0006

  • Claim: Pri4R leaves open whether 4D point-track supervision should be used at larger pretraining scale or with explicit test-time geometric inputs.
  • Stance: gap | Confidence: direct
  • Paper: 2603.01549 Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D Representation
  • Locator: VI Discussion, Limitations, and Future Work
  • Evidence: The conclusion says Pri4R was evaluated mainly as fine-tuning on demonstrations and small real-world rollouts, and suggests that pretraining-scale 3D point-track supervision or explicit test-time computation could further improve robustness.
  • Quote: “We presented Pri4R, a framework that enhances the world dynamics understanding of VLA models through privileged 4D representations. By supervising the model to predict 3D point tracks during training, we demonstrated that VLA backbones can develop a more physically-aware context, leading to improved control performance without any inference-time overhead. Our results across various benchmarks suggest that capturing the spatiotemporal evolution of a scene is a critical component for robust robot”
  • Authors: jisoo-kim; jungbin-cho; sanghyeok-chu

EA-WMEVAL-READ-0004

  • Claim: Robotic world-model evaluation should move beyond visual fidelity toward action-conditioned reliability, including physical adherence, action-following fidelity, and optimism-bias detection.
  • Stance: gap | Confidence: direct
  • Paper: 2605.29360 MiraBench: Evaluating Action-Conditioned Reliability in Robotic World Models
  • Locator: Abstract (full-text section)
  • Evidence: The paper frames existing evaluations as weak evidence for whether action-conditioned predictions are reliable, then defines MiraBench around physics adherence, action fidelity, and failure-case optimism bias.
  • Quote: “Abstract Action-conditioned world models are increasingly used as scalable simulators for robot learning, yet current evaluations provide limited evidence that their predictions are reliable under the actions they condition on. Existing benchmarks largely emphasize visual fidelity, leaving unclear whether predicted futures are physically plausible, faithful to commanded actions, and calibrated to failure when actions should not succeed. We introduce MiraBench , a hierarchical benchmark that defi”
  • Authors: tianzhuo-yang; zihan-shen; zirui-mi; et al.

EA-VLABREAK-2026-0001

  • Claim: H-WM 用低频符号逻辑转移维持全局顺序,用潜在视觉子目标把逻辑状态落到感知空间,再由高频 VLA 执行动作 chunk。
  • Stance: support | Confidence: direct
  • Paper: 2602.11291 H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Model
  • Locator: IV-C Hierarchical World Model Guidance for VLA
  • Evidence: 方法定义了逻辑世界模型、视觉世界模型、低层 VLA 和子任务完成检测的两时间尺度接口。
  • Quote: “The hierarchical information at multiple abstraction level enables the VLA to maintain consistency with long-horizon task structure while remaining responsive to local visual feedback.”
  • Authors: jinbang-huang; wenyuan-chen; zhiyuan-li; et al.

EA-VLABREAK-2026-0004

  • Claim: StructVLA 把稠密视频未来压缩成由夹爪转换和运动转折点定义的稀疏结构化帧,再将这种规划表征迁移到低层动作生成。
  • Stance: support | Confidence: direct
  • Paper: 2603.12553 Beyond Dense Futures: World Models as Structured Planners for Robotic Manipulation
  • Locator: pages 5-8, Sections 3.1-3.3
  • Evidence: 方法段给出动力学里程碑抽取和 planner-to-action 两阶段优化的完整链路。
  • Quote: “Structured frames provide compact progress anchors that connect task intent to executable motion phases.”
  • Authors: minghao-jin; mozheng-liao; mingfei-han; et al.

EA-ALIGN-READ-0015

  • Claim: 失败恢复可以把认知层(failure mode/recovery stage 判断与 reward 选择)与控制层(residual 纠正)分开;VLM 的失败分类错误由此成为与感知误差独立的认知误差来源。
  • Stance: support | Confidence: direct
  • Paper: 2606.09630 ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action Policies
  • Locator: 1 Introduction
  • Evidence: ReCoVLA 用外部 VLM 只推断 failure type、recovery stage、active entities、confidence 和 reward mask,不直接生成动作;确定性 reward compiler 做实体 grounding 与 stage gates,residual policy 在冻结 VLA latents 上学纠正。Limitations 明确列出 VLM failure-classification mistakes 与 perception errors、sim-to-real mismatch 并列为失败来源。
  • Quote: “Instead, it produces a structured recovery descriptor containing the failure type, recovery stage, active entities, confidence, and reward mask.”
  • Authors: haodi-hu; chung-ta-huang; jing-liu; et al.

EA-VLABREAK-2026-0002

  • Claim: 在五个 5-7 步 LIBERO-LoHo 任务上,双层逻辑+潜在视觉引导比仅逻辑引导高 16.4 个成功率百分点,也高于像素级生成引导。
  • Stance: conditional | Confidence: direct
  • Paper: 2602.11291 H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Model
  • Locator: VI Results
  • Evidence: H-WM 为 64.8%,logic-only 为 48.4%,H-WM-Stable-Diffusion 为 54.4%。
  • Quote: “Incorporating visual guidance yields consistent additional gains, providing more than 10% further improvement in Q-score and 17% in success rate.”
  • Authors: jinbang-huang; wenyuan-chen; zhiyuan-li; et al.

EA-VLABREAK-2026-0005

  • Claim: 在论文覆盖的设置中,StructVLA 的长时程改进同时出现在 LIBERO-Long 和 Franka 实机 tidy-up,但证据范围仍限于少量夹爪操作任务。
  • Stance: conditional | Confidence: direct
  • Paper: 2603.12553 Beyond Dense Futures: World Models as Structured Planners for Robotic Manipulation
  • Locator: page 11
  • Evidence: LIBERO 平均为 94.8%;实机 tidy-up 为 8/10,相同表面的 UniVLA 为 4/10。
  • Quote: “StructVLA completes8/10trials,comparedwith4/10forUniVLAand2/10forSpatialVLA, indicating stronger stability over extended execution.”
  • Authors: minghao-jin; mozheng-liao; mingfei-han; et al.

EA-ALIGN-READ-0006

  • Claim: ERVLA 的大规模实验表明,有效的具身 CoT 需要落到末端执行器运动或图像空间轨迹等动作相关表示,而将显式 CoT 作为自回归动作前缀会在推理时累积误差。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.03784 Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 摘要同时给出了动作相关 grounding 的有效性与 autoregressive action prefix 的 compounding-error 限制。
  • Quote: “Abstract Embodied chain-of-thought (CoT) aims to bridge linguistic reasoning with robotic control, yet its effective form and integration remain underexplored. In this paper, we revisit embodied CoT for robotic control at an unprecedented scale. We curate the largest embodied CoT corpus to date, comprising 978,743 trajectories, 226.3M samples, and 2592.5 hours of data. Through extensive experiments, we show that effective CoT must ground high-level semantic understaning in concrete linguistic ac”
  • Authors: nan-sun; yuan-zhang; yongkun-yang; et al.

EA-VLABREAK-2026-0003

  • Claim: H-WM 的分层收益以额外训练与系统复杂度、以及任务可被结构化逻辑表示为前提。
  • Stance: limit | Confidence: direct
  • Paper: 2602.11291 H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Model
  • Locator: VII Conclusion
  • Evidence: 结论明确列出额外组件/训练阶段的代价,以及对符号化状态的依赖。
  • Quote: “The logical world model depends on structured logical state representations, which assume that the task can be meaningfully formulated in a symbolic logical space.”
  • Authors: jinbang-huang; wenyuan-chen; zhiyuan-li; et al.

EA-ALIGN-READ-0001

  • Claim: A recorded robot action is not a universal supervision signal: the same command can produce different motions across controllers, embodiments, hardware units, and deployment-time dynamics.
  • Stance: limit | Confidence: direct
  • Paper: 2606.24049 SPACE: Enabling Learning from Cross-Robot Data Toward Generalist Policies
  • Locator: 3.2 Inconsistency of Control Commands across Robots
  • Evidence: SPACE predicts Cartesian state deltas as a shared end-effector-space representation and uses an action adapter to convert them into robot-specific control commands, improving cross-robot and dynamics-shift robustness.
  • Quote: “Recent work has scaled robot learning by training policies on data from multiple embodiments [ 27 , 23 , 32 ] , often using the Cartesian delta action space [ 23 , 32 ] since it is less dependent on robot-specific kinematics and invariant to base-frame translation [ 18 , 14 ] . In practice, this is typically realized by predicting Cartesian delta control commands that are fed to the underlying robot controller [ 23 , 32 ] . Figure 2: Different robots (e.g., UR5 vs. Franka Research 3) require dif”
  • Authors: haeone-lee

EA-ALIGN-READ-0003

  • Claim: Discrete action tokenization is a compact interface for autoregressive VLA policies, but decoding fixed tokens back into continuous robot controls is a bottleneck when the same token must mean different controls under different robot states and contacts.
  • Stance: limit | Confidence: direct
  • Paper: 2606.30113 SA-VLA: State-aware tokenizer for improving Vision-Language-Action Models' performance
  • Locator: Abstract (full-text section)
  • Evidence: SA-VLA conditions action-token decoding on proprioceptive state via adapters or cross-attention, reporting improved RoboTwin and zero-shot sim-to-real success over tokenizer baselines.
  • Quote: “Abstract Discrete action tokenization provides a compact interface for autoregressive VLA policies, but accurately recovering continuous robot actions from discrete codes remains challenging. Existing tokenizers typically map each discrete code to a fixed continuous action prototype, ignoring the robot’s current proprioceptive state. This limitation is particularly pronounced in manipulation, where the same action token may require different continuous controls under different joint configuratio”
  • Authors: tengyue-jiang; chunpu-xu; jiayue-kang; et al.

EA-ALIGN-READ-0004

  • Claim: Offline VLA indicators can fail to transfer to stable real-robot behavior when action semantics, coordinate frames, temporal modality alignment, image preprocessing, and dataset coverage are not controlled together.
  • Stance: limit | Confidence: direct
  • Paper: 2606.30456 Vision-Language-Action Models: Experimental Insights from a Real-World UR5 Platform
  • Locator: Abstract (full-text section)
  • Evidence: The UR5 study reports a gap between offline indicators and unstable closed-loop physical behavior, attributing it to data-model-control pipeline consistency rather than model capacity alone.
  • Quote: “Instead, it is strongly influenced by a combination of factors, including action semantics, coordinate frame conventions, temporal alignment between modalities, image preprocessing consistency, and dataset coverage and quality. These observations lead to a key interpretation: the successful deployment of VLA systems in real-world settings depends less on incremental improvements in model capacity and more on precise control of the entire data–model–control pipeline.”
  • Authors: mathilde-hochedel; marc-lalonde

EA-ALIGN-READ-0009

  • Claim: TACO 用触觉感知世界模型联合预测未来视频和力序列,再将真实失败附近状态转成带纠正动作的想象片段,用于接触丰富任务的 VLA 后训练。
  • Stance: limit | Confidence: direct
  • Paper: 2607.02840 TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training
  • Locator: 5 Conclusion and Limitations
  • Evidence: 结论的 Recognize–Imagine–Label 回路明确连接了真实失败、视频—力联合想象与纠正动作标注。
  • Quote: “We presented TACO, a tactile-aware world-model-driven framework for scalable VLA post-training in contact-rich manipulation. Following a Recognize–Imagine–Label loop, TACO converts real-world failures into imagined corrections without repeated human intervention: a tactile-aware world model jointly denoises future video and force sequences, while a unified progress-action model recognizes failure-adjacent states and labels imagined segments with corrective actions. To incorporate this supervisio”
  • Authors: shengbang-liu; yueru-jia; yuyang-yan; et al.

EA-VLABREAK-2026-0006

  • Claim: 在完整 LIBERO 闭环扫描中,BadWAM 的黑盒动作攻击将高成功率 WAM 从 96.5% 降至 43.1%,且失败对空间与长时程任务尤为严重。
  • Stance: limit | Confidence: direct
  • Paper: 2607.15207 BadWAM: When World-Action Models Dream Right but Act Wrong
  • Locator: 5.2 BadWAM Reliably Induces Task Failures
  • Evidence: 主实验在 40 个 LIBERO 任务、每任务 20 次试验上使用闭环攻击,并报告任务族级下降。
  • Quote: “On the action-only WAM, the action-only attack lowers success to 43.1%, a 53.4% drop.”
  • Authors: qi-li; xingyi-yang; xinchao-wang

EA-VLABREAK-2026-0007

  • Claim: 对 WAM 的安全监测不能只检查‘想象的未来是否看起来合理’,还必须验证未来与实际执行动作在闭环中是否同步。
  • Stance: limit | Confidence: direct
  • Paper: 2607.15207 BadWAM: When World-Action Models Dream Right but Act Wrong
  • Locator: 5.8 What Do These Results Imply for WAM Safety?
  • Evidence: 想象保持攻击在 40 个任务中有 39 个降低未来漂移,同时保留显著攻击强度。
  • Quote: “The relevant security property is not plausibility of the imagined future in isolation, but synchronization between the imagined future and the action that will actually be executed.”
  • Authors: qi-li; xingyi-yang; xinchao-wang

References

Interactive Wiki

进入交互阅读

静态页采用默认版本;v0、v0、v0可在同一交互 Wiki 中阅读,不建立相互竞争的索引页。

Method and limits

研究方法、证据边界与引用说明

本页由当前结算的证据层自动派生,不改写已结算 run。论文只有在完整非 OCR 全文、结构化论文笔记和 claim-support audit 通过后,才进入接受证据。

页面结论是对所列论文与研究窗口的综合,不等同于无限外推。引用本专题时,请同时保留本页 canonical 地址、访问日期,并在需要时回溯原论文。