世界模型训练是否有必要接监督信号还是走纯端到端

• Time range: 2026-02-09..2026-08-09
• Events: 18
• 每个事件一节,标题即锚点;trace-map 中的 event 链接跳转到这里。

EA-WMDATA-READ-0007

• Claim: A robot world model can be trained from a moderate-sized robot interaction dataset and then used to generate policy-training demonstrations, but its value depends on interaction-consistent long-horizon rollouts and sim-real correlation.
• Stance: support | Confidence: direct
• Paper: 2603.08546 Interactive World Simulator for Robot Policy Training and Evaluation
• Locator: IV-C Data Generation for Policy Training
• Evidence: The paper builds an Interactive World Simulator from a moderate-sized robot interaction dataset, reports world-model-generated policy data comparable to the same amount of real-world data, and evaluates sim-real performance correlation.
• Quote: “Notably, policies trained on 100% world simulator data perform comparably to those trained on an equivalent volume of real-robot expert data. This suggests that our simulator can generate data with quality similar to that of real-world demonstrations.”
• Authors: yixuan-wang; rhythm-syed; fangyu-wu; et al.

EA-WMDATA-READ-0009

• Claim: World-model training and post-training data should include dense corrective trajectories around failure-prone states, not only successful demonstrations.
• Stance: support | Confidence: direct
• Paper: 2604.21741 Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training
• Locator: 3.5 Collecting Corrective Trajectories for Post-Training
• Evidence: Hi-WM rolls policies inside a world model, lets humans intervene when rollouts become incorrect or failure-prone, caches and branches failure states, and adds corrective trajectories back into the training set for post-training.
• Quote: “Within Hi-WM, post-training data are collected in a closed-loop inside the interactive world model. The pre-trained policy first runs in the world model from the current observation and generates a rollout. When the rollout enters unfamiliar or failure-prone states, a human operator intervenes through the hardware-agnostic interface and provides corrective actions. The world model then continues the rollout from the current state using these human actions. Once the rollout has been guided back t”
• Authors: yaxuan-li; zhongyi-zhou; yefei-chen; et al.

EA-WMDATA-READ-0002

• Claim: GEM-4D supports geometry-feature distillation as a way to make video world models more actionable for robot manipulation without adding inference-time cost.
• Stance: support | Confidence: direct
• Paper: 2605.22882 GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation
• Locator: Abstract (full-text section)
• Evidence: The model distills 4D geometry foundation-model representations into a video backbone during training, discards the geometry branch at inference, and uses an inverse dynamics module to convert generated rollouts into executable trajectories; the paper reports real-world manipulation success improving from 61% to 81%.
• Quote: “Abstract Video world models can generate realistic futures from a single instruction, but they often fail to track the same physical points consistently across time. As a result, the generated videos appear plausible, yet lack the physical grounding required for reliable action execution, such as robot manipulation. We present GEM-4D , a geometry-grounded video world model that resolves this limitation by injecting dense 4D correspondence supervision distilled from a pretrained geometry foundati”
• Authors: kaichen-zhou; yuzhen-chen; fangneng-zhan; et al.

EA-WMDATA-READ-0008

• Claim: Robot videos can be converted into richer world-model supervision by pairing RGB with depth and pseudo 3D scene-flow targets, training models to represent current 3D structure and short-horizon future evolution rather than only behavior-cloning actions.
• Stance: support | Confidence: direct
• Paper: 2605.20752 GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic Manipulation
• Locator: 3.4 GaussianDream Training and Efficient Inference
• Evidence: GaussianDream trains current Gaussian reconstruction and future Gaussian prediction heads with RGB rendering, depth, and pseudo 3D scene-flow supervision, then retains only a compact prefix for control at inference.
• Quote: “GaussianDream follows an asymmetric strategy: dense Gaussian reconstruction and prediction supervise training, while only the compact prefix is retained for online control. Stage I: GaussianDream pretraining. We first train the reconstruction and prediction heads without action learning. For each demonstration sequence, RGB frames are paired with pseudo depth and pseudo 3D scene-flow targets constructed from adjacent frames. The GaussianDream objective combines current reconstruction and future”
• Authors: zijian-zhang; yuqing-jiang; qian-cheng; et al.

EA-WMDATA-READ-0001

• Claim: τ0-WM 使用真实机器人遥操作、UMI 式交互、人类第一视角视频与 rollout/失败轨迹的异构语料，并用按模态的监督掩码训练统一视频—动作世界模型。
• Stance: support | Confidence: direct
• Paper: 2606.01027 $τ_0$-WM: A Unified Video-Action World Model for Robotic Manipulation
• Locator: Abstract (full-text section)
• Evidence: 摘要直接报告了异构数据组成与 modality-specific supervision masks。
• Quote: “Abstract Robotic manipulation requires models that generate executable actions while anticipating and evaluating their future consequences before physical execution. We present -World Model ( -WM), a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone, -WM provides two complementary interfaces. First, a video action model jointly predicts future visual lat”
• Authors: pengfei-zhou; shengcong-chen; di-chen; et al.

EA-4D-READ-0013

• Claim: Dream-Tac将动作块、未来视觉和未来触觉放进同一世界动作建模目标，以弥补单纯视觉未来对接触互动线索的不足。
• Stance: support | Confidence: direct
• Paper: 2606.08737 Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
• Locator: 3.1. Problem Formulation
• Evidence: 问题建模段先定义动作与视觉未来的联合分布，再明确把未来触觉纳入联合预测目标。
• Quote: “Building on these two formulations, a world action model combines action prediction and future observation prediction into a unified framework. Specifically, it jointly models (3) or equivalently factorizes the joint distribution as (4) where future visual prediction provides predictive structure for action generation. However, in contact-rich manipulation, vision alone is often insufficient to capture physical interaction cues. To address this limitation, we introduce Dream-Tac, an enhanced wor”
• Authors: yunfan-lou; yifan-ye; yankai-fu; et al.

EA-4D-READ-0003

• Claim: WEAVER defines useful robot world models by three joint requirements: fidelity to real outcomes, temporal consistency over long horizons, and enough efficiency for evaluation, improvement, and planning.
• Stance: support | Confidence: direct
• Paper: 2606.13672 $\texttt{WEAVER}$, Better, Faster, Longer: An Effective World Model for Robotic Manipulation
• Locator: 3 WEAVER : World Estimation Across Views for Embodied Reasoning
• Evidence: The paper argues that manipulation world models must satisfy fidelity, consistency, and efficiency together, then designs a multi-view latent world model with reward/value prediction to support policy evaluation, synthetic policy improvement, and test-time planning.
• Quote: “Figure 2 : WEAVER Architecture. Left: The world model encodes memory, history, and action sequences to image future rollouts in latent space. Middle: The latent verifier, equipped with reward and critic heads, selects samples with high advantage to steer the policy distribution. Right: Decoded generation corresponding to different outcomes of action sequences. We now describe the key ingredients in WEAVER : a robot world model designed to support policy evaluation, policy improvement, and test-t”
• Authors: arnav-kumar-jain; yilin-wu; jesse-farebrother; et al.

EA-WMDATA-READ-0015

• Claim: WEAVER defines useful robot world models by three joint requirements: fidelity to real outcomes, temporal consistency over long horizons, and enough efficiency for evaluation, improvement, and planning.
• Stance: support | Confidence: direct
• Paper: 2606.13672 $\texttt{WEAVER}$, Better, Faster, Longer: An Effective World Model for Robotic Manipulation
• Locator: 3 WEAVER : World Estimation Across Views for Embodied Reasoning
• Evidence: The paper argues that manipulation world models must satisfy fidelity, consistency, and efficiency together, then designs a multi-view latent world model with reward/value prediction to support policy evaluation, synthetic policy improvement, and test-time planning.
• Quote: “Figure 2 : WEAVER Architecture. Left: The world model encodes memory, history, and action sequences to image future rollouts in latent space. Middle: The latent verifier, equipped with reward and critic heads, selects samples with high advantage to steer the policy distribution. Right: Decoded generation corresponding to different outcomes of action sequences. We now describe the key ingredients in WEAVER : a robot world model designed to support policy evaluation, policy improvement, and test-t”
• Authors: arnav-kumar-jain; yilin-wu; jesse-farebrother; et al.

EA-4D-READ-0011

• Claim: 如果目标包含扰动恢复，数据集必须显式收集nominal demonstrations和recovery interaction data；否则模型很难学习接触丢失后的再建立。
• Stance: conditional | Confidence: direct
• Paper: 2606.11184 TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
• Locator: IV-B 2 Perturbation-Aware Evaluation
• Evidence: TacForeSight在perturbation-aware设置中额外收集恢复示教，让外部扰动发生在执行中并要求示教者重新建立稳定接触；完整模型在三个扰动任务上报告90%、85%、85%平均完成/成功分数。
• Quote: “Policies in this setting are trained using both nominal demonstrations and recovery interaction data.”
• Authors: yujie-zang; yuhang-zheng; xian-nie; et al.

EA-WMDATA-READ-0006

• Claim: World-model training and post-training objectives should be tied to downstream action quality rather than intermediate video fidelity, because later denoising can become less action-relevant or physically unreliable.
• Stance: limit | Confidence: direct
• Paper: 2605.27947 SANTS: A State-Adaptive Scheduler for World Action Models
• Locator: Abstract (full-text section)
• Evidence: SANTS reports that fully denoised video is not always the best action condition, trains a scheduler with a path-level reward after action generation, and explicitly optimizes downstream action quality rather than video fidelity.
• Quote: “Abstract World Action Models (WAMs) improve robot manipulation by using video-based future representations to condition action generation. In pixel-space WAMs, however, the best action condition is not necessarily the fully denoised video. Controlled denoising-depth scans show that video refinement can reduce action error up to a state-dependent point, after which the gain may saturate or even reverse when late predictions become less action-relevant or physically unreliable. This suggests that”
• Authors: sants-authors

EA-4D-READ-0001

• Claim: Pri4R leaves open whether 4D point-track supervision should be used at larger pretraining scale or with explicit test-time geometric inputs.
• Stance: gap | Confidence: direct
• Paper: 2603.01549 Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D Representation
• Locator: VI Discussion, Limitations, and Future Work
• Evidence: The conclusion says Pri4R was evaluated mainly as fine-tuning on demonstrations and small real-world rollouts, and suggests that pretraining-scale 3D point-track supervision or explicit test-time computation could further improve robustness.
• Quote: “We presented Pri4R, a framework that enhances the world dynamics understanding of VLA models through privileged 4D representations. By supervising the model to predict 3D point tracks during training, we demonstrated that VLA backbones can develop a more physically-aware context, leading to improved control performance without any inference-time overhead. Our results across various benchmarks suggest that capturing the spatiotemporal evolution of a scene is a critical component for robust robot”
• Authors: jisoo-kim; jungbin-cho; sanghyeok-chu; et al.

EA-WMDATA-READ-0010

• Claim: Pri4R leaves open whether 4D point-track supervision should be used at larger pretraining scale or with explicit test-time geometric inputs.
• Stance: gap | Confidence: direct
• Paper: 2603.01549 Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D Representation
• Locator: VI Discussion, Limitations, and Future Work
• Evidence: The conclusion says Pri4R was evaluated mainly as fine-tuning on demonstrations and small real-world rollouts, and suggests that pretraining-scale 3D point-track supervision or explicit test-time computation could further improve robustness.
• Quote: “We presented Pri4R, a framework that enhances the world dynamics understanding of VLA models through privileged 4D representations. By supervising the model to predict 3D point tracks during training, we demonstrated that VLA backbones can develop a more physically-aware context, leading to improved control performance without any inference-time overhead. Our results across various benchmarks suggest that capturing the spatiotemporal evolution of a scene is a critical component for robust robot”
• Authors: jisoo-kim; jungbin-cho; sanghyeok-chu

EA-WMEVAL-READ-0005

• Claim: GEM-4D supports geometry-feature distillation as a way to make video world models more actionable for robot manipulation without adding inference-time cost.
• Stance: support | Confidence: direct
• Paper: 2605.22882 GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation
• Locator: Abstract (full-text section)
• Evidence: The model distills 4D geometry foundation-model representations into a video backbone during training, discards the geometry branch at inference, and uses an inverse dynamics module to convert generated rollouts into executable trajectories; the paper reports real-world manipulation success improving from 61% to 81%.
• Quote: “Abstract Video world models can generate realistic futures from a single instruction, but they often fail to track the same physical points consistently across time. As a result, the generated videos appear plausible, yet lack the physical grounding required for reliable action execution, such as robot manipulation. We present GEM-4D , a geometry-grounded video world model that resolves this limitation by injecting dense 4D correspondence supervision distilled from a pretrained geometry foundati”
• Authors: kaichen-zhou; yuzhen-chen; fangneng-zhan; et al.

EA-WMEVAL-READ-0010

• Claim: WEAVER defines useful robot world models by three joint requirements: fidelity to real outcomes, temporal consistency over long horizons, and enough efficiency for evaluation, improvement, and planning.
• Stance: support | Confidence: direct
• Paper: 2606.13672 $\texttt{WEAVER}$, Better, Faster, Longer: An Effective World Model for Robotic Manipulation
• Locator: 3 WEAVER : World Estimation Across Views for Embodied Reasoning
• Evidence: The paper argues that manipulation world models must satisfy fidelity, consistency, and efficiency together, then designs a multi-view latent world model with reward/value prediction to support policy evaluation, synthetic policy improvement, and test-time planning.
• Quote: “Figure 2 : WEAVER Architecture. Left: The world model encodes memory, history, and action sequences to image future rollouts in latent space. Middle: The latent verifier, equipped with reward and critic heads, selects samples with high advantage to steer the policy distribution. Right: Decoded generation corresponding to different outcomes of action sequences. We now describe the key ingredients in WEAVER : a robot world model designed to support policy evaluation, policy improvement, and test-t”
• Authors: arnav-kumar-jain; yilin-wu; jesse-farebrother; et al.

EA-WMEVAL-READ-0015

• Claim: World-model training and post-training objectives should be tied to downstream action quality rather than intermediate video fidelity, because later denoising can become less action-relevant or physically unreliable.
• Stance: limit | Confidence: direct
• Paper: 2605.27947 SANTS: A State-Adaptive Scheduler for World Action Models
• Locator: Abstract (full-text section)
• Evidence: SANTS reports that fully denoised video is not always the best action condition, trains a scheduler with a path-level reward after action generation, and explicitly optimizes downstream action quality rather than video fidelity.
• Quote: “Abstract World Action Models (WAMs) improve robot manipulation by using video-based future representations to condition action generation. In pixel-space WAMs, however, the best action condition is not necessarily the fully denoised video. Controlled denoising-depth scans show that video refinement can reduce action error up to a state-dependent point, after which the gain may saturate or even reverse when late predictions become less action-relevant or physically unreliable. This suggests that”
• Authors: sants-authors

EA-WMEVAL-READ-0004

• Claim: Robotic world-model evaluation should move beyond visual fidelity toward action-conditioned reliability, including physical adherence, action-following fidelity, and optimism-bias detection.
• Stance: gap | Confidence: direct
• Paper: 2605.29360 MiraBench: Evaluating Action-Conditioned Reliability in Robotic World Models
• Locator: Abstract (full-text section)
• Evidence: The paper frames existing evaluations as weak evidence for whether action-conditioned predictions are reliable, then defines MiraBench around physics adherence, action fidelity, and failure-case optimism bias.
• Quote: “Abstract Action-conditioned world models are increasingly used as scalable simulators for robot learning, yet current evaluations provide limited evidence that their predictions are reliable under the actions they condition on. Existing benchmarks largely emphasize visual fidelity, leaving unclear whether predicted futures are physically plausible, faithful to commanded actions, and calibrated to failure when actions should not succeed. We introduce MiraBench , a hierarchical benchmark that defi”
• Authors: tianzhuo-yang; zihan-shen; zirui-mi; et al.

EA-VLABREAK-2026-0006

• Claim: 在完整 LIBERO 闭环扫描中，BadWAM 的黑盒动作攻击将高成功率 WAM 从 96.5% 降至 43.1%，且失败对空间与长时程任务尤为严重。
• Stance: limit | Confidence: direct
• Paper: 2607.15207 BadWAM: When World-Action Models Dream Right but Act Wrong
• Locator: 5.2 BadWAM Reliably Induces Task Failures
• Evidence: 主实验在 40 个 LIBERO 任务、每任务 20 次试验上使用闭环攻击，并报告任务族级下降。
• Quote: “On the action-only WAM, the action-only attack lowers success to 43.1%, a 53.4% drop.”
• Authors: qi-li; xingyi-yang; xinchao-wang

EA-VLABREAK-2026-0007

• Claim: 对 WAM 的安全监测不能只检查‘想象的未来是否看起来合理’，还必须验证未来与实际执行动作在闭环中是否同步。
• Stance: limit | Confidence: direct
• Paper: 2607.15207 BadWAM: When World-Action Models Dream Right but Act Wrong
• Locator: 5.8 What Do These Results Imply for WAM Safety?
• Evidence: 想象保持攻击在 40 个任务中有 39 个降低未来漂移，同时保留显著攻击强度。
• Quote: “The relevant security property is not plausibility of the imagined future in isolation, but synchronization between the imagined future and the action that will actually be executed.”
• Authors: qi-li; xingyi-yang; xinchao-wang

References

• 2603.01549 Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D Representation (2026-03-02)
• 2603.08546 Interactive World Simulator for Robot Policy Training and Evaluation (2026-03-09)
• 2604.21741 Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training (2026-04-23)
• 2605.20752 GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic Manipulation (2026-05-20)
• 2605.22882 GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation (2026-05-20)
• 2605.27947 SANTS: A State-Adaptive Scheduler for World Action Models (2026-05-27)
• 2605.29360 MiraBench: Evaluating Action-Conditioned Reliability in Robotic World Models (2026-05-28)
• 2606.01027 $τ_0$-WM: A Unified Video-Action World Model for Robotic Manipulation (2026-05-31)
• 2606.08737 Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation (2026-06-07)
• 2606.11184 TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation (2026-06-09)
• 2606.13672 $\texttt{WEAVER}$, Better, Faster, Longer: An Effective World Model for Robotic Manipulation (2026-06-11)
• 2607.15207 BadWAM: When World-Action Models Dream Right but Act Wrong (2026-07-16T17:04:15Z)