Embodied AI Evidence Hub具身智能证据知识库

世界模型与评测 · World Models and Evaluation

触觉世界模型

Tactile World Models

更新于

30 秒结论

先读这个判断

版本说明：本轮以 15 篇可获取完整正文的论文为论证主干，逐篇核对问题、方法、结果与限制；未能取得可读全文的论文不再承担正文结论。

精读论文
15
正式证据事件
15
知识单元
EA-DATA, EA-EVAL, EA-MODEL, EA-SENSOR
审计状态
当前结算 run 已通过仓库校验

v0 · 完整正文

研究综述

研究边界

版本说明：本轮以 15 篇可获取完整正文的论文为论证主干，逐篇核对问题、方法、结果与限制；未能取得可读全文的论文不再承担正文结论。

本文聚焦“触觉世界模型”：机器人在接触丰富操作中，利用视觉、触觉、力/力矩、动作、语言或机器人状态来预测未来接触演化，并把预测用于规划、策略生成、推理期修正或闭环反射控制。时间范围为 2026 年 1 月 14 日至 7 月 14 日。本次范围综述从 867 条去重候选中核验 124 篇可用全文，并以 23 篇直接相关论文构成证据主干；检索覆盖世界模型、触觉表征、数据采集、控制接口、限制与迁移方向。

候选数量、全文数量、关键维度和连续低新增轮次均满足停止条件。无法取得可核验正文的候选不支撑结论，扫描件明确不在本次探索范围内。

核心结论

“触觉世界模型”不是把触觉图像塞进视觉世界模型，而是要建模接触状态如何随动作变化。现有证据显示，真正有用的触觉世界模型至少要同时处理四件事：接触变量表征、时间同步数据、动作条件预测、闭环使用接口；如果只增加一个触觉模态而不解决表征兼容和评测闭环，收益会不稳定（ContactWorld, Visuo-Tactile World Models）。

问题结构

第一层是感知变量。RGB 看不到接触力、摩擦、滑移、材料和被遮挡局部几何；

本轮论文进一步把这个差异落到世界模型上：Visuo-Tactile World Models 用触觉 grounding 改善物体持续性和物理一致性，TacForeSight 用 wrist force/torque 条件化未来触觉潜在状态，HapTile 还把 marker displacement 作为显式接触几何和滑移线索保存（Visuo-Tactile World Models, TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation, HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation Learning）。

第二层是表征与预测。ContactWorld 的关键结论是，触觉世界模型的收益依赖表征结构和跨模态兼容；点云和触觉力场组合在其 基准 中给出最强平均规划表现，但真实机器人附录也显示 TacDepth/TacFF 不总是稳定增益，原因包括标定和力推断噪声。Dream-Tac 则说明触觉信号是事件驱动、稀疏且随接触变化突增的，因此使用 contact gate 和 contact-aware attention，而不是把触觉当作持续视觉通道（ , ContactWorld: What Matters in Vision-Tactile World Models for Contact-Rich Manipulation, Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation）。

第三层是数据工程。Visuo-Tactile World Models 的小规模但完整数据包括 124 条遥操作演示、约 112k datapoints、8 个接触任务、成功和失败轨迹、本体状态、外部视频与四个 Digit 360 指尖视频；OmniVTA 扩展到 21,879 条轨迹、86 任务、126 对象和多传感器；HapTile 提供 1,726 条演示、38 任务、9 类技能、语言指令、15Hz 同步视觉/触觉/状态/动作；

第四层是闭环使用。OmniVTA 把预测的接触演化接到 60Hz reflexive tactile controller；TacForeSight 把预测触觉 潜在状态 作为 anticipatory contact priors；ViTaL 把 visuo-触觉潜在状态 world model 用于推理期候选动作验证；AT-VLA 则把视觉语言慢流和触觉快流分离，使用 3:1 快慢流比例处理接触反馈。

主要共识

共识一：视觉世界模型在接触丰富任务中缺关键状态。遮挡、视觉别名、局部接触、滑移和微小力变化，会让纯视觉预测在“最后几厘米”失真；触觉提供的是局部物理状态，不只是另一个图像视角。Visuo-Tactile World Models、TacForeSight、HapTile 等论文从模型、预测和数据三侧直接支持这点（Visuo-Tactile World Models, TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation, HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation Learning）。

共识二：模型必须显式尊重时间。触觉是接触事件驱动的，高速力/力矩可以先于指尖触觉变化，未来触觉 潜在状态 比当前触觉拼接更适合做 proactive control；同时长时域规划会放大接触预测误差。

这一结论在 TacForeSight、ContactWorld 和 Dream-Tac 中分别表现为 force-conditioned tactile foresight、long-horizon tactile importance 和 contact-gated attention（ ContactWorld: What Matters in Vision-Tactile World Models for Contact-Rich Manipulation, TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation, Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation）。

共识三：数据质量的定义必须包括闭环可用性。数据采集要同时管理硬件、同步、标定、动作、元数据和质量控制；TAMEn 对“成功演示不能代表可执行机器人行为”的批评尤其重要，因为接触丰富任务的临界失败往往来自力积累、早期滑移、局部变形或支撑不稳，这些状态只有真实物理交互和恢复数据才能覆盖（TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks）。

条件与分歧

触觉不是无条件增益。ContactWorld 的真实实验显示，点云对阀门旋拧很强，TacRGB 对图像视角有帮助，但 TacDepth/TacFF 在真实设置中并不总是稳定；作者把一部分原因归到传感器标定、深度和力推断噪声。这个结果提醒我们：触觉表示、视觉表示和动作模型 潜在状态 必须兼容，否则“多模态”可能变成噪声注入（ ContactWorld: What Matters in Vision-Tactile World Models for Contact-Rich Manipulation;）。

在线触觉与离线触觉蒸馏代表两条部署路线。OmniVTA、TacForeSight、ViTaL 和 AT-VLA 偏向在线感知和实时修正；HapticVLA 则主张通过安全接触奖励与 tactile distillation，在推理期不依赖触觉传感器也能获得触觉 aware 行为。前者适合扰动丰富、接触不可预测的场景，后者适合硬件成本或跨平台复现压力大的场景；

推理期使用世界模型会引入新的误差链。ViTaL 明确指出 潜在状态 world model 的保真度会影响候选动作验证，尤其是细微接触事件；触觉编码器的预训练规模也远小于现代视觉语言模型。这意味着“用世界模型挑动作”并不自动比行为克隆安全，除非评测覆盖 compounding error、接触误报/漏报和高频控制延迟（ Inference-time Policy Steering via Vision and Touch;）。

未解决问题

第三，触觉世界模型尚未形成统一的数据规范。HapTile 强调语言、动作、同步和 marker motion，TAMEn 强调可执行性与恢复，Visuo-Tactile World Models 强调成功/失败轨迹，OmniVTA 强调多任务多传感器。

对后续研究的启发

如果要做“触觉世界模型”，建议把研究对象定义为“动作条件的接触未来预测”，而不是“视觉世界模型加触觉输入”。最低可行模型可以预测短时域 触觉潜在状态、contact mask/force-field 或 marker displacement，并把预测接到 MPC、policy steering 或动作 head；

TacForeSight 和 Dream-Tac 分别给了 force-conditioned 潜在状态 prediction 与 world-action generation 两种参考形态（ TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation, Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation; model formulation follows from current strongest methods）。

数据采集上，建议设计“触觉世界模型数据五件套”：时间同步的视觉/触觉/动作/本体状态，至少一种 force 或 tactile-derived contact geometry，成功与失败演示，扰动恢复片段，机器人侧可执行性与闭环 replay 检查。

这个建议直接来自 Visuo-Tactile World Models、HapTile 和 TAMEn 对模型、数据与可执行性的共同约束（Visuo-Tactile World Models, HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation Learning, TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks）。

中心判断的工程化表达

触觉世界模型应被定义为“动作条件下的接触状态预测器”，而不是增加触觉输入的视频模型。它至少要回答三件事：接触是否建立，接触如何随动作变化，预测能否在控制截止前改变动作。若只重建触觉图像，却不能改善滑移检测、恢复或安全施力，就没有证明世界模型的控制价值。

数据单元应围绕接触事件组织。每个片段需要同步视觉、触觉、力矩、本体状态和动作，标出首次接触、稳定抓取、滑移、脱离与恢复。成功演示之外，还要采集扰动和失败邻域；这些片段决定模型是否只会复现理想路径，还是能在真实偏差中修正。

模型接口也应分时标。语言与视觉负责低频任务理解，触觉和力负责高频局部修正；快速通道不必重复完整语义推理，慢速通道也不能忽略接触告警。分层结构能降低延迟，却要求两条通道共享时间和动作状态，否则融合会制造新的错误。

评测应从四级展开：预测能否识别接触事件，短时动作能否降低滑移或过力，长时任务能否稳定完成，跨传感器与材料是否仍然校准。表征分数可以帮助诊断，但不能替代真实控制结果。尤其要报告错误拒绝与不确定性，避免世界模型在细微接触上自信地给出错误建议。

部署路线可以分成在线感知和离线蒸馏。在线触觉适合扰动不可预测、接触风险高的任务；离线蒸馏适合硬件受限、环境稳定的场景。两者的选择取决于失败代价与状态可预测性，而不是单纯比较模型规模。

精读复核后的证据重心

移出无法取得完整正文的几篇论文后，领域主线仍然清晰，但评测语言应更谨慎。ContactWorld 表明触觉并非“加了就好”，其效果依赖视觉与触觉表征在空间结构、时间连续性和跨模态动力学上是否兼容；Dream-Tac 把未来视觉、未来触觉和动作块统一为世界动作目标，但它仍然是特定传感器与任务上的结果。

数据与评测两端也更具体。TAMEn 将动捕精度模式、VR 便携模式与人在环恢复数据组合成金字塔数据配方；SoftVTBench 则说明只看任务成功会忽略滑移、掉落和过度形变等安全错误。因此，触觉世界模型的最小完整链条是：同步接触数据、可执行的未来预测、扰动恢复与独立安全指标。

References

Accepted evidence

去重论文引用

  1. AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action ModelsXiaoqi Li, Muhe Cai, Jiadong Xu, Juan Zhu, Hongwei Fan, Yan Shen, Guangrui Ren, Hao Dong · 2026
  2. ContactWorld: What Matters in Vision-Tactile World Models for Contact-Rich ManipulationZhiyuan Zhang, Pokuang Zhou, Kaidi Zhang, Adeesh Desai, Temitope Amosa, Davood Soleymanzadeh, Jiuzhou Lei, Minghui Zheng et al. · 2026
  3. Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot ManipulationYunfan Lou, Yifan Ye, Yankai Fu, Jun Cen, Xiaowei Chi, Yaoxu Lyu, Peidong Jia, Sirui Han et al. · 2026
  4. HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation LearningAmirhosein Alian, Yongqiang Zhao, Shiyi Gu, Xuyang Zhang, Zhuo Chen, Christopher E. Mower, Haitham Bou-Ammar, Shan Luo · 2026
  5. Inference-time Policy Steering via Vision and TouchYilin Wu, Zilin Si, Zeynep Temel, Oliver Kroemer, Andrea Bajcsy · 2026
  6. Multisensory Continual Learning: Adapting Pretrained Visuomotor Policies to ForceJaden Clark, Changhao Wang, Yihuai Gao, Seongheon Hong, Hojung Choi, Mark Cutkosky, Yifan Hou, Shuran Song · 2026
  7. PATCH: Action-Chunk-Conditioned Latent Patch Innovation Monitoring for Robot ManipulationYanan Zhou, Ranpeng Qiu, Yincong Chen, Jiajie Cui, Weiming Zhi · 2026
  8. RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous ManipulationShengcheng Luo · 2026
  9. SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable ObjectsBowen Jing, Mingxin Wang, Ruiyang Hao, Chenchen Ge, Hanwen Shen, Junjie He, Yang Cui, Yiming Hou et al. · 2026
  10. TacForeSight: Force-Guided Tactile World Model for Contact-Rich ManipulationYujie Zang, Yuhang Zheng, Xian Nie, Yupeng Zheng, Shuai Tian, Songen Gu, Chen Gao, Zining Wang et al. · 2026
  11. Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric AttentionSiyu Wu, Linjing You, Junjie Zhu, Yaozu Liu, Changhao Zhang, Jian Liu, Weiqiang Wang, Qi Li et al. · 2026
  12. TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich TasksLongyan Wu, Jieji Ren, Chenghang Jiang, Junxi Zhou, Shijia Peng, Ran Huang, Guoying Gu, Li Chen et al. · 2026
  13. Uncertainty Quantification for Flow-Based Vision-Language-Action ModelsRalf Römer · 2026
  14. Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model SimulatorsChristian Oefinger · 2026
  15. Visuo-Tactile World ModelsCarolina Higuera, Sergio Arnaud, Byron Boots, Mustafa Mukadam, Francois Robert Hogan, Franziska Meier · 2026
完整证据附录（15 条正式证据事件）
  • Time range: 2026-01-14..2026-07-14
  • Events: 15
  • 每个事件一节,标题即锚点;trace-map 中的 event 链接跳转到这里。

EA-TWM-READ-0008

  • Claim: TAMEn 用动捕精度模式与 VR 便携模式平衡数据质量和环境多样性，并把人在环的触觉可视化恢复数据纳入金字塔式数据配方。
  • Stance: support | Confidence: direct
  • Paper: 2604.07335 TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks
  • Locator: Abstract (full-text section)
  • Evidence: 摘要明确列出精度/便携双模式采集、触觉恢复遥操作和人在环恢复数据。
  • Quote: “To balance data quality and environmental diversity, we implement a dual-mode acquisition pipeline: a precision mode leveraging motion capture for high-fidelity demonstrations, and a portable mode utilizing VR-based tracking for in-the-wild acquisition and tactile-visualized recovery teleoperation. Building on this hardware, we unify large-scale tactile pretraining, task-specific bimanual demonstrations, and human-in-the-loop recovery data into a pyramid-structured data regime, enabling closed-l”
  • Authors: longyan-wu; jieji-ren; chenghang-jiang; et al.

EA-TWM-READ-0005

  • Claim: 触觉数据的有效形态不止原始 tactile image，还包括 marker displacement 等显式接触几何与滑移特征。
  • Stance: support | Confidence: direct
  • Paper: 2606.04825 HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation Learning
  • Locator: 4.2 Vision-Based Tactile Sensing and Marker Tracking
  • Evidence: HapTile 的每个夹爪手指安装视觉触觉传感器，接触会带来图像变化和 marker displacement；论文把 marker-motion 信号保存进数据集并用于 haptic feedback，实验也比较 vision-only、vision+tactile image 与 vision+tactile+marker 表征。
  • Quote: “We smooth the signal with separate rise and release factors, producing a stable tactile motion estimate. The resulting marker-motion signal is stored in HapTile and is also used to drive haptic feedback.”
  • Authors: amirhosein-alian; yongqiang-zhao; shiyi-gu; et al.

EA-TWM-READ-0002

  • Claim: 触觉世界模型可以被扩展为同时生成未来视觉、未来触觉和动作 chunk 的世界动作模型。
  • Stance: support | Confidence: direct
  • Paper: 2606.08737 Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: Dream-Tac 把 world action model 扩展到触觉，联合建模当前视觉、触觉、语言指令下的未来视觉观测、未来触觉观测和动作 chunk，并加入 contact-gated visuotactile fusion 与 contact-aware attention bias。
  • Quote: “In this paper, we propose Dream-Tac, a unified Tactile-World Action Model that jointly models actions, future visual observations, and tactile dynamics. Specifically, Dream-Tac introduces (i) contact-gated visuotactile fusion to selectively integrate tactile signals and (ii) a contact-aware attention bias to better regulate cross-modal interactions during manipulation.”
  • Authors: yunfan-lou; yifan-ye; yankai-fu; et al.

EA-TWM-READ-0006

  • Claim: 腕部六维力/力矩可作为未来触觉 latent 的先行条件，用于预测短时域接触变化。
  • Stance: support | Confidence: direct
  • Paper: 2606.11184 TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
  • Locator: IV-D 1 World Model Conditioning
  • Evidence: TacForeSight 的 TacForceWM 从双指触觉观测出发，以高频腕部 force/torque 为条件预测短时域触觉 latent dynamics；ablation 中 wrist wrench 条件的未来触觉预测优于无条件版本，MSE 从 0.027 降到 0.017，cosine 从 0.954 提升到 0.992。
  • Quote: “We first ablate the conditioning modality used in the tactile world model. We compare four conditioning designs: no external condition, RGB image condition, robot state condition, and wrist wrench condition. Prediction quality is evaluated using Mean Squared Error (MSE), cosine similarity (Cos), and symmetric KL divergence (KL sym ), which respectively measure element-wise prediction error, latent directional consistency, and latent distribution consistency. As shown in Table II , wrist wrench c”
  • Authors: yujie-zang; yuhang-zheng; xian-nie; et al.

EA-TWM-READ-0003

  • Claim: 在接触丰富操作中，触觉世界模型的关键不是简单增加模态，而是让表征同时具备空间结构、时间连续性和跨模态兼容性。
  • Stance: support | Confidence: direct
  • Paper: 2606.13877 ContactWorld: What Matters in Vision-Tactile World Models for Contact-Rich Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: ContactWorld 在 12 个接触丰富任务上比较视觉与触觉表征；点云把平均规划成功率从腕部视角 20.7% 和前视 22.0% 提升到 32.1%，点云加触觉力场进一步到 36.1%。作者强调触觉效果取决于跨模态表征兼容，而非模态数量本身。
  • Quote: “Abstract Contact-rich manipulation requires world models to reason over complex contact dynamics from multimodal sensory observations. However, it remains unclear which representation properties fundamentally support stable long-horizon planning in contact-rich settings. In this paper, we present ContactWorld, a benchmark and systematic empirical study of vision-tactile world models spanning 12 contact-rich manipulation tasks, including insertion, disassembly, screwing, and exploratory interacti”
  • Authors: zhiyuan-zhang; pokuang-zhou; kaidi-zhang; et al.

EA-TWM-READ-0004

  • Claim: 触觉世界模型也可以在推理期作为候选动作验证器，而不只是训练期的动态模型。
  • Stance: support | Confidence: direct
  • Paper: 2606.14981 Inference-time Policy Steering via Vision and Touch
  • Locator: 5 Experiments
  • Evidence: ViTaL 学习 visuo-tactile latent world model，结合视觉和文本条件触觉 verifier，对候选动作进行长时域视觉模式选择和短时域触觉 refinement；真实机器人任务包括 wiping、insertion 和 pipette transfer。
  • Quote: “We evaluate ViTaL on three real-world contact-rich manipulation tasks. Our experiments are designed to answer three questions: whether multimodal visuo-tactile steering improves policy over unimodal guidance (Sec. 5.1 ); whether a learned latent world model and semantically aligned verifier provides reliable rewards for predicted outcomes (Sec. 5.2 ); and whether the proposed bi-level optimization balances performance and efficiency compared to naive visual-tactile fusion (Sec. 5.3 ). Robot Setu”
  • Authors: yilin-wu; zilin-si; zeynep-temel; et al.

EA-TWM-READ-0015

  • Claim: Flow-based VLA 在真实部署时缺少置信度机制会形成安全与恢复缺口；velocity-field disagreement 可把动作预测的不可靠性转成失败检测和主动微调信号。
  • Stance: support | Confidence: direct
  • Paper: 2606.18043 Uncertainty Quantification for Flow-Based Vision-Language-Action Models
  • Locator: Abstract (full-text section)
  • Evidence: 作者将真实非平稳环境中的分布外场景描述为 VLA 可能“无预警失败”的关键限制，并提出用小 ensemble 的 velocity-field disagreement 量化 epistemic uncertainty；LIBERO 实验显示该不确定性与下游表现、失败检测和主动采样相关。
  • Quote: “This presents a critical limitation for real-world deployment in non-stationary environments, where models inevitably encounter scenarios outside their pretraining distribution and may fail without warning. To address this, we derive an efficient method for quantifying epistemic uncertainty in flow-matching models by leveraging velocity-field disagreement (VFD) across a small ensemble.”
  • Authors: ralf-rmer

EA-TWM-READ-0001

  • Claim: VT-WM 的训练序列同步记录腕部位姿、关节位置、外部视觉和两个指尖触觉视频，并使用时间戳对齐后降采样训练。
  • Stance: conditional | Confidence: direct
  • Paper: 2602.06001 Visuo-Tactile World Models
  • Locator: B.0.1 Training dataset
  • Evidence: 训练数据段明确列出了同步的本体状态、外部视频与双指触觉视频数据流。
  • Quote: “Each sequence contains multimodal data streams: proprioceptive information (wrist pose, joint positions), exocentric video from the camera, and video from each Digit 360 fingertip sensor. All data streams were synchronized using timestamps and downsampled to 6 FPS for training the world model. Our training dataset for V-WM and VT-WM consists of 124 demonstrations totaling 112k datapoints, with each demonstration averaging 40 seconds. For validation, we use 26 demonstrations spanning all tasks, c”
  • Authors: carolina-higuera; sergio-arnaud; byron-boots; et al.

EA-TWM-READ-0007

  • Claim: 触觉信息用于 VLA 或世界模型时，低频视觉语言理解和高频触觉反馈应在架构上分离。
  • Stance: conditional | Confidence: direct
  • Paper: 2605.07308 AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action Models
  • Locator: 5 Conclusion
  • Evidence: AT-VLA 把系统分为慢速视觉语言流和快速触觉流，慢速流负责任务理解和视觉定位，快速流以高频处理触觉反馈；作者采用 3:1 的快慢流频率比，并在真实接触丰富任务中验证 adaptive tactile injection、tactile gate、adaptive cross-attention 和 reaction dual-stream 的作用。
  • Quote: “In summary, AT-VLA introduces an adaptive framework that seamlessly integrates tactile sensing into vision-language-action models. Through the Adaptive Tactile Injection mechanism, AT-VLA dynamically balances pretrained visual-language knowledge with newly learned tactile representations, preserving model integrity while enhancing action precision. The Tactile Reaction Dual-Stream mechanism further enables rapid, high-frequency tactile responses by decoupling slow perceptual reasoning from fast”
  • Authors: xiaoqi-li; muhe-cai; jiadong-xu; et al.

EA-TWM-READ-0014

  • Claim: 在视觉遮挡或视觉退化下，显式把触觉接触投影到图像域可以改善空间对齐；但这种收益依赖对运动学和标定误差导致的空间不确定性建模。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.08765 RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 作者称视觉观测不可靠或被遮挡时，稀疏异构触觉与稠密视觉表示的对齐是核心挑战；方法使用正运动学和相机标定投影触觉传感器位置，并用力调制高斯 saliency maps 建模运动学和标定误差带来的空间不确定性。
  • Quote: “We then render force-modulated Gaussian saliency maps to model spatial uncertainty arising from kinematic and calibration errors.”
  • Authors: shengcheng-luo

EA-TWM-READ-0010

  • Claim: 触觉不是简单加 token 就能提升；接触任务中的 RGB 未来可能视觉上合理但物理上不完整，而无约束触觉注入还可能污染视频和动作预测。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.26663 Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出 insertion、assembly、search、reorientation 依赖 slip、jamming、contact normals 和小对齐误差，这些状态在 RGB 中弱可见或不可见；同时他们定义 tactile pollution：无约束触觉 token 注入会迫使视觉 dynamics model 吸收稀疏局部事件式接触信号，从而退化视频和动作预测。
  • Quote: “In contact-rich manipulation, however, visually plausible futures can be physically incomplete: insertion, assembly, search, and reorientation often depend on slip, jamming, contact normals, or small alignment errors that are weakly visible or hidden in RGB. A natural solution is to predict future tactile states, however, we identify tactile pollution , a failure mode where unconstrained tactile-token injection degrades video and action prediction by forcing a visual dynamics model to absorb spa”
  • Authors: siyu-wu; linjing-you; junjie-zhu; et al.

EA-TWM-READ-0011

  • Claim: 力觉/触觉/音频能揭示图像不可直接观测的交互状态；但这些模态硬件和任务依赖强，所以需要在少量多传感数据上适配既有视觉策略，而不是预训练时穷尽所有传感器。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.30988 Multisensory Continual Learning: Adapting Pretrained Visuomotor Policies to Force
  • Locator: Abstract (full-text section)
  • Evidence: 作者称接触丰富任务常依赖 vision 之外的 sensory data，force、tactile 或 audio feedback 能揭示 images 中不可直接观察的 interaction states；但这些模态 hardware- and task-specific，且大规模多传感数据稀缺。他们提出 MuSe，将 limited multisensory data 融入 pretrained vision-only policies，并以 force-torque sensing 做真实任务案例。
  • Quote: “Abstract Robot manipulation often depends on sensory data beyond vision, especially in contact-rich tasks where force, tactile, or audio feedback reveals interaction states not directly visible from images. Yet such modalities are hardware- and task-specific, and large multisensory datasets remain scarce, making it impractical to pretrain policies with every sensor they may encounter. We study multi-sensory continual learning : adapting a pretrained robot policy to new tasks with newly introduce”
  • Authors: jaden-clark; changhao-wang; yihuai-gao; et al.

EA-TWM-READ-0009

  • Claim: 全局视觉异常、帧级变化或策略不确定性不足以判断感知误差是否会影响当前动作；部署监控需要把异常限定到 action chunk 将要使用的执行走廊。
  • Stance: limit | Confidence: direct
  • Paper: 2606.16690 PATCH: Action-Chunk-Conditioned Latent Patch Innovation Monitoring for Robot Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出开放工作空间中移动物体、瞬时遮挡和目标运动附近扰动会让部署脆弱；现有 runtime monitors 往往依赖全局 observation anomalies、policy uncertainty 或 frame-level visual changes，难以区分任务相关执行风险和无害视觉变化。PATCH 通过 active action chunk 的 projected execution corridor 累计持续残差作为介入信号。
  • Quote: “Abstract Learning-based manipulation policies have made substantial progress in real-world robot manipulation, particularly for short-horizon action generation. However, deployment in open workspaces remains fragile under unexpected local scene dynamics, such as moving objects, transient occlusions, or disturbances near the intended motion. Existing runtime monitors often rely on global observation anomalies, policy uncertainty, or frame-level visual changes, and struggle to distinguish task-rel”
  • Authors: yanan-zhou; ranpeng-qiu; yincong-chen; et al.

EA-TWM-READ-0012

  • Claim: 只看任务完成会低估传感器感知误差的真实风险；柔性物操作需要把安全交互、滑移/掉落和过度形变作为独立评测目标。
  • Stance: limit | Confidence: direct
  • Paper: 2607.04234 SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects
  • Locator: 1 Introduction
  • Evidence: 作者指出现有 manipulation benchmarks 多以 success 为中心，很少评估执行过程是否物理安全；SoftVTBench 分开报告 Goal Success 和 Safety Success，后者要求无掉落并限制峰值形变。实验显示 success-only evaluation 会显著高估策略表现，而触觉感知可改善 Safety Success 并降低物体形变。
  • Quote: “Experiments show that success-only evaluation substantially overstates policy performance, as a large fraction of goal-completing rollouts violate physical safety, and that adding tactile sensing improves Safety Success while keeping Goal Success comparable and reduces object deformation during execution.”
  • Authors: bowen-jing; mingxin-wang; ruiyang-hao; et al.

EA-TWM-READ-0013

  • Claim: 世界模型/生成式仿真器不能仅凭视觉逼真度作为具身策略评测裁判；必须验证其是否按动作正确响应，并建立 admissibility/validation 梯度后才能把闭环判决当证据。
  • Stance: gap | Confidence: direct
  • Paper: 2607.07196 Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出机器人中 World Models 越来越被用于模拟动作后果并给出 success/safety verdict，但视频生成指标如 FVD 奖励视觉真实感，却忽略世界是否对 policy actions 正确响应；他们主张作为 test oracle 的 WM 需要先通过 accreditation，并提出 L0-L4 admissibility ladder。
  • Quote: “Abstract Across robotics, World Models (WMs) are increasingly used to evaluate action policies by simulating the consequences of actions in an imagined world, and returning a success or safety verdict. Yet a verdict is only as trustworthy as the WM that produced it, and the WM itself needs to be certified. In video-generation WMs , fidelity metrics such as Fréchet Video Distance (FVD) reward visual realism, but ignore whether the world responds correctly to the policy’s actions, including those”
  • Authors: christian-oefinger

References

Interactive Wiki

进入交互阅读

静态页采用默认版本；v0、v0、v0可在同一交互 Wiki 中阅读，不建立相互竞争的索引页。

用 AI 继续研究 v0 v0 v0

Method and limits

研究方法、证据边界与引用说明

本页由当前结算的证据层自动派生，不改写已结算 run。论文只有在完整非 OCR 全文、结构化论文笔记和 claim-support audit 通过后，才进入接受证据。

页面结论是对所列论文与研究窗口的综合，不等同于无限外推。引用本专题时，请同时保留本页 canonical 地址、访问日期，并在需要时回溯原论文。