多模态感知 · Multimodal Perception

近一年触觉在具身机器人领域的发展

Progress in Tactile Sensing for Robotics

更新于

30 秒结论

先读这个判断

本文考察 2025 年 7 月 20 日至 2026 年 7 月 20 日的触觉机器人研究,重点回答:触觉是否已经从专用传感器演示,进入可复用的具身学习与控制栈?本轮采用 scoping review,复用并增量审核了 867 篇候选、124 篇可读全文,最终接纳 23 篇完整非 OCR 全文和 24 条证据事件。范围覆盖触觉数据与表征、视觉—触觉世界模型、VLA/策略融合、灵巧与全身操作、评测及部署边界。

精读论文
23
正式证据事件
24
知识单元
EA-SENSOR, EA-DATA, EA-MODEL, EA-EVAL
审计状态
当前结算 run 已通过仓库校验

v0 · 完整正文

研究综述

研究边界

本文考察 2025 年 7 月 20 日至 2026 年 7 月 20 日的触觉机器人研究,重点回答:触觉是否已经从专用传感器演示,进入可复用的具身学习与控制栈?本轮采用 scoping review,复用并增量审核了 867 篇候选、124 篇可读全文,最终接纳 23 篇完整非 OCR 全文和 24 条证据事件。范围覆盖触觉数据与表征、视觉—触觉世界模型、VLA/策略融合、灵巧与全身操作、评测及部署边界。

这里的“触觉”包括局部压力/形变、视觉触觉图像、marker displacement、滑移和接触状态;腕部六维力/力矩作为低维全局受力信号单列,但在多篇系统中与触觉共同组成反馈链。本文不讨论面向人的触觉反馈、医疗假肢或纯材料器件性能,也不把候选论文、摘要或网页信息当成已验证证据。

中心判断

近一年最实质的进展,不是某一种触觉传感器突然“解决了机器人触觉”,而是触觉开始成为一条跨越数据、预测和控制的接触执行栈:接触前由视觉与语言提供任务语义和全局几何,接触中由动作条件世界模型预测触觉演化,接触后由高频触觉通道纠正滑移、卡滞、力不匹配和抓持不稳。这一判断可以被证伪:如果触觉只能提高离线重建或分类指标,却不能在遮挡、精密装配、软物体或扰动条件下改善闭环成功、安全或恢复,它仍只是感知附件。

现有证据支持“栈正在形成”,但不支持“通用触觉基础模型已经成熟”。硬件异构、时间同步、跨传感器迁移、长期漂移以及闭环评测仍是决定性约束。

一、数据底座从触觉图像扩展为接触状态序列

过去的常见做法是把一帧触觉图像附加到视觉策略。近一年的数据设计更接近“可重放的接触过程”:同步记录视觉、双指触觉、腕部位姿、关节状态和动作,并把时间对齐本身视为学习条件。Visuo-Tactile World Models 的训练序列就是这种同步多流结构;HapTile 进一步保留 marker displacement,使接触几何和滑移不再隐没在原始图像中。UMI-FT 则说明,在接触丰富任务里,便携采集只有同时记录力/力矩、深度、位姿与抓持力,才可能超越视觉—轨迹监督的上限。

规模也开始上升。HT-Bench 汇集约 1000 万 RGB 帧、780 万触觉帧和 226 项任务,并用检索、掩码补全、视觉到触觉合成和触觉帧预测分别测试空间结构、跨模态对齐和时间动态。Deform360 则把软物体的多视角全局运动、局部触觉形变和 3D 轨迹放进同一数据闭环。这表明“触觉数据”正在从单传感器读数变为视觉—触觉—动作—几何的同步事件流。

但规模化尚未等于通用化。HT-Bench 明确没有覆盖指尖光学触觉、力/力矩、皮肤式 taxel 和非手部本体,且四项任务仍是表征层评测,没有直接测量真实机器人闭环性能。可以推断:下一阶段的核心不是继续堆帧数,而是让数据模式能够跨传感器实例、跨本体和跨维护周期对齐;这一推断若被大规模跨硬件闭环迁移实验否定,就需要修正。

二、模型架构从“多一个 token”转向分时标接触控制

触觉和视觉的时间结构不同。视觉与语言适合较低频的物体识别、任务理解和接触前规划;滑移、卡滞与力突变要求更快反应。AT-VLA 因而把低频视觉语言流与高频触觉流分开,而不是把触觉简单拼到统一 token 序列。TouchWorld 更进一步,将视觉语言子任务规划、触觉世界模型预测、视觉—触觉动作生成和高频触觉残差纠正分成层级,使“预期接触”与“即时反射”同时存在。

这解释了为什么有些多模态模型增益有限。Tactile-WAM 指出,触觉是稀疏、局部、事件驱动的信号;无约束注入会让视觉 dynamics model 吸收不匹配的统计结构,反而污染视频和动作预测。真正的融合问题不是“有没有触觉”,而是:何时触发、注入哪一层、以什么空间参考系对齐,以及它是否改变当前动作 chunk。

RGB-S 展示了另一条路线:把稀疏触觉接触投影到图像域,并显式建模运动学与标定误差带来的空间不确定性。它说明视觉—触觉对齐不是抽象 embedding 相似度问题,而是接触点、相机和机器人运动学之间的几何问题。

三、触觉世界模型从“生成未来”走向“筛选和纠正动作”

世界模型是近一年触觉研究最明显的汇合点。Dream-Tac 联合生成未来视觉、未来触觉和动作 chunk;TacForeSight 用腕部六维力/力矩作为未来触觉 latent 的先行条件,说明全局受力和局部接触分布具有互补时间结构。ContactWorld 的比较则表明,收益来自空间结构、时间连续性和跨模态兼容性,而不是模态数量本身。

更重要的转变发生在推理期。Inference-time Policy Steering via Vision and Touch 用视觉—触觉 latent world model 在候选动作之间先做长时域视觉筛选,再以短时域触觉细化。触觉世界模型因此不再只是训练期的生成器,而成为动作验证器。TACO 又把失败附近的真实状态转成带纠正动作的想象片段,用于接触丰富任务的 VLA 后训练。两者共同指向“预测—验证—纠正”闭环。

不过,生成触觉逼真并不能自动获得控制权。世界模型若要充当评测裁判,必须先证明它按动作正确响应,而不是只生成视觉上可信的未来;Validate the Dream Before You Trust Its Verdict 因而主张建立可采信等级。对触觉系统而言,这意味着必须分别验证对象持久性、接触状态、动作忠实度和真实闭环相关性。

四、触觉的作用域从末端执行器扩到全身控制

触觉正在越过“手指传感器”的边界。在四足 loco-manipulation 中,接触变化会同时影响夹爪、手臂轨迹和身体稳定。Learning Tactile-Aware Quadrupedal Loco-Manipulation Policies 报告,低层加入 tactile-command tracking 后,插入成功率由 0.70 提升到 0.85,重定向加插入由 0.60 提升到 0.80。TAC-LOCO 在 10 次匹配真机试验中取得 90% 动态全身操作成功率,对照固定夹爪基线为 50%。

这些结果的意义不只在数字,而在控制接口:触觉不再在高层回答“是否接触”,而是进入低层命令跟踪和全身协调。适用边界也很明确——样本规模小、机器人与传感器特定,不能外推为所有 humanoid 或移动操作任务的通用收益。

五、评测正在从“完成任务”转向“安全地完成接触过程”

接触任务只报告成功率,会把压坏、过力、滑移和掉落隐藏在“成功”中。SoftVTBench 将 Goal Success 与 Safety Success 分开,要求无掉落并限制峰值形变,直接显示 success-only evaluation 会高估策略表现。部署监控也在转向局部执行走廊:PATCH 指出,全局视觉异常或帧级变化无法判断异常是否会影响当前动作;风险必须限定到 active action chunk 即将经过的区域。

因此,触觉项目的最小评测集不应只有任务成功率,还应包括接触检测延迟、滑移/掉落、过力、峰值形变、扰动恢复、跨传感器实例退化、模态缺失退化和风险—覆盖。没有这些指标,触觉模型很容易在离线数据上显得先进,却无法解释真机为什么更可靠。

条件、分歧与研究空白

第一,没有“最佳触觉传感器”的普适答案。本轮跨传感器、同任务、同策略预算的完整闭环比较仍不足;传感器空间分辨率、剪切力能力、材料摩擦、安装位置和维护成本必须相对于任务选择。

第二,跨硬件泛化仍是薄弱环节。触觉数据格式、接触面积、坐标系和噪声形态高度依赖传感器与本体。当前更现实的路线,是像Multisensory Continual Learning 那样,用少量目标硬件的多传感数据适配既有视觉策略,而不是假设预训练一次覆盖所有传感器。

第三,长期部署证据不足。标定漂移、表面磨损、换件后重标定、温度与材料变化鲜少进入主流基准。这里应区分“本轮未充分覆盖”与“论文明确声明开放问题”:前者是本综述的覆盖边界,不能冒充领域共识。

第四,触觉不能替代柔顺控制与机构设计。视觉、触觉、力控、阻抗和末端执行器共同决定最后一厘米的可执行性;任何单模态模型若缺少机械顺应性和动作约束,都可能把更好的状态估计转化为更大的接触力。

对研究与系统设计的含义

一个可操作的触觉栈应按四层验收:数据层记录同步视觉、触觉、力/力矩、动作和接触事件;表征层验证空间、跨模态和时间结构;策略层分离慢速语义规划、动作级预测和快速触觉纠错;部署层报告安全过程、恢复、漂移和跨实例退化。只有后一层出现稳定增益,前面三层的创新才算转化为具身能力。

结论

近一年触觉机器人研究已经越过“感知到接触”的阶段,开始回答“如何预测、验证并调节接触”。最有价值的变化是触觉进入分时标策略、世界模型和全身控制,并推动评测从任务终态走向接触过程安全。与此同时,通用触觉基础模型仍受硬件异构、同步标定、跨实例迁移和闭环证据约束。更稳妥的判断是:触觉的基础设施化刚刚开始,领先系统已经展示了局部闭环价值,但领域尚未完成从论文装置到可维护、可比较、可部署标准件的跨越。

References

Accepted evidence

去重论文引用

  1. AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action ModelsXiaoqi Li, Muhe Cai, Jiadong Xu, Juan Zhu, Hongwei Fan, Yan Shen, Guangrui Ren, Hao Dong · 2026
  2. ContactWorld: What Matters in Vision-Tactile World Models for Contact-Rich ManipulationZhiyuan Zhang, Pokuang Zhou, Kaidi Zhang, Adeesh Desai, Temitope Amosa, Davood Soleymanzadeh, Jiuzhou Lei, Minghui Zheng et al. · 2026
  3. Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World ModelsHongyu Li, Wanjia Fu, Xiaoyan Cong, Zekun Li, Binghao Huang, Hanxiao Jiang, Xintong He, Yiqing Liang et al. · 2026
  4. Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot ManipulationYunfan Lou, Yifan Ye, Yankai Fu, Jun Cen, Xiaowei Chi, Yaoxu Lyu, Peidong Jia, Sirui Han et al. · 2026
  5. HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation LearningAmirhosein Alian, Yongqiang Zhao, Shiyi Gu, Xuyang Zhang, Zhuo Chen, Christopher E. Mower, Haitham Bou-Ammar, Shan Luo · 2026
  6. HT-Bench: Benchmarking and Learning Dexterous Full-Hand Tactile Representations with Egocentric VisionYuzhe Huang, Jiaping Wu, Jiaming Jiang, Hezhe Lin, Aikebaier Aierken, Yunlong Wang, Kun Cheng, Ziyuan Jiao et al. · 2026
  7. In-the-Wild Compliant Manipulation with UMI-FTHojung Choi, Yifan Hou, Chuer Pan, Seongheon Hong, Austin Patel, Xiaomeng Xu, Mark R. Cutkosky, Shuran Song · 2026
  8. Inference-time Policy Steering via Vision and TouchYilin Wu, Zilin Si, Zeynep Temel, Oliver Kroemer, Andrea Bajcsy · 2026
  9. Learning Tactile-Aware Quadrupedal Loco-Manipulation PoliciesPokuang Zhou, Yuhao Zhou, Quan Khanh Luu, Seungho Han, Heng Zhang, Binghao Huang, Yunzhu Li, Arash Ajoudani et al. · 2026
  10. Multisensory Continual Learning: Adapting Pretrained Visuomotor Policies to ForceJaden Clark, Changhao Wang, Yihuai Gao, Seongheon Hong, Hojung Choi, Mark Cutkosky, Yifan Hou, Shuran Song · 2026
  11. PATCH: Action-Chunk-Conditioned Latent Patch Innovation Monitoring for Robot ManipulationYanan Zhou, Ranpeng Qiu, Yincong Chen, Jiajie Cui, Weiming Zhi · 2026
  12. RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous ManipulationShengcheng Luo · 2026
  13. SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable ObjectsBowen Jing, Mingxin Wang, Ruiyang Hao, Chenchen Ge, Hanwen Shen, Junjie He, Yang Cui, Yiming Hou et al. · 2026
  14. TAC-LOCO: Unified Whole-Body Control for Quadrupedal TACtile-Informed LOCO-ManipulationMuqun Hu, Yuhao Zhou, Kabir Ray Malik, Chi Lin, Won Suk Lee, Yu She, Yan Gu · 2026
  15. TacForeSight: Force-Guided Tactile World Model for Contact-Rich ManipulationYujie Zang, Yuhang Zheng, Xian Nie, Yupeng Zheng, Shuai Tian, Songen Gu, Chen Gao, Zining Wang et al. · 2026
  16. TACO: TActile World Model as a Self-COrrector forScalable VLA Post-TrainingShengbang Liu, Yueru Jia, Yuyang Yan, Jiaming Liu, Xinran Zhang, Qiuxuan Feng, Yandong Guo, Shiji Zhou et al. · 2026
  17. Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric AttentionSiyu Wu, Linjing You, Junjie Zhu, Yaozu Liu, Changhao Zhang, Jian Liu, Weiqiang Wang, Qi Li et al. · 2026
  18. TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich TasksLongyan Wu, Jieji Ren, Chenghang Jiang, Junxi Zhou, Shijia Peng, Ran Huang, Guoying Gu, Li Chen et al. · 2026
  19. TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous ManipulationJianyi Zhou, Feiyang Hong, Yunhao Li, Yicheng Zhao, Yongjue Cen, Zirui Liu, Jiakang Huang, Zirui Chen et al. · 2026
  20. Uncertainty Quantification for Flow-Based Vision-Language-Action ModelsRalf Römer · 2026
  21. Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model SimulatorsChristian Oefinger · 2026
  22. Visuo-Tactile World ModelsCarolina Higuera, Sergio Arnaud, Byron Boots, Mustafa Mukadam, Francois Robert Hogan, Franziska Meier · 2026
  23. You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile ContactPengfei Ye, Yuxiang Ma, Haonan Chen, Guangming Wang, Yixiong Jing, Brian Sheil, Yilun Du, Edward Adelson · 2026
完整证据附录(24 条正式证据事件)
  • Time range: 2025-07-20..2026-07-20
  • Events: 24
  • 每个事件一节,标题即锚点;trace-map 中的 event 链接跳转到这里。

EA-TWM-READ-0008

  • Claim: TAMEn 用动捕精度模式与 VR 便携模式平衡数据质量和环境多样性,并把人在环的触觉可视化恢复数据纳入金字塔式数据配方。
  • Stance: support | Confidence: direct
  • Paper: 2604.07335 TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks
  • Locator: Abstract (full-text section)
  • Evidence: 摘要明确列出精度/便携双模式采集、触觉恢复遥操作和人在环恢复数据。
  • Quote: “To balance data quality and environmental diversity, we implement a dual-mode acquisition pipeline: a precision mode leveraging motion capture for high-fidelity demonstrations, and a portable mode utilizing VR-based tracking for in-the-wild acquisition and tactile-visualized recovery teleoperation. Building on this hardware, we unify large-scale tactile pretraining, task-specific bimanual demonstrations, and human-in-the-loop recovery data into a pyramid-structured data regime, enabling closed-l”
  • Authors: longyan-wu; jieji-ren; chenghang-jiang; et al.

EA-TWM-READ-0005

  • Claim: 触觉数据的有效形态不止原始 tactile image,还包括 marker displacement 等显式接触几何与滑移特征。
  • Stance: support | Confidence: direct
  • Paper: 2606.04825 HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation Learning
  • Locator: 4.2 Vision-Based Tactile Sensing and Marker Tracking
  • Evidence: HapTile 的每个夹爪手指安装视觉触觉传感器,接触会带来图像变化和 marker displacement;论文把 marker-motion 信号保存进数据集并用于 haptic feedback,实验也比较 vision-only、vision+tactile image 与 vision+tactile+marker 表征。
  • Quote: “We smooth the signal with separate rise and release factors, producing a stable tactile motion estimate. The resulting marker-motion signal is stored in HapTile and is also used to drive haptic feedback.”
  • Authors: amirhosein-alian; yongqiang-zhao; shiyi-gu; et al.

EA-TWM-READ-0002

  • Claim: 触觉世界模型可以被扩展为同时生成未来视觉、未来触觉和动作 chunk 的世界动作模型。
  • Stance: support | Confidence: direct
  • Paper: 2606.08737 Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: Dream-Tac 把 world action model 扩展到触觉,联合建模当前视觉、触觉、语言指令下的未来视觉观测、未来触觉观测和动作 chunk,并加入 contact-gated visuotactile fusion 与 contact-aware attention bias。
  • Quote: “In this paper, we propose Dream-Tac, a unified Tactile-World Action Model that jointly models actions, future visual observations, and tactile dynamics. Specifically, Dream-Tac introduces (i) contact-gated visuotactile fusion to selectively integrate tactile signals and (ii) a contact-aware attention bias to better regulate cross-modal interactions during manipulation.”
  • Authors: yunfan-lou; yifan-ye; yankai-fu; et al.

EA-TWM-READ-0006

  • Claim: 腕部六维力/力矩可作为未来触觉 latent 的先行条件,用于预测短时域接触变化。
  • Stance: support | Confidence: direct
  • Paper: 2606.11184 TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
  • Locator: IV-D 1 World Model Conditioning
  • Evidence: TacForeSight 的 TacForceWM 从双指触觉观测出发,以高频腕部 force/torque 为条件预测短时域触觉 latent dynamics;ablation 中 wrist wrench 条件的未来触觉预测优于无条件版本,MSE 从 0.027 降到 0.017,cosine 从 0.954 提升到 0.992。
  • Quote: “We first ablate the conditioning modality used in the tactile world model. We compare four conditioning designs: no external condition, RGB image condition, robot state condition, and wrist wrench condition. Prediction quality is evaluated using Mean Squared Error (MSE), cosine similarity (Cos), and symmetric KL divergence (KL sym ), which respectively measure element-wise prediction error, latent directional consistency, and latent distribution consistency. As shown in Table II , wrist wrench c”
  • Authors: yujie-zang; yuhang-zheng; xian-nie; et al.

EA-TWM-READ-0003

  • Claim: 在接触丰富操作中,触觉世界模型的关键不是简单增加模态,而是让表征同时具备空间结构、时间连续性和跨模态兼容性。
  • Stance: support | Confidence: direct
  • Paper: 2606.13877 ContactWorld: What Matters in Vision-Tactile World Models for Contact-Rich Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: ContactWorld 在 12 个接触丰富任务上比较视觉与触觉表征;点云把平均规划成功率从腕部视角 20.7% 和前视 22.0% 提升到 32.1%,点云加触觉力场进一步到 36.1%。作者强调触觉效果取决于跨模态表征兼容,而非模态数量本身。
  • Quote: “Abstract Contact-rich manipulation requires world models to reason over complex contact dynamics from multimodal sensory observations. However, it remains unclear which representation properties fundamentally support stable long-horizon planning in contact-rich settings. In this paper, we present ContactWorld, a benchmark and systematic empirical study of vision-tactile world models spanning 12 contact-rich manipulation tasks, including insertion, disassembly, screwing, and exploratory interacti”
  • Authors: zhiyuan-zhang; pokuang-zhou; kaidi-zhang; et al.

EA-TWM-READ-0004

  • Claim: 触觉世界模型也可以在推理期作为候选动作验证器,而不只是训练期的动态模型。
  • Stance: support | Confidence: direct
  • Paper: 2606.14981 Inference-time Policy Steering via Vision and Touch
  • Locator: 5 Experiments
  • Evidence: ViTaL 学习 visuo-tactile latent world model,结合视觉和文本条件触觉 verifier,对候选动作进行长时域视觉模式选择和短时域触觉 refinement;真实机器人任务包括 wiping、insertion 和 pipette transfer。
  • Quote: “We evaluate ViTaL on three real-world contact-rich manipulation tasks. Our experiments are designed to answer three questions: whether multimodal visuo-tactile steering improves policy over unimodal guidance (Sec. 5.1 ); whether a learned latent world model and semantically aligned verifier provides reliable rewards for predicted outcomes (Sec. 5.2 ); and whether the proposed bi-level optimization balances performance and efficiency compared to naive visual-tactile fusion (Sec. 5.3 ). Robot Setu”
  • Authors: yilin-wu; zilin-si; zeynep-temel; et al.

EA-TWM-READ-0015

  • Claim: Flow-based VLA 在真实部署时缺少置信度机制会形成安全与恢复缺口;velocity-field disagreement 可把动作预测的不可靠性转成失败检测和主动微调信号。
  • Stance: support | Confidence: direct
  • Paper: 2606.18043 Uncertainty Quantification for Flow-Based Vision-Language-Action Models
  • Locator: Abstract (full-text section)
  • Evidence: 作者将真实非平稳环境中的分布外场景描述为 VLA 可能“无预警失败”的关键限制,并提出用小 ensemble 的 velocity-field disagreement 量化 epistemic uncertainty;LIBERO 实验显示该不确定性与下游表现、失败检测和主动采样相关。
  • Quote: “This presents a critical limitation for real-world deployment in non-stationary environments, where models inevitably encounter scenarios outside their pretraining distribution and may fail without warning. To address this, we derive an efficient method for quantifying epistemic uncertainty in flow-matching models by leveraging velocity-field disagreement (VFD) across a small ensemble.”
  • Authors: ralf-rmer

EA-UMI-READ-0015

  • Claim: 软物体和形变场景中的数据质量主要受可观测性约束:必须同时记录多视角全局运动、触觉局部形变和可用于评测的 3D 轨迹。
  • Stance: support | Confidence: direct
  • Paper: 2607.05390 Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Models
  • Locator: Abstract (full-text section)
  • Evidence: 论文认为形变物体有高维状态和复杂材料属性,接触诱发的局部形变常被末端执行器或物体遮挡;已有数据集常缺对象多样性、依赖合成数据,或缺高保真标注与接触形变。Deform360 采集 198 个日常物体、1,980 个交互序列、215 小时以上数据、41 个环视相机和双臂触觉 UMI gripper,并用 markerless 3D tracking 提取稠密几何与运动。
  • Quote: “To address this, we present Deform360, a large-scale visuotactile dataset featuring 198 daily-life objects, 1,980 interaction sequences, and over 215 hours of observations from 41 surround-view cameras and bimanual tactile grippers to capture both global motion and contact-induced local deformations. Leveraging a novel markerless visuotactile 3D tracking pipeline to extract dense geometry and motion, we systematically evaluate current state-of-the-art world models, comparing 2D video models agai”
  • Authors: hongyu-li; wanjia-fu; xiaoyan-cong; et al.

EA-UMI-READ-0002

  • Claim: UMI-style data is useful for contact-rich manipulation when the collection interface records force/torque, depth, pose, and grasp-force signals; the paper also implies that vision/trajectory-only data is insufficient for force-sensitive tasks.
  • Stance: conditional | Confidence: direct
  • Paper: 2601.09988 In-the-Wild Compliant Manipulation with UMI-FT
  • Locator: Abstract (full-text section)
  • Evidence: The HTML full text reports that UMI-FT mounts compact six-axis force/torque sensors on each finger, uses multimodal demonstrations to train adaptive compliance policies, and shows diverse in-the-wild data outperforming limited scene-diversity data in a skewer task.
  • Quote: “Abstract Many manipulation tasks require careful force modulation. With insufficient force the task may fail, while excessive force could cause damage. The high cost, bulky size and fragility of commercial force/torque (F/T) sensors have limited large-scale, force-aware policy learning. We introduce UMI-FT, a handheld data-collection platform that mounts compact, six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose. Using the mult”
  • Authors: hojung-choi; yifan-hou; chuer-pan; et al.

EA-TWM-READ-0001

  • Claim: VT-WM 的训练序列同步记录腕部位姿、关节位置、外部视觉和两个指尖触觉视频,并使用时间戳对齐后降采样训练。
  • Stance: conditional | Confidence: direct
  • Paper: 2602.06001 Visuo-Tactile World Models
  • Locator: B.0.1 Training dataset
  • Evidence: 训练数据段明确列出了同步的本体状态、外部视频与双指触觉视频数据流。
  • Quote: “Each sequence contains multimodal data streams: proprioceptive information (wrist pose, joint positions), exocentric video from the camera, and video from each Digit 360 fingertip sensor. All data streams were synchronized using timestamps and downsampled to 6 FPS for training the world model. Our training dataset for V-WM and VT-WM consists of 124 demonstrations totaling 112k datapoints, with each demonstration averaging 40 seconds. For validation, we use 26 demonstrations spanning all tasks, c”
  • Authors: carolina-higuera; sergio-arnaud; byron-boots; et al.

EA-TWM-READ-0007

  • Claim: 触觉信息用于 VLA 或世界模型时,低频视觉语言理解和高频触觉反馈应在架构上分离。
  • Stance: conditional | Confidence: direct
  • Paper: 2605.07308 AT-VLA: Adaptive Tactile Injection for Enhanced Feedback Reaction in Vision-Language-Action Models
  • Locator: 5 Conclusion
  • Evidence: AT-VLA 把系统分为慢速视觉语言流和快速触觉流,慢速流负责任务理解和视觉定位,快速流以高频处理触觉反馈;作者采用 3:1 的快慢流频率比,并在真实接触丰富任务中验证 adaptive tactile injection、tactile gate、adaptive cross-attention 和 reaction dual-stream 的作用。
  • Quote: “In summary, AT-VLA introduces an adaptive framework that seamlessly integrates tactile sensing into vision-language-action models. Through the Adaptive Tactile Injection mechanism, AT-VLA dynamically balances pretrained visual-language knowledge with newly learned tactile representations, preserving model integrity while enhancing action precision. The Tactile Reaction Dual-Stream mechanism further enables rapid, high-frequency tactile responses by decoupling slow perceptual reasoning from fast”
  • Authors: xiaoqi-li; muhe-cai; jiadong-xu; et al.

EA-TWM-READ-0014

  • Claim: 在视觉遮挡或视觉退化下,显式把触觉接触投影到图像域可以改善空间对齐;但这种收益依赖对运动学和标定误差导致的空间不确定性建模。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.08765 RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 作者称视觉观测不可靠或被遮挡时,稀疏异构触觉与稠密视觉表示的对齐是核心挑战;方法使用正运动学和相机标定投影触觉传感器位置,并用力调制高斯 saliency maps 建模运动学和标定误差带来的空间不确定性。
  • Quote: “We then render force-modulated Gaussian saliency maps to model spatial uncertainty arising from kinematic and calibration errors.”
  • Authors: shengcheng-luo

EA-TWM-READ-0010

  • Claim: 触觉不是简单加 token 就能提升;接触任务中的 RGB 未来可能视觉上合理但物理上不完整,而无约束触觉注入还可能污染视频和动作预测。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.26663 Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出 insertion、assembly、search、reorientation 依赖 slip、jamming、contact normals 和小对齐误差,这些状态在 RGB 中弱可见或不可见;同时他们定义 tactile pollution:无约束触觉 token 注入会迫使视觉 dynamics model 吸收稀疏局部事件式接触信号,从而退化视频和动作预测。
  • Quote: “In contact-rich manipulation, however, visually plausible futures can be physically incomplete: insertion, assembly, search, and reorientation often depend on slip, jamming, contact normals, or small alignment errors that are weakly visible or hidden in RGB. A natural solution is to predict future tactile states, however, we identify tactile pollution , a failure mode where unconstrained tactile-token injection degrades video and action prediction by forcing a visual dynamics model to absorb spa”
  • Authors: siyu-wu; linjing-you; junjie-zhu; et al.

EA-TWM-READ-0011

  • Claim: 力觉/触觉/音频能揭示图像不可直接观测的交互状态;但这些模态硬件和任务依赖强,所以需要在少量多传感数据上适配既有视觉策略,而不是预训练时穷尽所有传感器。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.30988 Multisensory Continual Learning: Adapting Pretrained Visuomotor Policies to Force
  • Locator: Abstract (full-text section)
  • Evidence: 作者称接触丰富任务常依赖 vision 之外的 sensory data,force、tactile 或 audio feedback 能揭示 images 中不可直接观察的 interaction states;但这些模态 hardware- and task-specific,且大规模多传感数据稀缺。他们提出 MuSe,将 limited multisensory data 融入 pretrained vision-only policies,并以 force-torque sensing 做真实任务案例。
  • Quote: “Abstract Robot manipulation often depends on sensory data beyond vision, especially in contact-rich tasks where force, tactile, or audio feedback reveals interaction states not directly visible from images. Yet such modalities are hardware- and task-specific, and large multisensory datasets remain scarce, making it impractical to pretrain policies with every sensor they may encounter. We study multi-sensory continual learning : adapting a pretrained robot policy to new tasks with newly introduce”
  • Authors: jaden-clark; changhao-wang; yihuai-gao; et al.

EA-TWM-READ-0009

  • Claim: 全局视觉异常、帧级变化或策略不确定性不足以判断感知误差是否会影响当前动作;部署监控需要把异常限定到 action chunk 将要使用的执行走廊。
  • Stance: limit | Confidence: direct
  • Paper: 2606.16690 PATCH: Action-Chunk-Conditioned Latent Patch Innovation Monitoring for Robot Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出开放工作空间中移动物体、瞬时遮挡和目标运动附近扰动会让部署脆弱;现有 runtime monitors 往往依赖全局 observation anomalies、policy uncertainty 或 frame-level visual changes,难以区分任务相关执行风险和无害视觉变化。PATCH 通过 active action chunk 的 projected execution corridor 累计持续残差作为介入信号。
  • Quote: “Abstract Learning-based manipulation policies have made substantial progress in real-world robot manipulation, particularly for short-horizon action generation. However, deployment in open workspaces remains fragile under unexpected local scene dynamics, such as moving objects, transient occlusions, or disturbances near the intended motion. Existing runtime monitors often rely on global observation anomalies, policy uncertainty, or frame-level visual changes, and struggle to distinguish task-rel”
  • Authors: yanan-zhou; ranpeng-qiu; yincong-chen; et al.

EA-TWM-READ-0012

  • Claim: 只看任务完成会低估传感器感知误差的真实风险;柔性物操作需要把安全交互、滑移/掉落和过度形变作为独立评测目标。
  • Stance: limit | Confidence: direct
  • Paper: 2607.04234 SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects
  • Locator: 1 Introduction
  • Evidence: 作者指出现有 manipulation benchmarks 多以 success 为中心,很少评估执行过程是否物理安全;SoftVTBench 分开报告 Goal Success 和 Safety Success,后者要求无掉落并限制峰值形变。实验显示 success-only evaluation 会显著高估策略表现,而触觉感知可改善 Safety Success 并降低物体形变。
  • Quote: “Experiments show that success-only evaluation substantially overstates policy performance, as a large fraction of goal-completing rollouts violate physical safety, and that adding tactile sensing improves Safety Success while keeping Goal Success comparable and reduces object deformation during execution.”
  • Authors: bowen-jing; mingxin-wang; ruiyang-hao; et al.

EA-TWM-READ-0013

  • Claim: 世界模型/生成式仿真器不能仅凭视觉逼真度作为具身策略评测裁判;必须验证其是否按动作正确响应,并建立 admissibility/validation 梯度后才能把闭环判决当证据。
  • Stance: gap | Confidence: direct
  • Paper: 2607.07196 Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出机器人中 World Models 越来越被用于模拟动作后果并给出 success/safety verdict,但视频生成指标如 FVD 奖励视觉真实感,却忽略世界是否对 policy actions 正确响应;他们主张作为 test oracle 的 WM 需要先通过 accreditation,并提出 L0-L4 admissibility ladder。
  • Quote: “Abstract Across robotics, World Models (WMs) are increasingly used to evaluate action policies by simulating the consequences of actions in an imagined world, and returning a success or safety verdict. Yet a verdict is only as trustworthy as the WM that produced it, and the WM itself needs to be certified. In video-generation WMs , fidelity metrics such as Fréchet Video Distance (FVD) reward visual realism, but ignore whether the world responds correctly to the policy’s actions, including those”
  • Authors: christian-oefinger

EA-LOCOMANIP-2026-0012

  • Claim: Adding tactile-command tracking at the low level raised insertion success from 0.70 to 0.85, full reorientation-plus-insertion from 0.60 to 0.80, and valve tightening from 0.80 to 0.85.
  • Stance: support | Confidence: direct
  • Paper: 2604.27224 Learning Tactile-Aware Quadrupedal Loco-Manipulation Policies
  • Locator: IV-B Experimental Results and Analyze
  • Evidence: The paper compares variants with the same tactile-aware high level but different low-level tactile tracking.
  • Quote: “Finally, comparing Baseline 3 (P3) with our full method demonstrates the benefit of incorporating tactile commands into the low-level policy: our method further increases the success rate from 0.70 to 0.85 on Task 1 insertion (+0.15), from 0.60 to 0.80 on Task 1 whole (+0.20), and from 0.80 to 0.85 on Task 2 (+0.05), and achieves 1.00 on Task 3 (compared to 0.20 for P1).”
  • Authors: pokuang-zhou; yuhao-zhou; quan-khanh-luu; et al.

EA-LOCOMANIP-2026-0021

  • Claim: In 10 matched hardware trials, tactile-informed TAC-LOCO achieved 90% dynamic loco-manipulation success versus 50% for Deep WBC with a fixed gripper.
  • Stance: support | Confidence: direct
  • Paper: 2607.10132 TAC-LOCO: Unified Whole-Body Control for Quadrupedal TACtile-Informed LOCO-Manipulation
  • Locator: 6.5 Baseline comparison
  • Evidence: The hardware baseline comparison isolates learned grasp regulation under the same command set.
  • Quote: “We conduct 10 hardware trials using the same set of loco-manipulation commands as in Sec. 6 . Table 4 compares the success rates of our policy and the baseline, showing that our tactile-informed policy achieves a substantially higher success rate. Figure 11 shows that the baseline suffers from gradual object slip during the task. Since the gripper width remains fixed, the policy cannot actively suppress slip once the external force changes. Table 4: Deployment success rate comparison with baseli”
  • Authors: muqun-hu; yuhao-zhou; kabir-ray-malik; et al.

EA-TACTILE-2026-0001

  • Claim: 近一年触觉表征研究开始从小规模单任务管线走向大规模全手触觉—第一视角配对数据和多任务、任务级 OOD 基准;HT-Bench 以约 1000 万 RGB 帧、780 万触觉帧和 226 项任务测量接触结构、跨模态对齐与时间动态。
  • Stance: support | Confidence: direct
  • Paper: 2606.19161 HT-Bench: Benchmarking and Learning Dexterous Full-Hand Tactile Representations with Egocentric Vision
  • Locator: Abstract; 3 HT-Bench: A Multi-Task Tactile Evaluation Benchmark
  • Evidence: 摘要和基准设计章节直接给出数据规模、四项评测任务与任务级 OOD 划分。
  • Quote: “comprising 10M RGB frames and 7.8M tactile frames collected across 226 tasks.”
  • Authors: yuzhe-huang; jiaping-wu; jiaming-jiang; et al.

EA-SENSORERR-READ-0010

  • Claim: 物体 6-DoF 位姿误差在遮挡、弱光、反光/透明表面下会让视觉方法失效;单次双触点触觉可作为视觉不可靠时的位姿观测补充。
  • Stance: support | Confidence: direct
  • Paper: 2606.28899 You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact
  • Locator: Abstract (full-text section)
  • Evidence: 作者明确指出视觉位姿估计常在遮挡、差光照、反光或透明表面下失败,并提出 tactile-only pose estimation:把触觉接触表示成局部 3D 点云,结合校准传感器位姿恢复完整 6-DoF object pose;实验在视觉不可靠时优于视觉和几何基线。
  • Quote: “Abstract Accurate 6-DoF object pose estimation is fundamental to robotic manipulation, yet vision-based methods often fail under occlusion, poor lighting, and reflective or transparent surfaces.”
  • Authors: pengfei-ye; yuxiang-ma; haonan-chen; et al.

EA-SENSORERR-READ-0012

  • Claim: 触觉在灵巧操作中补足视觉/语言无法稳定观测的接触隐变量;滑移、力不匹配、接触稳定性等局部误差需要比语义规划更快的反馈通道。
  • Stance: support | Confidence: direct
  • Paper: 2607.07287 TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 作者把日常灵巧操作的误差来源明确落在滑移、错位、不稳定抓取和力不匹配上,并指出视觉/语言不能可靠揭示力、滑移和接触稳定性;其分层策略将视觉语言子任务规划、触觉世界模型预测和高频触觉残差修正分开。
  • Quote: “Vision and language provide semantic and geometric guidance, but they cannot reliably reveal hidden contact states such as force, slip, and contact stability.”
  • Authors: jianyi-zhou; feiyang-hong; yunhao-li; et al.

EA-TACTILE-2026-0002

  • Claim: HT-Bench 的进步仍停留在表征层:当前四项任务没有直接测量真实机器人闭环操作,因此不能据此宣称策略或部署收益。
  • Stance: limit | Confidence: direct
  • Paper: 2606.19161 HT-Bench: Benchmarking and Learning Dexterous Full-Hand Tactile Representations with Egocentric Vision
  • Locator: 6 Limitations and Future Work
  • Evidence: 作者在限制章节明确列出硬件/本体覆盖和闭环下游评测缺失。
  • Quote: “While these tasks assess structural, cross-modal, and temporal understanding, they do not directly measure downstream robotic performance.”
  • Authors: yuzhe-huang; jiaping-wu; jiaming-jiang; et al.

EA-SENSORERR-READ-0001

  • Claim: TACO 用触觉感知世界模型联合预测未来视频和力序列,再将真实失败附近状态转成带纠正动作的想象片段,用于接触丰富任务的 VLA 后训练。
  • Stance: limit | Confidence: direct
  • Paper: 2607.02840 TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training
  • Locator: 5 Conclusion and Limitations
  • Evidence: 结论的 Recognize–Imagine–Label 回路明确连接了真实失败、视频—力联合想象与纠正动作标注。
  • Quote: “We presented TACO, a tactile-aware world-model-driven framework for scalable VLA post-training in contact-rich manipulation. Following a Recognize–Imagine–Label loop, TACO converts real-world failures into imagined corrections without repeated human intervention: a tactile-aware world model jointly denoises future video and force sequences, while a unified progress-action model recognizes failure-adjacent states and labels imagined segments with corrective actions. To incorporate this supervisio”
  • Authors: shengbang-liu; yueru-jia; yuyang-yan; et al.

References

Interactive Wiki

进入交互阅读

静态页采用默认版本;v0、v0、v0可在同一交互 Wiki 中阅读,不建立相互竞争的索引页。

Method and limits

研究方法、证据边界与引用说明

本页由当前结算的证据层自动派生,不改写已结算 run。论文只有在完整非 OCR 全文、结构化论文笔记和 claim-support audit 通过后,才进入接受证据。

页面结论是对所列论文与研究窗口的综合,不等同于无限外推。引用本专题时,请同时保留本页 canonical 地址、访问日期,并在需要时回溯原论文。