Embodied AI Evidence Hub具身智能证据知识库

多模态感知 · Multimodal Perception

近一年具身感知问题是否已有效解决

Has Embodied Perception Been Solved?

更新于

30 秒结论

先读这个判断

本文考察 2025 年 7 月 14 日至 2026 年 7 月 14 日公开的具身感知研究，证据集中在视觉—语言—动作模型、三维与时序表征、主动感知、触觉/力觉、事件视觉、不确定性与闭环评测。这里不讨论导航规划本身是否成功，只问一个更基础的问题：机器人能否把真实世界转化为足以支持当前动作的、及时且可校验的状态表征。

精读论文
19
正式证据事件
19
知识单元
EA-SENSOR, EA-4D, EA-EVAL
审计状态
当前结算 run 已通过仓库校验

v0 · 完整正文

研究综述

研究边界

本文考察 2025 年 7 月 14 日至 2026 年 7 月 14 日公开的具身感知研究，证据集中在视觉—语言—动作模型、三维与时序表征、主动感知、触觉/力觉、事件视觉、不确定性与闭环评测。这里不讨论导航规划本身是否成功，只问一个更基础的问题：机器人能否把真实世界转化为足以支持当前动作的、及时且可校验的状态表征。

因此，“有效解决”不能等同于图像分类、目标检测或单帧位姿估计达到高分。至少还要满足：关键状态在任务需要时可观测；跨视角与跨时间保持一致；缺失、遮挡和分布外输入能被识别；多模态融合不会引入更大污染；误判能够通过行动复核并撤销；最终收益能反映在闭环成功、安全与恢复上。本次选取 19 篇具有全文定位证据的代表论文。它能支持主要能力边界判断，但不代表所有传感器与机器人形态。

中心判断

具身感知尚未作为通用能力得到有效解决，但在“刚体、可见、静态、已标定、短时”的子空间里，视觉主导的感知已经相当成熟。 未解决部分不再主要表现为“模型认不出常见物体”，而是机器人不知道当前观测缺了什么、不知道预测何时不可信，也无法稳定获得接触、摩擦、滑移、遮挡后状态和动作后果等行动相关变量。过去一年的进展更像是在补齐观测闭环，而不是证明这一闭环已经闭合。

一、二维语义进步没有消除物理状态的不可观测性

视觉基础模型已经能提供强语义先验，但物理操作要求的不只是“这是什么”。深度、尺度、六自由度位姿、接触位置、摩擦、柔性形变和滑移决定动作能否执行，其中许多变量在普通 RGB 中根本不可直接观测。

Lift3D-VLA 的出发点正是二维视觉预训练在几何约束、遮挡、接触和动态一致性上会丢失信息；显式三维几何与动态表征能改善操作。类似地，3PoinTr 表明点轨迹监督仍受可见性、遮挡、深度估计和本体分割约束，短暂消失的关键物体点不能简单从监督中删除。这两项工作支持三维与时序建模的价值，也反向说明单帧视觉语义没有解决行动状态估计。

触觉证据更直接。You Only Touch Once 利用单次触觉接触补充弱光、遮挡、反光或透明表面下不可靠的视觉位姿估计；TouchWorld 把滑移、力不匹配和接触稳定性视为需要快速反馈的隐变量。若这些状态已经能被视觉稳定获得，触觉通道就不会持续带来显著收益。因而，过去一年的多模态进展本身就是“感知仍不完整”的正面证据。

二、主动感知解决了“去哪里看”，还没有解决“如何持续相信”

具身感知与离线视觉的根本差别，是观测分布由动作决定。机器人可以通过移动相机、改变抓取姿态或触碰物体来制造新证据。这使主动感知成为重要进展。

OA-NBV 证明移动机器人能够选择下一最佳视点以绕开人体遮挡；ActiveVLA 也显示主动改变观察视角可以改善精细三维操作。但两个结果都有明确边界：前者主要处理单步视点选择，不等于完整的多视图状态融合；后者在最困难的组合泛化任务上仍显著退化。

这一区分很重要。一次“再看一眼”可以消除局部歧义，但通用具身感知还要求系统把多个不同时刻、不同坐标系、不同质量的观测合并成可持续更新的信念，并在新证据到来时撤销旧判断。当前论文更多证明了主动取证可行，尚未证明长期证据管理已经稳定。

三、多模态不是把更多传感器接到同一个模型

视觉、触觉、力觉和事件流提供互补信息，但也带来时间同步、空间标定、采样频率和置信度不一致。HapTile 强调，多模态示教必须处理时间戳、动作—状态一致性、触觉标记跟踪与回合切分，否则模型会混淆接触信号和动作真值。RGB-S 把触觉接触投影到图像域，在遮挡条件下改善空间对齐，但收益依赖于显式建模运动学和标定误差。

更值得警惕的是“模态污染”。Tactile-WAM 指出，接触任务中的视觉未来可能看起来合理却物理上不完整，但无约束地加入触觉也会污染视频与动作预测。这说明融合不是信息量的单调增加：一个错位、延迟或硬件特定的模态，可能比缺失模态更危险。

事件视觉提供另一种互补路线。Event-VLA 利用事件流对照明变化更鲁棒、对运动更敏感的特性改善动作预测。该结果支持在特定失效模式下补模态，而不是得出“RGB 已被替代”或“所有任务都应多传感”的结论。

四、感知可靠性的核心缺口转向“不确定性是否可用于行动”

机器人不可能在所有输入上都看对，因此更现实的问题是：系统是否知道自己可能看错，并据此停止、复核或恢复。近一年多项工作开始把置信度放回闭环。Perturbation-Based Uncertainty 从隐藏激活扰动中提取分布外风险信号；Uncertainty Quantification for Flow-Based VLA 用速度场分歧识别动作预测不可靠性。

但全局异常分数仍不够。PATCH 指出，整幅图像的异常或帧级变化，未必与当前动作将要经过的执行区域有关；监控必须绑定当前动作段。这意味着“模型能输出一个 uncertainty”不等于感知可靠性已经解决，只有当不确定性能改变停止、换视角、降速或恢复决策时，它才成为有效感知能力。

开放词汇模型还暴露了系统性误导问题。Rethinking Embodied Navigation via Relational Inductive Bias 显示，相似外观产生的误检、难以撤销的静态先验和缺少行动验证，会持续污染地图与决策。VLM-LLM Navigation Bottlenecks 则发现，独立感知精度提升会出现边际饱和，而误检与框形变仍能主导失败。因此，平均精度提高与高风险错误消失是两个不同命题。

五、成功率掩盖了感知是否“安全有效”

如果机器人最后完成了任务，传统指标往往判定成功；但柔性物可能已经过度形变，物体可能短暂滑落，夹爪可能施加过大力量。SoftVTBench 明确把安全交互、滑移、掉落与形变从任务完成中分离。它提醒我们：若评测只看终态，感知对接触风险的失败会被控制器偶然补偿所掩盖。

TACO 和 TacForeSight 进一步说明，视觉难以发现的局部接触扰动需要触觉或力反馈，并且扰动恢复必须有专门数据。由此可以推断，感知的验收单位应从“单帧预测”转向“动作段内状态是否足以避免不可恢复失败”。

条件、分歧与可证伪边界

在以下条件下，“具身感知已基本可用”是合理工程判断：目标为常见刚体；光照稳定；视角和遮挡受控；相机、机械臂与工具坐标系经过标定；动作速度低；任务依赖的接触变量少；运行时长不足以累积明显漂移。固定工位分拣、标准件抓取和可视化良好的简单装配，可能已经进入这个区域。

结论不适用于透明或反光物、严重遮挡、软体与易碎物、紧配合插入、动态人机交互、跨传感器换件以及长期无人运行。若未来系统能在这些条件下跨硬件复现，并满足以下三点，本判断才会被推翻：第一，在分布外与缺失模态下保持校准良好的置信度；第二，主动获取新证据并撤销错误状态；第三，多模态收益在真实闭环安全、恢复和连续运行指标上稳定成立。

可操作的感知验收框架

层级需要回答的问题建议指标
可观测性任务关键状态是否被任一模态看到遮挡恢复率、接触/滑移检出、六自由度位姿误差
对齐与融合多模态是否同步且不互相污染时间残差、标定投影误差、模态缺失退化、融合消融
置信度系统是否知道何时不可信校准误差、分布外检出、误停与漏停
主动验证能否通过动作消除歧义再观测成功率、错误证据撤销率、信息增益
闭环结果感知改善是否真正改变行为安全成功、恢复率、滑移/掉落、过力、人工接管

研究空白与下一步

第一，长期漂移、磨损、传感器换件和跨实例泛化仍缺少统一基准。第二，多数主动感知工作关注下一视点，长期信念更新与反证机制仍弱。第三，不确定性方法需要从离线相关性走向真实动作干预，证明它能减少碰撞、过力和不可恢复失败。第四，多模态研究需要报告“何时不该融合”，包括错位模态对已有视觉策略的负迁移。

结论

近一年文献不支持“具身感知已经有效解决”的总体判断。它支持一个分层结论：常见刚体和受控视觉条件下的语义与几何感知已较成熟；开放世界中的遮挡恢复、接触状态、跨模态对齐、置信度、主动验证与长期维护仍未闭合。真正的突破不只是让机器人看见更多，而是让它知道当前状态是否足以行动、何时需要补证据，以及如何在看错之后安全地修正。

References

Accepted evidence

去重论文引用

  1. 3PoinTr: 3D Point Tracks for Learning Manipulation from Unconstrained Human VideosAdam Hung, Bardienus Pieter Duisterhof, Jeffrey Ichnowski · 2026
  2. ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic ManipulationZhenyang Liu · 2026
  3. Event-VLA: Action-Conditioned Event Fusion for Robust Vision-Language-Action ModelJiaxin Liu, Xun Xu, Zhenhao Zhang, Hanqing Wang, Ruiqi Chen, Shi Chang, Weiyu Guo, Laurent Kneip · 2026
  4. Exploring Bottlenecks in VLM-LLM Navigation: How 3D Scene Understanding Capability Impacts Zero-Shot VLNZiyi Xia · 2026
  5. HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation LearningAmirhosein Alian, Yongqiang Zhao, Shiyi Gu, Xuyang Zhang, Zhuo Chen, Christopher E. Mower, Haitham Bou-Ammar, Shan Luo · 2026
  6. Lift3D-VLA: Lifting VLA Models to 3D Geometry and Dynamics-Aware ManipulationJiaming Liu, Qingpo Wuwu, Nuowei Han, Hao Chen, Zhuoyang Liu, Fan Fei, Yueru Jia, Chenyang Gu et al. · 2026
  7. Multisensory Continual Learning: Adapting Pretrained Visuomotor Policies to ForceJaden Clark, Changhao Wang, Yihuai Gao, Seongheon Hong, Hojung Choi, Mark Cutkosky, Yifan Hou, Shuran Song · 2026
  8. OA-NBV: Occlusion-Aware Next-Best-View Planning for Human-Centered Active Perception on Mobile RobotsBoxun Hu · 2026
  9. PATCH: Action-Chunk-Conditioned Latent Patch Innovation Monitoring for Robot ManipulationYanan Zhou, Ranpeng Qiu, Yincong Chen, Jiajie Cui, Weiming Zhi · 2026
  10. Perturbation-Based Uncertainty for Failure Detection in Vision-Language-Action ModelsYousung Lee, Dongsoo Har · 2026
  11. Rethinking Embodied Navigation via Relational Inductive BiasWeitao An · 2026
  12. RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous ManipulationShengcheng Luo · 2026
  13. SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable ObjectsBowen Jing, Mingxin Wang, Ruiyang Hao, Chenchen Ge, Hanwen Shen, Junjie He, Yang Cui, Yiming Hou et al. · 2026
  14. TacForeSight: Force-Guided Tactile World Model for Contact-Rich ManipulationYujie Zang, Yuhang Zheng, Xian Nie, Yupeng Zheng, Shuai Tian, Songen Gu, Chen Gao, Zining Wang et al. · 2026
  15. TACO: TActile World Model as a Self-COrrector forScalable VLA Post-TrainingShengbang Liu, Yueru Jia, Yuyang Yan, Jiaming Liu, Xinran Zhang, Qiuxuan Feng, Yandong Guo, Shiji Zhou et al. · 2026
  16. Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric AttentionSiyu Wu · 2026
  17. TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous ManipulationJianyi Zhou, Feiyang Hong, Yunhao Li, Yicheng Zhao, Yongjue Cen, Zirui Liu, Jiakang Huang, Zirui Chen et al. · 2026
  18. Uncertainty Quantification for Flow-Based Vision-Language-Action ModelsRalf Römer, Maximilian Seeliger, Saida Liu, Ben Sturgis, Marco Bagatella, Daniel Marta, Andreas Krause, Angela P. Schoellig · 2026
  19. You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile ContactPengfei Ye, Yuxiang Ma, Haonan Chen, Guangming Wang, Yixiong Jing, Brian Sheil, Yilun Du, Edward Adelson · 2026
完整证据附录（19 条正式证据事件）
  • Time range: 2025-07-14..2026-07-14
  • Events: 19
  • 每个事件一节,标题即锚点;trace-map 中的 event 链接跳转到这里。

ERR-PVC-READ-0015

  • Claim: Lift3D-VLA 指出，纯 2D VLA 难以保真地表达可达性、遮挡、接触和随时间演化的几何约束，而现有 2D‑3D 转换又会损失几何保真度。
  • Stance: support | Confidence: direct
  • Paper: 2607.06564 Lift3D-VLA: Lifting VLA Models to 3D Geometry and Dynamics-Aware Manipulation
  • Locator: I Introduction
  • Evidence: 引言将操作需求归结为显式 3D 结构与时间一致性，并说明纯 2D 管线及有损的跨模态变换会削弱这些约束。
  • Quote: “Despite this progress, robotic manipulation fundamentally requires spatial reasoning in the physical world [ 57 , 82 , 12 , 13 , 59 ] : the robot must infer 3D structure, reason about geometric relationships (e.g., reachability, occlusion, and contact), and plan actions that remain temporally consistent as the geometry evolves. Purely 2D VLA pipelines often struggle to reliably capture these geometric constraints, particularly in cluttered or dynamic environments. A natural direction is to expli”
  • Authors: jiaming-liu; qingpo-wuwu; nuowei-han; et al.

EA-4D-READ-0012

  • Claim: 3PoinTr 保留含暂时遮挡点的轨迹，只屏蔽不可见的单个点—时间损失，因而能继续利用操作中任务关键的物体点监督。
  • Stance: conditional | Confidence: direct
  • Paper: 2603.08485 3PoinTr: 3D Point Tracks for Learning Manipulation from Unconstrained Human Videos
  • Locator: 4.3 Results: 3D Point Track Prediction
  • Evidence: 结果段对比了删除整条不可见轨迹的基线与仅屏蔽不可见 point-timestep 损失的 3PoinTr。
  • Quote: “The primary advantage of 3PoinTr is that it trains on data General Flow ignores. Real-world points are often temporarily occluded; General Flow removes any trajectory with invisible point-timestep pairs during preprocessing, whereas 3PoinTr retains all trajectories and masks losses for individual invisible point-timestep pairs. This provides additional supervision over task-critical object points that are temporarily occluded during manipulation. For example, in the Throw Away Paper task, every”
  • Authors: adam-hung; bardienus-pieter-duisterhof; jeffrey-ichnowski

EA-TWM-READ-0012

  • Claim: 只看任务完成会低估传感器感知误差的真实风险；柔性物操作需要把安全交互、滑移/掉落和过度形变作为独立评测目标。
  • Stance: limit | Confidence: direct
  • Paper: 2607.04234 SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects
  • Locator: 1 Introduction
  • Evidence: 作者指出现有 manipulation benchmarks 多以 success 为中心，很少评估执行过程是否物理安全；SoftVTBench 分开报告 Goal Success 和 Safety Success，后者要求无掉落并限制峰值形变。实验显示 success-only evaluation 会显著高估策略表现，而触觉感知可改善 Safety Success 并降低物体形变。
  • Quote: “Experiments show that success-only evaluation substantially overstates policy performance, as a large fraction of goal-completing rollouts violate physical safety, and that adding tactile sensing improves Safety Success while keeping Goal Success comparable and reduces object deformation during execution.”
  • Authors: bowen-jing; mingxin-wang; ruiyang-hao; et al.

EA-SENSORERR-READ-0008

  • Claim: Flow-based VLA 在真实部署时缺少置信度机制会形成安全与恢复缺口；velocity-field disagreement 可把动作预测的不可靠性转成失败检测和主动微调信号。
  • Stance: support | Confidence: direct
  • Paper: 2606.18043 Uncertainty Quantification for Flow-Based Vision-Language-Action Models
  • Locator: Abstract (full-text section)
  • Evidence: 作者将真实非平稳环境中的分布外场景描述为 VLA 可能“无预警失败”的关键限制，并提出用小 ensemble 的 velocity-field disagreement 量化 epistemic uncertainty；LIBERO 实验显示该不确定性与下游表现、失败检测和主动采样相关。
  • Quote: “This presents a critical limitation for real-world deployment in non-stationary environments, where models inevitably encounter scenarios outside their pretraining distribution and may fail without warning. To address this, we derive an efficient method for quantifying epistemic uncertainty in flow-matching models by leveraging velocity-field disagreement (VFD) across a small ensemble.”
  • Authors: ralf-rmer; maximilian-seeliger; saida-liu; et al.

EA-SENSORERR-READ-0009

  • Claim: VLA 的感知-动作误差不只来自传感器本身，也来自分布外观测下模型无法给出可靠置信度；隐藏激活扰动产生的 epistemic signal 可用于失败检测。
  • Stance: support | Confidence: direct
  • Paper: 2606.20754 Perturbation-Based Uncertainty for Failure Detection in Vision-Language-Action Models
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出现代 VLA 常用回归或 flow-based action generation，缺少显式预测概率；他们通过对 transformer hidden activations 注入高斯扰动，利用扰动后动作预测分歧估计不确定性，并在 LIBERO/LIBERO-PRO 的分布偏移下提升失败检测。
  • Quote: “Specifically, we inject Gaussian perturbations into transformer hidden activations and estimate epistemic signals from disagreement across perturbed action predictions. Experiments on LIBERO and LIBERO-PRO show that perturbation-based uncertainty consistently improves failure detection under distribution shift compared to sampling-based uncertainty, providing a practical uncertainty signal for VLA models.”
  • Authors: yousung-lee; dongsoo-har

EA-SENSORERR-READ-0010

  • Claim: 物体 6-DoF 位姿误差在遮挡、弱光、反光/透明表面下会让视觉方法失效；单次双触点触觉可作为视觉不可靠时的位姿观测补充。
  • Stance: support | Confidence: direct
  • Paper: 2606.28899 You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact
  • Locator: Abstract (full-text section)
  • Evidence: 作者明确指出视觉位姿估计常在遮挡、差光照、反光或透明表面下失败，并提出 tactile-only pose estimation：把触觉接触表示成局部 3D 点云，结合校准传感器位姿恢复完整 6-DoF object pose；实验在视觉不可靠时优于视觉和几何基线。
  • Quote: “Abstract Accurate 6-DoF object pose estimation is fundamental to robotic manipulation, yet vision-based methods often fail under occlusion, poor lighting, and reflective or transparent surfaces.”
  • Authors: pengfei-ye; yuxiang-ma; haonan-chen; et al.

EA-SENSORERR-READ-0011

  • Claim: RGB-centric VLA 在照明变化导致的可见性退化下会暴露鲁棒性问题；事件流作为对照明更鲁棒、对运动敏感的补充观测，可以改善不同可见性水平下的动作预测。
  • Stance: support | Confidence: direct
  • Paper: 2606.29384 Event-VLA: Action-Conditioned Event Fusion for Robust Vision-Language-Action Model
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出现有 VLA 往往假设稳定明亮的室内环境，而真实操作中 illumination shifts 会造成 degraded RGB observations；Event-VLA 将 degraded visibility 定义为 RGB-centric policies 的鲁棒性问题，并通过 action-query routing 将 event streams 融入 action representation，仿真和真实部署实验显示在不同可见性下保持更强鲁棒性。
  • Quote: “Abstract Vision-Language-Action (VLA) models have become an important paradigm of embodied AI. However, existing VLA models typically assume well-lit and stable indoor settings, while real-world embodied manipulation may involve degraded RGB observations caused by illumination shifts, posing critical challenges for robust robotic manipulation. To address this gap, we propose Event-VLA , an event-enhanced VLA framework for generalizable manipulation across varying illumination conditions. We form”
  • Authors: jiaxin-liu; xun-xu; zhenhao-zhang; et al.

EA-SENSORERR-READ-0012

  • Claim: 触觉在灵巧操作中补足视觉/语言无法稳定观测的接触隐变量；滑移、力不匹配、接触稳定性等局部误差需要比语义规划更快的反馈通道。
  • Stance: support | Confidence: direct
  • Paper: 2607.07287 TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 作者把日常灵巧操作的误差来源明确落在滑移、错位、不稳定抓取和力不匹配上，并指出视觉/语言不能可靠揭示力、滑移和接触稳定性；其分层策略将视觉语言子任务规划、触觉世界模型预测和高频触觉残差修正分开。
  • Quote: “Vision and language provide semantic and geometric guidance, but they cannot reliably reveal hidden contact states such as force, slip, and contact stability.”
  • Authors: jianyi-zhou; feiyang-hong; yunhao-li; et al.

EA-PNAV-2026-0002

  • Claim: 主动感知能改善固定视角VLA，但并未解决通用感知；论文在最难的组合泛化任务上仍报告明显退化。
  • Stance: conditional | Confidence: direct
  • Paper: 2601.08325 ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulation
  • Locator: 4.1 Experimental Results
  • Evidence: 结果段在报告总体领先的同时明确指出最难L4任务性能下降。
  • Quote: “Although performance decreases on the most difficult L4 tasks (1.2%), ActiveVLA still demonstrates promising long-horizon reasoning and strong 3D perception for precise manipulation.”
  • Authors: zhenyang-liu

EA-PNAV-2026-0003

  • Claim: OA-NBV证明机器人可以主动绕开遮挡获得更好观察，但作者明确把能力限定为单步视点选择，而非完整多视图感知。
  • Stance: conditional | Confidence: direct
  • Paper: 2603.11072 OA-NBV: Occlusion-Aware Next-Best-View Planning for Human-Centered Active Perception on Mobile Robots
  • Locator: V-B Limitations and future work.
  • Evidence: 限制段直接划定即时单步观测与完整多视图任务之间的边界。
  • Quote: “Finally, OA-NBV targets single-step viewpoint selection for immediate observation quality, rather than multi-view reconstruction.”
  • Authors: boxun-hu

EA-PNAV-2026-0004

  • Claim: 对零样本VLN而言，感知并非简单地“越准越已解决”：独立精度会出现边际饱和，而误检和框形变仍是关键失败源。
  • Stance: conditional | Confidence: direct
  • Paper: 2605.14801 Exploring Bottlenecks in VLM-LLM Navigation: How 3D Scene Understanding Capability Impacts Zero-Shot VLN
  • Locator: IV CONCLUSIONS
  • Evidence: 结论直接同时报告感知饱和和两类仍关键的误差。
  • Quote: “Our analysis also revealed that false positives and distorted bounding box aspect ratios are critical factors affecting navigation performance, and that a small set of core navigation-relevant object categories is sufficient for successful navigation.”
  • Authors: ziyi-xia

EA-SENSORERR-READ-0014

  • Claim: HapTile 的数据质量控制在机器人控制环中同步全部模态，检查空轨迹、损坏轨迹和时间戳缺口，并验证动作—状态一致性。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.04825 HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation Learning
  • Locator: 3.2 Synchronization and Data Quality Control
  • Evidence: 数据质量段明确记录了控制环同步、时间戳缺口检查、损坏轨迹剔除和 action-state consistency 检查。
  • Quote: “All data modalities are synchronized through the robot control loop. For policy learning, actions are converted to a unified 7D end-effector delta representation (1) where are translational deltas, are rotational deltas, and is the gripper command. This decouples learning from the exact robot configuration, enabling cross-embodiment by focusing the policy on local contact adjustment from tactile feedback. Several quality checks are applied to every collected trajectory. Empty or corrupted trajec”
  • Authors: amirhosein-alian; yongqiang-zhao; shiyi-gu; et al.

EA-SENSORERR-READ-0007

  • Claim: 在视觉遮挡或视觉退化下，显式把触觉接触投影到图像域可以改善空间对齐；但这种收益依赖对运动学和标定误差导致的空间不确定性建模。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.08765 RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 作者称视觉观测不可靠或被遮挡时，稀疏异构触觉与稠密视觉表示的对齐是核心挑战；方法使用正运动学和相机标定投影触觉传感器位置，并用力调制高斯 saliency maps 建模运动学和标定误差带来的空间不确定性。
  • Quote: “We then render force-modulated Gaussian saliency maps to model spatial uncertainty arising from kinematic and calibration errors.”
  • Authors: shengcheng-luo

EA-SENSORERR-READ-0015

  • Claim: 如果目标包含扰动恢复，数据集必须显式收集nominal demonstrations和recovery interaction data；否则模型很难学习接触丢失后的再建立。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.11184 TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation
  • Locator: IV-B 2 Perturbation-Aware Evaluation
  • Evidence: TacForeSight在perturbation-aware设置中额外收集恢复示教，让外部扰动发生在执行中并要求示教者重新建立稳定接触；完整模型在三个扰动任务上报告90%、85%、85%平均完成/成功分数。
  • Quote: “Policies in this setting are trained using both nominal demonstrations and recovery interaction data.”
  • Authors: yujie-zang; yuhang-zheng; xian-nie; et al.

EA-SENSORERR-READ-0003

  • Claim: 触觉不是简单加 token 就能提升；接触任务中的 RGB 未来可能视觉上合理但物理上不完整，而无约束触觉注入还可能污染视频和动作预测。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.26663 Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出 insertion、assembly、search、reorientation 依赖 slip、jamming、contact normals 和小对齐误差，这些状态在 RGB 中弱可见或不可见；同时他们定义 tactile pollution：无约束触觉 token 注入会迫使视觉 dynamics model 吸收稀疏局部事件式接触信号，从而退化视频和动作预测。
  • Quote: “In contact-rich manipulation, however, visually plausible futures can be physically incomplete: insertion, assembly, search, and reorientation often depend on slip, jamming, contact normals, or small alignment errors that are weakly visible or hidden in RGB. A natural solution is to predict future tactile states, however, we identify tactile pollution , a failure mode where unconstrained tactile-token injection degrades video and action prediction by forcing a visual dynamics model to absorb spa”
  • Authors: siyu-wu

EA-SENSORERR-READ-0004

  • Claim: 力觉/触觉/音频能揭示图像不可直接观测的交互状态；但这些模态硬件和任务依赖强，所以需要在少量多传感数据上适配既有视觉策略，而不是预训练时穷尽所有传感器。
  • Stance: conditional | Confidence: direct
  • Paper: 2606.30988 Multisensory Continual Learning: Adapting Pretrained Visuomotor Policies to Force
  • Locator: Abstract (full-text section)
  • Evidence: 作者称接触丰富任务常依赖 vision 之外的 sensory data，force、tactile 或 audio feedback 能揭示 images 中不可直接观察的 interaction states；但这些模态 hardware- and task-specific，且大规模多传感数据稀缺。他们提出 MuSe，将 limited multisensory data 融入 pretrained vision-only policies，并以 force-torque sensing 做真实任务案例。
  • Quote: “Abstract Robot manipulation often depends on sensory data beyond vision, especially in contact-rich tasks where force, tactile, or audio feedback reveals interaction states not directly visible from images. Yet such modalities are hardware- and task-specific, and large multisensory datasets remain scarce, making it impractical to pretrain policies with every sensor they may encounter. We study multi-sensory continual learning : adapting a pretrained robot policy to new tasks with newly introduce”
  • Authors: jaden-clark; changhao-wang; yihuai-gao; et al.

EA-PNAV-2026-0013

  • Claim: 开放词汇感知错误会形成系统性误导并持续污染地图与导航决策，因此标准检测能力并不等于具身感知已解决。
  • Stance: limit | Confidence: direct
  • Paper: 2606.10348 Rethinking Embodied Navigation via Relational Inductive Bias
  • Locator: Abstract > first paragraph
  • Evidence: 引言直接描述视觉相似、静态先验和缺少动作验证导致的持续污染。
  • Quote: “visual similarity can induce false positives, static priors are difficult to update once contradicted, and the lack of embodied verification may lead to repeated failed exploration, continuously contaminating map updates and navigation decisions.”
  • Authors: weitao-an

EA-SENSORERR-READ-0002

  • Claim: 全局视觉异常、帧级变化或策略不确定性不足以判断感知误差是否会影响当前动作；部署监控需要把异常限定到 action chunk 将要使用的执行走廊。
  • Stance: limit | Confidence: direct
  • Paper: 2606.16690 PATCH: Action-Chunk-Conditioned Latent Patch Innovation Monitoring for Robot Manipulation
  • Locator: Abstract (full-text section)
  • Evidence: 作者指出开放工作空间中移动物体、瞬时遮挡和目标运动附近扰动会让部署脆弱；现有 runtime monitors 往往依赖全局 observation anomalies、policy uncertainty 或 frame-level visual changes，难以区分任务相关执行风险和无害视觉变化。PATCH 通过 active action chunk 的 projected execution corridor 累计持续残差作为介入信号。
  • Quote: “Abstract Learning-based manipulation policies have made substantial progress in real-world robot manipulation, particularly for short-horizon action generation. However, deployment in open workspaces remains fragile under unexpected local scene dynamics, such as moving objects, transient occlusions, or disturbances near the intended motion. Existing runtime monitors often rely on global observation anomalies, policy uncertainty, or frame-level visual changes, and struggle to distinguish task-rel”
  • Authors: yanan-zhou; ranpeng-qiu; yincong-chen; et al.

EA-SENSORERR-READ-0001

  • Claim: TACO 用触觉感知世界模型联合预测未来视频和力序列，再将真实失败附近状态转成带纠正动作的想象片段，用于接触丰富任务的 VLA 后训练。
  • Stance: limit | Confidence: direct
  • Paper: 2607.02840 TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training
  • Locator: 5 Conclusion and Limitations
  • Evidence: 结论的 Recognize–Imagine–Label 回路明确连接了真实失败、视频—力联合想象与纠正动作标注。
  • Quote: “We presented TACO, a tactile-aware world-model-driven framework for scalable VLA post-training in contact-rich manipulation. Following a Recognize–Imagine–Label loop, TACO converts real-world failures into imagined corrections without repeated human intervention: a tactile-aware world model jointly denoises future video and force sequences, while a unified progress-action model recognizes failure-adjacent states and labels imagined segments with corrective actions. To incorporate this supervisio”
  • Authors: shengbang-liu; yueru-jia; yuyang-yan; et al.

References

Interactive Wiki

进入交互阅读

静态页采用默认版本；v0、v0、v0可在同一交互 Wiki 中阅读，不建立相互竞争的索引页。

用 AI 继续研究 v0 v0 v0

Method and limits

研究方法、证据边界与引用说明

本页由当前结算的证据层自动派生，不改写已结算 run。论文只有在完整非 OCR 全文、结构化论文笔记和 claim-support audit 通过后，才进入接受证据。

页面结论是对所列论文与研究窗口的综合，不等同于无限外推。引用本专题时，请同时保留本页 canonical 地址、访问日期，并在需要时回溯原论文。