多模态感知 · Multimodal Perception

近一年具身感知与导航是否已解决

Have Embodied Perception and Navigation Been Solved?

更新于

30 秒结论

先读这个判断

本文讨论的是 2025 年 7 月 14 日至 2026 年 7 月 14 日公开文献中的具身感知与导航,不把图像分类、离线定位或单一仿真榜单的高分直接等同于机器人能力。这里的“解决”至少包含五个条件:在闭环中连续工作,能迁移到未见环境,面对遮挡、动态主体和物理扰动仍可靠,失败后能够发现并恢复,并且在真实硬件上满足延迟与安全约束。

精读论文
15
正式证据事件
15
知识单元
EA-SENSOR, EA-EVAL, ERR-EMBODIED
审计状态
当前结算 run 已通过仓库校验

v0 · 完整正文

研究综述

研究边界

本文讨论的是 2025 年 7 月 14 日至 2026 年 7 月 14 日公开文献中的具身感知与导航,不把图像分类、离线定位或单一仿真榜单的高分直接等同于机器人能力。这里的“解决”至少包含五个条件:在闭环中连续工作,能迁移到未见环境,面对遮挡、动态主体和物理扰动仍可靠,失败后能够发现并恢复,并且在真实硬件上满足延迟与安全约束。

本次范围研究覆盖 511 篇候选论文,恢复 39 篇全文,并对其中 15 篇代表性论文做了逐段核验。它足以判断主要失败机制和能力边界,但不能证明所有机器人形态、所有传感器配置或所有导航任务都具有同一结论。

中心判断

感知和导航都没有作为“开放世界通用能力”被解决;被显著推进的是若干有明确边界的子问题。静态或弱动态环境中的语义识别、地图辅助定位、短程目标导航和受控条件下的主动观测,已经从“能否做到”转向“如何提高效率”。但一旦任务要求长期运行、跨视角记忆、动态交互、真实碰撞约束和异常恢复,性能仍会系统性退化。更准确的说法是:感知与导航的局部模块正在成熟,闭环可靠性尚未闭合。

一、为什么会出现“已经解决”的印象

过去一年有三类进展非常醒目。第一,视觉语言模型把开放词汇语义直接带入导航,使机器人不再完全依赖封闭类别表。第二,三维场景图、拓扑先验和动态语义图改善了长程搜索与重规划。第三,主动感知和端侧蒸馏开始处理固定视角与推理延迟问题。MSGNav 展示了多模态三维场景图对零样本导航的价值,CausalNav 把动态语义图用于真实户外长距离导航,LocalNav 则说明前沿模型能力可以被压缩到端侧运行。

这些结果是真进步,却容易造成统计口径错位:论文通常回答“某个方法在给定任务上是否优于基线”,而“问题是否解决”要求回答“系统在未见、动态、长时、受物理约束的环境里是否仍然可靠”。前者的正结果并不能自动推出后者。

二、感知的剩余问题不是再提高一次识别精度

具身感知与静态视觉最大的不同,是观测由动作产生。机器人当前看不到什么、下一步去哪里看、错误观测如何进入记忆,都会改变后续决策。因此,感知是否解决不能只看单帧准确率。

主动观测确实可以修复一部分固定视角缺陷。ActiveVLA 通过主动感知改善精细三维操作,OA-NBV 让移动机器人选择能绕开人体遮挡的下一视点。但两篇论文都保留了清晰边界:前者在最难的组合泛化任务上仍明显退化,后者解决的是单步视点选择,而非持续多视图融合。这说明“会移动相机”是必要能力,还不是稳定世界状态估计。

更关键的是,感知误差会形成有结构的污染。Rethinking Embodied Navigation via Relational Inductive Bias 指出,开放词汇感知的视觉相似性误检、难以撤销的静态先验和缺少行动验证,会持续污染地图与导航决策。VLM-LLM Navigation Bottlenecks 进一步显示,独立感知精度提高会出现边际饱和,而误检和目标框形变仍能主导失败。由此可见,真正缺失的是带不确定性的时序融合、证据撤销和主动验证,而不只是更大的视觉主干。

社会导航把这一问题放得更大。Ratatouille 的专家能利用微妙的人体姿态和意图线索,机器人却主要看到位置并承受感知延迟。这里的障碍不是“看见一个人”,而是及时估计对方将如何运动,并把这种估计安全地写入动作决策。

三、导航的剩余问题是从路线生成到物理闭环的落差

近年的语言导航系统已经能生成看起来合理的高层路线,但合理路线并不保证可执行。VLNVerse 给出了很直接的对照:思维链提示能改善放松碰撞约束的传送式设置,到了严格物理条件下性能仍然很低,碰撞成为主要瓶颈。这不是语言推理完全无效,而是高层语义正确没有覆盖局部几何、动力学和控制误差。

这种落差在开放世界基准上更明显。UAV-ON 中所有受测方法的碰撞率都超过 30%,远高于真实部署可接受水平。NaviTrace 则发现视觉语言模型与人类和近似预言机基线仍有显著差距,目标定位是主导失败模式;即使给出正确目标,路径塑形也仍落后于人类。导航尚未解决的不是某一个规划算法,而是目标定位、局部避障、动作后果预测和路径执行的联合误差。

连续任务还揭示了更深的认知缺口。城市空域导航基准显示,模型在关键决策分叉处的失败横跨几何理解、跨视角对应、动作后果想象和长期记忆,而不是单一视觉分类错误。HaltNav 的真实测试也表明,系统需要显式异常检测、轻量结构先验和重规划;缺少这些机制的基线在目标式指令和临时阻塞下会出现灾难性退化。

四、感知与导航不能被分开宣告“解决”

感知和导航构成一个反馈环:感知形成状态,状态驱动动作,动作决定下一次观测。误检会把错误对象写进地图,错误地图会引导重复探索;不良视角会导致目标定位失败,错误动作又可能使目标永远离开视野。因而,离线感知分数和规划成功率只描述了环中的两个截面。

近一年文献中也出现了缩小这一落差的路线。ReaDy-Go 用真实到仿真的动态场景支持特定环境迁移,CausalNav 强调长时动态语义建模,HaltNav 把异常检测与重规划纳入系统。但它们各自仍受训练环境数量、极端光照天气、长时一致性或动态主体多样性限制。这些工作更像“把闭环补齐”的证据,而不是“闭环已经补齐”的证据。

条件、分歧与可证伪边界

“尚未解决”不应被理解为所有子任务都不成熟。在以下条件同时成立时,可以把导航看作工程上接近可用:环境静态或低动态、地图或拓扑先验可靠、目标类别明确、视角与光照变化有限、速度较低、任务时长较短,并允许人工初始化或失败重启。仓储巡检、固定园区配送和结构化室内搜索中的一部分任务已经接近这一状态。

结论会被两类未来证据推翻。第一,系统在未见真实环境中持续运行数十小时,面对遮挡、移动障碍、照明和传感器扰动仍保持低碰撞、低近失事件和稳定到达率。第二,系统能在错误地图、目标误检和临时阻塞后自主诊断、撤销错误证据并恢复任务,而且不依赖隐藏的人类干预。若这些结果能被跨平台复现,就可以开始讨论“问题已基本解决”。

当前证据还存在一个评价层缺口。Robot Policy Evaluation for Sim-to-Real Transfer 指出,标准化且可扩展的仿真到真实基准仍然不足。因此,不同论文间的成功率并不能直接横向拼接,更不能从一次真实演示推导长期可靠性。

可操作的研究与评测框架

层级不应只看更有判别力的指标
感知单帧检测精度遮挡后重识别、跨视角一致性、错误证据撤销、时延下状态有效性
导航单次成功率碰撞与近失事件、路径执行误差、恢复成功率、无人工干预时长
系统仿真排行榜未见真实环境迁移、极端条件退化曲线、平均故障间隔、跨平台复现

研究设计上,应优先记录“第一处可证伪偏离点”:目标是否实际可见,状态是否被正确写入记忆,计划是否使用了正确状态,动作是否可执行,控制器是否实现了动作。这样才能判断失败来自感知、记忆、推理、规划还是控制,避免把所有问题都归因于视觉模型。

研究空白与下一步

第一,需要从短回合成功率转向长期可靠性。多数论文仍缺少小时级连续运行、累计地图漂移和故障恢复统计。第二,需要把动态环境中的人类意图、物体运动和传感延迟统一纳入状态估计,而不是把动态主体当作瞬时障碍物。第三,需要建立跨仿真、真实环境和不同机器人形态的统一压力测试,公开近失事件、人工接管和失败重试。第四,需要评估“错误如何被发现与撤销”,因为开放词汇感知带来的系统性误导比随机噪声更危险。

结论

近一年文献不支持“具身感知和导航已经解决”的笼统说法。它支持的是一个更精确也更有用的判断:受控条件下的语义感知、地图组织和短程导航正在快速工程化;开放世界中的主动感知、物理安全、长期记忆、动态交互与失败恢复仍是核心研究问题。下一阶段的胜负不只取决于模型能否找到目标,而取决于机器人能否知道自己何时看错、走错,并在真实世界中安全地改正。

References

Accepted evidence

去重论文引用

  1. ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic ManipulationZhenyang Liu · 2026
  2. CausalNav: A Long-term Embodied Navigation System for Autonomous Mobile Robots in Dynamic Outdoor ScenariosHongbo Duan · 2026
  3. Exploring Bottlenecks in VLM-LLM Navigation: How 3D Scene Understanding Capability Impacts Zero-Shot VLNZiyi Xia · 2026
  4. HaltNav: Reactive Visual Halting over Lightweight Topological Priors for Robust Vision-Language NavigationZihui Yu · 2026
  5. How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban AirspaceBaining Zhao · 2026
  6. LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal NavigationNicolas Baumann · 2026
  7. MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied NavigationXun Huang · 2025
  8. NaviTrace: Evaluating Embodied Navigation of Vision-Language ModelsTim Windecker · 2025
  9. OA-NBV: Occlusion-Aware Next-Best-View Planning for Human-Centered Active Perception on Mobile RobotsBoxun Hu · 2026
  10. Ratatouille: Imitation Learning Ingredients for Real-world Social Robot NavigationJames R. Han · 2025
  11. ReaDy-Go: Real-to-Sim Dynamic 3D Gaussian Splatting Simulation for Environment-Specific Visual Navigation with Moving ObstaclesSeungyeon Yoo · 2026
  12. Rethinking Embodied Navigation via Relational Inductive BiasWeitao An · 2026
  13. Robot Policy Evaluation for Sim-to-Real Transfer: A Benchmarking PerspectiveXuning Yang · 2025
  14. UAV-ON: A Benchmark for Open-World Object Goal Navigation with Aerial AgentsJianqiang Xiao · 2025
  15. VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and EvaluationSihao Lin · 2025
完整证据附录(15 条正式证据事件)
  • Time range: 2025-07-14..2026-07-14
  • Events: 15
  • 每个事件一节,标题即锚点;trace-map 中的 event 链接跳转到这里。

EA-PNAV-2026-0002

  • Claim: 主动感知能改善固定视角VLA,但并未解决通用感知;论文在最难的组合泛化任务上仍报告明显退化。
  • Stance: conditional | Confidence: direct
  • Paper: 2601.08325 ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulation
  • Locator: 4.1 Experimental Results
  • Evidence: 结果段在报告总体领先的同时明确指出最难L4任务性能下降。
  • Quote: “Although performance decreases on the most difficult L4 tasks (1.2%), ActiveVLA still demonstrates promising long-horizon reasoning and strong 3D perception for precise manipulation.”
  • Authors: zhenyang-liu

EA-PNAV-2026-0011

  • Claim: ReaDy-Go支持环境特定的动态sim-to-real路线,但作者仍要求扩大训练环境,并引入安全学习以应对更密集、多样和激进的动态主体。
  • Stance: conditional | Confidence: direct
  • Paper: 2602.11575 ReaDy-Go: Real-to-Sim Dynamic 3D Gaussian Splatting Simulation for Environment-Specific Visual Navigation with Moving Obstacles
  • Locator: V Conclusion
  • Evidence: 结论把泛化潜力与后续扩环境、处理复杂动态体的必要性并列。
  • Quote: “Future work includes expanding training environments to strengthen generalization and integrating safe reinforcement learning methods to handle more challenging scenarios involving diverse dynamic agents beyond humans, dense dynamic obstacle settings, and aggressive human motions.”
  • Authors: seungyeon-yoo

EA-PNAV-2026-0003

  • Claim: OA-NBV证明机器人可以主动绕开遮挡获得更好观察,但作者明确把能力限定为单步视点选择,而非完整多视图感知。
  • Stance: conditional | Confidence: direct
  • Paper: 2603.11072 OA-NBV: Occlusion-Aware Next-Best-View Planning for Human-Centered Active Perception on Mobile Robots
  • Locator: V-B Limitations and future work.
  • Evidence: 限制段直接划定即时单步观测与完整多视图任务之间的边界。
  • Quote: “Finally, OA-NBV targets single-step viewpoint selection for immediate observation quality, rather than multi-view reconstruction.”
  • Authors: boxun-hu

EA-PNAV-2026-0012

  • Claim: 真实VLN鲁棒性依赖显式结构先验、异常检测和重规划;没有这些机制的基线在目标式指令或阻塞下会出现灾难性退化。
  • Stance: conditional | Confidence: direct
  • Paper: 2603.12696 HaltNav: Reactive Visual Halting over Lightweight Topological Priors for Robust Vision-Language Navigation
  • Locator: IV-C Real-World Evaluation Results
  • Evidence: 实机结果段直接比较基线在目标式和障碍注入条件下的崩溃。
  • Quote: “The real world dramatically amplifies both failure modes observed in simulation. Under goal-only instructions (L2-B), InternVLA-N1 collapses to 0% SR and StreamVLN to 13.33%, far below their simulation counterparts. Under obstacle injection, both baselines drop to 0% SR across all conditions—even StreamVLN, which retained 37.50% in simulation.”
  • Authors: zihui-yu

EA-PNAV-2026-0004

  • Claim: 对零样本VLN而言,感知并非简单地“越准越已解决”:独立精度会出现边际饱和,而误检和框形变仍是关键失败源。
  • Stance: conditional | Confidence: direct
  • Paper: 2605.14801 Exploring Bottlenecks in VLM-LLM Navigation: How 3D Scene Understanding Capability Impacts Zero-Shot VLN
  • Locator: IV CONCLUSIONS
  • Evidence: 结论直接同时报告感知饱和和两类仍关键的误差。
  • Quote: “Our analysis also revealed that false positives and distorted bounding box aspect ratios are critical factors affecting navigation performance, and that a small set of core navigation-relevant object categories is sufficient for successful navigation.”
  • Authors: ziyi-xia

EA-PNAV-2026-0015

  • Claim: 开放世界航空ObjectNav远未解决:基准中所有方法的碰撞率都超过真实部署可接受水平,语义探索尚未转化为安全控制。
  • Stance: limit | Confidence: direct
  • Paper: 2508.00288 UAV-ON: A Benchmark for Open-World Object Goal Navigation with Aerial Agents
  • Locator: 6.2 Results
  • Evidence: 结果段直接指出所有方法的高碰撞率和真实部署不可接受性。
  • Quote: “Notably, all methods exhibit collision rates exceeding 30%, which would be unacceptable in real-world UAV deployments where safety is critical. Such high collision rates pose serious risks in physical environments, where hardware damage and human safety must be considered.”
  • Authors: jianqiang-xiao

EA-PNAV-2026-0009

  • Claim: 真实社会导航的进步仍依赖受限的人体状态表征;论文明确指出机器人缺少专家可用的人类意图线索,并受到感知延迟影响。
  • Stance: limit | Confidence: direct
  • Paper: 2509.17204 Ratatouille: Imitation Learning Ingredients for Real-world Social Robot Navigation
  • Locator: VI Limitations and Future Work
  • Evidence: 限制段直接比较专家与机器人可见信息,说明剩余感知鸿沟。
  • Quote: “The expert can infer human intent from subtle body cues and benefits from perfect perception without delays, whereas the robot only observes human positions.”
  • Authors: james-r-han

EA-PNAV-2026-0010

  • Claim: 当前VLM导航仍存在显著人类差距,且目标定位是主导失败模式;这说明基础视觉语言能力尚未等价为可靠空间行动。
  • Stance: limit | Confidence: direct
  • Paper: 2510.26909 NaviTrace: Evaluating Embodied Navigation of Vision-Language Models
  • Locator: IV-D Summary of Key Findings
  • Evidence: 关键发现段同时报告人类差距和目标定位主导失败。
  • Quote: “(1) Large human performance gap. Across all four embodiments and task categories, VLM scores are substantially worse than both human and oracle-like baselines, highlighting significant room for improvement (see Fig. ˜ 4 and Table ˜ III ). (2) Goal localization is the dominant failure mode. When models predict only the goal location and we connect it with a straight line, scores are similar to full-trace predictions.”
  • Authors: tim-windecker

EA-PNAV-2026-0001

  • Claim: MSGNav的结果不能说明零样本导航已解决:作者明确指出VFM/VLM延迟阻碍实时部署,且最后一公里仅被缓解而未被彻底解决。
  • Stance: limit | Confidence: direct
  • Paper: 2511.10376 MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigation
  • Locator: 5 Conclusion
  • Evidence: 结论段直接给出两项残余问题,构成对榜单增益的部署边界。
  • Quote: “Despite these advantages of MSGNav , scene graph-based methods still face low inference efficiency due to the latency of VFMs and VLMs, suggesting future work on faster graph construction and inference for real‑time deployment. Additionally, while the “last-mile” problem has been mitigated by the VVD module, it is not fully resolved.”
  • Authors: xun-huang

EA-PNAV-2026-0007

  • Claim: 现有VLN的高层推理并未克服物理执行:即使CoT改善理想化传送设置,严格物理条件下性能仍低,碰撞是主要瓶颈。
  • Stance: limit | Confidence: direct
  • Paper: 2512.19021 VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluation
  • Locator: 5.3 Zero-shot Performance on VLNVerse
  • Evidence: 零样本实验把同一代理放入传送和严格物理设置,直接暴露碰撞导致的退化。
  • Quote: “Strikingly, when we introduce CoT prompting, performance in the Tel-Hop setting improves further, yet performance in the Strict setting remains low. This result strongly confirms that physical collision is the primary bottleneck.”
  • Authors: sihao-lin

EA-PNAV-2026-0008

  • Claim: CausalNav说明动态语义图可显著推进户外长距离导航,但作者仍把扩展性、极端光照天气和长时一致性列为未解决限制。
  • Stance: limit | Confidence: direct
  • Paper: 2601.01872 CausalNav: A Long-term Embodied Navigation System for Autonomous Mobile Robots in Dynamic Outdoor Scenarios
  • Locator: V Conclusion and Future Work
  • Evidence: 结论的限制句直接界定了方法在真实动态场景之外的缺口。
  • Quote: “While effective in dynamic scenes, CausalNav still faces limitations in scalability, robustness under extreme lighting or weather, and long-horizon consistency.”
  • Authors: hongbo-duan

EA-PNAV-2026-0005

  • Claim: 当前LMM的连续空间行动仍远未解决:失败跨越几何感知、跨视角理解、动作后果想象和长期记忆,而非单一视觉分类误差。
  • Stance: limit | Confidence: direct
  • Paper: 2604.07973 How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspace
  • Locator: 7. Conclusion
  • Evidence: 结论把关键决策分叉后的发散归结为四类相互耦合的能力缺口。
  • Quote: “By examining navigation completion progress curves, we identify a critical decision bifurcation (CDB) phenomenon, in which navigation errors of LMMs do not grow gradually but instead deviate after a pivotal decision point. From the CDB viewpoint, we derive that LMMs exhibit emerging reasoning and action capabilities but still suffer from four key limitations: insufficient geometric perception, limited cross-view understanding, lack of spatial imagination about action consequences, and weak long-”
  • Authors: baining-zhao

EA-PNAV-2026-0013

  • Claim: 开放词汇感知错误会形成系统性误导并持续污染地图与导航决策,因此标准检测能力并不等于具身感知已解决。
  • Stance: limit | Confidence: direct
  • Paper: 2606.10348 Rethinking Embodied Navigation via Relational Inductive Bias
  • Locator: Abstract > first paragraph
  • Evidence: 引言直接描述视觉相似、静态先验和缺少动作验证导致的持续污染。
  • Quote: “visual similarity can induce false positives, static priors are difficult to update once contradicted, and the lack of embodied verification may lead to repeated failed exploration, continuously contaminating map updates and navigation decisions.”
  • Authors: weitao-an

EA-PNAV-2026-0014

  • Claim: 端侧VLM可显著降低导航推理延迟,但基于场景图的ObjectNav仍无法原生表示瞬态组合语义,动态线索可能在稀疏查询间丢失。
  • Stance: limit | Confidence: direct
  • Paper: 2606.27871 LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation
  • Locator: 5 Conclusion
  • Evidence: 结论的限制段直接说明场景图的时空语义缺口。
  • Quote: “Limitations SG -based ObjectNav approaches are spatio-temporally limited. While the SG updates between VLM queries, the SG cannot inherently bind transient, compositional semantics (e.g., distinguishing a generic ”chair” from a ”chair with a person on it” ).”
  • Authors: nicolas-baumann

EA-PNAV-2026-0006

  • Claim: 缺乏标准化、可扩展的sim-to-real基准本身就是关键瓶颈,因此模拟榜单分数不足以宣告感知或导航已经解决。
  • Stance: gap | Confidence: direct
  • Paper: 2508.11117 Robot Policy Evaluation for Sim-to-Real Transfer: A Benchmarking Perspective
  • Locator: I Introduction
  • Evidence: 引言直接把标准化sim-real可迁移性基准的缺失称为关键瓶颈。
  • Quote: “The absence of a standardized, scalable robotic benchmark for sim-to-real transferability presents a critical bottleneck for visual policy for robotics.”
  • Authors: xuning-yang

References

Interactive Wiki

进入交互阅读

静态页采用默认版本;v0、v0、v0可在同一交互 Wiki 中阅读,不建立相互竞争的索引页。

Method and limits

研究方法、证据边界与引用说明

本页由当前结算的证据层自动派生,不改写已结算 run。论文只有在完整非 OCR 全文、结构化论文笔记和 claim-support audit 通过后,才进入接受证据。

页面结论是对所列论文与研究窗口的综合,不等同于无限外推。引用本专题时,请同时保留本页 canonical 地址、访问日期,并在需要时回溯原论文。