ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning

Abstract (EN)

Vision-language navigation (VLN) for UAVs demands grounding free-form instructions into 6-DoF flight under partial observability. While Vision-Language-Action (VLA) models excel at semantic reasoning, they suffer from brittleness due to geometric inconsistency and dynamics mismatch. To address this, we propose ImagineUAV, an imagination-driven framework leveraging cascaded world-action modeling. Instead of direct regression, ImagineUAV employs a latent video diffusion model to generate instruction-conditioned future observations, explicitly imagining environmental evolution, from which 6-DoF motions are inferred via an action extractor. A kinodynamic planner then refines these estimates into collision-free trajectories. Additionally, a step-distilled inference pipeline ensures real-time execution. With only 1.3B parameters, ImagineUAV outperforms prior VLN and VLA baselines on benchmarks and real-world flights, validating the practicality of imagination-driven aerial navigation.

摘要 (ZH)

无人机视觉语言导航(VLN)要求在部分可观测条件下将自由形式的指令映射到六自由度飞行。虽然视觉-语言-动作(VLA)模型在语义推理方面表现出色,但由于几何不一致性和动力学不匹配,它们存在鲁棒性不足的问题。为此,我们提出ImagineUAV——一种利用级联世界-动作建模的想象驱动框架。ImagineUAV并非直接回归,而是采用潜在视频扩散模型生成指令条件下的未来观测,明确想象环境演化过程,进而通过动作提取器推断六自由度运动。随后,动力学规划器将这些估计优化为无碰撞轨迹。此外,一种逐步蒸馏的推理流程确保了实时执行。仅凭13亿参数,ImagineUAV在基准测试和真实飞行中均优于先前的VLN与VLA基线方法,验证了想象驱动空中导航的实用性。

← Back