Automated Gaze-based Behavioral Segmentation and Temporal Representation for Bridge Inspection in Unconstrained 3D Environments

Abstract (EN)

Visual bridge inspection is a knowledge-intensive task in which inspectors coordinate visual search, spatial navigation, structural reasoning, and defect identification and documentation. It is a central maintenance task for bridges and a key basis for safety assessments, yet its results are susceptible to individual subjectivity. While eye-tracking-based behavioral studies quantify underlying processes, existing research often imposes restrictive simplifications to reduce environmental complexity, thereby compromising ecological validity. This study proposes an automated data analytics framework for converting multimodal inspection data into an inspection mode time series. Unconstrained 3D gaze, head-movement, drone navigation, and scene geometry data are segmented into temporal windows and classified into three functional modes: global scanning, local inspection, and navigation. The resulting temporal representation enables the extraction of interpretable behavioral descriptors, including transition probabilities, dwell times, transition entropy, fixation measures, and spatial revisit metrics. A feasibility study using a virtual bridge inspection platform demonstrates that the proposed representation captures meaningful differences in inspection strategy and reveals exploratory relationships with inspection performance. This study contributes a framework for human-informed computer-aided infrastructure inspection systems, inspector training, and data-driven assessment of constructed facilities.

摘要 (ZH)

视觉桥梁检查是一项知识密集型任务,检查人员需协调视觉搜索、空间导航、结构推理及缺陷识别与记录。作为桥梁核心维护任务和安全评估的关键依据,其结果易受个体主观性影响。基于眼动追踪的行为研究虽能量化潜在过程,但现有研究常施加限制性简化以降低环境复杂性,从而损害生态效度。本研究提出一种自动化数据分析框架,将多模态检查数据转换为检查模式时间序列。非约束三维眼动、头部运动、无人机导航及场景几何数据被分割为时间窗口,并分类为三种功能模式:全局扫描、局部检查和导航。由此生成的时间表征可提取可解释的行为描述符,包括状态转移概率、驻留时间、转移熵、注视测量及空间重访指标。基于虚拟桥梁检查平台的可行性研究表明,该表征能捕捉检查策略的显著差异,并揭示其与检查绩效的探索性关联。本研究为基于人类认知的计算机辅助基础设施检查系统、检查人员培训及建成设施数据驱动评估提供了框架支撑。

← Back