• 论文
主办单位:煤炭科学研究总院有限公司、中国煤炭学会学术期刊工作委员会
基于图像与点云融合的巷道锚护孔位识别定位方法
  • Title

    Roadway anchor hole recognition and positioning method based on image and point cloud fusion

  • 作者

    王宏伟李进闫志蕊郭军军张夫净李超

  • Author

    WANG Hongwei;LI Jin;YAN Zhirui;GUO Junjun;ZHANG Fujing;LI Chao

  • 单位

    太原理工大学 机械与运载工程学院太原理工大学 山西省煤矿智能装备工程研究中心太原理工大学 矿业工程学院

  • Organization
    College of Mechanical and Vehicle Engineering, Taiyuan University of Technology
    Center of Shanxi Engineering Research for Coal Mine Intelligent Equipment, Taiyuan University of Technology
    College of Mining Engineering, Taiyuan University of Technology
  • 摘要

    煤矿掘进巷道锚护位置的精准识别与定位是钻锚机器人实现智能永久支护亟需突破的关键技术。笔者提出一种基于视觉图像与激光点云融合的巷道锚护孔位智能识别定位方法,包括图像目标识别、点云图像特征融合和定位坐标提取3个步骤:①针对煤矿井下低照度、水雾和粉尘等环境因素导致的锚孔轮廓成像模糊的问题,采用IA(Image-Adaptive)-SimAM-YOLOv7-tiny 网络对巷道待锚护孔位进行视觉识别,该网络能够自适应地增强图像亮度和对比度,恢复锚孔边缘的高频信息,并使模型重点关注锚孔特征,提高锚孔检测的成功率;②求解激光雷达和工业相机联合标定的外参矩阵,将图像检测的锚孔边界框通过透视投影关系生成锥形感兴趣区域(Region Of Interest,ROI),获得对应的目标点云团簇;③采用点云处理算法提取锚护孔位边界点云,获得孔位中心坐标及其法向量,并通过坐标深度差比较判断锚孔识别的正确性。文中搭建了锚杆台车机械臂钻孔定位系统,对算法自主定位的精度以及准确度进行验证,试验结果表明:IA-SimAM-YOLOv7-tiny模型的平均精度均值(Mean Average Precision,mAP)为87.3%,较YOLOv7-tiny模型提高了4.6%;提出的融合算法定位误差为3 mm,单锚孔情况下系统平均识别时间为0.77 s,与单一视觉方法相比,采用激光与视觉多源融合不仅可以降低环境和小样本训练对定位性能的影响,而且可以获得锚护孔位的法向量,为机械臂调整钻孔位姿实现精准锚固提供依据。

  • Abstract

    The accurate identification and positioning of the anchor position of the coal mine tunneling roadway is the key technology that the drilling anchor robot urgently needs to break through to achieve intelligent permanent support. An intelligent identification and positioning method for roadway anchor hole position based on visual image and laser point cloud fusion is proposed, which includes three steps: image target recognition, point cloud image feature fusion, and positioning coordinate extraction: ① In order to solve the problem of blurred image of anchor hole contour caused by environmental factors such as low illumination, water mist and dust in coal mines, the IA (Image-Adaptive)-SimAM-YOLOv7-tiny network was used to visually identify the position of the anchor hole to be anchored in the roadway, which can adaptively enhance the image brightness and contrast, recover the high-frequency information at the edge of the anchor hole, and make the model focus on the characteristics of the anchor hole, so as to improve the success rate of anchor hole detection. ② The Region of Interest (ROI) of image detection is generated by perspective projection relationship to generate a cone-shaped area of interest to obtain the corresponding target point cloud cluster; ③ The point cloud processing algorithm is used to extract the anchor hole boundary point cloud, obtain the central coordinates of the hole position and its normal vector, and judge the correctness of the anchor hole recognition by comparing the coordinate depth difference. In this paper, a drilling and positioning system for bolting trolley manipulator is built to verify the accuracy and accuracy of the algorithm's autonomous positioning, and the experimental results show that the mean average precision (mAP) of the IA-SimAM-YOLOv7-tiny model is 87.3%, which is 4.6% higher than that of the YOLOv7-tiny model. Compared with the single vision method, the fusion algorithm proposed in this paper has a positioning error of 3 mm, and the average recognition time of the system in the case of a single anchor hole is 0.77 s, compared with the single vision method, the fusion of laser and visual multi-source can not only reduce the influence of environment and small sample training on positioning performance, but also obtain the normal vector of the anchor hole position, which provides a basis for the manipulator to adjust the drilling posture to achieve accurate anchoring.

  • 关键词

    锚孔精准定位图像识别点云处理激光雷达和相机联合标定数据融合

  • KeyWords

    accurate positioning of anchor hole;image recognition;point cloud processing;joint calibration of lidar and camera;data fusion

  • 基金项目(Foundation)
    山西省揭榜招标资助项目(20201101008);山西省基础研究计划资助项目(202203021212275);山西省重点研发计划资助项目(202102100401015)
  • DOI
  • 引用格式
    王宏伟,李 进,闫志蕊,等. 基于图像与点云融合的巷道锚护孔位识别定位方法[J]. 煤炭科学技术,2024,52(5):249−261.
  • Citation
    WANG Hongwei,LI Jin,YAN Zhirui,et al. Roadway anchor hole recognition and positioning method based on image and point cloud fusion[J]. Coal Science and Technology,2024,52(5):249−261.
  • 图表

    Table1

    消融试验结果
    模型 精度值/
    %
    召回率/
    %
    mAP/
    %
    参数量/
    MB
    检测速度/
    ( \( \text{s}·{\text{img}}^{-1} \) )
    YOLOv7-tiny 94.1 93.3 82.7 12.3 0.08
    Faster-RCNN 76.3 98.2 75.5 113.4 0.45
    YOLOv5s 92.3 93.5 81.2 14.4 0.14
    YOLOv7 95.1 95.4 83.2 74.8 0.19
    IA-YOLOv7-tiny 95.4 95.6 84.8 12.5 0.25
    SimAM-YOLOv7-tiny 96.1 96.5 85.5 12.3 0.09
    IA-SimAM-YOLOv7-tiny 97.2 98.6 87.3 18.4 0.27

    Table2

    SimAM注意力机制的添加位置效果对比
    融合方式 不同评价方式下检测效果/%
    精度值 召回率 mAP
    Baseline 94.1 93.3 82.7
    Block-1 94.8 95.4 83.3
    Block-2 93.6 93.1 82.1
    Block-3 95.3 95.9 84.1
    Ours 96.1 96.5 85.5

    Table3

    不同场景下算法定位情况对比
    场景序号 识别算法 实际坐标/m 检测坐标/m 误差/mm 锚孔法向量 识别时间/s
    a 改进的YOLOv7-tiny+Depth (1.1100,0.1340,0.0980) (1.1099,0.1327,0.1081) (1.1,1.3,2.1) 0.43
    改进的YOLOv7-tiny+Lidar (1.1080,0.1670,0.0860) (1.1092,0.1653,0.0888) (1.2,1.7,1.8) (−0.8744,−0.4027,0.2703) 0.77
    b 改进的YOLOv7-tiny+Depth (1.1270,0.1300,0.1260) (1.1317,0.1351,0.1302) (4.7,5.1,4.2) 0.43
    改进的YOLOv7-tiny+Lidar (1.1280,0.1680,0.1190) (1.1302,0.1695,0.1176) (2.2,1.5,1.4) (−0.9879,−0.1518,−0.0312) 0.75
    c 改进的YOLOv7-tiny+Depth (1.5930,0.0870,0.1500) (1.5950,0.0855,0.1516) (2.0,1.5,1.6) 0.44
    (1.5893,0,0632,0.619)
    改进的YOLOv7-tiny+Lidar (1.5960,0.1160,0.1410) (1.5948,0.1172,0.1427) (1.2,0.8,1.7) (0.9874,0.1579,0) 1.28
相关问题

主办单位:煤炭科学研究总院有限公司 中国煤炭学会学术期刊工作委员会

©版权所有2015 煤炭科学研究总院有限公司 地址:北京市朝阳区和平里青年沟东路煤炭大厦 邮编:100013
京ICP备05086979号-16技术支持: 云智互联
Baidu
map