• 论文
主办单位:煤炭科学研究总院有限公司、中国煤炭学会学术期刊工作委员会
面向多人−多机复杂协作任务的煤矿XR智能运维系统
  • Title

    Research on coal mine XR intelligent operation and maintenance system for complex collaborative tasks involving multiple humans and multiple robots

  • 作者

    王学文刘曙光王雪松谢嘉成刘京铨王禧龙

  • Author

    WANG Xuewen;LIU Shuguang;WANG Xuesong;XIE Jiacheng;LIU Jingquan;WANG Xilong

  • 单位

    太原理工大学 机械与运载工程学院煤矿综采装备山西省重点实验室智能采矿装备技术全国重点实验室

  • Organization
    College of Mechanical and Vehicle Engineering, Taiyuan University of Technology
    Shanxi Key Laboratory of Fully-Mechanized Coal Mining Equipment
    National Key Laboratory of Intelligent Mining Equipment Technology
  • 摘要

    随着煤矿智能化的发展与煤矿机器人的研发应用,煤矿操作员与煤矿机器人之间的高效协作对于井下复杂任务起到至关重要的作用。为优化多工种煤矿操作员与多机器人协作的复杂运行关系,基于数字孪生理念与在XR领域的长期实践,开展面向多人−多机复杂协作任务的煤矿XR智能运维系统设计与关键技术研究。首先,针对复杂任务中2类煤矿操作员(包括集控操作员与就地操作员)与2类煤矿机器人(包括探测机器人与作业机器人)协作的典型场景,设计了系统总体架构,将系统划分为物理子系统、VR运维子系统与AR运维子系统3部分,并对各部分的内容、功能以及3部分之间的协同运行关系进行介绍;然后,对系统涉及的VR运维子系统构建、AR运维子系统构建以及通讯网络构建等关键技术进行剖析,对各关键技术对应的解决方案进行了探讨,并实现了2类煤矿操作员、2类煤矿机器人与VR/AR运维子系统的集成运行;最后,在实验室环境下模拟井下复杂环境搭建了试验场地,在试验场地中设定了任务点与具体任务,对系统的可行性与有效性进行测试验证。试验结果表明,煤矿XR智能运维系统能够在不同复杂任务对应的多人−多机协作情形中发挥作用。通过VR运维子系统与AR运维子系统的协同运行,可实现虚拟空间和物理空间的协同感知、决策与控制,能够在虚拟空间中对物理空间的复杂任务进行迭代、优化和验算,形成了人−人、人−机、机−机交互协作的智能运行模式。

  • Abstract

    With the development of coal mine intelligence and the application of coal mine robots, an efficient collaboration between coal mine operators and coal mine robots plays a crucial role in the execution of complex underground tasks. To optimize the complex operational relationship of multiple coal mine operators and multiple robots, based on the concept of digital twin and extensive experience in the XR field, the research is conducted on the design and key technologies of XR intelligent operation and maintenance system for complex collaborative tasks involving multiple humans and multiple robots in coal mines. Firstly, for a typical scenario of collaboration between two types of coal mine operators (i.e central control operators and field control operators) and two types of coal mine robots (i.e. detection robots and operating robots) in complex tasks, the overall system architecture is designed. The system is divided into three parts: the physical subsystem, VR operation and maintenance subsystem, and AR operation and maintenance subsystem. The content, functions, and collaborative operation relationships among these three parts are introduced. Then, an analysis of key technologies related to the VR operation and maintenance subsystem, AR operation and maintenance subsystem, and communication networking is carried out. The solutions corresponding to each key technology are discussed, and the integration and operation of the two types of coal mine operators, two types of coal mine robots, and VR/AR operation and maintenance subsystem are implemented. Finally, a laboratory environment simulating complex underground conditions is set up to create a test site, where the task points and specific tasks are defined. The feasibility and effectiveness of the system are tested and verified. The experimental results show that the coal mine XR intelligent operation and maintenance system is able to function in collaborative situations between multiple humans and multiple robots corresponding to different complex tasks. Through the collaborative operation of the VR operation and maintenance subsystem and the AR operation and maintenance subsystem, the collaborative perception, decision-making, and control between virtual space and physical space can be achieved. This allows for the iterative optimization and verification of complex tasks in a physical space from a virtual space, forming an intelligent operational mode of human-human, human-robot, and robot-robot interactive collaboration.

  • 关键词

    煤矿智能化煤矿机器人煤矿操作员人机协作XR智能运维数字孪生

  • KeyWords

    coal mine intelligence;coal mine robots;coal mine operators;human-robot collaboration;XR intelligent operation and maintenance;digital twin

  • 基金项目(Foundation)
    中央引导地方科技发展资金资助项目(YDZJSX2022A014); 山西省留学人员科技活动择优重点资助项目(20230008); 山西省回国留学人员科研资助项目(2023-71)
  • DOI
  • 引用格式
    王学文,刘曙光,王雪松,等. 面向多人−多机复杂协作任务的煤矿XR智能运维系统[J]. 煤炭学报,2024,49(4):2124−2140.
  • Citation
    WANG Xuewen,LIU Shuguang,WANG Xuesong,et al. Research on coal mine XR intelligent operation and maintenance system for complex collaborative tasks involving multiple humans and multiple robots[J]. Journal of China Coal Society,2024,49(4):2124−2140.
  • 相关文章
  • 图表
    VR运维子系统各模块耦合运行关系
    VR运维子系统各模块耦合运行关系
    系统协同运行关系
    系统协同运行关系
    人机状态监测界面
    人机状态监测界面
    整体任务场景的构建过程
    整体任务场景的构建过程
    基于5W2H的人机任务分配
    基于5W2H的人机任务分配
    全局与局部导航
    全局与局部导航
    AR对象虚实映射
    AR对象虚实映射
    多模态交互模型
    多模态交互模型
    WebRTC连接建立过程
    WebRTC连接建立过程
    ROS与Unity3d双向通讯
    ROS与Unity3d双向通讯
    基于PUN的Unity3d跨平台通讯过程
    基于PUN的Unity3d跨平台通讯过程
    感知−决策−控制一体化架构
    感知−决策−控制一体化架构
    2类煤矿操作员与2类煤矿机器人
    2类煤矿操作员与2类煤矿机器人
    任务点布置
    任务点布置
    液压支架试验台控制任务
    液压支架试验台控制任务
    路障清除任务规划与执行过程
    路障清除任务规划与执行过程
    采煤机截齿维修任务
    采煤机截齿维修任务

    Table1

    优化前后A*算法规划时间对比
    组别 优化前规划
    时间/ms
    优化后规划
    时间/ms
    时间缩短/%
    1 25.73 5.12 80.10
    2 21.18 4.28 79.79
    3 27.34 5.50 79.88
    4 25.42 5.11 79.90
    5 19.11 3.98 79.17
    6 23.87 4.83 79.77
    7 21.46 4.39 79.54
    8 25.38 5.09 79.94
    平均 79.76

    Table2

    ROS各通信机制的优缺点
    通信机制 优点 缺点
    主题 广播/订阅机制,
    异步传输,延迟低
    如果主题发布频率高,可能会导致消息堆积,从而影响稳定性
    服务 请求/响应模式,
    同步传输,稳定性高
    等待响应可能会引入一些延迟
    动作 提供长时间任务支撑,允许节点发送目标并接收状态反馈 不适用于对实时性要求
    较高的任务
    参数服务器 分布式参数存储
    系统,通信延迟较低,稳定性也很高
    参数通常为静态,不适合频
    繁变化的实时数据
相关问题

主办单位:煤炭科学研究总院有限公司 中国煤炭学会学术期刊工作委员会

©版权所有2015 煤炭科学研究总院有限公司 地址:北京市朝阳区和平里青年沟东路煤炭大厦 邮编:100013
京ICP备05086979号-16技术支持: 云智互联
Baidu
map