mireabulletinRussian Technological JournalRussian Technological Journal2782-32102500-316XRTU MIREA10.32362/2500-316X-2022-10-5-38-48mireabulletin-567Research ArticleРОБОТИЗИРОВАННЫЕ КОМПЛЕКСЫ И СИСТЕМЫ. ТЕХНОЛОГИИ ДИСТАНЦИОННОГО ЗОНДИРОВАНИЯ НЕРАЗРУШАЮЩЕГО КОНТРОЛЯMULTIPLE ROBOTS (ROBOTIC CENTERS) AND SYSTEMS. REMOTE SENSING AND NON-DESTRUCTIVE TESTINGТрекер объектов на спортивных мероприятиях3D object tracker for sports eventshttps://orcid.org/0000-0002-1219-5090ВолковаМ. А.VolkovaM. A.

Волкова Мария Александровна - старший преподаватель, кафедра проблем управления Института искусственного интеллекта.

119454, Москва, пр-т Вернадского, д. 78.

Scopus Author ID 57194215422, SPIN-код РИНЦ 5939-6811

Maria A. Volkova - Senior Lecturer, Control Problems Department, Institute of Artificial Intelligence, MIREA -Russian Technological University.

78, Vernadskogo pr., Moscow, 119454.

Scopus Author ID 57194215422, RSCI SPIN-code 5939-6811

volkova_m@mirea.ruhttps://orcid.org/0000-0003-3353-9945РомановМ. П.RomanovM. P.

Романов Михаил Петрович – доктор технических наук, профессор, директор Института искусственного интеллекта.

119454, Москва, пр-т Вернадского, д. 78.

Scopus Author ID 14046079000, SPIN-код РИНЦ 5823-8795

Mikhail P. Romanov - Dr. Sci. (Eng.), Professor, Director of the Institute of Artificial Intelligence, MIREA - Russian Technological University.

78, Vernadskogo pr., Moscow, 119454.

Scopus Author ID 14046079000, RSCI SPIN-code 5823-8795

m_romanov@mirea.ruhttps://orcid.org/0000-0003-0701-7529БычковА. М.BychkovA. M.

Бычков Александр Михайлович - ассистент, кафедра проблем управления Института искусственного интеллекта.

119454, Москва, пр-т Вернадского, д. 78.

Alexander M. Bychkov - Assistant, Control Problems Department, Institute of Artificial Intelligence, MIREA -Russian Technological University.

78, Vernadskogo pr., Moscow, 119454.

bychkov@mirea.ruМИРЭА - Российский технологический университетРоссияMIREA - Russian Technological UniversityRussian Federation2022201020221053848Copyright © Волкова М.А., Романов М.П., Бычков А.М., 20222022Волкова М.А., Романов М.П., Бычков А.М.Volkova M.A., Romanov M.P., Bychkov A.M.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.rtj-mirea.ru/jour/article/view/567Цели

Цели. На сегодняшний день спорт является одной из наиболее перспективных областей для применения систем слежения за объектами. Большинство методов, на базе которых реализованы эти системы, ориентированы на отслеживание движущихся объектов в двумерной плоскости, например, для локализации игроков на поле, а также на их идентификацию по различным признакам. С развитием дрон-рейсинга актуальной стала задача определения положения в трехмерной системе координат. Целями работы являются разработка программно-алгоритмического обеспечения метода, позволяющего отслеживать траекторию движущихся объектов в трехмерном пространстве, абстрагированного от способа сегментации данных, и тестирование предложенного решения для оценки качества работы трекера.

Методы

Методы. На основе проведенного обзора и анализа современных методов отслеживания траекторий движения был выбран метод сопоставления информации о скорости и положении объектов.

Результаты

Результаты. Предложена структура программно-алгоритмического обеспечения трекера движущихся объектов на спортивных мероприятиях и представлены результаты экспериментальных исследований на общедоступном датасете APIDIS, который включает в себя фрагменты видеозаписи баскетбольной игры, где по критерию качества отслеживания MOTA был получен показатель 0.858. Также были проведены эксперименты с использованием предложенного авторами датасета с пролетом FPV квадрокоптера по трассе. В результате по полученным с трекера данным была восстановлена траектория полета дрона в трехмерном пространстве.

Выводы

Выводы. Результаты проведенных экспериментальных исследований показали, что предложенное решение позволяет отслеживать траекторию полета квадрокоптера в трехмерной (мировой) системе координат, а также подходит для слежения за объектами на спортивных мероприятиях.

Objectives

Objectives. Sports events are currently among the most promising areas for the application of tracking systems. In most cases, such systems are designed to track moving objects in a two-dimensional plane, e.g., players on the field, as well as to identify them by various features. However, as new sports such as drone racing are developed, the problem of determining the position of an object in a three-dimensional coordinate system becomes relevant. The aim of the present work was to develop algorithms and software for a method to perform 3D tracking of moving objects, regardless of the data segmentation technique, and to test this method to estimate the tracking quality.

Methods

Methods. A method for matching information on the speed and position of objects was selected based on a review and analysis of contemporary tracking methods.

Results

Results. The structure of a set of algorithms comprising software for a moving-object tracker for sports events is proposed. Experimental studies were performed on the publicly available APIDIS dataset, where a MOTA metric of 0.858 was obtained. The flight of an FPV quadcopter along a track was also tracked according to the proposed dataset; the 3D path of the drone flight was reconstructed using the tracker data.

Conclusions

Conclusions. The results of the experimental studies, which demonstrated the feasibility of using the proposed method to track a quadcopter flight trajectory in a three-dimensional world coordinate system, is also showed that the method is suitable for tracking objects at sports events.

трекерслежение за движущимися объектамиFPV квадрокоптеропределение положениясистема слеженияtrackermoving object trackingFPV quadcopterlocalizationtracking systemРабота выполнена при финансовой поддержке РТУ МИРЭА в рамках гранта «Инновации в реализации приоритетных направлений развития науки и технологий» (НИЧ 28/24).This research is supported by the RTU MIREA grant “Innovations in the implementation of priority areas for the science and technology development,” project No. 28/24.ReferencesZein Y., Darwiche M., Mokhiamar O. GPS tracking system for autonomous vehicles. Alexandria Eng. J. 2018;57(4): 3127-3137. https://doi.org/10.1016/j.aej.2017.12.002Zein Y., Darwiche M., Mokhiamar O. GPS tracking system for autonomous vehicles. Alexandria Eng. J. 2018;57(4): 3127-3137. https://doi.org/10.1016/j.aej.2017.12.002Yu K., et al. Deep learning-based traffic safety solution for a mixture of autonomous and manual vehicles in a 5G-enabled intelligent transportation system. IEEE Transactions on Intelligent Transportation Systems. 2020;22(7): 4337-4347. https://doi.org/10.1109/TITS.2020.3042504Yu K., et al. Deep learning-based traffic safety solution for a mixture of autonomous and manual vehicles in a 5G-enabled intelligent transportation system. IEEE Transactions on Intelligent Transportation Systems. 2020;22(7): 4337-4347. https://doi.org/10.1109/TITS.2020.3042504Ryan B.J., et al. COVID-19 contact tracing solutions for mass gatherings. Disaster Medicine and Public Health Preparedness. 2021;15(3):e1-e7. https://doi.org/10.1017/dmp.2020.241Ryan B.J., et al. COVID-19 contact tracing solutions for mass gatherings. Disaster Medicine and Public Health Preparedness. 2021;15(3):e1-e7. https://doi.org/10.1017/dmp.2020.241Khan S., et al. Implementing traceability systems in specific supply chain management (SCM) through critical success factors (CSFs). Sustainability. 2018;10(1):204. https://doi.org/10.3390/su10010204Khan S., et al. Implementing traceability systems in specific supply chain management (SCM) through critical success factors (CSFs). Sustainability. 2018;10(1):204. https://doi.org/10.3390/su10010204Cioppa A., Deliege A., Magera F., Giancola S., Barnich O., Ghanem B., Van Droogenbroeck M. Camera calibration and player localization in soccerNet-v2 and investigation of their representations for action spotting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2021. P. 4537-4546. https://doi.org/10.1109/CVPRW53098.2021.00511Cioppa A., Deliege A., Magera F., Giancola S., Barnich O., Ghanem B., Van Droogenbroeck M. Camera calibration and player localization in soccerNet-v2 and investigation of their representations for action spotting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2021. P. 4537-4546. https://doi.org/10.1109/CVPRW53098.2021.00511Kong L., Zhu M., Ran N., Liu Q., He R. Online multiple athlete tracking with pose-based long-term temporal dependencies. Sensors. 2020;21(1):197. https://doi.org/10.3390/s21010197Kong L., Zhu M., Ran N., Liu Q., He R. Online multiple athlete tracking with pose-based long-term temporal dependencies. Sensors. 2020;21(1):197. https://doi.org/10.3390/s21010197Liu J., Tong X., Li W., Wang T., Zhang Y., Wang H. Automatic player detection, labeling and tracking in broadcast soccer video. Pattern Recognition Lett. 2009;30(2):103-113. https://doi.org/10.1016/j.patrec.2008.02.011Liu J., Tong X., Li W., Wang T., Zhang Y., Wang H. Automatic player detection, labeling and tracking in broadcast soccer video. Pattern Recognition Lett. 2009;30(2):103-113. https://doi.org/10.1016/j.patrec.2008.02.011Possegger H., Sternig S., Mauthner T., Roth P.M., Bischof H. Robust real-time tracking of multiple objects by volumetric mass densities. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013. P. 2395-2402. https://doi.org/10.1109/CVPR.2013.310Possegger H., Sternig S., Mauthner T., Roth P.M., Bischof H. Robust real-time tracking of multiple objects by volumetric mass densities. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013. P. 2395-2402. https://doi.org/10.1109/CVPR.2013.310Bialkowski A., Lucey P., Carr P., Denman S., Matthews I., Sridharan S. Recognising team activities from noisy data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2013. P. 984-990. https://doi.org/10.1109/CVPRW.2013.143Bialkowski A., Lucey P., Carr P., Denman S., Matthews I., Sridharan S. Recognising team activities from noisy data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2013. P. 984-990. https://doi.org/10.1109/CVPRW.2013.143Foehn P., Brescianini D., Kaufmann E., et al. AlphaPilot: Autonomous drone racing. Auton. Robot. 2022;46(1): 307-320. https://doi.org/10.1007/s10514-021-10011-yFoehn P., Brescianini D., Kaufmann E., et al. AlphaPilot: Autonomous drone racing. Auton. Robot. 2022;46(1): 307-320. https://doi.org/10.1007/s10514-021-10011-ySpica R., Cristofalo E., Wang Z., Montijano E., Schwager M. A real-time game theoretic planner for autonomous two-player drone racing. IEEE Transactions on Robotics. 2020;36(5):1389-1403. https://doi.org/10.1109/TRO.2020.2994881Spica R., Cristofalo E., Wang Z., Montijano E., Schwager M. A real-time game theoretic planner for autonomous two-player drone racing. IEEE Transactions on Robotics. 2020;36(5):1389-1403. https://doi.org/10.1109/TRO.2020.2994881Kaufmann E., et al. Beauty and the beast: Optimal methods meet learning for drone racing. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE; 2019. P. 690-696. https://doi.org/10.1109/ICRA.2019.8793631Kaufmann E., et al. Beauty and the beast: Optimal methods meet learning for drone racing. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE; 2019. P. 690-696. https://doi.org/10.1109/ICRA.2019.8793631Волкова М.А., Романов А.М., Романов М.П. Распределенная система локализации объектов в рабочей зоне модульного реконфигурируемого мобильного робота. Мехатроника, автоматизация, управление. 2021;22(12):634-643. https://doi.org/10.17587/mau.22.634-643VolkovaM.A., RomanovA.M., Romanov M.P. Distributed system for objects localization in the working area of a modular reconfigurable mobile robot. Mekhatronika, Avtomatizatsiya, Upravlenie. 2021;22(12):634-643 (in Russ.). https://doi.org/10.17587/mau.22.634-643Dai-Hong J., Lei D., Dan L., San-You Z. Moving-object tracking algorithm based on PCA-SIFT and optimization for underground coal mines. IEEE Access. 2019;7: 35556-35563. https://doi.org/10.1109/ACCESS.2019.2899362Dai-Hong J., Lei D., Dan L., San-You Z. Moving-object tracking algorithm based on PCA-SIFT and optimization for underground coal mines. IEEE Access. 2019;7: 35556-35563. https://doi.org/10.1109/ACCESS.2019.2899362Матвеев И.А., Чигринский В.В. Оптимизация работы системы слежения, основанной на сети камер. Известия Российской академии наук. Теория и системы управления. 2020;4:110-124. https://doi.org/10.31857/S0002338820040125Chigrinskii V.V., Matveev I.A. Optimization of a tracking system based on a network of cameras. J. Comput. Syst. Sci. Int. 2020;59(4):583-597. https://doi.org/10.1134/S1064230720040127Lan X., Ye M., Shao R., Zhong B., Yuen P.C., Zhou H. Learning modality-consistency feature templates: A robust RGB-infrared tracking system. IEEE Transactions on Industrial Electronics.2019;66(12):9887-9897. https://doi.org/10.1109/TIE.2019.2898618Lan X., Ye M., Shao R., Zhong B., Yuen P.C., Zhou H. Learning modality-consistency feature templates: A robust RGB-infrared tracking system. IEEE Transactions on Industrial Electronics.2019;66(12):9887-9897. https://doi.org/10.1109/TIE.2019.2898618Bao Q., Liu W., Cheng Y., Zhou B., Mei T. Pose-guided tracking-by-detection: Robust multi-person pose tracking. IEEE Transactions on Multimedia. 2020;23:161-175. https://doi.org/10.1109/TMM.2020.2980194Bao Q., Liu W., Cheng Y., Zhou B., Mei T. Pose-guided tracking-by-detection: Robust multi-person pose tracking. IEEE Transactions on Multimedia. 2020;23:161-175. https://doi.org/10.1109/TMM.2020.2980194Wang Z., Li M., Lu Y., Bao Y., Li Z., Zhao J. Effective multiple pedestrian tracking system in video surveillance with monocular stationary camera. Expert Systems with Applications. 2021;178:114992. https://doi.org/10.1016/j.eswa.2021.114992Wang Z., Li M., Lu Y., Bao Y., Li Z., Zhao J. Effective multiple pedestrian tracking system in video surveillance with monocular stationary camera. Expert Systems with Applications. 2021;178:114992. https://doi.org/10.1016/j.eswa.2021.114992Linke D., Link D., Lames M. Football-specific validity of TRACAB's optical video tracking systems. PLoSONE. 2020;15(3):e0230179. https://doi.org/10.1371/journal.pone.0230179Linke D., Link D., Lames M. Football-specific validity of TRACAB's optical video tracking systems. PLoSONE. 2020;15(3):e0230179. https://doi.org/10.1371/journal.pone.0230179Pons E., Garda-Calvo T., Resta R., Blanco H., Lopez del Campo R., D^az Garda J., Pulido J.J. A comparison of a GPS device and a multi-camera video technology during official soccer matches: Agreement between systems. PLoSONE. 2019;14(8):e0220729. https://doi.org/10.1371/journal.pone.0220729Pons E., Garda-Calvo T., Resta R., Blanco H., Lopez del Campo R., D^az Garda J., Pulido J.J. A comparison of a GPS device and a multi-camera video technology during official soccer matches: Agreement between systems. PLoSONE. 2019;14(8):e0220729. https://doi.org/10.1371/journal.pone.0220729Nishikawa Y., Sato H., Ozawa J. Multiple sports player tracking system based on graph optimization using lowcost cameras. In: 2018 IEEE International Conference on Consumer Electronics (ICCE). IEEE; 2018. P. 1-4. https://doi.org/10.1109/ICCE.2018.8326126Nishikawa Y., Sato H., Ozawa J. Multiple sports player tracking system based on graph optimization using lowcost cameras. In: 2018 IEEE International Conference on Consumer Electronics (ICCE). IEEE; 2018. P. 1-4. https://doi.org/10.1109/ICCE.2018.8326126Hui Q. Motion video tracking technology in sports training based on Mean-Shift algorithm. J. Supercomput. 2019;75(9):6021-6037. https://doi.org/10.1007/s11227-019-02898-3Hui Q. Motion video tracking technology in sports training based on Mean-Shift algorithm. J. Supercomput. 2019;75(9):6021-6037. https://doi.org/10.1007/s11227-019-02898-3Taj M., Cavallaro A. Distributed and decentralized multicamera tracking. IEEE Signal Processing Magazine. 2011;28(3):46-58. https://doi.org/10.1109/MSP.2011.940281Taj M., Cavallaro A. Distributed and decentralized multicamera tracking. IEEE Signal Processing Magazine. 2011;28(3):46-58. https://doi.org/10.1109/MSP.2011.940281Taj M., Cavallaro A. Simultaneous detection and tracking with multiple cameras. In: Cipolla R., Battiato S., Farinella G. (Eds.). Machine Learning for Computer Vision. Studies in Computational Intelligence. Berlin, Heidelberg: Springer; 2013. V. 411. P. 197-214. https://doi.org/10.1007/978-3-642-28661-2_8Taj M., Cavallaro A. Simultaneous detection and tracking with multiple cameras. In: Cipolla R., Battiato S., Farinella G. (Eds.). Machine Learning for Computer Vision. Studies in Computational Intelligence. Berlin, Heidelberg: Springer; 2013. V. 411. P. 197-214. https://doi.org/10.1007/978-3-642-28661-2_8Liang Q., Wu W., Yang Y., Zhang R., Peng Y., Xu M. Multi-player tracking for multi-view sports videos with improved k-shortest path algorithm. Appl. Sci. 2020;10(3):864. https://doi.org/10.3390/app10030864Liang Q., Wu W., Yang Y., Zhang R., Peng Y., Xu M. Multi-player tracking for multi-view sports videos with improved k-shortest path algorithm. Appl. Sci. 2020;10(3):864. https://doi.org/10.3390/app10030864Yang Y., Xu M., Wu W., Zhang R., Peng Y. 3D multiview basketball players detection and localization based on probabilistic occupancy. In: 2018 Digital Image Computing: Techniques and Applications (DICTA). IEEE; 2018. P. 1-8. https://doi.org/10.1109/DICTA.2018.8615798Yang Y., Xu M., Wu W., Zhang R., Peng Y. 3D multiview basketball players detection and localization based on probabilistic occupancy. In: 2018 Digital Image Computing: Techniques and Applications (DICTA). IEEE; 2018. P. 1-8. https://doi.org/10.1109/DICTA.2018.8615798Delmerico J., Cieslewski T., Rebecq H., Faessler M., Scaramuzza D. Are we ready for autonomous drone racing? The UZH-FPV drone racing dataset. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE; 2019. P. 6713-6719. https://doi.org/10.1109/ICRA.2019.8793887Delmerico J., Cieslewski T., Rebecq H., Faessler M., Scaramuzza D. Are we ready for autonomous drone racing? The UZH-FPV drone racing dataset. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE; 2019. P. 6713-6719. https://doi.org/10.1109/ICRA.2019.8793887Chen J., Wu D., Song P., Deng F., He Y., Pang S. Multiview triangulation: Systematic comparison and an improved method. IEEE Access. 2020;8:21017-21027. https://doi.org/10.1109/ACCESS.2020.2969082Chen J., Wu D., Song P., Deng F., He Y., Pang S. Multiview triangulation: Systematic comparison and an improved method. IEEE Access. 2020;8:21017-21027. https://doi.org/10.1109/ACCESS.2020.2969082Romanov A.M., et al. Modular reconfigurable robot distributed computing system for tracking multiple objects. IEEE Systems J. 2021;15(1):802-813. https://doi.org/10.1109/JSYST.2020.2990921Romanov A.M., et al. Modular reconfigurable robot distributed computing system for tracking multiple objects. IEEE Systems J. 2021;15(1):802-813. https://doi.org/10.1109/JSYST.2020.2990921Delannay D., Danhier N., De Vleeschouwer C. Detection and recognition of sports(wo)men from multiple views. In: 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC). IEEE; 2009. Р. 1-7. https://doi.org/10.1109/ICDSC.2009.5289407Delannay D., Danhier N., De Vleeschouwer C. Detection and recognition of sports(wo)men from multiple views. In: 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC). IEEE; 2009. Р. 1-7. https://doi.org/10.1109/ICDSC.2009.5289407Byeon M., et al. Variational inference for 3-D localization and tracking of multiple targets using multiple cameras. IEEE Transactions on Neural Networks and Learning Systems. 2019;30(11):3260-3274. https://doi.org/10.1109/TNNLS.2018.2890526Byeon M., et al. Variational inference for 3-D localization and tracking of multiple targets using multiple cameras. IEEE Transactions on Neural Networks and Learning Systems. 2019;30(11):3260-3274. https://doi.org/10.1109/TNNLS.2018.2890526Zhang R., et al. Multi-camera multi-player tracking with deep player identification in sports video. Pattern Recognition. 2020;102:107260. https://doi.org/10.1016/j.patcog.2020.107260Zhang R., et al. Multi-camera multi-player tracking with deep player identification in sports video. Pattern Recognition. 2020;102:107260. https://doi.org/10.1016/j.patcog.2020.107260

The authors declare that there are no conflicts of interest present.