3D object tracker for sports events

Objectives. Sports events are currently among the most promising areas for the application of tracking systems. In most cases, such systems are designed to track moving objects in a two-dimensional plane, e.g., players on the field, as well as to identify them by various features. However, as new sports such as drone racing are developed, the problem of determining the position of an object in a three-dimensional coordinate system becomes relevant. The aim of the present work was to develop algorithms and software for a method to perform 3D tracking of moving objects, regardless of the data segmentation technique, and to test this method to estimate the tracking quality.

Methods. A method for matching information on the speed and position of objects was selected based on a review and analysis of contemporary tracking methods.

Results. The structure of a set of algorithms comprising software for a moving-object tracker for sports events is proposed. Experimental studies were performed on the publicly available APIDIS dataset, where a MOTA metric of 0.858 was obtained. The flight of an FPV quadcopter along a track was also tracked according to the proposed dataset; the 3D path of the drone flight was reconstructed using the tracker data.

Conclusions. The results of the experimental studies, which demonstrated the feasibility of using the proposed method to track a quadcopter flight trajectory in a three-dimensional world coordinate system, is also showed that the method is suitable for tracking objects at sports events.

1. Zein Y., Darwiche M., Mokhiamar O. GPS tracking system for autonomous vehicles. Alexandria Eng. J. 2018;57(4): 3127-3137. https://doi.org/10.1016/j.aej.2017.12.002

2. Yu K., et al. Deep learning-based traffic safety solution for a mixture of autonomous and manual vehicles in a 5G-enabled intelligent transportation system. IEEE Transactions on Intelligent Transportation Systems. 2020;22(7): 4337-4347. https://doi.org/10.1109/TITS.2020.3042504

3. Ryan B.J., et al. COVID-19 contact tracing solutions for mass gatherings. Disaster Medicine and Public Health Preparedness. 2021;15(3):e1-e7. https://doi.org/10.1017/dmp.2020.241

4. Khan S., et al. Implementing traceability systems in specific supply chain management (SCM) through critical success factors (CSFs). Sustainability. 2018;10(1):204. https://doi.org/10.3390/su10010204

5. Cioppa A., Deliege A., Magera F., Giancola S., Barnich O., Ghanem B., Van Droogenbroeck M. Camera calibration and player localization in soccerNet-v2 and investigation of their representations for action spotting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2021. P. 4537-4546. https://doi.org/10.1109/CVPRW53098.2021.00511

6. Kong L., Zhu M., Ran N., Liu Q., He R. Online multiple athlete tracking with pose-based long-term temporal dependencies. Sensors. 2020;21(1):197. https://doi.org/10.3390/s21010197

7. Liu J., Tong X., Li W., Wang T., Zhang Y., Wang H. Automatic player detection, labeling and tracking in broadcast soccer video. Pattern Recognition Lett. 2009;30(2):103-113. https://doi.org/10.1016/j.patrec.2008.02.011

8. Possegger H., Sternig S., Mauthner T., Roth P.M., Bischof H. Robust real-time tracking of multiple objects by volumetric mass densities. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013. P. 2395-2402. https://doi.org/10.1109/CVPR.2013.310

9. Bialkowski A., Lucey P., Carr P., Denman S., Matthews I., Sridharan S. Recognising team activities from noisy data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2013. P. 984-990. https://doi.org/10.1109/CVPRW.2013.143

10. Foehn P., Brescianini D., Kaufmann E., et al. AlphaPilot: Autonomous drone racing. Auton. Robot. 2022;46(1): 307-320. https://doi.org/10.1007/s10514-021-10011-y

11. Spica R., Cristofalo E., Wang Z., Montijano E., Schwager M. A real-time game theoretic planner for autonomous two-player drone racing. IEEE Transactions on Robotics. 2020;36(5):1389-1403. https://doi.org/10.1109/TRO.2020.2994881

12. Kaufmann E., et al. Beauty and the beast: Optimal methods meet learning for drone racing. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE; 2019. P. 690-696. https://doi.org/10.1109/ICRA.2019.8793631

13. VolkovaM.A., RomanovA.M., Romanov M.P. Distributed system for objects localization in the working area of a modular reconfigurable mobile robot. Mekhatronika, Avtomatizatsiya, Upravlenie. 2021;22(12):634-643 (in Russ.). https://doi.org/10.17587/mau.22.634-643

14. Dai-Hong J., Lei D., Dan L., San-You Z. Moving-object tracking algorithm based on PCA-SIFT and optimization for underground coal mines. IEEE Access. 2019;7: 35556-35563. https://doi.org/10.1109/ACCESS.2019.2899362

15. Chigrinskii V.V., Matveev I.A. Optimization of a tracking system based on a network of cameras. J. Comput. Syst. Sci. Int. 2020;59(4):583-597. https://doi.org/10.1134/S1064230720040127

16. Lan X., Ye M., Shao R., Zhong B., Yuen P.C., Zhou H. Learning modality-consistency feature templates: A robust RGB-infrared tracking system. IEEE Transactions on Industrial Electronics.2019;66(12):9887-9897. https://doi.org/10.1109/TIE.2019.2898618

17. Bao Q., Liu W., Cheng Y., Zhou B., Mei T. Pose-guided tracking-by-detection: Robust multi-person pose tracking. IEEE Transactions on Multimedia. 2020;23:161-175. https://doi.org/10.1109/TMM.2020.2980194

18. Wang Z., Li M., Lu Y., Bao Y., Li Z., Zhao J. Effective multiple pedestrian tracking system in video surveillance with monocular stationary camera. Expert Systems with Applications. 2021;178:114992. https://doi.org/10.1016/j.eswa.2021.114992

19. Linke D., Link D., Lames M. Football-specific validity of TRACAB's optical video tracking systems. PLoSONE. 2020;15(3):e0230179. https://doi.org/10.1371/journal.pone.0230179

20. Pons E., Garda-Calvo T., Resta R., Blanco H., Lopez del Campo R., D^az Garda J., Pulido J.J. A comparison of a GPS device and a multi-camera video technology during official soccer matches: Agreement between systems. PLoSONE. 2019;14(8):e0220729. https://doi.org/10.1371/journal.pone.0220729

21. Nishikawa Y., Sato H., Ozawa J. Multiple sports player tracking system based on graph optimization using lowcost cameras. In: 2018 IEEE International Conference on Consumer Electronics (ICCE). IEEE; 2018. P. 1-4. https://doi.org/10.1109/ICCE.2018.8326126

22. Hui Q. Motion video tracking technology in sports training based on Mean-Shift algorithm. J. Supercomput. 2019;75(9):6021-6037. https://doi.org/10.1007/s11227-019-02898-3

23. Taj M., Cavallaro A. Distributed and decentralized multicamera tracking. IEEE Signal Processing Magazine. 2011;28(3):46-58. https://doi.org/10.1109/MSP.2011.940281

24. Taj M., Cavallaro A. Simultaneous detection and tracking with multiple cameras. In: Cipolla R., Battiato S., Farinella G. (Eds.). Machine Learning for Computer Vision. Studies in Computational Intelligence. Berlin, Heidelberg: Springer; 2013. V. 411. P. 197-214. https://doi.org/10.1007/978-3-642-28661-2_8

25. Liang Q., Wu W., Yang Y., Zhang R., Peng Y., Xu M. Multi-player tracking for multi-view sports videos with improved k-shortest path algorithm. Appl. Sci. 2020;10(3):864. https://doi.org/10.3390/app10030864

26. Yang Y., Xu M., Wu W., Zhang R., Peng Y. 3D multiview basketball players detection and localization based on probabilistic occupancy. In: 2018 Digital Image Computing: Techniques and Applications (DICTA). IEEE; 2018. P. 1-8. https://doi.org/10.1109/DICTA.2018.8615798

27. Delmerico J., Cieslewski T., Rebecq H., Faessler M., Scaramuzza D. Are we ready for autonomous drone racing? The UZH-FPV drone racing dataset. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE; 2019. P. 6713-6719. https://doi.org/10.1109/ICRA.2019.8793887

28. Chen J., Wu D., Song P., Deng F., He Y., Pang S. Multiview triangulation: Systematic comparison and an improved method. IEEE Access. 2020;8:21017-21027. https://doi.org/10.1109/ACCESS.2020.2969082

29. Romanov A.M., et al. Modular reconfigurable robot distributed computing system for tracking multiple objects. IEEE Systems J. 2021;15(1):802-813. https://doi.org/10.1109/JSYST.2020.2990921

30. Delannay D., Danhier N., De Vleeschouwer C. Detection and recognition of sports(wo)men from multiple views. In: 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC). IEEE; 2009. Р. 1-7. https://doi.org/10.1109/ICDSC.2009.5289407

31. Byeon M., et al. Variational inference for 3-D localization and tracking of multiple targets using multiple cameras. IEEE Transactions on Neural Networks and Learning Systems. 2019;30(11):3260-3274. https://doi.org/10.1109/TNNLS.2018.2890526

32. Zhang R., et al. Multi-camera multi-player tracking with deep player identification in sports video. Pattern Recognition. 2020;102:107260. https://doi.org/10.1016/j.patcog.2020.107260