Algorithms for the visual analysis of an environment by an autonomous mobile robot for area cleanup

Objectives. At present, increasing rates of pollution of vast areas by various types of household waste are becoming an increasingly serious problem. In this connection, the creation of a robotic complex capable of performing autonomous litter collection functions becomes an urgent need. One of the key components of such a complex comprises a vision system for detecting and interacting with target objects. The purpose of this work is to develop the underlying algorithmics for the vision system of robots executing area cleaning functions.

Methods. Within the framework ofthe proposed structure ofthe system for visual analysis ofthe external environment, algorithms for detecting and classifying objects of various appearance have been developed using convolutional neural networks. The neural network detector was set up by gradient descent on the open dataset of TACO training samples. To determine the geometric parameters of a surface in the field of view of the robot and estimate the coordinates of objects on the ground, a homography matrix was formed to take into account information about the characteristics and location of the video camera.

Results. The developed software and algorithms for a mobile robot equipped with a monocular video camera are capable of implementing the functions of neural network detection and classification of litter objects in the frame, as well as projection of found objects on a terrain map for their subsequent collection.

Conclusions. Experimental studies have shown that the developed system of visual analysis of the external environment of an autonomous mobile robot has sufficient efficiency to solve the tasks of detecting litter in the field of view of an autonomous mobile robot.

1. Черняева Т.К. Актуальные проблемы влияния отходов производства и потребления на объекты окружающей среды и состояние здоровья населения (обзор). Гигиена и санитария. 2013;3:32–35. [Chernyaeva T.K. Actual problems of the impact of production and consumption waste on environment and public health (review of literature). Gigiena i sanitariya = Hygiene and Sanitation. 2013;3:32–35 (in Russ.).]

2. Bansal S., Patel S., Shah I., Patel A., Makwana J., Thakker R. AGDC: Automatic Garbage Detection and Collection. arXiv preprint arXiv:1908.05849. 2019. https://doi.org/10.48550/arXiv.1908.05849

3. Howard A.G., Zhu M., Chen B., Kalenichenko D., Wang W., Weyand T., Andreetto M., Adam H. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv preprint arXiv:1704.04861. 2017. https://doi.org/10.48550/arXiv.1704.04861

4. Карпов В.Э. ПИД-управление в нестрогом изложении. М.: НИИ информационных технологий; 2012. 34 с. URL: http://radioservice.at.ua/_fr/0/Karpov_mobline1. pdf [Karpov V.E. PID-upravlenie v nestrogom izlozhenii (PID-Control in a Loose Presentation). Moscow; 2012. 34 р. (in Russ.). URL: http://radioservice.at.ua/_fr/0/Karpov_mobline1.pdf]

5. Salmador A., Cid J.P., Novelle I.R. Intelligent Garbage Classifier. Int. J. Interact. Multimedia Artif. Intell. 2008;1(1):31–36.

6. Xu Z., Ji X., Wang M., Sun X. Edge detection algorithm of medical image based on Canny operator. J. Phys.: Conf. Ser. 2021;1955(1):012080. https://doi.org/10.1088/1742-6596/1955/1/012080

7. Ming-Kuei Hu. Visual pattern recognition by moment invariants. IRE Transactions on Inform. Theory. 1962;8(2): 179–187. https://doi.org/10.1109/TIT.1962.1057692

8. El-ghazal A., Basir O., Belkasim S. A novel curvature-based shape Fourier Descriptor. 2008 15th IEEE International Conference on Image Processing (ICIP). 2008. P. 953–956. https://doi.org/10.1109/ICIP.2008.4711914

9. Pascal V., Yoshua B. K-Local Hyperplane and Convex Distance Nearest Neighbor Algorithms. In: DietterichT.G., Becker S., Ghahramani Z. (Eds.). Advances in Neural Information Processing Systems. The MIT Press; 2001. V. 14. P. 985–992.

10. Deng J., Xuan X., Wang W., Li Z., Yao H., Wang Z. A review of research on object detection based on deep learning. J. Phys.: Conf. Ser. 2020;1684(1):012028. https://doi.org/10.1088/1742-6596/1684/1/012028

11. Бредихин А.И. Алгоритмы обучения сверточных нейронных сетей. Вестник Югорского государственного университета. 2019;1(52):41–54. https://doi.org/10.17816/byusu20190141-54 [Bredikhin A.I. Training algorithms for convolutional neural networks. Vestnik Yugorskogo gosudarstvennogo universiteta = Yugra State University Bulletin. 2019;1(52):41–54 (in Russ.). https://doi.org/10.17816/byusu20190141-54 ]

12. Proença P.F., Simões P. TACO: Trash Annotations in Context for Litter Detection. arXiv preprint arXiv: 2003.06975. 2020. https://doi.org/10.48550/arXiv.2003.06975

13. Bochkovskiy A., Wang C.Y., Liao H.Y.M. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv preprint arXiv: 2004.10934. 2020. https://doi.org/10.48550/arXiv.2004.10934

14. Liu L., Özsu M.T. (Eds.). Encyclopedia of Database Systems. New York, NY, USA: Springer; 2009. V. 6. https://doi.org/10.1007/978-0-387-39940-9

15. Yasuda T. (Ed.) Multi-Robot Systems, Trends and Development. London, United Kingdom: IntechOpen; 2011. 598 р. https://doi.org/10.5772/544