Modernization of a statistical data processing algorithm for integrated background monitoring

Objectives. Since the 1980s, the Integrated Background Monitoring Network (IBMoN) operated by RosHydroMet[2] in the Russian Federation has been collecting experimental data on background levels of pollutants in environmental media, including atmospheric air, precipitation, surface water, soil, and vegetation. As part of its operations, IBMoN regularly updates recommendations and regulatory documents regarding chemical analysis methods and requirements for monitoring site placement every 5–10 years. However, the statistical framework of the network has not been revised since 1995 (Algorithm 1995). This paper sets out to modernize selected steps of the statistical algorithm for processing data on air quality obtained from IBMoN.

Methods. In the study, correlation analysis, trend analysis, descriptive mathematical statistics, and structural algorithmization methods were used.

Results. Based on a review of Algorithm 1995 steps requiring modernization, a structure for the updated data processing algorithm version, Algorithm 2025, was developed. This new algorithm will enable the joint analysis of RosHydroMet data to mitigate the impact of noise and fluctuations in the time series of background concentrations of air pollutants. The effectiveness of the proposed methods for modernizing selected steps of Algorithm 1995 was substantiated.

Conclusions. Since it is not possible to obtain data in the format required by RosHydroMet without modifying some steps of the Algorithm 1995, this algorithm requires modernization. As a result of modernizing the selected steps, noise can be reduced by up to 70% to eliminate the main types of random and systematic errors caused by the human factor. However, future work is required to identify suitable approaches for modernizing the remaining steps that are not covered in this study.

1. Gromov S.А., Paramonov S.G. Current status and prospects for the development of integrated background monitoring of environmental pollution. Problemy ehkologicheskogo monitoringa i modelirovaniya ehkosistem = Problems of Ecological Monitoring and Ecosystem Modeling. 2015;26(1):205–221 (in Russ.). https://elibrary.ru/uybmod

2. Burtseva L.V., Golubeva N.I., Konkova E.S. Development of quality assurance methods of heavy metal measurements in the atmospheric air and precipitations in the integrated background monitoring system of roshydromet. Sistemy kontrolya okruzhayushchei sredy = Monitoring Systems of Environment (MSoE). 2019;4(38):20–26 (in Russ.). https://doi.org/10.33075/2220-5861-2019-4-20-26

3. Chernogaeva G.M. (Ed). Obzor sostoyaniya i zagryazneniya okruzhayushchei sredy v Rossiiskoi Federatsii za 2021 god (Review of the State and Pollution of the Environment in the Russian Federation for 2021) . Moscow: RosHydroMet; 2022, 219 p. (in Russ.). https://www.meteorf.gov.ru/upload/iblock/dc8/Obzor_2021.pdf

4. Chernogaeva G.M. (Ed). Tendentsii i dinamika sostoyaniya i zagryazneniya okruzhayushchei sredy v Rossiiskoi Federatsii po dannym mnogoletnego monitoringa za poslednie 10 let: Analiticheskii obzor (Trends and Dynamics of the State and Pollution of the Environment in the Russian Federation According to Long-Term Monitoring Data over the Past 10 Years: Analytical Review). Moscow: RosHydroMet; 2017, 47 p. (in Russ.). ISBN 978-5-9500881-2-4

5. Burtseva L.V., Konkova E.S. Assessment of atmospheric pollution with copper in background areas of the Europe. Problemy ehkologicheskogo monitoringa i modelirovaniya ehkosistem = Problems of Ecological Monitoring and Ecosystem Modeling. 2020;31(1-2):14–33 (in Russ.). https://doi.org/10.21513/0207-2564-2020-1-14-33

6. Salmi T., Maata A., Antilla P., Ruoho-Airola T., Amnell T. Detecting Trends of Annual Values of Atmospheric Pollutants by the Mann–Kendall Test and Sen’s Slope estimates—the Excel Template Application Makesens. Report code: FMI-AQ-31. Helsinki: Finnish Meteorological Institute; 2002, 36 p.

7. Perikhanyan Ye., Shahnazaryan G., Gabrielyan A. Long term trends of wet deposition and atmospheric concentrations of nitrogen and sulfur compounds at EMEP site in Armenia. J. Atmos. Chem. 2020;77(3):101–116. https://doi.org/10.1007/s10874-020-09408-3

8. Pavlova K.V. Application of Spearman’s rank correlation coefficient in statistical research. Bezopasnost’. Upravlenie. Iskusstvennyi intellekt. 2023;3(3):28–32 (in Russ.). https://elibrary.ru/ubhmwg

9. Golobokova L.P., Berdashkinova O.I., Loskutova M.A., et al. Long-term Studies of Atmospheric Aerosol Chemical Composition at the Cape Baranov Ice Base Station. Atmos. Ocean Opt. 202437(1):14–23. https://doi.org/10.1134/S1024856023700033 [Original Russian Text: Golobokova L.P., Berdashkinova O.I., Loskutova M.A., Rize D.D., Onishchuk N.A., Sakerin S.M., Turchinovich Yu.S. Long-term Studies of Atmospheric Aerosol Chemical Composition at the Cape Baranov Ice Base Station. Optika atmosfery i okeana. 2023;36(11):874–882 (in Russ.). https://doi.org/10.15372/AOO2023110210 ]

10. Kukkonen J., Konttinen M., Bremer P., Salmi T., Saari H. The seasonal variation of urban air quality in northern European conditions. Int. J. Environ. Pollut. 2000;14(1-6):480–487. https://doi.org/10.1504/IJEP.2000.000571

11. Zareba M., Weglinska E., Danek T. Air pollution seasons in urban moderate climate areas through big data analytics. Sci. Rep. 2024;14(1):3058. https://doi.org/10.1038/s41598-024-52733-w

12. Sköld M. Trend detection with non-detects in long-term monitoring, a mixed model approach. Environ. Monit. Assess. 2023;195(6):663. https://doi.org/10.1007/s10661-023-11285-8

13. Ehksperiandova L.P., Belikov K.N., Khimchenko S.V., et al. Once again about determination and detection limits. J. Anal. Chem. 2010;65(3):223–228. https://doi.org/10.1134/S1061934810030020 [Original Russian Text: Ehksperiandova L.P., Belikov K.N., Khimchenko S.V., Blank T.A. Once again about determination and detection limits. Zhurnal analiticheskoi khimii. 2010;65(3):229–234 (in Russ.). https://elibrary.ru/kkilqd ]

14. Gilbert R.O. Statistical Methods for Environmental Pollution Monitoring. New York: Van Nostrand Reinhold Company Inc.; 1987, 334 p.

15. Rozenberg G.S., Shitikov V.K., Brusilovskii G.N. Ehkologicheskoe prognozirovanie (Funktsional’nye prediktory vremennykh ryadov) (Environmental Forecasting (Functional Predictors of Time Series). Tol’yatti; 1994, 182 р. (In Russ.). https://elibrary.ru/zumjsp

16. Zheksenbaeva A.K. Statistical characteristics of rainfall in northern Kazakhstan during the vegetative period. Nauka, novye tekhnologii i innovatsii Kyrgyzstana = Science, New Technologies and Innovations of Kyrgyzstan. 2016;5:110–117 (in Russ.). https://elibrary.ru/wdfhgb

17. Bruce P., Bruce A. Prakticheskaya statistika dlya spetsialistov Data Science (Practical Statistics for Data Scientists) : transl. from Engl. BKHV-Petersburg; 2018, 302 p. (In Russ.). ISBN 978-5-9775-3974-6 [Bruce P., Bruce A. Practical Statistics for Data Scientists. O’Reilly; 2017, 368 p.]

18. Bokov V.N., Vorobyev V.N., Serebritskyi I.A. Evaluation of the background pollution of atmosphere in St. Petersburg and suburbs. Uchenye zapiski Rossiiskogo gosudarstvennogo gidrometeorologicheskogo universiteta = Proceedings of the Russian State Hydrometeorological University. 2011;20:36–41 (in Russ.). https://elibrary.ru/onoexv

19. Lenskaya O.Yu., Kulyabova A.B. On the methods of analysis of irregular observations of the composition of atmospheric air pollution. Vestnik Chelyabinskogo gosudarstvennogo univrsiteta = Bulletin of Chelyabinsk State University. 2007;6:80–84 (in Russ.). https://elibrary.ru/mvmwmp

20. Van Zoest V.M., Stein A., Hoek G. Outlier Detection in Urban Air Quality Sensor Networks. Water, Air, and Soil Pollution. 2018;229(4):111. https://doi.org/10.1007/s11270-018-3756-7