mireabulletinRussian Technological JournalRussian Technological Journal2782-32102500-316XRTU MIREA10.32362/2500-316X-2024-12-4-106-116WDYUFJmireabulletin-967Research ArticleМАТЕМАТИЧЕСКОЕ МОДЕЛИРОВАНИЕMATHEMATICAL MODELINGАнализ нейросетевых моделей для прогнозирования временных рядовNeural network analysis in time series forecastinghttps://orcid.org/0009-0000-2019-2642ПашшоевБ.PashshoevB.

Пашшоев Бахтиёржон, студент

119454, Москва, пр-т Вернадского, д. 78

Bakhtierzhon Pashshoev, Student

78, Vernadskogo pr., Moscow, 119454

bahtiyorposhshoev@gmail.comhttps://orcid.org/0000-0001-5325-6198ПетрусевичД. А.PetrusevichD. A.

Петрусевич Денис Андреевич, к.ф.-м.н., доцент, кафедра высшей математики, Институт искусственного интеллекта

119454, Москва, пр-т Вернадского, д. 78

Scopus Author ID 55900513600, ResearcherID AAA-6661-2020

Denis A. Petrusevich, Cand. Sci. (Phys.-Math.), Associate Professor, Higher Mathematics Department, Institute of Artificial Intelligence

78, Vernadskogo pr., Moscow, 119454

Scopus Author ID 55900513600, ResearcherID AAA-6661-2020

petrusevich@mirea.ruФГБОУ ВО «МИРЭА – Российский технологический университет»РоссияMIREA – Russian Technological UniversityRussian Federation202405082024124106–116Copyright © Пашшоев Б., Петрусевич Д.А., 20242024Пашшоев Б., Петрусевич Д.А.Pashshoev B., Petrusevich D.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.rtj-mirea.ru/jour/article/view/967

Цели. Основная цель работы – построить нейросетевые модели временных рядов (LSTM, GRU, RNN) и сравнить результаты прогнозирования с их помощью между собой и с результатами стандартных моделей (ARIMA, ETS), чтобы выяснить, в каких случаях следует пользоваться определенной группой моделей.Методы. Проведен обзор нейросетевых моделей, рассмотрена структура моделей RNN, LSTM, GRU. Они используются для моделирования временных рядов российской макроэкономической статистики. Качество подстройки моделей под данные и качество прогнозов сравниваются в эксперименте. Нейросетевые и стандартные модели могут применяться как для всего ряда целиком, так и для его частей (тренд и сезонность). При построении прогноза на несколько временных промежутков вперед рассматриваются два подхода: построение прогноза сразу на весь промежуток и пошаговый прогноз. Так появляется несколько комбинаций моделей, которые могут использоваться для прогнозирования. Эти подходы проанализированы в вычислительном эксперименте.Результаты. Проведено несколько экспериментов, в которых построены и сравниваются по близости прогноза к данным ряда в тестовом периоде стандартные (ARIMA, ETS, LOESS) и нейросетевые модели (LSTM, GRU, RNN).Выводы. Для сезонных временных рядов модели на основе нейронных сетей превзошли по точности прогноза на тестовый период времени стандартные модели ARIMA, ETS. Одношаговый прогноз вычислительно менее эффективен, чем интегральный прогноз на весь целевой период, но точно указать, для каких рядов какой именно подход оказывается лучшим по качеству, не удается. Комбинированные модели (нейронные сети для тренда, ARIMA – для сезонности) почти всегда дают хороший результат. При прогнозировании несезонного гетероскедастичного ряда курса акций лучшие результаты показали стандартные подходы (метод LOESS и модель ETS). 

Objectives. To build neural network models of time series (LSTM, GRU, RNN) and compare the results of forecasting with their mutual help and the results of standard models (ARIMA, ETS), in order to ascertain in which cases a certain group of models should be used.Methods. The paper provides a review of neural network models and considers the structure of RNN, LSTM, and GRU models. They are used for modeling time series in Russian macroeconomic statistics. The quality of model adjustment to the data and the quality of forecasts are compared experimentally. Neural network and standard models can be used both for the entire series and for its parts (trend and seasonality). When building a forecast for several time intervals in the future, two approaches are considered: building a forecast for the entire interval at once, and step-by-step forecasting. In this way there are several combinations of models that can be used for forecasting. These approaches are analyzed in the computational experiment.Results. Several experiments have been conducted in which standard (ARIMA, ETS, LOESS) and neural network models (LSTM, GRU, RNN) are built and compared in terms of proximity of the forecast to the series data in the test period.Conclusions. In the case of seasonal time series, models based on neural networks surpassed the standard ARIMA and ETS models in terms of forecast accuracy for the test period. The single-step forecast is computationally less efficient than the integral forecast for the entire target period. However, it is not possible to accurately indicate which approach is the best in terms of quality for a given series. Combined models (neural networks for trend, ARIMA for seasonality) almost always give good results. When forecasting a non-seasonal heteroskedastic series of share price, the standard approaches (LOESS method and ETS model) showed the best results.

динамические рядымакроэкономическая статистикаGRULSTMRNNDNNвременные рядыdynamic seriesmacroeconomic statisticsGRULSTMRNNDNNtime seriesReferencesHyndman R.J., Athanasopoulos G. Forecasting: principles and practice. 3rd ed. OTexts; 2021. 442 p. ISBN-13 978-0987507136Hyndman R.J., Athanasopoulos G. Forecasting: principles and practice. 3rd ed. OTexts; 2021. 442 p. ISBN-13 978-0987507136Stock J.H., Watson M.W. Introduction to Econometrics. 3rd ed. Pearson; 2019. ISBN-13 978-9352863501Stock J.H., Watson M.W. Introduction to Econometrics. 3rd ed. Pearson; 2019. ISBN-13 978-9352863501Калугин Т.Р., Ким А.К., Петрусевич Д.А. Анализ моделей ADL(p, q), используемых для описания связей между временными рядами. Russ. Technol. J. 2020;8(2):7–22. https://doi.org/10.32362/2500-316X-2020-8-2-7-22Kalugin T.R., Kim A.K., Petrusevich D.A. Analysis of the high order ADL(p, q) models used to describe connections between time series. Russ. Technol. J. 2020;8(2):7–22 (in Russ.). https://doi.org/10.32362/2500-316X-2020-8-2-7-22Petrusevich D. Improvement of time series forecasting quality by means of multiple models prediction averaging. In: Proceedings of the Third International Workshop on Modeling, Information Processing and Computing (MIP: Computing-2021). 2021;2899:109–117. https://doi.org/10.47813/dnit-mip3/2021-2899-109-117Petrusevich D. Improvement of time series forecasting quality by means of multiple models prediction averaging. In: Proceedings of the Third International Workshop on Modeling, Information Processing and Computing (MIP: Computing-2021). 2021;2899:109–117. https://doi.org/10.47813/dnit-mip3/2021-2899-109-117Beletskaya N., Petrusevich D. Linear combinations of time series models with minimal forecast variance. J. Commun. Technol. Electron. 2023;67(1):144–158. https://doi.org/10.1134/S1064226922130022Beletskaya N., Petrusevich D. Linear combinations of time series models with minimal forecast variance. J. Commun. Technol. Electron. 2023;67(1):144–158. https://doi.org/10.1134/S1064226922130022Box G., Jenkins G. Time Series Analysis: Forecast and Management. John Wiley & Sons; 2015. 712 p. ISBN 978-11185674918Box G., Jenkins G. Time Series Analysis: Forecast and Management. John Wiley & Sons; 2015. 712 p. ISBN 978-11185674918Haykin S. Neural Networks and Learning Machines. Pearson Education; 2011. 936 p. ISBN 978-0133002553Haykin S. Neural Networks and Learning Machines. Pearson Education; 2011. 936 p. ISBN 978-0133002553Shi J., Jain M., Narasimhan G. Time Series Forecasting (TSF) Using Various Deep Learning Models. 2022. URL: https://arxiv.org/abs/2204.11115v1, https://doi.org/10.48550/arXiv.2204.11115Shi J., Jain M., Narasimhan G. Time Series Forecasting (TSF) Using Various Deep Learning Models. 2022. URL: https://arxiv.org/abs/2204.11115v1, https://doi.org/10.48550/arXiv.2204.11115Amalou I., Mouhni N., Abdali A. Multivariate time series prediction by RNN architectures for energy consumption forecasting. Energy Rep. 2022;8:1084–1091. https://doi.org/10.1016/j.egyr.2022.07.139Amalou I., Mouhni N., Abdali A. Multivariate time series prediction by RNN architectures for energy consumption forecasting. Energy Rep. 2022;8:1084–1091. https://doi.org/10.1016/j.egyr.2022.07.139Aseeri A. Effective RNN-Based forecasting methodology design for improving short-term power load forecasts: application to large-scale power-grid time series. J. Computational Sci. 2023;68(4):101984. https://doi.org/10.1016/j.jocs.2023.101984Aseeri A. Effective RNN-Based forecasting methodology design for improving short-term power load forecasts: application to large-scale power-grid time series. J. Computational Sci. 2023;68(4):101984. https://doi.org/10.1016/j.jocs.2023.101984Ning Y., Kazemi H., Tahmasebi P. A comparative machine learning study for time series oil production forecasting: ARIMA, LSTM, and Prophet. Comput. Geosci. 2022;164(1):105126. https://doi.org/10.1016/j.cageo.2022.105126Ning Y., Kazemi H., Tahmasebi P. A comparative machine learning study for time series oil production forecasting: ARIMA, LSTM, and Prophet. Comput. Geosci. 2022;164(1):105126. https://doi.org/10.1016/j.cageo.2022.105126Wang P., Zheng X., Ai G., Liu D., Zhu B. Time series prediction for the epidemic trends of COVID-19 using the improved LSTM deep learning method: Case studies in Russia, Peru and Iran. Chaos, Solitons & Fractals. 2020;140:110214. https://doi.org/10.1016/j.chaos.2020.110214Wang P., Zheng X., Ai G., Liu D., Zhu B. Time series prediction for the epidemic trends of COVID-19 using the improved LSTM deep learning method: Case studies in Russia, Peru and Iran. Chaos, Solitons & Fractals. 2020;140:110214. https://doi.org/10.1016/j.chaos.2020.110214Arunkumar K.E., Kalaga D.V., Kumar M.S., Kawaji M., Brenza T.M. Comparative analysis of Gated Recurrent Units (GRU), long Short-Term memory (LSTM) cells, autoregressive Integrated moving average (ARIMA), seasonal autoregressive Integrated moving average (SARIMA) for forecasting COVID-19 trends. Alexandria Eng. J. 2022;61(10):7585–7603. https://doi.org/10.1016/j.aej.2022.01.011Arunkumar K.E., Kalaga D.V., Kumar M.S., Kawaji M., Brenza T.M. Comparative analysis of Gated Recurrent Units (GRU), long Short-Term memory (LSTM) cells, autoregressive Integrated moving average (ARIMA), seasonal autoregressive Integrated moving average (SARIMA) for forecasting COVID-19 trends. Alexandria Eng. J. 2022;61(10):7585–7603. https://doi.org/10.1016/j.aej.2022.01.011Kumar B., Sunil, Yadav N. A novel hybrid model combining βSARMA and LSTM for time series forecasting. Appl. Soft Comput. 2023;134:110019. https://doi.org/10.1016/j.asoc.2023.110019Kumar B., Sunil, Yadav N. A novel hybrid model combining βSARMA and LSTM for time series forecasting. Appl. Soft Comput. 2023;134:110019. https://doi.org/10.1016/j.asoc.2023.110019Abebe M., Noh Y., Kang Y.-J., Seo C., Kim D., Seo J. Ship trajectory planning for collision avoidance using hybrid ARIMA-LSTM models. Ocean Eng. 2022;256:111527. https://doi.org/10.1016/j.oceaneng.2022.111527Abebe M., Noh Y., Kang Y.-J., Seo C., Kim D., Seo J. Ship trajectory planning for collision avoidance using hybrid ARIMA-LSTM models. Ocean Eng. 2022;256:111527. https://doi.org/10.1016/j.oceaneng.2022.111527Cascone L., Sadiq S., Ullah S., Mirjalili S., Ur H., Siddiqui R., Umer M. Predicting household electric power consumption using multi-step time series with convolutional LSTM. Big Data Res. 2023;31:100360. https://doi.org/10.1016/j.bdr.2022.100360Cascone L., Sadiq S., Ullah S., Mirjalili S., Ur H., Siddiqui R., Umer M. Predicting household electric power consumption using multi-step time series with convolutional LSTM. Big Data Res. 2023;31:100360. https://doi.org/10.1016/j.bdr.2022.100360Wang H., Zhang Y., Liang J., Liu L. DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for time series prediction. Neural Netw. 2023;157:240–256. https://doi.org/10.1016/j.neunet.2022.10.009Wang H., Zhang Y., Liang J., Liu L. DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for time series prediction. Neural Netw. 2023;157:240–256. https://doi.org/10.1016/j.neunet.2022.10.009Zhao L., Mo C., Ma J., Chen Z., Yao C. LSTM-MFCN: A time series classifier based on multi-scale spatial–temporal features. Computer Commun. 2022;182(3):52–59. https://doi.org/10.1016/j.comcom.2021.10.036Zhao L., Mo C., Ma J., Chen Z., Yao C. LSTM-MFCN: A time series classifier based on multi-scale spatial–temporal features. Computer Commun. 2022;182(3):52–59. https://doi.org/10.1016/j.comcom.2021.10.036Rasjid Z.E., Setiawan R., Effendi A. A Comparison: Prediction of Death and Infected COVID-19 Cases in Indonesia Using Time Series Smoothing and LSTM Neural Network. Procedia Comput. Sci. 2021;179(5):982–988. http://doi.org/10.1016/j.procs.2021.01.102Rasjid Z.E., Setiawan R., Effendi A. A Comparison: Prediction of Death and Infected COVID-19 Cases in Indonesia Using Time Series Smoothing and LSTM Neural Network. Procedia Comput. Sci. 2021;179(5):982–988. http://doi.org/10.1016/j.procs.2021.01.102Dubey A.K., Kumar A., García-Díaz V., Sharma A.K., Kanhaiya K. Study and analysis of SARIMA and LSTM in forecasting time series data. Sustain. Energy Technol. Assess. 2021;47:101474. https://doi.org/10.1016/j.seta.2021.101474Dubey A.K., Kumar A., García-Díaz V., Sharma A.K., Kanhaiya K. Study and analysis of SARIMA and LSTM in forecasting time series data. Sustain. Energy Technol. Assess. 2021;47:101474. https://doi.org/10.1016/j.seta.2021.101474Wu Z., Yin H., He H., Li Y. Dynamic-LSTM hybrid models to improve seasonal drought predictions over China. J. Hydrol. 2022;615:128706. https://doi.org/10.1016/j.jhydrol.2022.128706Wu Z., Yin H., He H., Li Y. Dynamic-LSTM hybrid models to improve seasonal drought predictions over China. J. Hydrol. 2022;615:128706. https://doi.org/10.1016/j.jhydrol.2022.128706Yan Y., Wang X., Ren F., Shao Z., Tian C. Wind speed prediction using a hybrid model of EEMD and LSTM considering seasonal features. Energy Rep. 2022;8:8965–8980. https://doi.org/10.1016/j.egyr.2022.07.007Yan Y., Wang X., Ren F., Shao Z., Tian C. Wind speed prediction using a hybrid model of EEMD and LSTM considering seasonal features. Energy Rep. 2022;8:8965–8980. https://doi.org/10.1016/j.egyr.2022.07.007Bian S., Wang Z., Song W., Zhou X. Feature extraction and classification of time-varying power load characteristics based on PCANet and CNN+Bi-LSTM algorithms. Electric Power Systems Research. 2023;217(6):109149. https://doi.org/10.1016/j.epsr.2023.109149Bian S., Wang Z., Song W., Zhou X. Feature extraction and classification of time-varying power load characteristics based on PCANet and CNN+Bi-LSTM algorithms. Electric Power Systems Research. 2023;217(6):109149. https://doi.org/10.1016/j.epsr.2023.109149Sangiorgio M., Dercole F. Robustness of LSTM neural networks for multi-step forecasting of chaotic time series. Chaos, Solitons & Fractals. 2020;139(8):10045. https://doi.org/10.1016/j.chaos.2020.110045Sangiorgio M., Dercole F. Robustness of LSTM neural networks for multi-step forecasting of chaotic time series. Chaos, Solitons & Fractals. 2020;139(8):10045. https://doi.org/10.1016/j.chaos.2020.110045Liu X., Lin Z., Feng Z. Short-term offshore wind speed forecast by seasonal ARIMA – A comparison against GRU and LSTM. Energy. 2021;227:120492. http://doi.org/10.1016/j.energy.2021.120492Liu X., Lin Z., Feng Z. Short-term offshore wind speed forecast by seasonal ARIMA – A comparison against GRU and LSTM. Energy. 2021;227:120492. http://doi.org/10.1016/j.energy.2021.120492Shahid F., Zameer A., Muneeb M. Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM. Chaos, Solitons & Fractals. 2020;140:110212. https://doi.org/10.1016/j.chaos.2020.110212Shahid F., Zameer A., Muneeb M. Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM. Chaos, Solitons & Fractals. 2020;140:110212. https://doi.org/10.1016/j.chaos.2020.110212Wang J., Wang P., Tian H., Tansey K., Liu J., Quan W. A deep learning framework combining CNN and GRU for improving wheat yield estimates using time series remotely sensed multi-variables. Comput. Electron. Agric. 2023;206(4):107705. https://doi.org/10.1016/j.compag.2023.107705Wang J., Wang P., Tian H., Tansey K., Liu J., Quan W. A deep learning framework combining CNN and GRU for improving wheat yield estimates using time series remotely sensed multi-variables. Comput. Electron. Agric. 2023;206(4):107705. https://doi.org/10.1016/j.compag.2023.107705Hua H., Liu M., Li Y., Deng S., Wang Q. An ensemble framework for short-term load forecasting based on parallel CNN and GRU with improved ResNet. Electric Power Syst. Res. 2023;216(3):109057. https://doi.org/10.1016/j.epsr.2022.109057Hua H., Liu M., Li Y., Deng S., Wang Q. An ensemble framework for short-term load forecasting based on parallel CNN and GRU with improved ResNet. Electric Power Syst. Res. 2023;216(3):109057. https://doi.org/10.1016/j.epsr.2022.109057Zhang D., Sun W., Dai Y., Liu K., Li W., Wang C. A hierarchical early kick detection method using a cascaded GRU network. Geoenergy Sci. Eng. 2023;222(3):211390. https://doi.org/10.1016/j.geoen.2022.211390Zhang D., Sun W., Dai Y., Liu K., Li W., Wang C. A hierarchical early kick detection method using a cascaded GRU network. Geoenergy Sci. Eng. 2023;222(3):211390. https://doi.org/10.1016/j.geoen.2022.211390Грамович Я.В., Мусатов Д.Ю., Петрусевич Д.А. Применение беггинга в прогнозировании временных рядов. Russ. Technol. J. 2024;12(1):101–110. https://doi.org/10.32362/2500-316X-2024-12-1-101-110Gramovich I.V., Musatov D.Yu., Petrusevich D.A. Implementation of bagging in time series forecasting. Russ. Technol. J. 2024;12(1):101–110 (in Russ.). https://doi.org/10.32362/2500-316X-2024-12-1-101-110]McLeod A., Li W. Diagnostic checking ARMA time series models using squared residual autocorrelations. J. Time Ser. Anal. 1983;4(4):269–273. https://doi.org/10.1111/j.1467-9892.1983.tb00373.xMcLeod A., Li W. Diagnostic checking ARMA time series models using squared residual autocorrelations. J. Time Ser. Anal. 1983;4(4):269–273. https://doi.org/10.1111/j.1467-9892.1983.tb00373.xMcLeod A., Li W. Diagnostic checking ARMA time series models using squared residual autocorrelations. J. Time Ser. Anal. 1983;4(4):269–273. https://doi.org/10.1111/j.1467-9892.1983.tb00373.x

The authors declare that there are no conflicts of interest present.