Neural network analysis in time series forecasting
https://doi.org/10.32362/2500-316X-2024-12-4-106-116
EDN: WDYUFJ
Abstract
Objectives. To build neural network models of time series (LSTM, GRU, RNN) and compare the results of forecasting with their mutual help and the results of standard models (ARIMA, ETS), in order to ascertain in which cases a certain group of models should be used.
Methods. The paper provides a review of neural network models and considers the structure of RNN, LSTM, and GRU models. They are used for modeling time series in Russian macroeconomic statistics. The quality of model adjustment to the data and the quality of forecasts are compared experimentally. Neural network and standard models can be used both for the entire series and for its parts (trend and seasonality). When building a forecast for several time intervals in the future, two approaches are considered: building a forecast for the entire interval at once, and step-by-step forecasting. In this way there are several combinations of models that can be used for forecasting. These approaches are analyzed in the computational experiment.
Results. Several experiments have been conducted in which standard (ARIMA, ETS, LOESS) and neural network models (LSTM, GRU, RNN) are built and compared in terms of proximity of the forecast to the series data in the test period.
Conclusions. In the case of seasonal time series, models based on neural networks surpassed the standard ARIMA and ETS models in terms of forecast accuracy for the test period. The single-step forecast is computationally less efficient than the integral forecast for the entire target period. However, it is not possible to accurately indicate which approach is the best in terms of quality for a given series. Combined models (neural networks for trend, ARIMA for seasonality) almost always give good results. When forecasting a non-seasonal heteroskedastic series of share price, the standard approaches (LOESS method and ETS model) showed the best results.
About the Authors
B. Pashshoev
MIREA – Russian Technological University
Russian Federation
Russian Federation
Bakhtierzhon Pashshoev, Student
78, Vernadskogo pr., Moscow, 119454
D. A. Petrusevich
MIREA – Russian Technological University
Russian Federation
Russian Federation
Denis A. Petrusevich, Cand. Sci. (Phys.-Math.), Associate Professor, Higher Mathematics Department, Institute of Artificial Intelligence
78, Vernadskogo pr., Moscow, 119454
Scopus Author ID 55900513600, ResearcherID AAA-6661-2020
References
1. Hyndman R.J., Athanasopoulos G. Forecasting: principles and practice. 3rd ed. OTexts; 2021. 442 p. ISBN-13 978-0987507136
2. Stock J.H., Watson M.W. Introduction to Econometrics. 3rd ed. Pearson; 2019. ISBN-13 978-9352863501
3. Kalugin T.R., Kim A.K., Petrusevich D.A. Analysis of the high order ADL(p, q) models used to describe connections between time series. Russ. Technol. J. 2020;8(2):7–22 (in Russ.). https://doi.org/10.32362/2500-316X-2020-8-2-7-22
4. Petrusevich D. Improvement of time series forecasting quality by means of multiple models prediction averaging. In: Proceedings of the Third International Workshop on Modeling, Information Processing and Computing (MIP: Computing-2021). 2021;2899:109–117. https://doi.org/10.47813/dnit-mip3/2021-2899-109-117
5. Beletskaya N., Petrusevich D. Linear combinations of time series models with minimal forecast variance. J. Commun. Technol. Electron. 2023;67(1):144–158. https://doi.org/10.1134/S1064226922130022
6. Box G., Jenkins G. Time Series Analysis: Forecast and Management. John Wiley & Sons; 2015. 712 p. ISBN 978-11185674918
7. Haykin S. Neural Networks and Learning Machines. Pearson Education; 2011. 936 p. ISBN 978-0133002553
8. Shi J., Jain M., Narasimhan G. Time Series Forecasting (TSF) Using Various Deep Learning Models. 2022. URL: https://arxiv.org/abs/2204.11115v1, https://doi.org/10.48550/arXiv.2204.11115
9. Amalou I., Mouhni N., Abdali A. Multivariate time series prediction by RNN architectures for energy consumption forecasting. Energy Rep. 2022;8:1084–1091. https://doi.org/10.1016/j.egyr.2022.07.139
10. Aseeri A. Effective RNN-Based forecasting methodology design for improving short-term power load forecasts: application to large-scale power-grid time series. J. Computational Sci. 2023;68(4):101984. https://doi.org/10.1016/j.jocs.2023.101984
11. Ning Y., Kazemi H., Tahmasebi P. A comparative machine learning study for time series oil production forecasting: ARIMA, LSTM, and Prophet. Comput. Geosci. 2022;164(1):105126. https://doi.org/10.1016/j.cageo.2022.105126
12. Wang P., Zheng X., Ai G., Liu D., Zhu B. Time series prediction for the epidemic trends of COVID-19 using the improved LSTM deep learning method: Case studies in Russia, Peru and Iran. Chaos, Solitons & Fractals. 2020;140:110214. https://doi.org/10.1016/j.chaos.2020.110214
13. Arunkumar K.E., Kalaga D.V., Kumar M.S., Kawaji M., Brenza T.M. Comparative analysis of Gated Recurrent Units (GRU), long Short-Term memory (LSTM) cells, autoregressive Integrated moving average (ARIMA), seasonal autoregressive Integrated moving average (SARIMA) for forecasting COVID-19 trends. Alexandria Eng. J. 2022;61(10):7585–7603. https://doi.org/10.1016/j.aej.2022.01.011
14. Kumar B., Sunil, Yadav N. A novel hybrid model combining βSARMA and LSTM for time series forecasting. Appl. Soft Comput. 2023;134:110019. https://doi.org/10.1016/j.asoc.2023.110019
15. Abebe M., Noh Y., Kang Y.-J., Seo C., Kim D., Seo J. Ship trajectory planning for collision avoidance using hybrid ARIMA-LSTM models. Ocean Eng. 2022;256:111527. https://doi.org/10.1016/j.oceaneng.2022.111527
16. Cascone L., Sadiq S., Ullah S., Mirjalili S., Ur H., Siddiqui R., Umer M. Predicting household electric power consumption using multi-step time series with convolutional LSTM. Big Data Res. 2023;31:100360. https://doi.org/10.1016/j.bdr.2022.100360
17. Wang H., Zhang Y., Liang J., Liu L. DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for time series prediction. Neural Netw. 2023;157:240–256. https://doi.org/10.1016/j.neunet.2022.10.009
18. Zhao L., Mo C., Ma J., Chen Z., Yao C. LSTM-MFCN: A time series classifier based on multi-scale spatial–temporal features. Computer Commun. 2022;182(3):52–59. https://doi.org/10.1016/j.comcom.2021.10.036
19. Rasjid Z.E., Setiawan R., Effendi A. A Comparison: Prediction of Death and Infected COVID-19 Cases in Indonesia Using Time Series Smoothing and LSTM Neural Network. Procedia Comput. Sci. 2021;179(5):982–988. http://doi.org/10.1016/j.procs.2021.01.102
20. Dubey A.K., Kumar A., García-Díaz V., Sharma A.K., Kanhaiya K. Study and analysis of SARIMA and LSTM in forecasting time series data. Sustain. Energy Technol. Assess. 2021;47:101474. https://doi.org/10.1016/j.seta.2021.101474
21. Wu Z., Yin H., He H., Li Y. Dynamic-LSTM hybrid models to improve seasonal drought predictions over China. J. Hydrol. 2022;615:128706. https://doi.org/10.1016/j.jhydrol.2022.128706
22. Yan Y., Wang X., Ren F., Shao Z., Tian C. Wind speed prediction using a hybrid model of EEMD and LSTM considering seasonal features. Energy Rep. 2022;8:8965–8980. https://doi.org/10.1016/j.egyr.2022.07.007
23. Bian S., Wang Z., Song W., Zhou X. Feature extraction and classification of time-varying power load characteristics based on PCANet and CNN+Bi-LSTM algorithms. Electric Power Systems Research. 2023;217(6):109149. https://doi.org/10.1016/j.epsr.2023.109149
24. Sangiorgio M., Dercole F. Robustness of LSTM neural networks for multi-step forecasting of chaotic time series. Chaos, Solitons & Fractals. 2020;139(8):10045. https://doi.org/10.1016/j.chaos.2020.110045
25. Liu X., Lin Z., Feng Z. Short-term offshore wind speed forecast by seasonal ARIMA – A comparison against GRU and LSTM. Energy. 2021;227:120492. http://doi.org/10.1016/j.energy.2021.120492
26. Shahid F., Zameer A., Muneeb M. Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM. Chaos, Solitons & Fractals. 2020;140:110212. https://doi.org/10.1016/j.chaos.2020.110212
27. Wang J., Wang P., Tian H., Tansey K., Liu J., Quan W. A deep learning framework combining CNN and GRU for improving wheat yield estimates using time series remotely sensed multi-variables. Comput. Electron. Agric. 2023;206(4):107705. https://doi.org/10.1016/j.compag.2023.107705
28. Hua H., Liu M., Li Y., Deng S., Wang Q. An ensemble framework for short-term load forecasting based on parallel CNN and GRU with improved ResNet. Electric Power Syst. Res. 2023;216(3):109057. https://doi.org/10.1016/j.epsr.2022.109057
29. Zhang D., Sun W., Dai Y., Liu K., Li W., Wang C. A hierarchical early kick detection method using a cascaded GRU network. Geoenergy Sci. Eng. 2023;222(3):211390. https://doi.org/10.1016/j.geoen.2022.211390
30. Gramovich I.V., Musatov D.Yu., Petrusevich D.A. Implementation of bagging in time series forecasting. Russ. Technol. J. 2024;12(1):101–110 (in Russ.). https://doi.org/10.32362/2500-316X-2024-12-1-101-110]McLeod A., Li W. Diagnostic checking ARMA time series models using squared residual autocorrelations. J. Time Ser. Anal. 1983;4(4):269–273. https://doi.org/10.1111/j.1467-9892.1983.tb00373.x
Supplementary files
|
|
1. DNN structure | |
| Subject | ||
| Type | Исследовательские инструменты | |
View
(145KB)
|
Indexing metadata ▾ | |
- The study aims to build neural network models of time series (LSTM, GRU, RNN) and compare the results of forecasting with their mutual help and the results of standard models (ARIMA, ETS), in order to ascertain in which cases a certain group of models should be used.
- In the case of seasonal time series, models based on neural networks surpassed the standard ARIMA and ETS models in terms of forecast accuracy for the test period.
- The single-step forecast is computationally less efficient than the integral forecast for the entire target period. However, it is not possible to accurately indicate which approach is the best in terms of quality for a given series.
- Combined models (neural networks for trend, ARIMA for seasonality) almost always give good results.
Review
For citations:
Pashshoev B., Petrusevich D.A. Neural network analysis in time series forecasting. Russian Technological Journal. 2024;12(4):106–116. https://doi.org/10.32362/2500-316X-2024-12-4-106-116. EDN: WDYUFJ
Views:
1472
Email this article 