Implementation of bagging in time series forecasting

Objectives. The purpose of the article is to build different models of bagging, to compare the accuracy of their forecasts for the test period against standard models, and to draw conclusions about the possibility of further use of the bagging technique in time series modeling.

Methods. This study examines the application of bagging to the random component of a time series formed after removing the trend and seasonal part. A bootstrapped series combining into a new random component is constructed. Based on the component thus obtained, a new model of the series is built. According to many authors, this approach allows the accuracy of the time series model to be improved by better estimating the distribution.

Results. The theoretical part summarizes the characteristics of the different bagging models. The difference between them comes down to the bias estimate obtained, since the measurements making up the bootstraps are not random. We present a computational experiment in which time series models are constructed using the index of monetary income of the population, the macroeconomic statistics of the Russian Federation, and the stock price of Sberbank. Forecasts for the test period obtained by standard, neural network and bagging-based models for some time series are compared in the computational experiment. In the simplest implementation, bagging showed results comparable to ARIMA and ETS standard models, while and slightly inferior to neural network models for seasonal series. In the case of non-seasonal series, the ARIMA and ETS standard models gave the best results, while bagging models gave close results. Both groups of models significantly surpassed the result of neural network models.

Conclusions. When using bagging, the best results are obtained when modeling seasonal time series. The quality of forecasts of seigniorage models is somewhat inferior to the quality of forecasts of neural network models, but is at the same level as that of standard ARIMA and ETS models. Bagging-based models should be used for time series modeling. Different functions over the values of the series when constructing bootstraps should be studied in future work.

1. Hyndman R.J., Athanasopoulos G. Forecasting: Principles and Practice. 2nd ed. Publisher OTexts; 2018. 382 p. ISBN 978-0-9875-0711-2

2. Özen K., Yildirim D. Application of bagging in day-ahead electricity price forecasting and factor augmentation. Energy Econ. 2021;103(2):105573. http://doi.org/10.1016/j.eneco.2021.105573

3. Meira E., Oliveira F.L.C., de Menezes L.M. Forecasting natural gas consumption using Bagging and modified regularization techniques. Energy Econ. 2022;106(3):105760. http://doi.org/10.1016/j.eneco.2021.105760

4. Kim D., Baek J-G. Bagging ensemble-based novel data generation method for univariate time series forecasting. Expert Syst. Appl. 2022;203(1):117366. http://doi.org/10.1016/j.eswa.2022.117366

5. Liu X., Liu A., Chen J.L., Li G. Impact of decomposition on time series bagging forecasting performance. Tourism Manag. 2023;97(2):104725. https://doi.org/10.1016/j.tourman.2023.104725

6. Beletskaya N., Petrusevich D. Linear Combinations of Time Series Models with Minimal Forecast Variance. J. Commun. Technol. Electron. 2023;67(1):144–158. http://doi.org/10.1134/S1064226922130022

7. Musatov D., Petrusevich D. Modeling of forecasts variance reduction at multiple time series prediction averaging with ARMA(1, q) functions. In: Proceedings of MIP Computing-V 2022: V International Scientific Workshop on Modeling, Information Processing and Computing. 2022;3091:1–11. https://doi.org/10.47813/dnit-mip5/2022-3091-1-11

8. Golov V.A., Petrusevich D.A. Data analysis methods in astronomic objects classification (Sloan Digital Sky Survey DR14). Russ. Technol. J. 2021;9(3):66–77 (in Russ.). https://doi.org/10.32362/2500-316X-2021-9-3-66-77

9. Carlstein E. The use of subseries methods for estimating the variance of a general statistic from a stationary time series. Ann. Statist. 1986;14(3):1171–1179. https://doi.org/10.1214/aos/1176350057

10. Carlstein E., Do K-A., Hall P., Hesterberg T., Künsch H.R. Matched-Block Bootstrap for Dependent Data. Research Report 74. Seminar für Statistik. ETH Zentrum. Zürich, Switzerland; 1995. 25 p.

11. Künsch H.R. The jackknife and the bootstrap for general stationary observations. Ann. Statist. 1989;17(3):1217–1261. https://doi.org/10.1214/aos/1176347265

12. Götze F., Künsch H.R. Second-order correctness of the blockwise bootstrap for stationary observations. Ann. Statist. 1996;24(5):1914–1933. http://doi.org/10.1214/aos/1069362303

13. Bühlman P., Künsch H.P. Block length selection in the bootstrap for time series. Comput. Stat. Data Anal. 1999;31(3): 295–310. https://doi.org/10.1016/S0167-9473(99)00014-6

14. Politis D.N., Romano J.P. The stationary bootstrap. J. Am. Stat. Assoc. 1994;89(428):1303–1313. https://doi.org/10.1080/01621459.1994.10476870

15. Politis D.N., White H. Automatic Block-Length Selection for the Dependent Bootstrap. Econometric Rev. 2004;23(1):53–70. https://doi.org/10.1081/ETC-120028836

16. Lahiri S.N. Theoretical comparison of block bootstrap methods. Ann. Statist. 1999;27(1):386–404. https://doi.org/10.1214/aos/1018031117

17. Bergmeir C., Hyndman R. J., Benitez J.M. Bagging exponential smoothing methods using STL decomposition and Box–Cox transformation. Int. J. Forecast. 2016;32(2):303–312. https://doi.org/10.1016/j.ijforecast.2015.07.002

18. Petropoulos F., Hyndman R.J., Bergmeir C. Exploring the sources of uncertainty: Why does bagging for time series forecasting work? Eur. J. Oper. Res. 2018;268(2):545–554. http://doi.org/10.1016/j.ejor.2018.01.045

19. Haykin S. Neural Networks and Learning Machines. Pearson Education; 2011. 936 p. ISBN 978-0-1330-0255-3

20. Box G., Jenkins G. Time Series Analysis: Forecast and Management. John Wiley & Sons; 2015. 712 p. ISBN 978-1-1185-6749-18

21. Rasjid Z.E., Setiawan R., Effendi A. A Comparison: Prediction of Death and Infected COVID-19 Cases in Indonesia Using Time Series Smoothing and LSTM Neural Network. Procedia Comput. Sci. 2021;179(5):982–988. http://doi.org/10.1016/j.procs.2021.01.102

22. Liu X., Lin Z., Feng Z. Short-term offshore wind speed forecast by seasonal ARIMA – A comparison against GRU and LSTM. Energy. 2021;227:120492. http://doi.org/10.1016/j.energy.2021.120492

23. Shahid F., Zameer A., Muneeb M. Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM. Chaos, Solitons & Fractals. 2020;140:110212. https://doi.org/10.1016/j.chaos.2020.110212

24. Amalou I., Mouhni N., Abdali A. Multivariate time series prediction by RNN architectures for energy consumption forecasting. Energy Rep. 2022;8:1084–1091. https://doi.org/10.1016/j.egyr.2022.07.139