Evaluation of autoregressive time series prediction using validity of cross-validation

Cross-Validation (CV) CV is a validation technique used to explain how well the estimated values from a fitted statistical model will generalize to the explanatory variables under study. Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved

Why Cross Validation technique is different for time series data? However, for predicting the time series data, this K-fold is not a straightforward because of the non-stationarity and serial correlation present in the data. K-fold CV is applicable for the autoregressive models with uncorrelated errors and have wide scope in predicting the model using Machine Learning methods. Standard procedure for model validation is the K- fold CV as in Regression Analysis and classification problem. Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved

Types of Cross Validation K-fold Cross Validation 01 K-fold Cross Validation for time series for purely autoregressive models particularly for the dependent case instead of applying to the independent case. Hold-out Cross Validation 02 Hold-out cross-validation technique is performed by splitting the data into training and test set. Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved

Model selection adopted in that paper is 5-fold Cross Validation using leave-one-out, non- dependent, and out-of-sample evaluation methods using AR(1) to AR(5) models and residuals have been taken care for the uncorrelated errors using Serial Correlation. 01 02 Three different experiments are carried out to forecast the AR model using Monte Carlo simulation. 03 The monthly seasonality from the ACF and PACF plots for the data considered for Monte- Carlo simulation is depicted in the following figure and the results of the experiment is discussed. 04 Applicability of the procedure using the yearly sun spot data available in R. The data involves 289 observations recorded from 1700 to 1988 and are depicted in the following figure. Contd... Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved

Discussions Auto correlation Relevant auto correlation is captured using Ljung-Box test with the residual series of 1560 model configurations. Method RMSE Out of 1560, 763 satisfies the Box test and the configuration which have minimum root mean square error is chosen. 5 fold CV 2,247 The 5-fold CV yields an error rate of 2.247 andout-of-sample (OOS) methodyields 2.281 and the CV procedure is not an over fitting model and the results are tabulated. 005 2,281 Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved

Summary Discussed the applicability of the K-fold Cross Validation procedure for the purely AR models beyond the usual practice and it is found to be useful when the residuals in the model is uncorrelated. 01 From the Monte-Carlo simulation, the K-fold classification yields better result than out-of- sample method for time series data and the real-time data explains the application of this using AR model with uncorrelated errors. 02 The over-fitting and under-fitting of the model is accessed using the Ljung-Box test and it results in there is no such issues occurred in the model. 03 Examines the applicability of the K-fold cross validation using neural network model with a monte-carlo simulations, & it can also be extended to the other regression models such as random forest and other machine learning techniques. 04 The interesting point is that if we use whatever regression model for these kinds of Time Series Data, this method still proves the K-fold cross validation is the best method. 05 Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved

PHONE NUMBER UK : +44-1143520021 INDIA : +91-4448137070 Freelancer EMAIL ADDRESS Consultant info@statswork.com Guest Blog Editor GET IN TOUCH WITH US CONTACT hr@workfoster.com Research Planning | Data Collection | Semantic Annotation | Business Analytics | Bio Statistics | Econometrics Copyright © 2019 Statswrok. All rights reserved

Evaluation of autoregressive time series prediction using validity of cross-validation