Automatic database of robust neural network forecasting / Saadi Ahmad Kamaruddin, Nor Azura Md. Ghani and Norazan Mohamed Ramli

Neural network has been widely known by its function as a universal approximator. Neural network is an efficient tool for forecasting financial time series as well as many other areas and has shown a great success, outperforming other forecasting techniques. The backpropagation algorithm is one of the most famous algorithms to train neural network based on the mean square error (MSE) of ordinary least squares (OLS). However, this algorithm is not always so robust in the presence of outliers, and this may ruin the entire neural network fit. Therefore, this automatic database is designed to provide an alternative for robust neural network forecasting using statistical robust estimators of M-estimators, Iterated Least Median Square (ILMedS) and Particle Swarm Optimization on Least Median Square (PSO-LMedS), replacing the MSE cost function to handle time series data with missing values, outliers and noise, which always exist in real-life-time series data. Artificial neural network (ANN) was implemented in the system because it has been proven to outperform many other conventional forecasting techniques. However, the most popular backpropagation algorithm which is based on Widrow-Hoff delta learning rule is not completely robust in the presence of outliers and this may cause false prediction of future values. The problem of outliers is suggested to be the main concern of robust statistics where multiple methods are possible to deal with the problem that has been proposed (Huber, 1981; Hampel, Ronchetti, Rousseeuw & Stahel, 1986). The idea is simple, when an actual underlying model steers away from the assumptions made for example normal error distribution, then there emerges the role of robust statistics to work accordingly (Rusiecki, 2012). Robust estimators in this case are reliable and efficient in administering outlying data, where they perform credibly enough to sample from popular statistical methods which are not gravely impacted by small departures from assumptions of the model (Rusiecki, 2012). The direct idea of making the conventional neural network learning algorithm more powerful towards outlying data is by replacing the mean square error (MSE) with a different symmetric and continuous cost function. This will result in a nonlinear influence function (Rusiecki, 2012) with the capability to cater the influence of large errors. This can only be performed by making the loss functions robust using the statistical robust methods to reduce the impact of outliers issues (Rusiecki, 2012; El-Melegy, Essai & Ali, 2009), where the usual outliers occurence in routine data ranges up to 10% or even more (Rusiecki, 2012; El-Melegy et al., 2009; Zhang, 1997)- this is the primary subject of this system. Most of the previous studies seek to improve the learning algorithm of backpropagation neural networks by adapting the M-estimators predominantly. All those previous endeavours focus only on the nonliear autoregressive inputs artificial neural network (NAR-ANN) model. Not much work had considered using a robust approach in improving the nonlinear autoregressive moving average inputs artificial neural network (NARMA-ANN) model - here comes the novelty of this system. The overall performance of the NARMA model is better than the NAR model (Zhang, 1997). Therefore, we implemented all kinds of Mestimators, Iterative Least Median Squares (ILMedS) and Particle Swarm Optimization on Least Median Squared (PSO-LMedS) aiming to reduce the errors caused by outlying data in both NAR-ANN and NARMA-ANN models. It is expected that the improved robust neural network time series models can be applied on all type of time series data for future forecasting purposes.