PSO and Linear LS for parameter estimation of NARMAX/NARMA/NARX models for non-linear data / Siti Muniroh Abdullah

System Identification, a discipline for constructing models from dynamic systems, consist of three major steps: structure selection, parameter estimation and model validation. The parameter estimation step is concerned with the estimation of model parameters once the structure is known. Typically, parameter estimation is performed using various types of Least Squares (LS) algorithms due to its stable and efficient numerical computation. However, LS methods suffers from oversmoothing properties (such as in curve fitting) and high sensitivity to outliers as error squaring significantly increases in magnitude. Additionally, when the range of data increases, it makes the nonlinear processes become more difficult. In this thesis, the Particle Swarm Optimization (PSO) is proposed for parameter estimation of a Nonlinear Autoregressive Moving Average with Exogeneous Inputs (NARMAX) and its derivatives ((NARX) and (NARMA)) of three datasets, which are Direct Current Motor (DCM), Flexible Robot Arm (FRA) and Mackey Glass (MG) system. PSO is a swarm-based search algorithm perform a stochastic search to explore the search space. Due to its stochastic nature, the algorithm does not inherit the numerical problems posed by the LLS algorithms. The proposed method is compared with three established conventional Linear Least Squares (LLS) solution methods : Normal Equation (NE), QR factorization (QR) and Singular Value Decomposition (SVD). Results suggest that the PSO algorithm is viable alternative to other established algorithms for LLS parameter estimation. In DCM NARX experiment, LLS is outperform in term of criterion fitness while PSO outperform in correlation violation values. For model fit test (MSE and R-squared) both methods perform similar. For FRA NARMAX, PSO outperform than LLS in term of criterion fitness, MSE and correlation violation while in R-squared analysis both methods perform similar. In the last experiment MG NARMA, LLS outperform than PSO in MSE value, criterion fitness and R -squared analysis while in term of correlation violation, PSO outperform than LLS. Additionally, the PSO algorithm was found to improve the correlation tests (reduction in correlation violation 22.22% in DCM NARX, 1.89% in FRA NARMAX and 10.46% in MG NARMA experiment) relative to the LLS algorithms.