Extension of RMIL conjugate gradient method for unconstrained optimization / Nur Idalisa Norddin

The Conjugate Gradient (CG) methods have significantly contributed to solving Unconstrained Optimization (UO) problems. This research focused on the modification of existing CG method of Rivaie, Mustafa, Ismail and Leong (RMIL). RMIL is ubiquitous for its effectiveness as an optimization technique, yet their significance remains to be defined and their full potential is yet to be realized. Even the global convergence theoretical is available for RMIL method, it only applies for the positive RMIL parameter. Indeed, the numerical performance of RMIL method is impressive regardless of its parameter sign. Much efforts have been made previously to increase the efficiency of RMIL method. Hence, this research proposed a CG search direction named NEW RMIL by combining the scaled negative gradient as initial direction and a third-term parameter. Sufficient Descent Condition (SDC) and global convergence qualities for both the exact and the strong Wolfe line search were demonstrated to exist in NEWRMIL algorithm. The experiments were performed by a total of 44 multi-dimensional mathematical test functions with various levels of complexity. When compared with the existing CG methods, NEWRMIL performs similarly under precise line search, while under Strong Wolfe line search NEWRMIL is superior and relatively faster convergence speed. Additionally, the practicality of NEWRMIL was demonstrated in solving multiple linear regression problems. The findings show that the NEWRMIL algorithm is the most efficient and has the minimum NOI and CPU time when compared to the direct technique and existing CG methods.