Impact of optimizer on the MLP-based models for student performance classification

Effectively predicting student academic performance is a critical challenge in engineering education, where enhancing the performance and generalization of machine learning models can significantly aid early intervention strategies, which are crucial for engineering students as they help identify and support those at risk of falling behind, ensuring better academic outcomes and retention in the challenging field. This study investigates the impact of different optimization algorithms on Multi-Layer Perceptron (MLP) models for student performance forecasting, utilizing a dataset of 99 student samples. Recursive Feature Elimination (RFE) was employed to select the most salient features, thereby reducing model complexity. Five optimizers, AdamW, AdaGrad, AmsGrad, Nadam, and SGD with Momentum were evaluated to assess their influence on convergence speed, stability, and generalization. Performance was gauged by the number of epochs for convergence and key metrics including accuracy, precision, recall, and F1-score. AdamW and Nadam demonstrated superior overall performance, converging rapidly with stable results. AdamW achieved the highest F1-score (86.95%), while both AdamW and Nadam attained the highest testing accuracy (80.0%). Conversely, SGD with Momentum underperformed, exhibiting signs of underfitting with the lowest accuracy (55.0%) and F1-score (47.05%). By combining RFE with a careful selection of adaptive optimizers, this research underscores a robust methodology for developing MLP models capable of effectively analyzing educational data. These findings highlight the balance between learning efficiency and predictive reliability, supporting data-driven decision-making in education. Future research will focus on validating these findings on larger datasets and exploring the impact of optimizer choice on fairness metrics in educational predictions.