Hyperparameter tuning in deep learning using NSGA-III: a Multi-Objective perspective

The rapid advancement of Deep Learning (DL) has introduced critical challenges in optimizing model performance while managing computational constraints. Traditional hyperparameter tuning methods such as manual search, grid search, and random search are inefficient and computationally expensive, failing to address real-world trade-offs among conflicting objectives. This thesis proposes and implements a novel framework: the Multi-Objective NSGA-III-DL model, wherein the Non-Dominated Sorting Genetic Algorithm III (NSGA-III) is directly infused into the deep learning optimization process. This integration enables simultaneous optimization of three pivotal objectives: model accuracy, F1-score, and model size—constituting a multi-objective optimization problem that remains underexplored in contemporary DL research. The proposed Multi-Objective NSGAIII-DL framework is benchmarked against six baseline optimization strategies—manual search, grid search, random search, Bayesian optimization, particle swarm optimization, and standard genetic algorithms using MNIST and CIFAR-10 datasets. Experimental results demonstrate that Multi-Objective NSGAIII-DL consistently dominates in composite performance metrics across various model architectures. For the MNIST dataset, Multi- Objective NSGAIII-DL achieves superior scores in MLP (2.9669), LeNet (2.9884), and CNN (2.9882) architectures; for CIFAR-10, it leads in MLP (2.0439), LeNet (2.2835), and CNN (2.4348) configurations. These findings underscore its adaptability across architectures of varying complexity and dataset difficulty. Multi-Objective NSGAIII-DL effectively navigates high-dimensional hyperparameter spaces, yielding Pareto-optimal solutions that strike an effective balance between accuracy and F1-score while minimizing model size. The framework reduces dependence on manual expertise and mitigates the biases inherent in conventional tuning methods. Detailed convergence analysis further reveals Multi-Objective NSGAIII-DL's superior optimization trajectory, characterized by more rapid and stable convergence towards optimal trade-offs. This research provides a robust methodological advancement for deploying efficient, scalable DL models in resource-constrained environments such as edge computing and embedded systems. The findings establish Multi-Objective NSGAIII-DL as a powerful, automated approach to multi-objective hyperparameter optimization, significantly enhancing the efficiency, adaptability, and practicality of deep learning systems.