Selective thermal and visible image fusion model for illumination-invariant ear recognition / Syed Mohd Zahid Syed Zainal Ariffin

Ear recognition has gained interest as a mode of biometrics due to its invariant to emotions and no-contact acquisition. It is, however, affected by illumination variations where visible images taken in bad illumination condition show significant quality degradation making feature extraction arduous. Several image enhancement methods were proposed by previous studies to normalize the illumination variation effect. Those enhancement methods, however, still did not provide significant improvement, especially for the image with very poor illumination condition. Therefore, this study proposed a new model for illumination invariant ear recognition using thermal imaging which is invariant to illumination variations. While invariant to illumination changes, thermal images lack details of features. The fusion of thermal and visible images was considered based on the reported successes in previous literature for facial recognition. Image fusion aims to complement the information of both images. A new thermal and visible ear images dataset was developed. Images in this dataset were acquired in different illumination conditions measured by lux. All images in the dataset were then classified into three illumination categories (i.e. dark, moderate and bright) based on specified criteria. Later, an approach to classify illumination using no-reference image quality assessment (NR-IQA) metrics (i.e., image entropy, standard deviation and average pixel value) was proposed. Experiment results supported the use of NR-IQA for illumination classification IQA. Performance of ear recognition was evaluated using several thermal and visible image fusion methods, which were simple average, weighted average, average discrete wavelet transform (DWT), weighted DWT, optimized DWT, principal component analysis (PCA) and non-subsampled contourlet transform (NSCT). Initially, the performance of thermal images outperformed all the fused images, contradicting the results from previous literature. Further experiments were conducted with well-illuminated images. Based on the findings, all DWT-based fused images obtained better accuracy rate compared to thermal images. This result showed that thermal and visible fusion improved recognition rate for the image with minimal illumination variations. Therefore, a selective thermal and visible image fusion model (SelF-TV) was proposed. In this model, images were classified into two illumination categories (i.e., good and poor) using NR-IQAs. Images in poor illumination underwent ear recognition process using only thermal images while images in good illumination were fused with their corresponding thermal images before the recognition process. The evaluation was done based on the ear identification test (one-to-many). The maximum recognition rate achieved using SelF-TV was 98.18% compared to 94.55% for recognition using thermal images. Ear verification test (one-to-one) was conducted to validate the result using optimized DWT fusion. The verification results confirmed the findings when selective optimized DWT fusion obtained the lowest equal error rate (EER) at 0.4625 over thermal images (0.5081) and conventional optimized DWT (0.5191).