High-performance Two-stage Face Recognition Algorithm for Reduced Illumination Effect

Face recognition, which is a method of identifying a person in a digital image, is widely applied to biometric-based authentication systems. A significant decrease in recognition rate is caused by extracted features that are affected by illumination. In an attempt to resolve this problem, in this paper, we present a two-stage algorithm, namely, a local binary pattern (LBP) followed by the algorithm Fisherface. As the first step of this work, a face image is converted to an LBP, which is then projected onto a low-dimensional feature space using Fisherface for subsequent classification and recognition. As a result, the outperformance of this work is demonstrated by a recognition rate of up to 96.45%, a figure far beyond 67.97% using the LBP histogram (LBPH), 84.69% using Fisherface, and 93.09% using the support vector machine (SVM) algorithm.


Introduction
Face recognition (1)(2)(3) is one of the computer vision techniques. As the first step, a face is detected and separated out of an image, and the feature thereof is then extracted for subsequent recognition. As one of the most common techniques, face recognition has been widely applied to personal identification, machine version, smart home access systems, and more.
In addition to its critical role in practical applications, face recognition has been drawing more academic attention than ever before over the years, and there are many published studies on face recognition accordingly. Existing recognition techniques in the literature can be mainly categorized into two types, namely, feature-matching-based (4)(5)(6) and machine-learning-based (7)(8)(9)(10)(11) techniques. Scale-invariant feature transform (SIFT), (4) speeded up robust features (SURF), (5) and local binary pattern histogram (LBPH) (6) are the representatives of the former, whereas the Eigenface technique using principal component analysis (PCA), (7) Fisherface using PCA and linear discriminant analysis (LDA), (8) the support vector machine (SVM) algorithm, (9,10) and the neural-network-based recognition algorithm (11) are those of the latter.
The performance of a wide variety of face recognition algorithms, including all those mentioned above, is susceptible to a number of factors, e.g., illumination variation, facial expression, and pose variation. As such, the combined use of a local binary pattern (LBP) and the algorithm Fisherface is presented herein as an effective way of significantly reducing the effect of illumination on face recognition, which leads to an improved recognition rate accordingly.

Proposed Approach
As the first step in this work, a face image is converted into an LBP, which is then projected to a low-dimensional feature space using Fisherface for classification and recognition. As a distinctive feature for texture classification, the LBP, proposed by Ojala et al., (12) is operated as illustrated in Fig. 1. In a 3 × 3 block, the value of the center pixel is subtracted from each value of 8 neighboring pixels. If the result is negative, then the binary 0 is assigned; otherwise, 1 is assigned. The process starts from the top left pixel of the 3 × 3 block as a center pixel and is repeated in a clockwise direction, until all the neighbors are done, and an eight-bit binary number is produced accordingly. Subsequently, the binary number 01011000 is converted into its equivalent decimal as the LBP code of the center pixel. The same process is repeated until the LBP of a face image is done. Figure 2 shows the LBPs corresponding to the face images with different illuminations.
Subsequently, Fisherface (8) is employed to transform the LBP profile into a low-dimensional feature space for more reliable pattern classification and recognition. As one of the well-known techniques for face recognition, Fisherface uses PCA and LDA to obtain a projection matrix, whereby an input vector is transformed into a low-dimensional feature vector.
Fisherface is described in detail as follows. Assume that there are N n-dimensional image vectors, represented as {x 1 , x 2 , …, x N }, each belonging to one of the C classes {X 1 , X 2 , …, X C }. Using a linear transformation, we subsequently transform x i to an m-dimensional feature vector y i , defined as where W is an n-by-m matrix with orthonormal columns and m < n. Fisherface aims to optimize W and then y i to improve the image vector classification and face recognition performance. W is then optimized via the introduction of 3 scatter matrices, i.e., the total scatter matrix S t , the between-class scatter matrix S b , and the within-class scatter matrix S w , respectively defined as where µ denotes the mean of all the samples x i , µ k represents the mean of the class X k , and N k symbolizes the number of samples in X k . Finally, the optimal solution W opt , as can be found in the previous study, (8) is given as  7), 2 stages are involved in dealing with the dimension reduction and classification, that is, PCA is followed by LDA. W opt is obtained after training and serves as the transfer matrix in Eq. (1) to transform an input vector to a feature vector for subsequent recognition.

Experimental Results
In this section, the performances of LBPH, Fisherface, SVM, and this work are compared in terms of recognition rate. A total of 2432 cropped images out of the Extended Yale face database B (13,14) are employed as test samples for the comparison. There are 38 human subjects involved, namely, C = 38, each having 64 differently illuminated face images.
All the test samples are divided into 2 parts, one for training purposes and the other for testing purposes. As listed in Table 1, 10 test cases are involved in this paper. Eleven out of the 64 face images of each human subject are chosen in the first case as the training samples, namely, a total of 11 × 38 = 418 face images (N = 418, N k = 11 for all k), and the number of face images is increased up to 20, that is, a total of 20 × 38 = 760 face images (N = 760, N k = 20 for all k) in the final case. Excluding the training samples, the remaining samples are employed for recognition testing.
Subsequently, Table 2 shows the recognition rates of all the test cases in Table 1 obtained using Fisherface, LBPH, SVM, and this proposal. This proposal is found to outperform its counterparts to a great extent at all events. In other words, the effect of illumination on face recognition can be significantly reduced. Other findings are that a higher recognition rate can be provided by this proposed method, but by using a smaller number of training samples than of those used in the counterpart methods, and that the training time and definitely the training cost can be reduced. Moreover, we found that the recognition rate increases with the number of training samples, as illustrated in Fig. 3, which is a graphical representation of Table 2, but tends to reach an upper bound toward the end.

Conclusion
A cascade of LBP and Fisherface is applied to face recognition problems herein in an attempt to reduce the effect of illumination. Our proposed method outperformed the other methods, showing a recognition rate of up to 96.45% in the case of N k = 20, a figure far beyond 67.97% obtained using LBPH, 84.69% using Fisherface, and 93.09% using SVM. This showed that a high-performance algorithm is considered to be a competitive candidate, particularly when addressing the face recognition issue with a complicated background.    Table 2.