Multimodal Biometric Identiﬁcation System based on Cascaded

Advanced of Fingerprint and Finger Vein Images and AND Rule at

Decision Level Fusion

El Mehdi Cherrat

, Rachid Alaoui

and Hassane Bouzahir

ISTI Laboratory, National School of Applied Sciences, Ibn Zohr University, Agadir, Morocco

ASTIMI Laboratory, Higher School of Technology- Sale-, Mohammed V University, Rabat, Morocco

Keywords:

Fingerprint Recognition; Finger Vein Recognition, Minutiae points, histogram of oriented gradients Feature,

Fusion.

Abstract:

The multimodal identiﬁcation system can integrate a variety of biometric characteristics. The main advan-

tage of multibiometric system against traditional single biometric is achieving the recognition process more

accurate and safe. In this paper, we will present a multimodal biometric recognition system that combines

ﬁngerprint and ﬁnger vein. The features in theses biometric traits are extracted to identify that individual is

genuine or impostor using minutia points for ﬁngerprint and Histogram of Oriented Gradient for ﬁnger vein.In

the ﬁrst stage, cascade multimodal biometric system based on the both biometric modalities is applied. In

the second stage, the fusion is accomplished at decision level method based on AND rule using multimodal

biometric recognition system. The simulation results have demonstrated that the proposed fusion algorithm

performs increase probability the accuracy to 99,85 than the other system based on unimodal characteristics.

1 INTRODUCTION

In recent years, the biometric system necessity has

been rapidly increased. The biometric recognition is

required reliable to distinguish one individual from

another using measurable morphological (such as ﬁn-

gerprint, face, iris, etc.) or behavioral (for example

voice, signature, etc.) features. With these charac-

teristics including being less susceptible to veriﬁca-

tion being stolen or forgotten. It is used for criminal

identiﬁcation, immigration and naturalization service,

securing access to buildings or personal objects, sup-

porting anonymous transactions, etc.(Cherrat et al.,

2017).

The most common biometric system is ﬁngerprint

recognition. It is considered an excellent biometric

modality for identiﬁcation or veriﬁcation the person,

especially in the latest smart phones and consumer de-

vices. Compared to other biometric traits, the ﬁnger

vein modality has achieved popularity in biometric

recognition because of the variety advantages given

by these systems for example, 1) the vein of each

person are completely unique and different 2) it is

identiﬁed as being less prone to modify with age and

growth (3 the ﬁnger veins biometric is easily acquired

using sensor capable of capturing or the NIR (Near-

Infrared) light source 4) the vein structure is hidden

inside the skin. Thus, the possibility of spoof the

human recognition system is very complex (Khellat-

Kihel et al., 2016).

The general structure of biometric recognition

system consists of four main steps. In the ﬁrst one,

the acquisition of biometric image is process of get-

ting a digitalized image of a person using speciﬁc cap-

turing device. In the second step, the pre-processing

is allowed to improve overall quality of the captured

image and to correct its orientation . After that, the

region of interest is localized. It is the process of ob-

tenting all important data needed for recognition. In

the next step, the features information are extracted

using different algorithms. In the last step, generally,

the matching of the extracted characteristics is applied

in order to perform the recognition of the person.

The multimodal biometrics combines two or more

different biometric modalities and reduces certain

limitations of systems based on one modality such

as spoof attacks,non-universality, noise in sensed

data,inter-class similarities and intra-class variations.

Thus, the recognition system based on fusion of

multibiometric is most recommended for signiﬁcantly

Cherrat, E., Alaoui, R. and Bouzahir, H.

Multimodal Biometric Identiﬁcation System based on Cascaded Advanced of Fingerprint and Finger Vein Images and AND Rule at Decision Level Fusion.

DOI: 10.5220/0009773701610166

In Proceedings of the 1st International Conference of Computer Science and Renewable Energies (ICCSRE 2018), pages 161-166

ISBN: 978-989-758-431-2

161

improving the system performance and reducing the

error rate the identiﬁcation or veriﬁcation of the in-

dividual. This fusion can be applied at the sen-

sor level, the feature extraction level, the matching-

score level, rank level and at the decision level. The

multimodal biometric systems are classiﬁed as multi-

instance, multi-sensor, multi-algorithm, multi-modal

and hybrid systems (Khellat-Kihel et al., 2016).

The rest of the paper is separated into four sec-

tions. In the section 2, the related works in the ﬁeld

are reviewed. Section 3 discusses the proposed algo-

rithm. Experimental results have been analysed and

discussed in Section 4. Finally, the conclusion is pre-

sented in the last section.

2 RELATED WORKS

Many techniques have been proposed of the multi-

modal biometrics system. Ross et al (Ross, 2003)

presented different levels of fusion and score level fu-

sion on the multimodal biometric system. Singh.al

(Singh et al., 2004) proposed biometric recognition

system based on face combining visible and thermal

Infrared (IR) images at sensor level. Son (Son and

Lee, 2005) have been subjected a fusion of face and

iris at feature level. Ross.al (Ross and Govindara-

jan, 2005) presented hand and face combined at fea-

ture level. Moreover, the experiments were applied

in three different scenarios. At the fusion of match

scores, Jain.al (Jain and Huang, 2004) proposed lin-

ear discriminant function and the decision trees. Dif-

ferent fusion techniques and normalization methods

of ﬁngerprint, hand geometry and palm-print biomet-

ric sources are achieved by Yang.al (Yang and Ma,

2007). Another multimodal biometric system based

on multi-instance iris recognition system using a fu-

sion of right iris and left iris for the same individ-

ual is studied by Wu.al. (Wu et al., 2007). Jain.al

(Jain et al., 1999) introduced a multimodal biomet-

ric system using face, ﬁngerprint, and voice. Yang.al

(Yang, 2018) presented a multi-biometric system can-

celable using ﬁngerprint and ﬁnger-vein, which com-

bines the minutia points of ﬁngerprint and ﬁnger-vein

image feature based on a feature-level of three fusion

techniques. The fusion multimodal biometric sys-

tem based on ﬁngerprint and ﬁnger-vein at score level

using four score fusion approaches (min score, max

score, simple sum, user weighting) and three score

normalization techniques (min-max, z-score, hyper-

bolic tangent) is developed by Vishi.al (Vishi and

Jøsang, 2017).

3 PROPOSED METHOD

In ﬁrst level of our algorithm, ﬁngerprint image is

enhanced using gabor ﬁlter technique, binarized and

passed to thinning algorithm. Then, the features

points are extracted using ridge ending and bifurca-

tion uniformly namely minutiae. In the ﬁnal step, the

comparison of minutiae information provided from

the registered database, and the query ﬁngerprint is

presented to the matching. In the second level, the

Linear Regression Line have been utilized to solve

the orientation of misalignments of ﬁnger vein im-

ages. Next, the region of interest of image is obtained

using canny method. After that, the histogram equal-

ization is applied to enhance the cropped ﬁnger vein

image.Furthermore, the features extracted is based on

Histogram of Oriented Gradient algorithm. Finally,

the score provided is compared and machted with

stored ﬁngervein templates stored in the database. If

the ﬁrst level is not identiﬁed then the second level

works. The fusion is applied at cascaded advanced

decision level. If the ﬁrst level is passed then the sec-

ond level avoids for matching ﬁngervein extracted. In

this section, we detail the proposed technique which

is illustrated in Figure 1. The details of each phase are

represented in the following.

Figure 1: Block diagram of proposed algorithm for ﬁnger

vein image recognition

3.1 Fingerprint Recognition

3.1.1 Image Enhancement

To overcome the background noise, non-uniform il-

lumination and low contrast of the ﬁngerprint image

captured, the preprocessing is important step for char-

acteristic extraction and then the matching. The mean

and variance are used to normalize and estimate the

orientation of input image. After that, the frequency

image is computed from which the region mask is

provided using block classiﬁcation of resulted im-

ICCSRE 2018 - International Conference of Computer Science and Renewable Energies

162

age. Then, gabor ﬁlters applied to normalized image

(Hong et al., 1998).

3.1.2 Feature Extraction

The features points are extracted from ﬁngerprint im-

age such as ridge ending and bifurcation uniformly

namely minutiae. Before extracting the minutiae, the

binarization method is applied using block with size

3x3. This process is transformed the 8 bits gray im-

age to 1 bit with 1 value for the valleys and 0 value

for ridges based on a given threshold. Next, mor-

phological technique processing (dilatation and ero-

sion) is used as postprocessing to achieve more com-

pact blocks for reducing the noise region. Moreover,

the thinning operation is applied to remove basically

the redundant pixels until having a single pixel width

(Cui and Yang, 2011). Finally, the bifurcation and

ending points are detected by computing black pixels

of 8-directional nearest for each pixel point in ﬁnger-

print image. If the central pixel is black and has 3

black values nearest, then this pixel is a bifurcation.

When the number of black nearest is just 1, the feature

point is represented ending. The connection number

(CN) for a given ridge can be represented in equa-

tion 1 . Hence, the minutea characteristic extraction

of ﬁngerprint pattern is represented by the following

parameters, 1) Type of the ridge, 2) x-coordinate, 3)

y-coordinate, 4) θ-orientation.

CN =

∑

i=0

− P

i+1

| (1)

where P

is the pixel value at index i and P

3.1.3 Features Mathching

The minutiae feature extraction of ﬁngerprint pattern

is represented by the type of ridge, the spatial coor-

dinates x, y and orientation of minutiae points. The

Euclidian distance is used to ﬁnd number of matched

two minutiae pairs. This distance is described as fol-

lows :

, M

) =

− x

)

+ (y

− y

) (2)

, M

) = min(|θ

−θ

|, 360−|θ

−θ

|) ≤ θ

(3)

where M

and M

are the extracted minutiae points

pairs from the template in the enrolled database and

the input query ﬁngerprint image respectively. A

(di-

rection difference between M

and M

) is smaller than

an angular tolerance θ

The similarity score S

based on minutiae points

between the queried and stored ﬁngerprint images is

calculated using equation (6).

M(M

, M

) =

(

1 i f (E

≤ r

)and(A

≤ θ

)

0 Otherwise

(4)

where r

is allowed the diffrence between M

and M

∑

i=1

∑

j=1

M(M

, M

) (5)

Fscore

(6)

where N

is the total matching of M

and M

. N

and

are total number of M

and M

respectively.

3.2 Fingervein Recognition

3.2.1 Orientation Correction

In this section, the obtained region of ﬁnger vein

images is needed to determine that images are ori-

ented correctly or not. The orientation corrected angle

can affect to accurately extract feature extraction and

matching. Thus, Linear Regression Line is applied to

compute the estimated orientation angle θ represented

in the ﬁgure 2. First, all middle points are represented

the line function of the ﬁnger vein image, which is de-

ﬁned in equation (7). Next, the orientation angle value

is calculated by using equation (8). Finally, these im-

ages are considered normal, if orientation angle value

is equal to 0, otherwise the ﬁnger vein image is not

correctly oriented. The ﬁgure 3 represents the results

of the orientation corrected angle.

Figure 2: Orientation angle detection.

y = ax + b (7)

a =

∑

i=1

− ¯x) − (y

− ¯y)

∑

i=1

− ¯x)

(8)

¯x =

∑

i=1

, ¯y =

∑

i=1

(9)

θ =











−arctan(a) i f (a < 0)

arctan(a) i f (a > 0)

0 i f (a = 0)

(10)

Multimodal Biometric Identiﬁcation System based on Cascaded Advanced of Fingerprint and Finger Vein Images and AND Rule at

Decision Level Fusion

163

where x

= 1,2,3....M, i= 1,2,3....M. Therefore, the ori-

entation angle value θ between the estimted line and

X-axis is calculted using equation (10).

(a) (b)

Figure 3: Orientation correction of ﬁnger vein image : (a)

ﬁnger vein image distortion oriented ; (b) ﬁnger vein image

oriented correctly

3.2.2 ROI Detection

When the ﬁnger vein image is correctly oriented,

the regions of interest will be obtained. This region

has the ridge and lines patterns of the ﬁnger vein

that is exploited for recognition. Canny technique is

the famous edge detector algorithm. It is developed

by (Canny, 1986). For this reason, Canny method

is adopted to extract the ROI of ﬁnger vein image.

Firstly, this technique is applied to obtain the edge

outline of ﬁnger vein image. After that, the inner rect-

angle is used to extract the ROI. The result of ROI of

ﬁnger vein image using Canny edge detector scheme

is shown in Figure 4.

(a) (b)

Figure 4: Illustration of ROI extraction and pre-processing

of ﬁnger vein image : (a) Original image ; (b) Canny

method ; (c) ROI detected ; (d) ROI pre-processing

3.2.3 Pre-processing

After extracted the ROI providing Canny edge detec-

tor, we evaluate the sharpness and contrast of ﬁnger

vein image. Thus, local histogram equalization is ap-

plied for image contrast enhancement (Kim, 2001).

The essential data of ﬁnger vein image can be shown

clearly which is represented in Figure 4.

3.2.4 Feature Extraction Method

HOG (Histogram of Oriented Gradients) descriptor

has shown outstanding success in recognition system.

HOG has been popular used as one of the better fea-

tures to acquire local shape points or the edge. For

this advantage, this technique is applied in our al-

gorithm for feature extraction in order to recognize

the person. The HOG orientation of each cell, small

connected areas, is separated. For better compensat-

ing the illumination, the normalized histogram is ob-

tained by accumulating a measure of the local his-

togram gradient orientation over blocks based on the

results to normalize each cell in the block. These his-

tograms are combined to represent the HOG feature

(Dalal, 2005). The process of extracting the HOG de-

scriptor is illustrated in Figure 5.

Figure 5: Illustration of HOG descriptor extraction

3.2.5 Features Comparison

The generated similarity score based on HOG fea-

tures Before to compute the generated similarity score

Hscore

based on HOG features, Hamming distances

is computed to match scores between the ﬁnger vein

template stored in database and the input test tem-

plate as calculated using equation (11). The similarity

score of HOG is given by equation (12).

∑

i=0

− F

| (11)

where F

and F

are the extracted HOG from the

template in the enrolled database and the input query

ﬁnger vein image respectively.

Hscore

= min(D

) (12)

4 EXPERIMENTS AND RESULTS

The experimental operation platform in this study

is described as follows: the host conﬁguration: CPU

ICCSRE 2018 - International Conference of Computer Science and Renewable Energies

164

Intel Core2 Duo at 2.00 GHz, RAM 3.00 GB, runtime

environment: Microsoft Visual Studio C++ 2013 with

OpenCV library. In order to validate the proposed al-

gorithm, the results have been tested on the VERA

Fingervein Database (Tome, 2015) and the public Fin-

gerprint Veriﬁcation Competition 2004 dataset (Maio

et al., 2004). The performance measure is accuracy

rate as deﬁned by equation (13).

Accuracy =

FAR +FRR

Total

NumAcc

(13)

where FAR (False Acceptance Rate) is the probabil-

ity of unauthorized users that are not recognized over

the total number tested, FRR (False Reject Rate) de-

scribes the percentage of authorized users that are

not recognized falsely to the total number tested and

Total

NumAcc

is the total number access.

Table 1: The accuracy rate for different recognition biomet-

ric system results.

Algorithms Accuracy Rate (%)

Fingerprint using Minutiae 96,93

Fingervein using HOG 97,45

Cascaded Multimodal 99,85

Cascaded Multimodal and And Rule 99,28

Figure 6: Recognition results with comparison of algo-

rithms at the accuracy rate.

Table 1 shows the performance of accuracy

rate based on single biometric system using ﬁnger-

print, ﬁngervein images and the cascaded multimodal

recognition biometric system using ﬁngerprint and

ﬁngervein. Figure 6 shows the ROC curves using dif-

ferent recognition methods. In comparison with sin-

gle biometric system, our proposed algorithm espe-

cially with the cascaded multimodal biometric system

using ﬁngerprint and ﬁngervein shows superior per-

formance in terms of accuracy rate with 99,85% with

where Fingerprint using minutiae points ﬁngervein

using HOG, cascaded multimodal and And Rule give

96,93% ,97,45% and 99,28% respectively. We can

conclude from these results that the cascaded multi-

modal recognition biometric system using ﬁngerprint

and ﬁngervein leads to an improvement in recognition

biometric system performance.

5 CONCLUSIONS

This paper presented multimodal biometric identiﬁ-

cation system based on cascaded advanced of ﬁnger-

print and ﬁnger vein images and AND rule at decision

level fusion in order to achieve accurate recognition

of the person. In ﬁrst step level, the ﬁngerprint im-

age is enhanced based on gabor ﬁlter algorithm, bina-

rized. Moreover it is passed to thinning technique, ex-

tract minutiae points and ﬁnally the matching. If the

matching score is greater than the given ﬁngerprint

threshold then recognition is stopped. Else the second

level is started with ﬁngervein image. The orientation

correction, ROI detection based on canny method and

local histogram equalization to improve the quality of

ﬁngervein image are applied.After that, the important

features are extracted using HOG method. In the next

level, the recognition of the both biometrics sources

are veriﬁed at decision level based on AND rule. The

results have shown that the proposed work performed

better in personal identiﬁcation rate than others based

solely on one algorithm. The proposed method can be

further extended by matching the features with other

metrics.

REFERENCES

Canny, J. (1986). A computational approach to edge de-

tection. IEEE Transactions on pattern analysis and

machine intelligence, (6), pp. 679-698.

Cherrat, E. M., Alaoui, R., Bouzahir, H., and Jenkal, W.

(2017). High density salt-and-pepper noise suppres-

sion using adaptive dual threshold decision based al-

gorithm in ﬁngerprint images. In 2017 Intelligent Sys-

tems and Computer Vision (ISCV), pages 1–4. IEEE.

Cui, F. and Yang, G. (2011). Score level fusion of ﬁnger-

print and ﬁnger vein recognition. Journal of Compu-

tational information systems, 7(16):5723–5731.

Dalal, N. (2005). Histograms of oriented gradients for hu-

man detection. IEEE Computer Society Conference

on, vol. 1, pp. 886-893.

Hong, L., Wan, Y., and Jain, A. (1998). Fingerprint image

enhancement: algorithm and performance evaluation.

IEEE transactions on pattern analysis and machine

intelligence, 20(8):777–789.

Jain, A. and Huang, J. (2004). Integrating independent

components and linear discriminant analysis for gen-

der classiﬁcation. In Sixth IEEE International Con-

ference on Automatic Face and Gesture Recognition,

2004. Proceedings., pages 159–163. IEEE.

Multimodal Biometric Identiﬁcation System based on Cascaded Advanced of Fingerprint and Finger Vein Images and AND Rule at

Decision Level Fusion

165

Jain, A. K., Hong, L., and Kulkarni, Y. (1999). A mul-

timodal biometric system using ﬁngerprint, face and

speech. In 2nd Int’l Conf. AVBPA, volume 10.

Khellat-Kihel, S., Abrishambaf, R., Monteiro, J. L., and

Benyettou, M. (2016). Multimodal fusion of the ﬁn-

ger vein, ﬁngerprint and the ﬁnger-knuckle-print us-

ing kernel ﬁsher analysis. Applied Soft Computing,

42:439–447.

Kim, J. (2001). An advanced contrast enhancement us-

ing partially overlapped sub-block histogram equal-

ization. IEEE transactions on circuits and systems for

video technology, 11(4), pp. 475-484.

Maio, D., Maltoni, D., Cappelli, R., Wayman, J. L., and

Jain, A. K. (2004). Fvc2004: Third ﬁngerprint veri-

ﬁcation competition. In International Conference on

Biometric Authentication, pages 1–7. Springer.

Ross, A. (2003). Information fusion in biometrics. Pattern

recognition letters, 24(13), pp. 2115-2125.

Ross, A. A. and Govindarajan, R. (2005). Feature level

fusion of hand and face biometrics. In Biometric

Technology for Human Identiﬁcation II, volume 5779,

pages 196–205. International Society for Optics and

Photonics.

Singh, S., Gyaourova, A., Bebis, G., and Pavlidis, I. (2004).

Infrared and visible image fusion for face recogni-

tion. In Biometric Technology for Human Identiﬁca-

tion, volume 5404, pages 585–597. International So-

ciety for Optics and Photonics.

Son, B. and Lee, Y. (2005). Biometric authentication sys-

tem using reduced joint feature vector of iris and

face. In International Conference on Audio-and

Video-Based Biometric Person Authentication, pages

513–522. Springer.

Tome, P. (2015). The 1st Competition on Counter Measures

to Finger Vein Spooﬁng Attacks. The 8th IAPR In-

ternational Conference on Biometrics (ICB), pp. 513-

518.

Vishi, K. and Jøsang, A. (2017). A new approach for multi-

biometric fusion based on subjective logic. In Pro-

ceedings of the 1st International Conference on Inter-

net of Things and Machine Learning, page 68. ACM.

Wu, X., Wang, K., Zhang, D., and Qi, N. (2007). Com-

bining left and right irises for personal authentication.

In International Workshop on Energy Minimization

Methods in Computer Vision and Pattern Recognition,

pages 145–152. Springer.

Yang, F. and Ma, B. (2007). A new mixed-mode bio-

metrics information fusion based-on ﬁngerprint, hand-

geometry and palm-print. In Fourth International

Conference on Image and Graphics (ICIG 2007),

pages 689–693. IEEE.

Yang, W. (2018). A ﬁngerprint and ﬁnger-vein based can-

celable multi-biometric system. Pattern Recognition,

78, pp. 242-251.

ICCSRE 2018 - International Conference of Computer Science and Renewable Energies

166