EvaBio Platform for the Evaluation Biometric System

Application to the Optimization of the Enrollment Process for Fingerprints Devices

B. Vibert

, Z. Yao

, S. Vernois

, J-M. Le Bars

, C. Charrier

and C. Rosenberger

ENSICAEN, 3UMR 6072 GREYC, F-14032 Caen, France

UNICAEN, 3UMR 6072 GREYC, F-14032 Caen, France

Keywords:

Evaluation of biometric systems, Evaluation platform, Minutiae template selection, Fingerprint quality metric.

Abstract:

Nowadays, when someone wants to make a payment with a smartcard, the user has to enter a pin code to be

identiﬁed. Only biometrics is able to authenticate a user; yet biometric information is sensitive. To ensure

the security and privacy of biometric data, OCC (On-Card-Comparison) has been proposed. This approach

consists in storing biometric data in a secure zone on a smartcard and computing the veriﬁcation decision in

a Secure Element (SE). The purpose of this paper is to propose an evaluation platform for testing biometric

systems such as the analysis of performance and security on biometric OCC. Based on two examples, we

illustrate its different uses in an operationnal context. The ﬁrst example focus on the ”Quality module” which

allows to choose the enrollment by considering the ﬁngerprint quality with one proposed metric. The second

one addresses the minutiae reduction of the ﬁngerprint template when the number of minutiae is higher than

expected by the OCC.

1 INTRODUCTION

Nowaday, biometrics is often used in our daily life,

(passport, border control, smartphone ...). This kind

of applications requires in general the use of large on-

line biometric databases which may cause many se-

curity and privacy problems. In order to avoid these

problems, the secure storage of biometric data and

OCC veriﬁcation are increasingly deployed on a SE

(Secure Element) such as the French passport chip.

The main beneﬁt of this solution is to avoid the trans-

mission of the biometric reference template of the

user. The user has also the control of its own biomet-

ric data stored in the SE. A secure element guarantees

many security issues of the biometric reference (con-

ﬁdentiality, integrity).

The SE is frequently used for several applications

such as border control or face to face bank payment.

Thus, to avoid misused identity for example, it be-

comes very important to deﬁne a general methodol-

ogy for evaluating these embedded systems. Some

standards have been proposed as guidelines of bio-

metric systems (ISO, c; ISO, b) but the deﬁnition of

a certiﬁcation process is not yet done. Our lab has

a strong link with industrial companies and many of

them want to compare and evaluate OCC or sensor

in order to choose the best. We need a platform em-

bedding performance and security tools to analyze ex-

isting biometric systems and to help research in this

area. This platform should be modular, able to pro-

cess embedded biometric systems and strongly con-

nected with existing standards. This is why we pro-

posed in this paper, this evaluation platform of bio-

metric OCC for analyzing its performance and secu-

rity.

The paper is organized as follows. Section 2 is de-

voted to the state-of-the-art of evaluation platform of

biometric systems. Section 3 describes the proposed

platform by emphasing on speciﬁc modules. In Sec-

tion 4, we illustrate the beneﬁt of the proposed plat-

form through two examples of uses cases. We con-

clude and give some perspectives in Section 5.

2 STATE-OF-THE-ART

In the literature, only few platform exist for assess-

ing the performance and security of biometric sys-

tems. We can cite the NIST platform (Grother et al.,

2011), which is used in many research competitions.

It allows researchers or manufacturers to test their

OCC or minutiae extractors, in term of interoperabil-

329

Vibert B., Yao Z., Vernois S., Bars J., Charrier C. and Rosenberger C..

EvaBio Platform for the Evaluation Biometric System - Application to the Optimization of the Enrollment Process for Fingerprints Devices.

DOI: 10.5220/0005244203290335

In Proceedings of the 1st International Conference on Information Systems Security and Privacy (ICISSP-2015), pages 329-335

ISBN: 978-989-758-081-9

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

ity. NIST reports disseminate information on FMR

(False Match Rate) and FNMR (False Non Match

Rate) for each OCC and extractor. The purpose of this

platform is to compare existing algorithms or systems

by a trusted third party.

We can also mention the online FVC-Ongoing

platform (Biolab, 2009) dedicated to algorithms for

ﬁngerprint veriﬁcation (evolution of the FCV com-

petitions). The platform offers multiple databases

grouped into two parts. The ﬁrst one (Fingerprint

Veriﬁcation) quantiﬁes both enrollment and veriﬁca-

tion modules, while the second one (ISO Fingerprint

Matching) quantiﬁes only the veriﬁcation module on

ISO Templates (ISO, a) based on minutiae. Perfor-

mance metrics are: the failure to acquire rate (FTA)

and the failure to enroll rate (FTE), the false non

match rate (FNMR) for a deﬁned false match rate

(FMR) and vice versa, the average enrollment and

veriﬁcation times, the maximum size required to store

the biometric template on the SE, the distribution of

legitimate and impostors users scores and the ROC

curve with the associated equal error rate (EER). The

main drawback of this platform is that it is necessary

to submit the executable or source code of the OCC

algorithm to the online platform which can cause con-

ﬁdentiality issues.

Another platform is actually in development

within the BEAT (Biometric Evaluation And Testing)

European project (Project, 2013). At the end of the

project, a framework to evaluate the performance of

biometric technologies using several metrics and cri-

teria (performance, vulnerabilities, privacy). The goal

of this project is to have a common platform for the

industrial and researchers to evaluate their products

and to have an independent and certiﬁed result with

common criteria. This platform is not yet released

actually and does not focus embedded biometric sys-

tems.

We have seen the main platforms in the state-of-

the-art and we have presented their possibilities and

drawbacks. However, no platform answer to our cri-

teriae (usability, modularity,...). This is why we have

decided to develop our platform we present here.

3 EvaBio PLATFORM

EvaBio platform permits to make the link between in-

dustrial companies and researcher. This strong link

with industrial permits to improve the platform to re-

spond to their needs and also permit to share results

with academics. Figure 1 summarize this idea.

Figure 1: Links between EvaBio platform with industrial

and research.

3.1 General Scheme

The general synopsis of the proposed platform is

given Figure 2. This evolution of our ﬁrst platform

(Vibert et al., 2013) allows us to have more functional

modules such as Sensor, Computing, Quality metrics,

Security for OCC and Audit. The new modules yield

to developers or researchers to have different kinds of

methods to evaluate OCC or to choose a sensor.

3.2 Modules

The platform is composed of different modules with

speciﬁc treatments, and all modules are independent.

This modularity allows us to modify a module with-

out changing the overall operation of the platform.

For example we can quantify the beneﬁt of quality

checking of the ﬁngerprint during the enrollment

process. The platform uses active mechanisms of

communication by event allowing multiple modules

simultaneously access data exchanged between the

client application and the OCC, thus offering ”on the

ﬂy” analysis of results. All the main modules such as

Core, Scenario, Performance, GUI interface

are explained in a previous paper(Vibert et al., 2013).

In this paper, only Sensor, Computing, Evaluation,

Fingerprint quality assessment modules are

described.

The Sensor module is a little platform which

permits to acquire real and fake ﬁngerprint databases

with real ﬁnger and speciﬁc protocols. This module

is used to evaluate the performance of a sensor and

to provide attacks on it. We also went on mortuary

to test if the sensor is able to acquire dead ﬁngerprint

(Vibert et al., 2014). This sensor platform could be

used in input on Core, to acquire in live one or more

ﬁngerprints to compare on an OCC algorithm.

Computing module yields to have a distributed

computation to improve the efﬁciency of the evalua-

tion of OCC. For example, from three OCC on three

smartcards, we are able to run three different tests

in parallel. We divide by three the evaluation time

within a campaign.

ICISSP2015-1stInternationalConferenceonInformationSystemsSecurityandPrivacy

330

Figure 2: General scheme of the EvaBio platform.

Fingerprint quality assessment module is

devoted to the quality metric of ﬁngerprint images

or minutiae templates. Fingerprint quality metric

is an auxiliary solution to guarantee the matching

performance by dropping the bad quality samples

(Grother and Tabassi, 2007) in both the enrollment

and matching sessions. This purpose can be simply

achieved because good ﬁngerprint quality could

provide more precise and reliable features. Obvi-

ously, this is also beneﬁcial to the OCC operations,

especially when it is necessary to consider minutiae

selection. There is a much higher probability that

a minutiae extractor can correctly localize minutiae

points within good quality images than that within

bad quality prints (Chen et al., 2005). Therefore,

a reduced minutiae template can preserve correctly

detected minutiae as much as possible rather than

the spurious points, and the performance could be

ensured as well. The quality metric module in the

platform is combined with a validation component

which allows the user to measure the performance

of variant metrics, which enables making a further

decision to choose an appropriate metric.

Figure 3, presents two examples of an im-

age quality metrics distribution involved by the

Fingerprint quality assessment module : 1)

NFIQ (Tabassi and Wilson, 2005) and 2) GREYC Q

metric. The NFIQ generates ﬁve quality levels from 1

to 5 (Figure 3(a)), where the best quality is indicated

by the lowest value and the maximum level denotes

(a)

(b)

Figure 3: Image quality distribution for NFIQ and Q.

the samples of very poor quality. GREYC Q metric

(El-Abed et al., 2011), estimates the quality of ﬁnger-

print with ﬁve score groups (Figure 3(b)), poor (0-20),

EvaBioPlatformfortheEvaluationBiometricSystem-ApplicationtotheOptimizationoftheEnrollmentProcessfor

FingerprintsDevices

331

bad (20-40), medium (40-60), good (60-80) and very

good (80-100). Such a continuous quality score could

generate a better distribution of sample qualities than

those using only few quality levels. This module is

also a modular unit so that other quality metrics are

also employable in the experiments.

The Evaluation module used metrics commonly

used in the literature and ISO (ISO, c) with more spe-

ciﬁc ones:

• False Match Rate (FMR): it measures how many

times the biometric data of a user provides pos-

itive veriﬁcations with biometric data of another

user.

• False Non Match Rate (FNMR): it measures how

many times the biometric data of a user gives a

negative veriﬁcation of biometric data with the

same user,

• Success rate of Attack: it measures the ratio of

successful attacks (number of positive result over

a number of transactions).

• Measuring Interoperability: it quantiﬁes the ratio

of successful tests when providing an ISO tem-

plate to the OCC.

• ROC Curve: It describes the behavior of the bio-

metric OCC for each value of the decision thresh-

old (from which a test is positive). This implies

that it is possible to obtain the comparison score

from the OCC or to set decision threshold. For

industrial OCCs, this is rarely the case but for re-

search ones, this information is always available.

• Veriﬁcation Time: we measure the time required

to achieve a OCC enrollment or to obtain a veri-

ﬁcation result (after sending the ADPU (Applica-

tion Data Protocol Unit deﬁned in (ISO, d)) to the

SE. It is also possible to generate several statistics

on computation times such as histogram veriﬁca-

tion time, average, minimum or maximum time.

4 EXAMPLE OF USES CASES

In this section, we present experimental results on

a commercial OCC with the selection of enrollment

template when we have the quality of the original im-

age.

4.1 Quality Checking During

Enrollment

In this study, a method which permits to choose an en-

rollment template with the best quality and the max-

imum number of minutiae accepted by the OCC has

been proposed. This approach is tested with NFIQ

and Q metrics, and we obtain a better result than be-

fore only with the selection of enrollment template.

Figure 4, illustrates how we choose a template with-

out quality selection 4(a) and when we use a quality

assessment process 4(b).

(a)

(b)

Figure 4: Selection protocol on the reference template en-

rollment with and without quality checking where one col-

umn corresponds to samples from an individual.

Concerning the protocol, the used biometric data

have been collected in earlier experiments with 39 in-

dividuals. We have made three capture sessions with

two ﬁngers : left and right index ﬁnger, with ﬁve cap-

tures per session per individual. In total, we captured

1170 ﬁngerprint images and ISO Compact Card tem-

plates. On each image, we compute the NFIQ and Q

quality metrics an processed by the evaluation mod-

ule of the EVABIO platform. To choose the reference

templates, we have selected, for each person, the tem-

plate with the best quality value considering the two

metrics and the most number of minutiae under the

maximum size allowed by the OCC.

The performance without template selection is

also computed (Vibert et al., 2013). In Table 1, we

present the results of the comparison on a commer-

cial OCC with or without quality selection.

Table 1: Performance values for each quality metric selec-

tion method.

FMR FNMR

Without selection 0.41% 17.36%

NFIQ selection 0.05% 14.36%

Q selection 0.003% 4.75%

As a conclusion of the performance, there is a

fairly good robustness to imposture for all methods

ICISSP2015-1stInternationalConferenceonInformationSystemsSecurityandPrivacy

332

and the FNMR is relatively good with Q selection but

not with others. We observed quite satisfactory results

without quality selection, in terms of false match rate

compared to those found in (Grother et al., 2011) and

a very good performance with quality selection. The

false non match rate appears too high, however by im-

proving the selection of the enrollment template, we

are able to reduce the FNMR around 10% with NFIQ

selection and an other 10% with Q selection in com-

parison with NFIQ. This experiment shows that the

selection of enrollment template is very important to

achieve good performance.

In the EVABIO platform, a quality metric is used

as ﬁlter to choose the reference template for each user

in the database. We can quantify the gain of opera-

tional quality checking during the enrollment process.

It also helps us to improve quality metrics by consid-

ering on different databases similar experiments de-

tailed in Table 1.

4.2 Minutiae Selection

We also have developed a module to reduce the ISO

Compact Card template when we have too many

minutiae. The truncation method deﬁned in the state-

of-the-art (Grother and Salamon, 2007) has been ini-

tially embedded in the module. To determine if the

truncation is the best method in computation time and

performance, two methods have been tested.

4.2.1 No Selection

The ﬁrst method keeps all minutiae from the initial

template. The performance associated to the initial

template is used as reference for the experimental re-

sults. We expect that reduced templates could lead to

lower performances than the initial template.

4.2.2 Selection by Truncation

This method is based on a simple truncation i.e., we

only keep minutiae from the initial template the ﬁrst

max

minutiae. The efﬁciency of this simple approach

depends on the method used to generate the ﬁnger-

print template. For many commercial biometrics sys-

tems, a ﬁngerprint template is generated with a spe-

ciﬁc method. It can be generated considering minu-

tiae with the ascending locations Y as for example.

In the case where multiple captures have been made,

high quality minutiae (always present in the different

captures as for example) can be placed at the begin-

ning of the template. Selecting the N

max

ﬁrst minutiae

could be in this case a very efﬁcient and simple.

4.2.3 Barycentre Selection

This method based on a pruning mechanism is sim-

ple and fast (few milliseconds). It has been proposed

by the NIST for minutiae selection in (Grother and

Salamon, 2007). It has been shown that minutiae lo-

cated near the core of a ﬁngerprint minutiae are the

most useful ones for the matching process (Weiwe

and Wang, 2002). Given a ﬁngerprint template, the

core location is usually unknown. However, the cen-

troid of minutiae can be a good estimate (when no

other information is available). This minutiae selec-

tion approach tends to only keep minutiae near the

centroid for this reason. We have four steps for the

computation process:

1. Compute the centroid of the minutiae from the ﬁn-

gerprint template (containing N

minutiae);

Centroid = (X

, Y

) =

(

∑

i=1

∑

i=1

) (1)

2. Compute the distance of each minutiae to the cen-

troid;

− X

)

+ (Y

−Y

)

, i = 1 : N

(2)

3. Sort in ascending order minutiae according to the

distance r

, i = 1 : N

;

4. Select the ﬁrst N

max

minutiae.

4.2.4 Performance Evaluation

Concerning the protocol, we have used the

FVC2002DB2 (Maio et al., 2002) database to

illustrate results. This database is composed of 8

ﬁngerprints per person and 100 individuals, with

a total of 800 ﬁngerprints. All minutiae templates

used in the experiments have been extracted using

the NBIS tool, MINDTCT (Watson et al., 2007)

from the NIST. In order to realize the matching of

ﬁngerprint templates, we used a very well known

minutiae matching algorithm proposed in 1997 by

Jain et al (Jain et al., 1997). This method consists of

an alignment stage (translation and rotation estima-

tion between the two templates to compare) and a

matching stage after transformation.

To evaluate the performance of minutiae selec-

tion algorithms, we use the AUC (Area Under the

Curve) metric since it is often considered as global

performance criterion. We use this value to quantify

the efﬁciency of a minutiae selection method. We

compute the AUC value for each selection method

with N

max

varying from 30 to 50 by step of 2. The

Conﬁdence Interval (CI) is also used to weight the

EvaBioPlatformfortheEvaluationBiometricSystem-ApplicationtotheOptimizationoftheEnrollmentProcessfor

FingerprintsDevices

333

Table 2: AUC values for each minutiae selection method for different values of N

max

on FVC2002DB2.

max

30 34 38 42 46 50

No selection (%) 11.2 ± 0.15 11.2 ± 0.15 11.2 ± 0.15 11.2 ± 0.15 11.2 ± 0.15 11.2± 0.15

Truncation (%) 10.2 ± 0.28 9.97 ± 0.2 9.29 ± 0.14 8.93 ± 0.11 9.41 ± 0.07 9.48 ± 0.05

Barycentre (%) 8.73 ± 0.31 9.01 ± 0.2 9.00 ± 0.15 9.26 ± 0.10 9.17 ± 0.07 9.47 ± 0.04

Figure 5: Example of minutiae selection on a ﬁngerprint sample: stars represent selected minutiae by the different methods.

For the barycenter selection approach, the green point represents the estimated CORE point (barycenter of minutiae).

Figure 6: Evolution of the AUC value face to minutiae selection on FVC2002DB2.

results of AUC value.

We show in Figure 5 the results of minutiae

selection using the two tested methods. Selected

minutiae are represented by a red star and others

with a blue circle. We can see with the barycentre

approach, select minutiae are near the estimated

CORE. With the truncation method, we loose the

right part of the template.

Table 2 details the AUC value for each minutiae

selection method for the FVC2002 DB2 database.

On this database, the two selection methods permit to

obtain a better performance (Cf. Figure 3) compared

to the initial template (no selection).

To conclude with this illustration, we observe than

barycenter is better than truncature method which is

even the standard method. This use case illustrate

ICISSP2015-1stInternationalConferenceonInformationSystemsSecurityandPrivacy

334

the interest of the EVABIO platform. The minutiae

selection method can be seen as a preprocessing to

the enrollment process in an operational application.

The platform allows to test in a very convenient way

other selection methods.

5 CONCLUSIONS

In this paper, we have presented the beneﬁts of the

EVABIO biometric evaluation platform, and we have

illustrated with two examples the capability of the

platform. The ﬁrst one quantiﬁes how the use of bio-

metric quality metrics on enrollment template selec-

tion is inﬂuenced performance. The second is a brief

comparative study of ﬁngerprint minutiae selection

algorithms. To conclude, we demonstrate the facil-

ity to obtain results with the proposed platform.

In perspective, we plan to develop new modules

to evaluate the OCC and sensor on smartphone and to

design new attacks on OCC and sensors. We will also

improve the scenario module to propose new tests.

REFERENCES

ISO/IEC 19795-2. information technology - biometric data

interchange format - part 2: Finger Minutiae data,

2004.

ISO/IEC 19795-7. information technology - biometric per-

formance testing and reporting - part 7: testing of on-

card biometric comparison algorithms, 2011.

ISO/IEC 2382-37. Information technology - vocabulary -

part 37: Biometrics, 2012.

ISO/IEC 7816-1 to 15: Identiﬁcation cards - Integrated cir-

cuit(s) cards with contacts(Parts 1 to 15). ISO/IEC,

http://www.iso.org.

Biolab (2009). FVConGoing. https://biolab.csr.unibo.it/

FVCOnGoing.

Chen, Y., Dass, S. C., and Jain, A. K. (2005). Fingerprint

quality indices for predicting authentication perfor-

mance. In Audio-and Video-Based Biometric Person

Authentication, pages 160–170. Springer.

El-Abed, M., Hemery, B., Charrier, C., Rosenberger, C.,

et al. (2011). Evaluation de la qualit

e de donn

ees

biom

etriques. Revue des Nouvelles Technologies de

l’information (RNTI), pages 1–22.

Grother, P. and Salamon, W. (2007). Interoperability of

the ISO/IEC 19794-2 compact card and 10 ISO/IEC

7816-11 match-on-card speciﬁcations 11.

Grother, P., Salamon, W., Watson, C., Indovina, M., and

Flanagan, P. (2011). Minex ii ”performance of ﬁn-

gerprint match-on-card algorithms” phase iv : report

NIST interagency report 7477 (revision ii).

Grother, P. and Tabassi, E. (2007). Performance of biomet-

ric quality measures. Pattern Analysis and Machine

Intelligence, IEEE Transactions on, 29(4):531–543.

Jain, A. K., Hong, L., Pankanti, S., and Bolle, R. (1997).

An identity-authentication system using ﬁngerprints.

Proceedings of the IEEE, 9:1365–1388.

Maio, D., Maltoni, D., Cappelli, R., Wayman, J. L., and

Jain, A. K. (2002). FVC2002: Second ﬁngerprint ver-

iﬁcation competition. In Pattern Recognition, 2002.

Proceedings. 16th International Conference on, vol-

ume 3, pages 811–814. IEEE.

Project, B. (2013). Beat project. https://www.beat-eu.org/.

Tabassi, E. and Wilson, C. L. (2005). A novel approach to

ﬁngerprint image quality. In Image Processing, 2005.

ICIP 2005. IEEE International Conference on, vol-

ume 2, pages II–37. IEEE.

Vibert, B., Lebouteiller, J., Keita, F., Rosenberger, C., et al.

(2014). Biometric sensor and match-on-card evalua-

tion platform. In International Biometric Performance

Testing Conference (IBPC).

Vibert, B., Rosenberger, C., and Ninassi, A. (2013). Secu-

rity and performance evaluation platform of biometric

match on card. In Computer and Information Technol-

ogy (WCCIT), 2013 World Congress on, pages 1–6.

IEEE.

Watson, C. I., Garris, M. D., Tabassi, E., Wilson, C. L., Mc-

cabe, R. M., Janet, S., and Ko, K. (2007). Users guide

to nist biometric image software (nbis). Technical re-

port, NIST.

Weiwe, Z. and Wang, Y. (2002). Core-based structure

matching algorithm of ﬁngerprint veriﬁcation. Inter-

national Conference on Pattern Recognition.

EvaBioPlatformfortheEvaluationBiometricSystem-ApplicationtotheOptimizationoftheEnrollmentProcessfor

FingerprintsDevices

335