Comparison of ART-2 and SOFM Based Neural

Network Veriﬁers

P. Mautner, V. Matousek, T. Marsalek, and O. Rohlik

University of West Bohemia,

Faculty of Applied Sciences,

Univerzitni 8,

CZ – 306 14 Plzen (Pilsen),

Czech Republic

Abstract. The Carpenter-Grosberg ART-2 and Kohonen Self-organizing Feature

Map (SOFM) have been developed for the clustering of input vectors and have

been commonly used as unsupervised learned classiﬁers. In this paper we de-

scribe the use of these neural network models for signature veriﬁcation. The bio-

metric data of all signatures were acquired by a special digital data acquisition

pen and fast wavelet transformation was used for feature extraction. The part of

genuine signature data was used for training both signature veriﬁers. The archi-

tecture of the veriﬁers and achieved results are discussed here and ideas for future

research are also suggested.

1 Introduction

Commercial systems designed for handwritten text acquisition use as an input device

a scanner, a pen with a tablet or a GPS-based pen with a (infrared or ultrasound) trans-

mitter and several receivers (see [11], [13], [14]). Obvious disadvantage of these devices

is the limited mobility of a system composed of two or more pen parts. System based

on optical (OTM) technology require additional light sources (usually built-in inside

the pen) in order to work properly which is intrusive an uncomfortable.

Under the BISP (Biometrical Smart Pen for Personal Identiﬁcation) project several

pen prototypes were constructed. These prototypes integrate all the electronic devices

needed for the data acquisition inside the pen and are ergonomic and non-invasive as

they do not emit light, sound, or electromagnetic radiation and provide a comfortable

feeling while writing.

In our paper, we focused on two famous neural network architectures: the Carpenter-

Grosberg ART-2 and Kohonen Self-organizing Feature Map (SOFM). These networks

can be used as the signature veriﬁers for the data acquisition pen developed under BiSP

project. The ﬁrst experiments with these neural network veriﬁers were published in [1],

the new feature set and results of new tests are presented here. The block scheme of the

developed system using neural network veriﬁer is shown in Figure 1. Short description

This work was supported by the Ministry of Education of the Czech Republic - Project No.

MSM235200005 and the Bilateral Czech-German Research Project No. TSR 007/99

Mautner P., Matousek V., Marsalek T. and Rohlik O. (2004).

Comparison of ART-2 and SOFM Based Neural Network Veriﬁers.

In Proceedings of the First International Workshop on Artiﬁcial Neural Networks: Data Preparation Techniques and Application Development, pages

41-48

DOI: 10.5220/0001149300410048

 SciTePress

of this pen can be found in Section 2, Section 3 deals with the pen output signal fea-

ture extraction method and Section 4 describes both neural network signature veriﬁers.

Results of veriﬁcation experiments, and possible future work are discussed in Section 5

and Section 6, respectively.

Fig. 1. Block Scheme of Signature Veriﬁcation System

2 Data Acquisition Device

As mentioned above, data acquisition is performed by a special electronic pen which

was built at the University of Applied Sciences in Regensburg during the spring 2002

(Figure 2). The pen consists of two pairs of mechanical sensors that measure the hori-

zontal and vertical movements of the ballpoint nib and a pressure sensor that is placed

in the top part of the pen. The pen produces a total of three signals (Figure 3). The upper

signal corresponds to the pressure sensor and the other two correspond to the horizontal

and vertical movements of the pen. The data were acquired while writing the signature

”Dobner” (Figure 3).

Four strain gauge sensors that measure the horizontal and vertical movements of the

pen are located near the pen nib and are placed orthogonally to each other. The signal

produced by the horizontal pair of sensors is called x and the one produced by the

Fig. 2. Digital Data Acquisition Pen

Fig. 3. Signature ”Dobner” and signals produced by the pen (reprinted from [1])

vertical sensors y. Each pair of sensors is connected to a Wheatstone bridge. Therefore

there is only one output signal corresponding to the horizontal movement of the pen (x)

and one corresponding to the vertical movement (y). We cannot provide more detailed

description of the pen because of the patent pending status.

3 Feature Extraction

Before the feature vector is evaluated from the output signals, only the active part of the

signature has to be determined. This is done from the ﬁrst difference of output signal z.

To determine the beginning (or the end) of the signature, the z signal is scanned from

left to right (or conversely) and the ﬁrst difference is evaluated. The beginning (or the

end) of the signature is determined if the value of the ﬁrst difference of signal z exceeds

the threshold value Θ at the ﬁrst occurrence and value of the signal is greater than the

reference value σ. The threshold values Θ and σ are determined according to the type

of the piezoelectric sensor used.

For the extraction of features from the signals, the fast wavelet transform (FWT)

was used. At ﬁrst, each signal of the signature was ﬁltered by an average ﬁlter, after-

wards it was decomposed by FWT and coefﬁcients of a

and d

, for m = 1, 2, ..., 5

were determined [6]. The Daubechies and the Coiﬂet wavelet families were tested for

decomposition, the 5-th order Daubechies wavelet gave the best result. Using of this

wavelet, the following features were evaluated:

• W

energy

- energy values kd

and ka

• W

statistic

- mean values and standard deviation of coefﬁcients d

and a

where d

and a

are the detailed and the approximation coefﬁcients of FWT in scale

m and 5, respectively.

4 Neural Network Signature Veriﬁers

The neural network models are commonly used for processing classiﬁcation problems.

But signature veriﬁcation differs from the general classiﬁcation problem. The goal of

the general classiﬁcation problem is to choose one class from several classes, whereas

the training data contain data from all classes. For our application all the training data

are genuine signatures and we have no data for the class of forgery signatures. This is

the reason why the frequently used supervised learned neural network model such as

multi-layer perceptron are not suitable for the signature veriﬁcation task.

Fig. 4. ART-2 signature veriﬁer (reprinted from [1])

4.1 Architecture of the ART-2 Neural Network Veriﬁer

The adaptive resonance theory (ART), developed by Carpenter and Grossberg, was de-

signed for clustering binary input vectors (ART-1) or continuous-valued input vectors

(ART-2). With regards to the features what we used for description of signals, the ART-

2 model is suitable for signature veriﬁcation. The general architecture and description

of the ART-2 network is not discussed here, for details see [7], [8].

The basic structure of the network veriﬁer is illustrated in Figure 4. The network

consists of two layers of processing elements labelled F

(input and interface units)

and F

(cluster units), each fully interconnected with the others, and supplemental unit

G and R (called gain control unit and reset unit), which are used to control the pro-

cessing of the input data vector and creating of the clusters.

The input and interface layer F

consists of six sub- layers (these are not illustrated

in Figure 4); each sub-layer has the same number of processing units as is the size of

the feature vector. The purpose of these sub-layers is to allow the ART-2 network to

process continuously varying inputs. Moreover, they normalize the components of the

feature vector and suppress the noise. The size of the F

layer (and hidden sub-layers)

was 18 in case of W

energy

features and 36 for W

statistic

features.

The clustering layer F

consists of two processing units only, the former (labelled

A) is active only if the feature vector corresponding to the genuine signature appears at

the input of the network, the latter (labelled F) is active in other cases. More clusters

are not enabled in our application.

4.2 Training and Veriﬁcation

As was mentioned above, only the data for the genuine signature are known. Moreover,

the number of template signatures cannot be too high because the acquisition of a large

training set, e.g. at a bank counter could be boring and unpleasant for the customer.

Hence only 5 signatures were used for the training of the ART neural network in our

application. For these signatures corresponding feature vectors were evaluated and re-

peatedly presented to the input layer of the network (the slow learning mode was used

for ART-2 network training). The parameters of the hidden sub-layers of F

and vigi-

lance parameter ρ were set so that only the unit labelled A of layer F

was active during

the whole training procedure (the network places the template signatures only in one

cluster and adapts the corresponding weights between F

and F

layers). When the

training is completed, the network is prepared for veriﬁcation. The parameters of F

sub-layers are not changed during the veriﬁcation, only the vigilance parameter ρ have

to be set properly to the genuine and the forgery signatures were set to right clusters.

The three methods of setting the vigilance parameter were tested in our work:

• manual setting M : vigilance parameter ρ is set to the ﬁxed value (ρ = 0.98) man-

ually, for all veriﬁcations,

• automatic setting A

: ρ

= min

} i = 1 · · · N

• automatic setting A

: ρ

i = 1 · · · N

In the equations above, N

is a number of training vectors and r

is activation level of

unit R (see Figure 4 and [7], [8] for detailed description and evaluation of r

). The best

results were achieved by automatic setting A

4.3 Architecture of the SOFM Neural Network Veriﬁer

The self-organizing feature map (SOFM) has been developed by Theuvo Kohonen and

it has been described in several research papers and books [3], [4]. The purpose of the

self-organizing feature map is basically to map a continuous high-dimensional space

into discrete space of lower dimension (usually 1 or 2). The principal architecture of

the SOFM is illustrated in Figure 5. The map contains one layer of neurons, arranged

in a two-dimensional grid, and two layers of connections. In the ﬁrst layer of connec-

tions, each element is fully connected (through weights) to all feature vector compo-

nents. Computations are feed-forward in the ﬁrst layer of connection: the network com-

putes the scalar product between the input vector F

and each of the neuron weight

vectors w

i,j

. The second layer of connections acts as a recurrent excitatory/inhibitory

network, the aim of which is to implement the winer-take-all strategy, e.g. only the neu-

ron with the highest activation level is selected and labelled as the best matching unit

(BMU). The weight vector of this neuron then corresponds to the vector which is the

most similar to the input feature vector F

Fig. 5. Principal architecture of SOFM signature veriﬁer

4.4 Training of the Signature Veriﬁer

As in the case of ART-2 network, only 5 signatures were used for training of the SOFM.

For these signatures the corresponding feature vectors were evaluated and repeatedly

presented to the input layer of the network. To train the SOFM, the sequential training

algorithm was used (see [3]). During the training, the output layer of SOFM is being

arranged according to the training data and clusters corresponding to the genuine signa-

tures are created. In the most cases, the genuine signatures are projected to one part of

the two-dimensional grid, the forgeries are then projected to the other parts. After the

training procedure the position of the BMU’s for the genuine signature feature vectors

are recorded and the location threshold l

is evaluated. This threshold is used to de-

cide whether the input feature vector corresponds to the genuine signature or forgery.

In some cases the feature vectors corresponding to the forgery signatures are mapped

to the location of genuine ones, i.e. distance between units signed as BMU’s during the

training process is smaller than location threshold. In this case, the genuine signatures

and forgeries are separated according to the quantization error, i.e. euclidian distance

between BMU’s weight vector and input feature vector F

. Quantization error threshold

is also evaluated during the training procedure.

4.5 Veriﬁcation Process

After the training of the SOFM and setting up corresponding parameters, i.e. location

thresholds l

, quantization error threshold Q

, the neural network is ready for veriﬁca-

tion. The overall veriﬁcation process consist of the following three steps:

1. the veriﬁed signature is scanned by the acquisition pen and corresponding feature

vector is evaluated,

2. the feature vector is passed through the SOFM and the location and quantization

error of BMU is evaluated,

3. the signature is classiﬁed as genuine if the both following rules are true:

min

d(L

, L

) ≤ l

(1)

err

≤ Q

(2)

where T

= { t

| i = 1, 2, 3} is the set of feature vectors t

of the signatures used

for training, L

and L

are the locations of BMU of the i-th training signature and the

unknown signature respectively.

5 Experimental Results

To test the veriﬁer, signatures by 10 authors were taken. For each author, 20 genuine

signatures and 36 skilled forgeries were recorded. The skilled forgeries were written by

three different authors (12 forgeries for each person). Moreover, the signatures of other

authors were used as the random forgeries. Both veriﬁers were also tested by these set

of random forgeries (totally 5040 signatures).

Sometimes the author was not satisﬁed with his/her own signature. The quality of

the signature depended on his/her physical and mental condition. In such a case the sig-

natures were classiﬁed as forgery. For the evaluation of such cases, the authors marked

their genuine signatures by a mark from a 1 - 4 scale (1 is the best form of signature).

For the veriﬁers training, only the ﬁve signatures labelled by mark 1 or 2 were chosen.

In case of ART-2 veriﬁer, the some parameters of input sub-layers have to be set up

(see [7]) before the training process. The FAR (False Accept Ratio) and FRR (False

Reject Ratio) strongly depends on the setting of these parameters. In our application

we set up the parameters experimentally, but next tests have to be performed to ﬁnd

optimal setting.

For the SOFM, different sizes of the output layer, different topologies and differ-

ent number of training steps were also tested. Finally the network size 30 × 30 units

with rectangular topology was chosen as a good compromise between the length of the

network training (100 training epochs) and the achieved results. The neural network

weight vectors w

were initially set up linearly (see [3]).

The results of veriﬁcation process for ART-2 and SOFM veriﬁers are presented in

Table 1.

Table 1. Results of veriﬁcation process

energy

statistic

FAR [%] FAR [%]

veriﬁer forgeries FRR [%] forgeries FRR [%]

skilled random skilled random

ART-2 8 5 14 4 2 12

SOFM 9.5 4.5 11.5 6 4 10

6 Conclusion and Future Work

We have shown the application of two types of artiﬁcial neural networks for signature

veriﬁcation. It can be seen (Tab. 1) that classiﬁcation of genuine and forgery signatures

is reliable if the parameters of networks are trained by a sufﬁcient number of dutifully

prepared training patterns. The achieved FRR and FAR are fully comparable with the

results obtained by standard statistical or structural methods, the wide range testing of

both types of ANNs will be carried out in the future.

In our future work, we plan to focus on the following tasks which could improve

the results of veriﬁcation process:

– including the new valuable features to the feature vector describing signature,

– optimal setting of the parameters of the input and interface layer of ART-2,

– optimal setting of the thresholds l

and Q

of SOFM veriﬁer,

– checking the possibility of the application of other neural networks (supervised or

unsupervised learned).

References

1. Mautner, P., Matousek, V., Rohlik, O., Kempf, J.: Signature Veriﬁcation Using Unsupervised

Learned Neural Networks, Proceeding of the 1st IAPR-TC3 Workshop on Artiﬁcial Neural

Network in Pattern Recognition, Florence, Italy, 2003

2. Plamondon, R., Lorrette, G.: Automatic Signature Veriﬁcations and Writer Identiﬁcation-The

State of the ART, Pattern Recognition, Vol 22, No. 2, 1989

3. Koivisto, L.: Quinquennial Report 1994-1998, Neural Network Research Centre and Labo-

ratory of Computer and Information science,FIN-02015, Finland, 1999

4. Fiesler, E., Beale, R.:Handbook of Neural computation, Oxford University Press, New York,

Oxford, 1997

5. Rohlik, O., et al.:A New Approach to Signature Veriﬁcation: Digital Data Acquisition Pen ,

Neural Network World, Vol. 11, No. 5, pp. 493-501,2001

6. Pitnerr. S, Kamarthi, S. V.: Feature Extraction from Wavelet coefﬁcients for Pattern Recog-

nition Tasks, IEEE Transactions on PAMI, Vol. 21, No. 1, 1999

7. Fausett, L.: Fundamentals of Neural Networks, Prentice-Hall, New Jersey, 1994.

8. Carpenter, G. A., Grossberg, S.: ART-2: Self-organization of Stable Category Recognition

Codes for Analog Input Patterns, Applied Optics, No. 26, pp. 4919-4930, 1987

9. Plamondon, R.: Progress in Automatic Signature Veriﬁcation, World Scientiﬁc Publishing

Co. Pte. Ltd., Singapore, 1994

10. Tseng, L. Y., Huang, T. H.: An on-line Chinese Veriﬁcation Scheme Based on the ART-1

Neural Network, in Int. Joint Conf. on Neural Networks, Baltimore, Maryland, Jun. 1992, pp

624-630

11. I-pen producer homepage,

http://www.inductum.com/

12. OTM technology overview

http://www.omtech.com/

13. E-pen documentation

http://www.mediaforte.com.sg/products/graphic/epen/

14. N-Scribe product documentation

http://www.n-scribe.com/