Watermark Embedding and Extraction Scheme Design by Two-stage

Optimization for Illegal Replication Detection of Two-dimensional

Barcodes

Satoshi Ono

, Kentaro Nakai

, Takeru Maehara

and Ryo Ikeda

Department of Information Science and Biomedical Engineering, Graduate School of Science and Engineering,

Kagoshima University, 1-21-40 Korimoto, Kagoshima 890-0065, Japan

Technical Division, Graduate School of Science and Engineering, Kagoshima University,

1-21-40 Korimoto, Kagoshima 890-0065, Japan

Keywords:

Watermark Optimization, Mobile Phone, Two-dimensional Barcode.

Abstract:

Recently, two-dimensional (2D) barcodes displayed on mobile phones are becoming used for authentiﬁcation

such as airplane boarding pass and online payment. Digital watermarking is promising technology to detect

illegal replication or fabrication of such 2D codes. However, due to geometric distortions and/or interferences

between patterns of camera sensors and screen pixels, watermark may not be sufﬁciently extracted from sub-

bands which are used when embedding the watermark. This paper proposes a two-stage optimization method

for watermark embedding and extracting scheme design. The proposed method discriminates frequency sub-

bands between for embedding and extraction, whereas general watermarking schemes extract the watermark

from the same subbands as while embedding. To evaluate actual image deterioration by digital-analogue con-

version with mobile phone screen and camera, the proposed method uses actual mobile phones to obtain real

images of valid and replicated 2D codes. Experimental result have shown that the proposed two-stage op-

timization of watermark embedding and extracting schemes improved watermark performance for 2D code

replication detection.

1 INTRODUCTION

Barcodes are used for object recognition and identiﬁ-

cation in various areas, such as production, logistics,

and commerce. Quick Response (QR) codes, a kind

of two-dimensional barcode (2D code) investigated

by Denso Wave Inc., are currently used in Japan as

a shorthand method of accessing (an ”analogue short-

cut”), for example a URL, an e-mail address, a phone

number, and so on. Most Japanese mobile phones are

equipped with a camera and QR code decoder. By

holding a mobile phone over QR codes printed on pa-

pers, billboards, television screens, or digital signage

monitors, users can decode the QR code and browse

Web sites or send e-mails without typing URLs or e-

mail addresses on their mobile phones.

In recent years, QR codes (and other 2D codes) are

becoming used for authentiﬁcation such as airplane

boarding passes and online payments. In particular,

over the past few years, 2D codes displayed on mobile

phone screens have become increasingly common as

a paperless veriﬁcation. In mobile phone apps such as

“Mobile AMC Application” by All Nippon Airways

and “Passbook” by Apple, the barcode is displayed on

the phone screen and a passenger holds his/her mobile

phone over a barcode reader. Such paperless tickets

and coupons promote reduction of environmental im-

pacts. Indeed, in 2007, the International Air Transport

Association standardized an automatic check-in sys-

tem using a 2D code to reduce expenses and indus-

trial waste by replacing boarding passes using mag-

netic tape with mobile phones.

However, illegal replication or fabrication of 2D

codes has not been considered to date. Fig. 1 shows

an example replication of a 2D code displayed on a

mobile phone screen by other mobile phone’s camera.

The importance of technologies for detecting counter-

feit and copied 2D code is rising, not only to avoid

ﬁnancial losses but also to enhance the security and

safety of social hubs such as airports and train sta-

tions. Although woven patterns are widely used in

paper documents to detect a copy, it is difﬁcult to dis-

tinguish between valid and counterfeit 2D codes dis-

played on mobile phone screens.

201

Ono S., Nakai K., Maehara T. and Ikeda R..

Watermark Embedding and Extraction Scheme Design by Two-stage Optimization for Illegal Replication Detection of Two-dimensional Barcodes.

DOI: 10.5220/0005358102010208

In Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP-2015), pages 201-208

ISBN: 978-989-758-091-8

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

Figure 1: Illegal replication using mobile phones.

LL3 LH3

LH2

HL3 HH3

LH1

HL2 HH2

HL1 HH1

Figure 2: DWT subbands and example DWT output.

Recently, digital watermarking has been widely

used for copyright protection and the detection of im-

age modiﬁcation and attacks. Digital watermarking

techniques can be divided into robust and fragile wa-

termarking. For instance, robust watermarking is used

to protect copyright information even when a water-

marked image is replicated or modiﬁed, while a frag-

ile watermark detects modiﬁcation and tampering at-

tacks by being destroyed by them. Semi-fragile wa-

termarking is used to detect modiﬁcations and attacks

on printed images, whereas fragile watermarking is

used to detect modiﬁcations and attacks by digital

image processing. Compared to robust and fragile

watermarking, little attention has been paid to semi-

fragile watermarking (Rey and Dugelay, 2002; Song

et al., 2001).

Ono et al. proposed a semi-fragile watermark-

ing scheme for color 2D barcodes that detects ille-

gal replication (Ono et al., 2011; Ono et al., 2013;

Ono et al., 2014). This method uses Discrete Wavelet

Transform (DWT) (Kundur and Hatzinakos, 1997;

Kundur and Hatzinakos, 1998), and embeds a wa-

termark image as a high-frequency component in the

oblique direction. Fig. 2 shows frequency subbands

and example output images of three-level DWT; HL,

LH and HH denotes high frequency coefﬁcients (sub-

bands) in horizontal, vertical and diagonal directions,

respectively, and LL is low image subbands. The digit

of subband name corresponds to the DWT level.

Illegal replication of printed images with the

above watermark by photocopiers destroys the wa-

termark because the photocopiers express colors with

dotted or diagonal striped patterns of primary col-

ors, which involve diagonal high frequency compo-

nent. Although the above semi-fragile watermarking

(a) Cover 2D code image (b) Watermarked image

LH1

LH2

LH3

The embedded watermark

The watermark extracted from

subband not used for embedding

LL3

HL3

HL2

HL1

HH2

HH1

Figure 3: Watermark extraction example in which water-

mark is extracted from not only subband used for embed-

ding the watermark (LH2) but also other subband (LH1).

method produces copy-detectable 2D barcodes, the

method is basically designed for barcodes on print

media.

Then, Ono et al. applied this scheme to 2D

code displayed on mobile phone screens, which was

achieved by subband selection and watermarking

strength optimization (Ono et al., 2014). Various

types of ﬂat display panels likely require different

watermarking schemes. Thus, to obtain actual valid

and counterfeit 2D code images, the solutions of this

method are evaluated on actual mobile phones. In ad-

dition, a ﬂexible watermark that can be used on var-

ious mobile phones is achieved by formulating the

watermark design as a multi-objective optimization

problem. Implementing multi-objective optimization

with real mobile phones is an effective design ap-

proach for both semi-fragile as well as inconspicuous

robust watermarks.

However, due to slight geometric distortions and

frequency gaps between screen pixels and camera

imaging sensors, watermark may not be sufﬁciently

extracted from subbands which are used when em-

bedding the watermark. Fig. 3 shows an example of

watermark extraction from a camera-captured image.

In this example, although the watermark is embedded

into LH2, the watermark is extracted from LH1 in ad-

dition to LH2. The above-described shows that ro-

bust watermark extraction should be performed by re-

ferring some additional subbands. However, the pre-

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

202

vious method (Ono et al., 2014) embeds watermark

into multiple subbands, e.g., HL1, HL2, and HH2 are

used at darker regions of a cover image. Therefore,

it is difﬁcult to determine an appropriate subband set

for extracting the watermark when it is embedded to

multiple subbands.

This paper proposes a method for simultaneous

optimization of watermark embedding and extract-

ing schemes. The proposed method discriminates

frequency subbands and strength between when em-

bedding and extraction, whereas general watermark-

ing schemes extract the watermark from the same

subbands as while embedding. Although it is ex-

pected that simultaneous optimization for watermark

embedding and extraction schemes allows to achieve

appropriate semi-fragile watermarking, the optimiza-

tion problem scale becomes larger. Therefore, the

proposed method performs a two-stage optimization,

in which watermarking subbands and strength are

optimized ﬁrst without distinction between during

embedding and extracting, and then subbands and

their strength peculiar to both embedding and ex-

traction are discriminated and optimized simultane-

ously. To evaluate actual image deterioration by

digital-analogue conversion with mobile phone screen

and camera, the proposed method uses actual mobile

phones to obtain valid and replicated 2D code images.

Experimental results have shown that the simul-

taneous optimization of watermark embedding and

extracting schemes improved watermark performance

for replication detection.

2 RELATED WORK

Many researchers of digital image copyright protec-

tion have adopted optimization techniques in their ro-

bust watermark designs. For example, Vahedi et al.

proposed a watermarking approach for color images

(Vahedi et al., 2012) using discrete wavelet trans-

form (DWT) analysis (Kundur and Hatzinakos, 1997;

Kundur and Hatzinakos, 1998). This method op-

timizes the watermark embed levels for subbands,

thereby improving watermark robustness to various

intentional and unintentional attacks while ensuring

a high level of perceptual quality. The objective func-

tion is the linear sum of three objectives: visual qual-

ity, robustness, and amount of embedded informa-

tion. Mingzhi and Yan (Mingzhi et al., 2013) pro-

posed a combined DWT and Discrete Cosine Trans-

form (DCT) scheme (Rao and Yip, 1990), based on a

watermarking scheme optimized by genetic algorithm

(GA) (Goldberg, 1989). In this method, ﬁtness calcu-

lations are performed on images corrupted by attacks,

such as JPEG compression, Gaussian ﬁltering, image

sharpening, and cropping. In the method of Chu et al.,

DWT is used to select appropriate zerotrees that pre-

serve both the cover image quality and the robust em-

bedded watermark (Chu et al., 2008). Huang et al. op-

timized both the watermarked image quality and the

robustness of the extracted watermarks by tabu search

(TS). This approach overcomes channel impairments

while ensuring copyright and ownership protection

(Huang et al., 2011). Another approach is singular

value decomposition and lifting wavelet decomposi-

tion (Loukhaoukha et al., 2014). In this method, the

subjective quality and wavelet analyses are performed

by a just noticeable distortion (JND) model and Sym-

4, respectively.

All the above methods attempt to resist deliber-

ate attacks by applying benchmarks to watermarked

digital media, and the solutions are evaluated by sim-

ulation.

3 THE PROPOSED METHOD

3.1 Overview

This paper proposes a design method for watermark

embedding and extracting scheme in order to distin-

guish genuine 2D codes and their replica. The target

2D codes are displayed on mobile phone screens, and

we assume that replication is performed by captur-

ing the genuine 2D code displayed on a mobile phone

with another mobile phone camera as shown in Fig. 1

. Semi-fragile watermark is necessary to discrimi-

nate between them; the watermark is extracted only

from genuine 2D codes, and the replication, which

is digital-analogue conversion (Ho et al., 2003), de-

stroys the watermark.

The basic idea of the proposed method is as fol-

lows:

1. The proposed method uses 3-level 2D Haar DWT

based watermark.

2. Different frequency subband sets are allowed to

be used during watermark embedding and extract-

ing.

3. Real-coded GA (Eshelman and Schaffer, 1993) is

used as an optimizer.

4. Two-stage optimization is adopted to ﬁnd a good

solution of the target high-dimension problem.

5. Actual mobile phones are used to evaluate semi-

fragileness of designed watermark schemes.

Digital copying of 2D code images can be prevented by

other digital technologies.

WatermarkEmbeddingandExtractionSchemeDesignbyTwo-stageOptimizationforIllegalReplicationDetectionof

Two-dimensionalBarcodes

203

Initialize population

Embed watermark

Calculate fitness

Crossover

Capture valid and

replicated images

Extract watermark

images

Evaluate individuals

(candidate solutions)

Mutation

Selection

Reproduce individuals

(only v

em,b,r

are changed and

ex,b,r

are set to the same as v

em,b,r

in the first stage (first T

generations);

and both v

em,b,r

and v

ex,b,r

are

simultaneously optimized

in the second stage (second T

generations))

Figure 4: Process ﬂow of the proposed method.

Display watermarked

2D code

MP3 replicates 2D code

displayed on MP2

Capture 2D code image

and extract the watermark

Decoder

(Cam2)

Decoder

(Cam1䠅

Design a semi-fragile watermark,

and generate 2D code

Mobile phone

(MP1)

Mobile phone

(MP2)

Mobile

phone (MP3)

Figure 5: Proposed watermarking evaluation system struc-

ture (with one mobile phone model).

The proposed method is essentially based on the

previous work (Ono et al., 2014), and the difference

between the previous work and the proposed method

is 1., 2., and 4. in the above list. The proposed method

realizes the semi-fragileness by subband selection and

strength adjustment; they are distinguished and dif-

ferent subbands and strength levels are used between

when embedding and extracting, whereas the previ-

ous work does not discriminate between them. To

promote the effectiveness of distinct schemes, 3-level

DWT is used in this paper. On the other hand, the dis-

tinct scheme design and 3-level DWT make the prob-

lem much larger than the previous work. Therefore,

the proposed method performs subband selection and

strength adjustment by two-stage optimization to ﬁnd

a good solution within little search cost.

3.2 Process Flow

Fig. 4 shows the process ﬂow of the proposed method

for watermarking scheme design. The proposed

method iterates solution candidates (individuals) gen-

eration and their evaluation according to general pro-

cess ﬂow of GA.

Unlike general non-linear programming algo-

rithms, GA is a multi-point search method in which

many individuals simultaneously look for a global op-

timum and interact with each other. The initial popu-

lation of individuals are generated randomly, and pop-

ulation recombination is performed by genetic opera-

tors; selection, mutation and crossover.

The proposed method evaluates individuals with

actual mobile phones. Fig. 5 shows the system for

individual evaluation. A watermarked image, which

is a phenotype of individual, is generated according

to chromosome of the individual. The above water-

marked image, which is regarded as a valid code, is

displayed on mobile phone MP

and captured by cam-

era Cam

After capturing the valid 2D code image, the valid

code is also displayed on MP

. Then, replication is

performed; the above 2D code displayed on MP

captured by a camera of MP

. The replicated 2D

code image is shown on MP

screen, and it is cap-

tured by camera Cam

. To avoid the inﬂuence of im-

age processing and compression of ﬁle format, MP

directly shows the captured image without recording.

The valid watermarking extraction process performed

by MP

and Cam

evaluates the robustness of the

watermark sufﬁciently to overcome the noise caused

by digital-analogue conversions via the mobile phone

screen and the camera. Moreover, the replication pro-

cess by MP

, MP

, and Cam

evaluates the fragile-

ness of the watermark sufﬁciently to be destroyed by

two digital-analogue conversions between MP3 and

MP2 and between MP2 and Cam

3.3 Optimization

3.3.1 Design Variables

Simultaneous optimization of watermark embedding

and extracting scheme that is a target task of the

proposed method involves both frequency band se-

lection and strength adjustment. Adequate subband

selection and embedding/extracting strength adjust-

ment are indispensable in semi-fragile watermarking.

Therefore, design variables are designed to cope with

both subband selection and strength adjustment. Vari-

able v

p,b,r

is a real value and ranges from 0 to 1, where

p denotes the process, embedding em or extracting ex,

b denotes a subband, and r denotes cover 2D code

region to be embedded. The subband set used in

this study comprises vertical direction subbands LH1,

LH2, LH3, horizontal direction subbands HL1, HL2,

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

204

Figure 6: Implemented evaluation system.

Bright module region

Dark module region

Edge region

Figure 7: Three regions of 2D codes: bright modules, dark

modules, and edge regions.

HL3, diagonal direction subbands HH1, HH2, HH3,

and low frequency subband LL3. In our method, the

image region is divided into three regions according

to the brightness of cover 2D code image; brighter

module region b, darker module region d, and edge

(intermediate bright) region e.

If v

p,b,r

is higher than 0.5, then subband b of image

region r is used to embed or extract the watermark.

The embedding/extracting strength is determined by

the following equation;

p,b,r

{

2 × (v

p,b,r

− 0.5) × L

max

if v

p,b,r

> 0.5

0 otherwise

(1)

If v

p,b,r

is below 0.5, subband b of image region r is

not used for embed/extraction.

The target problem has totally 60 dimensions,

which is considerably larger problem than the prob-

lem with 21 dimensions in the previous work (Ono

et al., 2014).

3.3.2 Objective Function

The proposed method designs semi-fragile water-

mark; the desirable semi-fragile watermark is ex-

tracted only from valid 2D code and not from repli-

cated one. Therefore, semi-fragileness is represented

by the watermark extraction accuracy difference be-

tween valid and replicated 2D codes. On the other

hand, there is a constraint that the cover 2D code of

the valid image must be decoded by a general 2D code

decoder even though the watermark is embedded. Be-

sides, it does not matter whether the cover 2D code of

Table 1: Parameter conﬁguration.

Parameters values

Population size 300

200 generations

Termination condition 200 + 200 generations

Crossover blend crossover (BLX-α)

(Eshelman and Schaffer, 1993)

α = 0.5

Mutation uniform random

Mutation rate 0.05

the replicated image can be decoded by the general

decode or not.

According to the above consideration, objective

function f (I) of individual (candidate solution) I is

deﬁned as follows:

f (I) =BCR(W,W

valid

) − BCR(W,W

replicated

)

− P(Y

valid

) (2)

where W denotes digital watermark image, and W

valid

and W

replicated

are extracted watermark images from

valid and replicated 2D code images, respectively.

BCR denotes the pixel-wise coincidence ratio be-

tween two images. P(Y

valid

) is a penalty function

which is calculated based on error correction usage

ratio of the captured cover 2D code Y

valid

3.4 Two-stage Optimization

As described in Sec. 3.3.1, the target problem in-

volves 60 design variables, in which it is not easy

to ﬁnd the global optimum. In addition, the sub-

bands which are used for embedding watermark are

essentially used when also extracting. Therefore, the

proposed method utilizes two-stage optimization. In

the ﬁrst stage, the embedding and extracting strength

em,b,r

and v

ex,b,r

are optimized without any distinc-

tion as v

em,b,r

= v

ex,b,r

to reduce the search space.

Then, in the second stage, all 60 variables are dis-

criminated and simultaneously optimized. The ﬁrst

stage takes ﬁrst T

generations of the search, and then

the second stage does during T

generations. The best

solution of the ﬁrst stage is used as part of initial solu-

tions of the second stage, resulting in an improvement

of both accuracy and search efﬁciency.

4 EVALUATION

4.1 Experimental Setup

To verify the effectiveness of our method, exper-

iments were conducted with actual mobile phones

(SHARP ISW16SH equipped with 4.6 inch 720 ×

WatermarkEmbeddingandExtractionSchemeDesignbyTwo-stageOptimizationforIllegalReplicationDetectionof

Two-dimensionalBarcodes

205

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0 50 100 150 200 250 300 350 400

Best fitness

Generations

embed (1st)

embed+extract (1st)

embed (2nd) embed+extract (2nd)

embed (3rd) embed+extract (3rd)

embed (mean) embed+extract (mean)

Figure 8: Fitness transitions during two-stage optimization.

1, 280 pixels S-CG LCD and 12 megapixels CMOS

sensor camera), as shown in Fig. 6. The tested cover

2D code and watermark images are shown in Fig. 3(a)

and (b). The watermarked 2D codes were captured

by a camera (Logicool HD Pro Webcam C920, 3

megapixels) to extract watermark.

We used Real-Coded Genetic Algorithm (RCGA)

whose parameters were conﬁgured as shown in Table

1. The ﬁrst-stage took 200 generations to evolve wa-

termark embedding subbands, and then the second-

stage did 200 generations to optimize extracting sub-

bands in addition to embedding ones. The maximum

watermark-embedding strength L

max

was set to 192.

Three individual runs performed in the ﬁrst stage

of the search, and then three individual runs of the

second stage performed. The three best individuals

obtained by the ﬁrst stage search were used as part of

initial population of the second stage search.

4.2 Experimental Results

Fig. 8 shows the transitions of the best individual in

the population during the two-stage optimization. The

dotted lines denote the ﬁtness values of independent

runs, and the solid line describes the averaged value.

Fig. 8 demonstrates that the second-stage search suc-

ceeded in ﬁnding better individuals than the best in-

dividual of the ﬁrst-stage. In addition, the ﬁtness of

the best individual kept increasing even at around 400

generations, whereas the ﬁrst-stage search converged

after 100 generations; longer generation in the second

stage would lead better solution.

The ﬁtness value of the best solution exceeded 0.3,

which was calculated from BCR values of the water-

mark images extracted from genuine and replicated

2D codes that were about 0.8 and 0.5, respectively.

This means that the watermark can be decoded only

from the genuine 2D code when the watermark 2D

image is encoded with some error correctable code

which corrects more than 20 percent of the code.

Figs. 9 and 10 show example watermark schemes

1st

and S

2nd

that correspond to best individuals (so-

lutions) in the ﬁrst- and second-stage, respectively. In

both solutions, watermark images from replicated im-

ages were destroyed and only slight bleeds were left,

whereas genuine successfully kept watermark pat-

terns. As shown in Figs. 9(i) and 10(i), in watermark

images extracted from original 2D codes, the water-

mark extracted by S

2nd

involved less noise (black pix-

els) at white modules of watermark 2D code than that

by S

1st

. In addition, in watermark images extracted

from replicas, S

2nd

successfully removed the horizon-

tal component of watermark better than S

1st

, as shown

in Figs. 9(j) and 10(j).

Table 2 shows the details of S

1st

and S

2nd

whose

outputs were shown in Figs. 9 and 10. Focusing on the

bright region of the cover 2D code image, the scheme

2nd

watermark into HH2, HL1, LH2, and LH3, whereas

the ﬁrst-stage solution S

1st

embedded the watermark

mainly into HL1; and S

2nd

reconstructed the water-

mark from all the highest frequency subbands HH1,

HL1, and LH1. This subband combination allows to

facilitate the watermark destroy by replication at re-

gions white circle modules of the covered 2D code.

In contrast, at dark region, S

2nd

weakly embedded

the watermark into HL1 only. However, the water-

mark was extracted from HH1 and HH3 in addition to

HL1. This is because watermark embedded into HL1

gave rise to irregular but frequent vertical pattern as

shown in Fig. 10(g), and this weak watermark was

easily destroyed by replication.

In the case of edge regions of the cover 2D code

image which were smaller than other regions, S

2nd

used HL1 only similar to the dark region, whereas S

1st

used HH1 and LH1. S

2nd

reconstructed the watermark

from HH1, HH2, HL1, and LH2. Similar tendency to

the dark module regions could be seen in the edge re-

gions.

5 CONCLUSIONS

This paper proposes a method for designing semi-

fragile watermark for detecting 2D code replication.

To realize a watermarking scheme with appropriate

semi-fragileness, we introduced a new optimization

technique as follows:

• Distinguishing subbands/strength levels between

when embedding and extracting the watermark,

which facilitates watermark reconstruction from

more subbands than used for embedding. This

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

206

(a) Watermarked 2D

code

(b) Captured image of

the original 2D code

the replicated 2D

code

(d) Extracted watermark

from the original

(e) Extracted watermark

from the replica

(f) Enlargement of (a) (g) Enlargement of (b) (h) Enlargement of (c) (i) Enlargement of (d) (j) Enlargement of (e)

Figure 9: Example watermark images extracted from orginal and replicated 2D code designed by the ﬁrst-stage optimization

(embedding scheme only).

(a) Watermarked 2D

code

(b) Captured image of

the original 2D code

the replicated 2D

code

(d) Extracted watermark

from the original

(e) Extracted watermark

from the replica

(f) Enlargement of (a) (g) Enlargement of (b) (h) Enlargement of (c) (i) Enlargement of (d) (j) Enlargement of (e)

Figure 10: Example watermark images extracted from orginal and replicated 2D code designed by the second-stage optimiza-

tion for embedding and extraction schemes.

Table 2: Obtained solutions by two-stage optimization.

HH1 HH2 HH3 HL1 HL2 HL3 LH1 LH2 LH3 LL3

Bright First-stage 0.00 0.00 0.01 1.00 0.00 0.00 0.00 0.00 0.00 0.00

region Second- embed 0.00 0.44 0.00 1.00 0.00 0.00 0.00 0.35 0.33 0.00

stage extract 0.99 0.00 0.00 0.94 0.00 0.00 1.00 0.00 0.00 0.00

Dark First-stage 0.54 0.00 1.00 0.36 0.00 0.00 0.00 0.00 0.00 0.00

region Second- embed 0.00 0.00 0.00 0.32 0.00 0.00 0.00 0.00 0.00 0.00

stage extract 0.08 0.00 0.54 0.93 0.00 0.00 0.00 0.00 0.00 0.00

Edge First-stage 0.95 0.00 0.00 0.99 0.00 0.00 0.00 0.00 0.00 0.00

region Second- embed 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00

stage extract 0.77 0.70 0.00 0.94 0.00 0.00 0.00 0.28 0.00 0.00

facilitates watermark extraction from images dis-

played on 2D code screens.

• Two-stage optimization, which allows to optimize

subband selection and watermarking strength in

both embedding and extracting scheme. Distin-

guishing embedding and extracting variables in

the latter half of the search allows to efﬁciently

ﬁnd good solutions.

• Actual mobile phone based solution evaluation in-

stead of simulation, which allows taking various

WatermarkEmbeddingandExtractionSchemeDesignbyTwo-stageOptimizationforIllegalReplicationDetectionof

Two-dimensionalBarcodes

207

factors such as optical transfer function, distor-

tion and various noise models into account with-

out constructing a simulation model.

In future, we plan to apply multi-objective opti-

mization (Ono et al., 2014) to make watermarking

schemes robust against mobile phone screen types.

ACKNOWLEDGEMENTS

Part of this work was supported by SCOPE

(142110001) of Ministry of Internal Affairs and Com-

munications (MIC), Japan, The authors also would

like to thank A-T Communications, Co.,LTD., and

DENSO WAVE Inc.

REFERENCES

Chu, S.-C., Huang, H.-C., Shi, Y., Wu, S.-Y., and Shieh, C.-

S. (2008). Genetic watermarking for zerotree-based

applications. Circuits, Systems & Signal Processing,

27(2):171–182.

Eshelman, L. J. and Schaffer, J. D. (1993). Real-coded

genetic algorithms and interval-schemata. In Foun-

dations of Genetic Algorithms, volume 2, pages 187–

202.

Goldberg, D. E. (1989). Genetic Algorithms in Search, Op-

timization, and Machine Learning. Addison Wesley,

Reading.

Ho, A. T. S., Shen, J., Tan, H. P., and Woon, J.

(2003). Security-printing authentication using digital

watermarking. SPIE International Technical Group

Newsletter on Electronic Imaging, 13.

Huang, H.-C., Chu, S.-C., Pan, J.-S., Huang, C.-Y.,

and Liao, B.-Y. (2011). Tabu search based multi-

watermarks embedding algorithm with multiple de-

scription coding. Information Sciences, 181(16):3379

– 3396.

Kundur, D. and Hatzinakos, D. (1997). A robust digital

image watermarking method using wavelet-based fu-

sion. In 4th IEEE Int’l Conf. Image Processing, pages

544–547.

Kundur, D. and Hatzinakos, D. (1998). Digital watermark-

ing using multiresolution wavelet decomposition. In

Acoustics, Speech and Signal Processing, 1998. Pro-

ceedings of the 1998 IEEE International Conference

on, volume 5, pages 2969 –2972 vol.5.

Loukhaoukha, K., Nabti, M., and Zebbiche, K. (2014).

A robust svd-based image watermarking using a

multi-objective particle swarm optimization. Opto-

Electronics Review, 22(1):45–54.

Mingzhi, C., Yan, L., Yajian, Z., and Min, L. (2013). A

combined dwt and dct watermarking scheme opti-

mized using genetic algorithm. Journal of Multime-

dia, 8(3).

Ono, S., Maehara, T., Nakai, K., Ikeda, R., and Taniguchi,

K. (2014). Semi-fragile watermark design for de-

tecting illegal two-dimensional barcodes by evolution-

ary multi-objective optimization. In Proceedings of

Genetic and Evolutionary Computation Conference

(GECCO) (Companion), pages 175–176.

Ono, S., Maehara, T., Sakaguchi, H., Taniyama, D., Ikeda,

R., and Nakayama, S. (2013). Self-adaptive nich-

ing differential evolution and its application to semi-

fragile watermarking for two-dimensional barcodes

on mobile phone screen. In Proceedings of Genetic

and Evolutionary Computation Conference (GECCO)

(Companion), pages 189–190.

Ono, S., Tsutsumi, M., and Nakayama, S. (2011). A

copy detection method for colored two-dimensional

code using digital watermarking. The IEICE transac-

tions on information and systems (Japanese edetion),

94(12):1971–1974.

Rao, K. R. and Yip, P. (1990). Discrete Cosine Trans-

form: Algorithms, Advantages, Applications. Aca-

demic Press Professional, Inc., San Diego, CA, USA.

Rey, C. and Dugelay, J.-L. (2002). A survey of watermark-

ing algorithms for image authentication. EURASIP

Journal on Applied Signal Processing, 6:613–621.

Song, Y., Liu, R., and Tan, T. (2001). Digital Watermark-

ing for Forgery Detetion in Printed Materials, volume

2195/2001, pages 403–410. Springer Verlag.

Vahedi, E., Zorooﬁ, R. A., and Shiva, M. (2012). To-

ward a new wavelet-based watermarking approach for

color images using bio-inspired optimization princi-

ples. Digital Signal Processing, 22(1):153 – 162.

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

208