An Intelligent System for Reconstructing the Ripped-up

Paper-Moneys

Nan-Hsing Chiu

, Chang-En Pu

and Ming-Chang Hsieh

Department of Information Management, Chien Hsin University of Science and Technology,

No.229, Jianxing Rd., Zhongli City, Taoyuan County 32097, Taiwan

Investigation Bureau, Ministry of Justice, No.74, Zhonghua Rd., Xindian Dist., New Taipei City, Taiwan

Keywords: Forensic Science, Decision Support Systems, Particle Swarm Optimization, Ripped-up Paper-Moneys.

Abstract: The paper-moneys may face the problems of shreds in the unexpected accidents or human negligence. The

reconstruction of ripped-up paper-moneys can demonstrate the evidences for decision makers or forensic

examiners in order to exchange the complete paper-moneys. However, the reconstruction of ripped-up

paper-moneys is very difficult on the basis of a lot of shreds with different distance factors that are

measured from neighbouring pieces. How to identify the suitable feature weight for each distance factor is a

critical issue for reconstructing the ripped-up paper-moneys. Particle swarm optimization is a search

algorithm which is successfully adopted for solving many combination optimization problems in many

fields. This study utilizes particle swarm optimization for exploring the proper feature weight for each

distance factor to improve the reconstructed abilities. The proposed approach demonstrates the

automatically reconstructing abilities which enhance the effects and efficiencies on the reconstruction of

ripped-up paper-moneys.

1 INTRODUCTION

The paper-moneys are one of currencies that are

often adopted in the market transactions. The paper-

moneys may face the problem to come to pieces

together with the other documents from the shredder

in the unexpected accidents. The reconstruction of

ripped-up paper-moneys is an important procedure

to show the evidences of the paper-moneys for

forensic examiners or decision makers. A shredder

usually shreds paper-moneys into many small

pieces. The reconstruction of a huge pile of ripped-

up paper-moneys is difficult for experts or

investigators. The reconstruction of ripped-up paper-

moneys is similar to the jigsaw puzzles problem for

reassembling the small pieces.

In 2004, Goldberg et al. proposed a global

approach to solve the problem of jigsaw puzzles

(Goldberg et al., 2004). They tried to apply

automated methods for solving the reconstruction

problems based on computer vision and artificial

intelligence techniques. Their results show that the

proposed approach is able to improve the

reconstruction abilities.

Bock et al. figured out that the majority of jigsaw

puzzles problems have been based on rather specific

shape and colour features, as well as the

relationships that may exist between several jigsaw

puzzle pieces (Bock et al., 2004). That is, the ripped-

up paper-moneys are hard to have the same shape in

comparison with the jigsaw puzzles problems.

Although some progresses have been made in

devising semiautomatic methods for reducing the

complexity of reconstruction problems, many

reconstruction problems still remain difficult or

unresolved (Smet, 2008). In practice, experts or

investigators often still resort to manual

reconstruction procedures or seek expensive

assistance to reconstruct the ripped-up paper-

moneys. They have made little scientific information

available about their methods in reconstruction.

The reconstruction of ripped-up or shredded

paper-moneys is time-consuming without science

approach. Therefore, this study proposes the particle

swarm optimization (PSO) approach to improve the

reconstruction of ripped-up paper-moneys. The PSO

approach is adopted to solve the combination

optimization problems on the basis of ripped-up

paper-moneys for enhancing the efficiencies and

effects of reconstruction.

395

Chiu N., Pu C. and Hsieh M..

An Intelligent System for Reconstructing the Ripped-up Paper-Moneys.

DOI: 10.5220/0004409903950400

In Proceedings of the 15th International Conference on Enterprise Information Systems (ICEIS-2013), pages 395-400

ISBN: 978-989-8565-59-4

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

2 RELATED WORK

Many approached have been proposed for solving

the reconstruction of ripped-up documents or

fragments. In 2002, Papaodysseus et al. presented

the best-first strategy for the global reconstruction of

fragments (Papaodysseus et al., 2002). Their

methods considered the local evaluations in

selecting the most similar matching pair for

reconstruction the fragmented wall paintings. Leitao

and Stolfi proposed a multi-scale method for the

reconstruction of two dimensional fragments (Leitao

and Stolfi, 2002). Their experiments demonstrated

that this technique is useful for matching a very

large number of two dimensional fragments. Zhu et

al. proposed a global approach for reconstructing the

ripped-up documents by first finding candidate

matches from document shreds using curve

matching (Zhu et al., 2008). Their results indicate

that the reconstruction of ripped-up documents is

possibly accomplished automatic up to 50 pieces.

Moghaddam and Cheriet introduced a novel

restoration and reconstruction method for document

images (Moghaddam and Cheriet, 2011). A

modified genetic algorithm search technique is used

to find similar patches on several degraded

document images. Their results are promising that

the proposed method can be easily generalized to

natural images whenever they contain structural

information. Lin & Fan-Chiang presented an image-

based technique for shredded document

reconstruction (Lin and Fan-Chiang, 2012). The

image-based techniques are first used to identify the

shred images with high spatial proximity and

evaluate the similarity between any pair of shreds.

Also, a graph-based algorithm is then used to derive

the best shred sorting result for document

reconstruction. Their results show that the proposed

method has correctly merged the majority of the

shredded document. It can also be adopted to reduce

the workload of a manual document reconstruction

process.

Richter et al. proposed an algorithmic framework

for the reassembly of shredded documents (Richter

et al., 2013). They used a support vector machine

classifier to find pairs of support points between

fragments which are suitable for aligning their

respective fragments. After identifying these points

of attachment, they iteratively aligned all fragments

into groups. Their experiments show that the

proposed algorithm is capable of reassembling pages

consisting of up to 32 pieces. The proposed

algorithm also yielded satisfactory results in the face

of multiple missing pieces.

Richter et al. proposed a graph-based approach to

reassembling manually shredded documents (Richter

et al., 2011). They evaluated a set of constraints that

takes into account shape- and content-based

information of each fragment. In their evaluation,

the results show the effectiveness of the proposed

approach in different scenarios. Perl et al. proposed

optical character recognition approach for strip

shredded document reconstruction (Perl et al., 2011).

The reconstruction methodology recognizes

characters at the stripes’ borders and matches them

subsequently. The optical character recognition

system is exploited for recognizing partially visible

characters by means of local features. Their results

show the ability of matching shredded documents

using the information of cut characters.

Butler et al. presented a visual analytic approach

to reconstructing shredded documents (Butler et al.,

2012). They represented the shredded pieces as time

series and applied nearest neighbour matching

techniques that enable matching both the contours of

shredded pieces as well as the content of shreds

themselves. Their approach combines the

advantages of automated methods and enables

expert input. They believe the need for interesting

problem solving strategies would become

paramount, which will hopefully spur more research

into visual analytic methods.

Deever and Gallagher introduced a semi-

automatic approach for crosscut shredded document

reassembly (Deever and Gallagher, 2012).

Automatic algorithms are proposed for computing

features and ranking potential matches for each

shred. Furthermore, a human-computer interface is

designed to allow semiautomatic assembly of the

shreds using the computed feature and match

information. Their experiments demonstrate that the

proposed approach can successful reconstruct the

multiple shredded documents. It also shows the

effectiveness of the proposed automatic algorithms.

The automatic reconstruction of shreds is a

typical application in the field of computer vision,

pattern recognition and image analysis which can be

approximately viewed as the jigsaw puzzle problem.

Yao & Shao proposed a shape and image merging

technique to solve jigsaw puzzles (Yao & Shao,

2003). They assumed that there exist four corner

points for the canonical jigsaw puzzle pieces, and

the boundary curve can be separated into four edges

at the four corner points. The pieces have smooth

edges with well-defined corners for the jigsaw

puzzles. The majority of the proposed approach has

been based on specific shapes and colour features, as

well as the relationships that may exist between

ICEIS2013-15thInternationalConferenceonEnterpriseInformationSystems

396

several jigsaw puzzle pieces.

The reconstruction of ripped-up paper-moneys is

somewhat similar to the problem of automatic

reconstruction of jigsaw puzzles. Solana et al.

focused on the discussion of feature matching for

solving the document reconstruction (Solana et al.,

2005). They pointed out that the act of ripping often

produces irregularities in the fragment contours.

There are no restrictions on the shapes of the shreds

and corners for ripped-up documents or shredded

paper-moneys. This characteristic is the major part

of differences between jigsaw puzzles and ripped-up

paper-moneys. That is, the proposed approaches for

solving jigsaw puzzles make it difficult to get a

perfect curve matching on the problems of ripped-up

documents or shredded paper-moneys. The manual

or semiautomatic reconstruction of the paper-

moneys is a complex issue. How to figure out the

suitable combinations of ripped-up paper-moneys is

a crucial issue for reducing the reconstruction time.

3 PARTICLE SWARM

OPTIMIZATION

PSO is inspired by the social behaviour of biological

creatures for searching suitable solutions. This kind

of searching behaviour is equivalent to search for

solutions in a real-valued search space that has been

solving combination problems in many fields.

Senthilnath et al. utilized discrete PSO approach for

matching the features on solving the multi-sensor

image registration problem (Senthilnath et al., 2013).

The discrete PSO is adopted to explore the three

corresponding points in the sensed and reference

images using multi-objective optimization of

distance and angle conditions through objective

switching technique. The experiments show that the

proposed technique is able to register the sensed

image with the reference image by matching the

corner points. The discrete PSO approach is also

superior to the other approaches based on the results

obtained in their case studies.

Kashyap and Misra proposed a cost estimation

model based on multi-objective PSO to tune the

parameters of the famous constructive cost model

(Kashyap and Misra, 2013). They have used PSO to

build a suitable model for software cost estimation

by tuning the constructive cost model (COCOMO)

parameters. This cost estimation model is integrated

with quality function deployment (QFD)

methodology to assist decision making in software

designing and development processes for improving

the quality. The combination of PSO and QFD

approach assists the project managers to efficiently

plan the overall software development life cycle of

the software product.

Xue et al. applied multi-objective PSO for

feature selection (Xue et al., 2013). They

investigated two PSO-based multi-objective feature

selection algorithms. The two multi-objective

algorithms are compared with different feature

selection methods on 12 benchmark data sets. Their

experimental results show that the proposed PSO-

based multi-objective algorithms can achieve more

and better feature subsets than other approaches in

most cases.

Lee and Kim utilized the integration of multi-

objective PSO with preference-based sort for solving

the footstep optimization of robots (Lee & Kim,

2013). The proposed approach was applied to the

path following footstep optimization for humanoid

robots and the footsteps optimized for predefined

paths were successfully obtained. Through their

experiments, it is certain that the PSO approach can

be applied to various kinds of real world

applications.

In 2009, Chiu et al. combined PSO and

constraint-based reasoning mechanisms to explore

suitable timetables of customer service department

for the timetables scheduling problems (Chiu et al.,

2009). Their experimental results showed that the

PSO and constraint-based reasoning can overcome

the efficiency and flexibly concern under constraints

in developing workforce timetables. Wang et al.

proposed a hybrid PSO algorithm which employs a

diversity enhancing mechanism and neighbourhood

search strategies to achieve a trade-off between

exploration and exploitation abilities (Wang et al.,

2013). A comprehensive experimental results show

that the proposed approach obtains a promising

performance.

Chiu examined the benefits of improving grey

relational classifier based on the PSO (Chiu, 2009).

The PSO approach was utilized to investigate the

best fit of weights in the grey relational analysis

approach for deriving a classifier with preferred

balance of misclassification rates. The results

presented that the proposed approach provides a

preferred balance of misclassification rates than the

models without using PSO approach.

In 2013, Kaveh et al. utilized PSO method to

solve binary-state multi-objective reliability

redundancy allocation problems (Kaveh et al.,

2013). Statistical analysis was supplied to compare

the performance of their proposed algorithms. The

proposed method showed relative preference in

AnIntelligentSystemforReconstructingtheRipped-upPaper-Moneys

397

comparison with the other competing methods. Chen

et al. developed PSO and neighbourhood search to

obtain the near-optimal solution (Chen et al., 2013).

Their experiments indicate that the developed

approach not only provides good quality solutions

within a reasonable amount of time but also

outperforms the classic branch method.

Chiu proposed an integrated decision networks to

combine the PSO and neural networks in providing

the summarized suggestion (Chiu, 2011). The PSO

approach was used to explore the proper

combinations of suggestions in the integrated

decision networks. The experimental results

demonstrated that this approach is superior to the

other approaches. It also can provide an appropriate

summary for decision makers. The PSO approach

demonstrates its abilities in investigating the suitable

solutions in different fields. Therefore, the PSO

approach is utilized for exploring the suitable

combinations of ripped-up paper-moneys to improve

the reconstruction abilities.

4 METHODOLOGY

Figure 1 shows the mechanism that applies PSO for

reconstructing the ripped-up paper-moneys. The

referenced database includes the digital image of

complete paper-moneys for reference. The domain

experts or investigators have to make sure each

ripped-up paper-money if it is real. They can further

identify the position of ripped-up paper-moneys in

comparison with the referenced database.

Figure 1: The mechanism of reconstruction.

The distance measurement calculates the relative

distances among ripped-up paper-moneys. For

different distance factors, the summarization of

weighted Euclidean distance is the general distance

between pairs of ripped-up paper-moneys. The

nearest distance is the neighbour of the ripped-up

paper-money being reconstruction. The evaluation

procedure evaluates the reconstruction results of

ripped-up paper-moneys. If the reconstruction

results are unacceptable, the PSO is utilized to

explore the feature weights of distance factors. As

the features weights are different, the summarized

distance from pairs of ripped-up paper-moneys may

be different. These repeated procedures produce the

results of finally reconstructed paper-moneys.

The measurements of different distance values

from neighbouring pieces of ripped-up paper-

moneys plays a crucial factor for successful

reconstruction of paper-moneys. The possible

feature weights of distance factors are represented as

a “particle” of X

for PSO. For n distance factors, the

particle has its feature weights of distance factors

=(x

, x

, ... , x

) and velocity v

=(x

, x

, ... , x

While flying in the problem search space, each

particle generates a new solution of feature weights

of distance factors using directed velocity vector.

Each particle modifies its velocity to find a better

feature weight of distance factors by applying its

own flying experience of P

best

and G

best

. The P

best

particle best weights found in its earlier flights, and

the G

best

is global best weights found of the particles

in the population. For n distance factors, the new

distance factors x

and new velocity v

are shown in

Eq. (1) and (2). rand() is a random value in the range

between 0 and 1. C

and C

are positive constants for

regulating the maximum step sizes for the particles

to fly towards P

best

and G

best

, respectively. w is a

constant during searching iteration. During the

iteration, the value of each particle is calculated

using this fitness function of x

new

. The best value of

each x

new

is kept as the local best value. For the best

value of all x

new

, the best value is kept as the G

best

These investigating processes are finished while the

fitness values of distance weights are satisfied.







∗









∗



()∗(











)+ 



∗()∗









)

(1)



















(2)

5 EXPERIMENTS

Two colour copies of complete paper-moneys from

Taiwan are simulated for the reconstruction of

Position

Identification

Evaluation

ReconstructedPaper‐Moneys

Measurethe

Distance

Confirmthe

Neighbouring

Fragments

Ripped‐Up

Paper‐Moneys

Referenced

atabase



PSO

ICEIS2013-15thInternationalConferenceonEnterpriseInformationSystems

398

ripped-up paper-moneys. These two complete paper-

moneys are shredded from a shredder. Figure 2

shows these two paper-moneys after shredding. For

these ripped-up paper-moneys, there are 280 pieces

being reconstruction.

Figure 2: The ripped-up paper-moneys.

One of paper-moneys after reassembling from

these ripped-up paper-moneys is shown in Figure 3.

As the piece of ripped-up paper-moneys may belong

to one or the other paper-moneys, the reconstruction

processes are adopted to explore the combination

optimization. The reconstruction results of these

ripped-up paper-moneys include some of spaces.

These spaces are the pieces of ripped-up paper-

moneys which are hard to identify the correct

positions or damaged from the shredder. As the

reconstruction of paper-moneys are based on the

identification of correct positions of ripped-up

paper-moneys, these results show that the

identification of correct positions for the ripped-up

paper-moneys plays a critical factor for the

reconstruction results.

Figure 3: The reconstructed results.

6 CONCLUSIONS

The reconstruction of ripped-up paper-moneys is

very important for forensic examiners or decision

makers in order to confirm the amount of paper-

moneys are shredded. The reconstruction of ripped-

up paper-moneys is very difficult if there are many

pieces of shreds being reconstructed. The automatic

identification of neighbour pieces from ripped-up

paper-moneys shows a potential approach for

improving the reconstruction abilities. The PSO is a

search algorithm which is utilized for exploring the

proper distance weights for automatic identification

of neighbour pieces in reconstruction of ripped-up

paper-moneys.

The study presents the automatically

reconstruction mechanism that provides a valuable

direction for domain experts or investigators in

processing of ripped-up paper-moneys. We present a

study for an automatic reconstruction of ripped-up

paper-moneys based on PSO approach. We are

encouraged by the results of the present study and

are interested in exploring if the use of different

search approaches or different paper-moneys in

order to validate the reconstruction abilities in the

future.

ACKNOWLEDGEMENTS

We would like to thank National Science Council,

Taiwan, Republic of China supporting this study

under the contract number NSC 101-2410-H-231-

006-.

REFERENCES

Bock, J. D., Smet, P. D., Philips, W., D’Haeyer, J., 2004.

Constructing the topological solution of jigsaw

puzzles. In IEEE Int. Conference on Image Processing

(ICIP 2004). Vol. 3, pp. 2127-2130.

Butler, P., Chakraborty, P., Ramakrishan, N., 2012. The

Deshredder: A visual analytic approach to

reconstructing shredded documents. IEEE Conference

on Visual Analytics Science and Technology (VAST).

pp. 113-122.

Chen, Y. Y., Cheng, C. Y., Wang, L. C., Chen, T. L., 2013.

A hybrid approach based on the variable neighborhood

search and particle swarm optimization for parallel

machine scheduling problems-A case study for solar

cell industry. Int. J. Production Economics. Vol. 141,

pp. 66-78.

Chiu, C., Wu, M. J. J., Tsai, Y. T., Chiu, N. H., Ho, M. S.

H., Shyu, H. J., 2009. Constrain-based particle swarm

AnIntelligentSystemforReconstructingtheRipped-upPaper-Moneys

399

optimization (CBPSO) for call center scheduling.

International Journal of Innovative Computing,

Information and Control. Vol. 5, No. 12A, pp. 4541-

4549.

Chiu, N. H., 2009. An early software quality classification

based on improved grey relational classifier. Expert

Systems with Applications. Vol. 36, pp. 10727-10734.

Chiu, N. H., 2011. Combining techniques for software

quality classification: an integrated decision network

approach. Expert Systems with Applications. Vol. 38,

No. 4, pp. 4618-4625.

Deever, A., Gallagher, A., 2012. Semi-automatic assembly

of real cross-cut shredded documents. 19

IEEE

International Conference on Image Processing (ICIP).

pp. 233-236.

Goldberg, D., Malon, C., Bern, M., 2004. A global

approach to automatic solution of jigsaw puzzles.

Comput. Geom. Theory Appl. Vol. 28, No. 2-3, pp.

165-174.

Kashyap, D., Misra, A. K., 2013. Software cost estimation

using Particle Swarm Optimization in the light of

Quality Function Deployment technique. 2013

International Conference on Computer

Communication and Informatics (ICCCI). pp. 1-8.

Kaveh, K. D., Abtahi, A. R., Tavana, M., 2013. A new

multi-objective particle swarm optimization method

for solving reliability redundancy allocation problems.

Reliability Engineering and System Safety. Vol. 111,

pp. 58-75.

Lee, K. B., Kim, J. H., 2013. Multiobjective Particle

Swarm Optimization with Preference-based Sort and

its Application to Path Following Footstep

Optimization for Humanoid Robots. IEEE

Transactions on Evolutionary Computation. Iss. 99,

pp. 1.

Leitao, H., Stolfi, J., 2002. A multi-scale method for the

reassembly of two dimensional fragmented objects.

IEEE Trans. Pattern Anal. Machine Intell. Vol. 24, pp.

1239-1251.

Lin, H. Y., Fan-Chiang, W. C., 2012. Reconstruction of

shredded document based on image feature matching.

Expert Systems with Applications. Vol. 39, Iss. 3, pp.

3324-3332.

Moghaddam, R. F., Cheriet, M., 2011. Beyond pixels and

regions: A non-local patch means (NLPM) method for

content-level restoration, enhancement, and

reconstruction of degraded document images. Pattern

Recognition. Vol. 44, Iss. 2, pp. 363-374.

Papaodysseus, C., Panagopoulos, T., Exarhos, M.,

Triantafillou, C., Fragoulis, D., Doumas, C., 2002.

Contour-shape based reconstruction of fragmented,

1600 B.C. wall paintings. IEEE Trans. Signal

Processing. Vol. 50, No. 6, pp. 1277-1288.

Perl, J., Diem, M., Kleber, F., Sablatnig, R., 2011. Strip

shredded document reconstruction using optical

character recognition. 4

International Conference on

Imaging for Crime Detection and Prevention (ICDP

2011). pp. 1-6.

Richter, F., Ries, C. X., Cebron, N., Lienhart, R., 2013.

Learning to Reassemble Shredded Documents. IEEE

Transactions on Multimedia. Vol. 15, Iss. 3, pp. 582-

593.

Richter, F., Ries, C. X., Lienhart, R., 2011. A graph

algorithmic framework for the assembly of shredded

documents. IEEE International Conference on

Multimedia and Expo (ICME). pp. 1-6.

Senthilnath, J., Omkar, S. N., Mani, V., Karthikeyan, T.,

2013. Multiobjective Discrete Particle Swarm

Optimization for Multisensor Image Alignment. IEEE

Geoscience and Remote Sensing Letters. pp. 1-5.

Smet, P. D., 2008. Reconstruction of ripped-up documents

using fragment stack analysis procedures. Forensic

Science International. Vol. 176, pp. 124-136.

Solana, C., Justino, E., Oliveira, L. S., Bortolozzi, F.,

2005. Document reconstruction based on feature

matching. Proc. 18th Brazilian Symp. Computer

Graphics and Image Processing. pp. 163-170.

Wang, H., Sun, H., Li, C., Rahnamayan, S., Pan, J., 2013.

Diversity enhanced particle swarm optimization with

neighborhood search. Information Sciences. Vol. 223,

pp. 119-135.

Xue, B., Zhang, M., Browne, W. N., 2013. Particle Swarm

Optimization for Feature Selection in Classification: A

Multi-Objective Approach. IEEE Transactions on

Cybernetics. Iss. 99, pp. 1-16.

Yao, F. H., Shao, G. F., 2003. A shape and image merging

technique to solve jigsaw puzzles. Pattern Recognition

Letters. Vol. 24, No. 12, pp. 1819-1835.

Zhu, L., Zhou, Z., Hu, D., 2008. Globally consistent

reconstruction of ripped-up documents. IEEE

Transactions on Pattern Analysis and Machine

Intelligence. Vol. 30, No. 1.

ICEIS2013-15thInternationalConferenceonEnterpriseInformationSystems

400