New Two-step Motion Estimation using Adjustable Partial Distortion

Advanced Selected Search Point and Early Termination for Two Step Motion

Yonghoon Kim and Jechang Jeong

Department of Electronics and Computer Engineering, Hanyang University,

Haengdangdong, Sungdonggu, Seoul, Republic of Korea

Keywords: Video Coding, Fast Motion Estimation, Two-step Motion Search.

Abstract: In this paper, we proposed an advanced two-step motion estimation using adjustable partial distortion for

fast motion estimation. We improve the two-step search by using relationship between neighboring and

current block. The proposed algorithm is 187 times faster than FS and 2.7 times faster than TS-EPDS

without negligible PSNR degradation. Therefore, it is suitable for real-time video implementation.

1 INTRODUCTION

Motion estimation is a commonly used technique in

video compression, because it can reduce temporal

redundancy between neighboring frames. All the

video standards, including H.263/H.264 and MPEG-

1/2/3 have used motion estimation. Full search (FS)

is considered as optimal block matching algorithm

(BMA), but it has computational complexity

problem which limits practical usage. To reduce the

computational burden, various methods have been

proposed to estimate motion vectors faster while

maintaining the accuracy.

Firstly, many algorithm trying to reduce the

number of search point, such as diamond search

(DS), four-step search (4SS) algorithms, Hexagon-

based search (HEXBS), and Enhanced hexagonal

search (EHS) algorithm.

Secondly, some algorithms aims at reducing the

sum of absolute difference (SAD) computation for

each candidate block by testing only part of pixels in

the block at the cost of coding quality such as

normalized partial distortion search (NPDS), and

Cheung and Po extended the NPDS to an adjustable

PDS (APDS).

Recently, hybrid algorithms have been proposed,

such as dual-halfway stop normalized PDS (DHS-

NPDS), fast motion estimation based on search

range adjustment, two-step edge based PDS (TS-

EPDS), and Adaptive two-step edge based PDS

(ATS-EPDS). These algorithms attempt to combine

search point reduction (or search range adjustment)

with enhanced PDS algorithm. ATS-EPDS shows

best performance on both PSNR and speed among

four algorithms, but two-step based search method

has a problem that causes video quality degradation

on specific sequences.

In this paper, new two-step motion estimation is

proposed not only to solve quality degradation in TS

and ATS methods but also achieve further reduction

of complexity adaptive. It reduces the complexity by

control the search point and enhanced PDS

algorithm.

2 CONVENTIONAL

ALGORITHMS

The most common BMA method for matching two

blocks use the SAD, which gives similar

performance of using mean square error (MSE), but

less computational burden. The SAD of N×N is

defined as:

(, , , )

(, ) ( , ),

SAD x y mx my

xiy j I ximxy jmy

−

=++−++++

∑∑

(1)

where and represent the pixel value in the current

and previous frame, respectively, while (x,y)

represents the coordinates of the upper left corner

Kim Y. and Jeong J..

New Two-step Motion Estimation using Adjustable Partial Distortion Search - Advanced Selected Search Point and Early Termination for Two Step

Motion Search.

DOI: 10.5220/0004066400850088

In Proceedings of the International Conference on Signal Processing and Multimedia Applications and Wireless Information Networks and Systems

(SIGMAP-2012), pages 85-88

ISBN: 978-989-8565-25-9

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

pixel of the current block, and (mx,my) is the

horizontal and vertical candidate component of the

motion vector.

Table 1: Offset from the upper left corner pixel of a block.

P (S

, T

) P (S

, T

)

1 (0,0) 9 (1,0)

2 (2,2) 10 (3,2)

3 (2,0) 11 (0,1)

4 (0,2) 12 (2,3)

5 (1,1) 13 (3,0)

6 (3,3) 14 (1,2)

7 (3,1) 15 (2,1)

8 (1,3) 16 (0,3)

2.1 APDS

The partial distortion search algorithm is terminated

early when the partial distortion is bigger than

minimum SAD, and the pth partial distortion D

M×M sub-block is defined as follow:

(, , , )

(, )( , ),

np pn p p

Dxymxmy

Ixisyjt I ximxty jmys

−

=++++−++++++

∑∑

(2)

where (s

) denotes the pixel coordinates of the

upper left corner pixel of a block.

Table 1 represents the offset of the pth partial

distortion from the upper left corner pixels of a

block. The NPDS divide 16×16 block into 4×4 sub-

blocks, but the APDS has one more step. To find

impossible candidate early, it divides only the first

4×4 sub-block into four 2×2 block. After every

partial distortion is calculated, it is compared with

minimum SAD value which is calculated during

search process.

(1 ) ,nN

αεε

=−⋅+⋅

(3)

mi n

DSAD

⋅>⋅

(4)

The APDS algorithm has adjustable control

parameter α, where N is block size and n is the

number of calculated pixels in the current block. If

the ε=0, then α=n and APDS gives the same

performance as NPDS. In other case k=1, then α=N

and APDS works like PDS. Accordingly, if k is

decreased, then APDS is faster, but there is quality

degradation. In this paper, α is defined as 0.4, which

shows best performance on both speed and quality.

2.2 Adaptive Two-step Search

To find best motion vector quickly, selected search

pattern is used. Adaptive two-step search consist of

two steps; First rough search, and second

concentrated search. figure 1 shows the selected

search pattern for first step, and it has dense center

area and sparse outside area. In case of search range

±16, center area is same, but outside area pattern is

just repeated. Two best motion vectors candidates,

best motion vector which has minimum SAD and

second best motion vector which has second

minimum SAD, is obtained from first step. In

second step, we use search range adjustment (SRA),

and adaptive search range D is defined as:

1 2

if MVS

otherwise

⎧

⎨

⎩

(5)

Where motion vector similarity (MVS) is defined as:

(_ _,_ _) .

MVS

AX cur mv up mv cur mv left mv

−−

(6)

MVS compares the best motion vector from first

step with motion vector of neighboring blocks. If

motion vectors of neighboring blocks are similar to

current best motion vector, then search range around

the best motion vector is decreased and second

motion vector candidate is ignored.

3 PROPOSED ALGORITHM

3.1 Advanced Selected Search Pattern

(ASSP)

In case of search range ±16, selected search point in

first step has 161 point, and we add two points,

which is motion vector of upper and left block, and

the number of maximum search point is 163, and

figure 1 represents the advance selected pattern. The

additional points are calculated right after (0,0) and

then, rest of points are calculated. This algorithm

increases the quality, in terms of PSNR, and reduces

the average complexity.

Figure 1: Selected search pattern and additive search

points.

SIGMAP2012-InternationalConferenceonSignalProcessingandMultimediaApplications

3.2 Early Termination

To reduce complexity, we use early termination in

first step. The early termination threshold T is

defined as follow:

(_min)/8 4

(_min)/4 2 ,

( _ min )/2

pred

SAD if MVM

TSAD ifMVM

SAD otherwise

⎧

⎪

⎨

⎪

⎩

(7)

(_ _ , _ _ ) ,

MVM

AX up mvx up mvy left mvx left mvy

(8)

min ( _ min _ min ) .

pred up left

SAD SAD SAD=+

(9)

It makes first step more quickly without quality

degradation.

3.3 Overall Algorithm

New two-step adjustable partial distortion search

(NTS-APDS) has two part of search step: first rough

search and second concentrated search. Each step is

summarized as follows:

Step 1:

Find the two best motion vectors candidate, with a

minimum SAD and a second minimum SAD, Using

advanced selected search pattern in figure 1 and

APDS. If current SAD_min is less than T, ignore the

rest search point and go to step 2.

Step 2:

If D is 1 then do (a), otherwise do (a) and (b).

(a) Select the best motion vector point as the search

center. Search the points, which are not searched for

in Step 1, within a ±D search area via APDS.

(b) Select the second best motion vector point as the

search center. Search the points, which are not

searched for in Step 1, within a ±D search area via

APDS.

4 EXPERIMENTAL RESULTS

We simulated proposed algorithm using the

luminance component of various CIF (352×288)

sequences. Akiyo, News, Mother&Daughter, and

Hall monitor have small motions compared with

other sequences whereas Coast guard, Mobil, and

Foreman have intermediate motions, and rest of the

sequences (Bus, Football, Stefan) have relatively

high motions. The spiral scanning is used as

searching method, and the search range is ±16 for

16×16 block size.

We have compared performance of our proposed

method with FS, APDS, TS-EPDS, and ATS-EPDS.

In order to compare the objective performances, the

PSNR, speed-up, and operation times are used.

Speed-up is defined as operations_of_FS divided by

operations_of_BMA. The total number of operations

is the sum of addition/subtraction, comparison,

absolution, and 8 time of multiplication/division.

The reason that multiplication has weight is it

requires more computations than others, and shift

operation has same weight as addition.

Figure 2 shows the performance of the speed-up

factor over various algorithms and figure 3 shows

the Encoding times of total frames of each sequence.

The proposed algorithm achieves highest speed-up

and encoding time reduction. From the Figure 2 and

Figure 3, we can see the number of operation and the

encoding time are closely related.

Figure 2: The Speed-up results of the APDS, the TS-

EPDS, the ATS-EPDS, and the proposed method,

respectively.

Figure 3: The Speed-up results of the APDS, the TS-

EPDS, the ATS-EPDS, and the proposed method,

respectively.

Table 2 represents the PSNR performance of

four algorithms. The PSNR performance of APDS

and proposed algorithm show almost same with FS,

NewTwo-stepMotionEstimationusingAdjustablePartialDistortionSearch-AdvancedSelectedSearchPointandEarly

TerminationforTwoStepMotionSearch

Table 2: Experimental result of PSNR performance.

Sequence (frames)

PSNR (difference with FS) [dB]

FS APDS TS-EPDS ATS-EPDS Proposed

1.Akiyo (300) 42.95 42.93 (-0.02) 42.93 (-0.02) 42.92 (-0.03) 42.93 (-0.02)

2.News (300) 36.91 36.87 (-0.04) 36.83 (-0.08) 36.83 (-0.08) 36.81 (-0.10)

3.Mother&daughter (300) 40.54 40.48 (-0.06) 40.51 (-0.02) 40.50 (-0.03) 40.47 (-0.07)

4.Hall monitor (300) 34.83 34.78 (-0.05) 34.78 (-0.05) 34.79 (-0.04) 34.76 (-0.07)

5.Coastguard (300) 30.68 30.68 (0.00) 30.67 (-0.01) 30.70 (0.02) 30.66 (-0.02)

6.Mobile (300) 25.18 25.16 (-0.02) 25.13 (-0.05) 25.13 (-0.05) 25.13 (-0.05)

7.Foreman (300) 32.33 32.34 (0.01) 32.29 (-0.05) 32.31 (-0.02) 32.35 (0.03)

8.Bus (300) 25.65 25.64 (-0.01) 24.93 (-0.72) 24.78 (-0.87) 25.36 (-0.29)

9.Football (150) 27.73 27.70 (-0.03) 27.74 (0.01) 27.83 (0.09) 28.13 (0.40)

10.Stefan (90) 24.92 24.92 (0.00) 24.73 (-0.19) 24.91 (-0.01) 25.34 (0.42)

average 32.17 32.15 (-0.02) 32.05 (-0.12) 32.07 (-0.10) 32.19 (0.02)

but TS-EPDS and ATS-EPDS have some significant

PSNR drop on Bus and Stefan sequences. The

proposed algorithm makes up for weak points, and it

show better quality on most sequences. This PSNR

improvement is from the ASSP by adding two

predicted point to original selected search pattern.

5 CONCLUSIONS

In this paper, we present an new two-step adjustable

partial distortion (NTS-APDS) for fast motion

estimation. By modifying the selected search pattern

and using early termination on two-step motion

estimation, we can extremely accelerate the

convergence speed to the best motion vector. The

experimental results show that the proposed

algorithm complements weak point of original two-

step search, and achieve better performance on both

computation reduction and encoding time than other

algorithms. The proposed algorithm is 187 times

faster than FS and 2.7 times faster than TS-EPDS on

average without video quality degradation.

Therefore, it is suitable for real-time implementation

for high quality digital video applications.

ACKNOWLEDGEMENTS

This research was supported by the MKE (The

Ministry of Knowledge Economy), Korea, under the

ITRC(Information Technology Research Center)

support program supervised by the NIPA(National

IT Industry Promotion Agency)" (NIPA-2012-

H0301-12-1011).

REFERENCES

S. Zhu and K-K. Ma, “A new diamond search algorithm

for fast block matching motion estimation,” IEEE

Trans. Image Processing, col. 92.no. 2, pp. 287-293,

Feb. 2000.

L. M. Po and W. C. Ma, “A novel four step search

algorithm for fast block motion estimation,” IEEE

Trans. Circuits Syst. Video Technol., vol. 6, no. 3, pp.-

313-317, Jun. 1996.

C. Zhu, X. Lin, and L. P. Chau, “Hexagon-based search

pattern for fast block motion estimation,” IEEE Trans.

Circuits Syst. Video Technol., vol. 12, no. 5, pp.

349?355, May 2002.

C. Zhu, X. Lin, and L. P. Chau, “Enhanced Hexagonal

Search for Fast Block Motion Estimation,” IEEE

Trans. Circuits Syst. Video Technol., vol. 14, no. 10,

October 2004.

C. K. Cheung and L. M. Po, “Normalized partial distortion

algorithm for block motion estimation,” IEEE Trans.

Circuits Syst. Video Technol., vol. 10, no. 3, pp. 417-

422, Apr. 2000.

C. K. Cheung and L. M. Po, “Adjustable partial distortion

search algorithm for fast block motion estimation,”

IEEE Trans. Circuits Syst. Video Technol., vol. 13, no.

1, pp. 100–110, Jan. 2003.

X. Yi and N. Ling, “Improved Normalized Partial

Distortion Search with dual-halfway-stop for rapid

block motion estimation,” IEEE Trans. on Multimedia,

vol. 9, no. 5, pp. 995-1003, Aug. 2007.

S. Lee, “Fast motion estimation based on search range

adjustment and matching point decimation,” IET

Image Processing, 4(1), pp.1-10 ,2010.

M. G. Sarwer, and Q. M. Jonathan Wu, "Efficient Two

Step Edge based Partial Distortion Search for Fast

Block Motion Estimation," IEEE Trans. Consumer

Electron., vol. 55, no. 4, pp.2154-2162 Nov. 2009.

Y. Kim and J. Jeong “Adaptive Two-Step Edge-Based

Partial Distortion Search Algorithm using Motion

Vector Prediction,” IEEE Trans. Consumer Electron.,

vol. 57, no. 2, pp.631-637 MAY. 2011.

SIGMAP2012-InternationalConferenceonSignalProcessingandMultimediaApplications