OPTICAL FLOW ESTIMATION WITH CONFIDENCE MEASURES

FOR SUPER-RESOLUTION BASED ON RECURSIVE ROBUST

TOTAL LEAST SQUARES

Tobias Schuchert and Fabian Oser

Fraunhofer IOSB, Fraunhoferstr. 1, Karlsruhe, Germany

Keywords:

Optical ﬂow, Motion estimation, Super resolution, Conﬁdence measures.

Abstract:

In this paper we propose a novel optical ﬂow estimation method accompanied by conﬁdence measures. Our

main goal is fast and highly accurate motion estimation in regions where information is available and a con-

ﬁdence measure which identiﬁes these regions. Therefore we extend the structure tensor method to robust

recursive total least squares (RRTLS) and run it on a GPU for real-time processing. Based on a coarse-to-ﬁne

framework we propagate not only the motion estimates to ﬁner scales but also the covariance matrices, which

may be used as conﬁdence measures. Experiments on synthetic data show the beneﬁts of our approach. We ap-

plied the RRTLS framework to a real-time super-resolution method for deforming objects which incorporates

the conﬁdence measures and demonstrates that propagating the covariances through the pyramid improves

super-resolution results.

1 INTRODUCTION

Many methods for motion estimation between two or

more images have been proposed in the last years. In

2007 the Middlebury benchmark (Baker et al., 2011)

was proposed, where to date about 50 motion esti-

mation methods have been compared (including im-

plementations of the well known optical ﬂow meth-

ods of (Horn and Schunck, 1981), and (Lucas and

Kanade, 1981)). The benchmark consists of several

synthetic and real world sequences and handles many

different use cases. Also different masks for disconti-

nuities and untextured regions are available and used

for comparing the algorithms for these special prob-

lems. For more details on optical ﬂow methods we

refer to the surveys in (Barron et al., 1994) and (We-

ickert et al., 2006).

Nevertheless there exist applications where it is

important to know, where an estimation is reliable and

where not. In other applications a conﬁdence measure

may help to improve further processing steps. For ex-

ample (1) a precise estimation of unknown motions,

e.g. cell growth, may be heavily affected by a smooth-

ness constraint, as small important motions may be

over-smoothed or (2) estimation of super-resolution

images from image sequences based on motion esti-

mates may include the reliability of the motion esti-

mates to weight the different image and motion pa-

rameters.

In this paper we present a total least squares (TLS)

framework for optical ﬂow estimation. In order to

cope with discontinuities and large motions we devel-

oped a robust recursive TLS in a coarse-to-ﬁne frame-

work. We estimate motion and the reliability of these

motions on each pyramid step and propagate the es-

timation results and the conﬁdence measures through

the different scales.

The Middlebury benchmark (Baker et al., 2011)

gives an overview of current state-of-the art motion

estimation techniques. Most of these techniques are

based on variational approaches which yield full ﬂow

ﬁelds and high accuracy. Moreover there exist ef-

ﬁcient parallel implementation frameworks for vari-

ational methods making them real-time efﬁcient on

current GPUs

(Werlberger et al., 2009).

Local methods consider only neighbouring im-

age regions and are therefore even more suited for

parallelization. However, only few local methods

are present on the Middlebury benchmark website.

Mostly as reference methods and with unsatisfying

results, although they proved to yield high accuracy

(cmp. (Haussecker and Spies, 1999)). Recently (Senst

et al., 2011) proposed a local method for optical ﬂow

estimation and feature tracking implemented on a

Graphics Processing Unit

463

Schuchert T. and Oser F. (2012).

OPTICAL FLOW ESTIMATION WITH CONFIDENCE MEASURES FOR SUPER-RESOLUTION BASED ON RECURSIVE ROBUST TOTAL LEAST

SQUARES.

In Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods, pages 463-469

DOI: 10.5220/0003785904630469

 SciTePress

GPU. This method is based on ordinary least squares

and does not provide conﬁdence measures.

To the best of our knowledge none of the methods

on the Middlebury website yields special reliability

measures for motion estimates. The publicly avail-

able version of (Werlberger et al., 2009) offers the

possibility to calculate a geometric reliability mea-

surement for each estimate. There the ﬂow ﬁeld is

compared in both directions for inconsistency and the

result is mapped to a probability distribution. Another

method is to compute the inverse of the variational

energy in a local region, to identify regions, where

the energy is still large (Bruhn and Weickert, 2006).

There exist a number of conﬁdence measures for lo-

cal methods, e.g. spatial coherence or corner measure

(cmp. (Haussecker and Spies, 1999)). (Kondermann,

2009) compared different conﬁdence measures for lo-

cal and global methods, but did not ﬁnd a satisfy-

ing one. Therefore Kondermann proposed two conﬁ-

dence measures based on motion statistics from sam-

ple data and a hypothesis test, which yield superior

results compared to the two methods mentioned be-

fore. These measures are applicable to all optical ﬂow

ﬁelds (local/global) afterwards. However, these mea-

sures must be trained before and are therefore no al-

ternatives in our case as learning motion statistics is

expensive and training data must be available.

Based on the structure tensor approach (Foerst-

ner and Guelch, 1987) we developed a novel ro-

bust recursive total least squares framework, which

allows accurate motion estimation in structured re-

gions and computing the covariance matrix of each

estimate. This framework is related to the robust to-

tal least squares method presented by (Bab-Hadiashar

and Suter, 1998), but compared to their method we

(1) combine the structure tensor approach with a ro-

bust function and do not use LMSOD or LMedS for

outlier detection, which speeds up computation with-

out loss of accuracy, (2) use a coarse-to-ﬁne frame-

work to handle large motions, and (3) use the covari-

ance matrix based on the approximation of (Nestares

et al., 2000) as a reliable conﬁdence measure. There-

fore we propagate not only the motion estimates but

also the covariance matrices through the different

pyramid steps, similar to a Kalman ﬁlter approach

(cmp. (Simoncelli, 1999)). In experiments we demon-

strate the applicability of our approach.

In the ﬁrst section of this paper the optical ﬂow

estimation framework is presented. This includes

how the structure tensor approach is combined with

a robust implementation and how it is integrated in

a coarse-to-ﬁne framework based on recursive total

least squares (TLS). Then in Sec. 3 computation and

distribution of covariance matrices on different scales

of the pyramid are shown. Experiments in Sec. 4 show

the accuracy of the optical ﬂow estimator on synthetic

sinusoidal sequences and the Middlebury benchmark

and is compared to the method of (Werlberger et al.,

2009), one of the fastest global estimator, which is

still ranked high in the Middlebury benchmark. Sec-

tion 5 shows application of the new framework for

super-resolution. We show how the covariance is in-

corporated into the super-resolution framework im-

proving the results on real images. A summary and

an outlook on future work follows in Sec. 6.

2 ESTIMATION FRAMEWORK

We present an extended version of the well known

structure tensor method (Foerstner and Guelch, 1987)

for optical ﬂow estimation. On the one hand we inte-

grate a robust function to handle outliers following

(Black and Anandan, 1996) and (Schuchert, 2010)

and on the other hand we include the renormaliza-

tion technique of (Kanatani, 1996) to avoid systematic

bias following (Chojnacki et al., 2001). This modiﬁed

structure tensor method is then embedded in a coarse-

to-ﬁne strategy based on recursive total least squares

(cmp. (Boley et al., 1996)). In the following we ex-

plain these steps in more detail.

The structure tensor approach is based on the

brightness constancy constraint equation (BCCE)

+ g

= g

p = 0 (1)

with image gradients g = (g

)

, optical ﬂow

u = (u

)

and parameter vector p = (p

, p

)

The parameter vector and the optical ﬂow vector are

related by u = 1/p

, p

)

. It is assumed that so-

lution vector

p approximately solves all constraint

equations in the local neighbourhood Λ with size N

and therefore g

p with i ∈ [1,N] only approximately

equals 0. We get

p = e

∀i ∈

{

1,...,N

}

(2)

with errors e

which have to be minimized by the

sought solution

p. We minimize e in a weighted 2-

norm

p = argmin kek

, with

kek

∑

i=1





=: p

(3)

where w

is the weight for the i-th constraint and

where matrix J is called structure tensor. In general

Gaussian weights are used to reduce the inﬂuence of

constraints far away from the center pixel. The addi-

tional constraint |p| = 1 is introduced to avoid the triv-

ial solution p = 0. An eigenvalue analysis yields the

ICPRAM 2012 - International Conference on Pattern Recognition Applications and Methods

464

minimum solution, i.e. the eigenvector to the smallest

eigenvalue of J.

In order to reduce the inﬂuence of outliers in the

solution we introduce a robust function ρ into the es-

timation

p = argmin kek

, with

kek

∑

i=1

ρ(e

,σ) =

∑

i=1



p,σ



(4)

with a variable parameter σ. As robust function we

choose the Lorentzian

Lor

(x,σ) = log



1 +







. (5)

where σ is adapted following (Black and Anandan,

1996). We iteratively solve the structure tensor

method and update the weights according to (4).

(Chojnacki et al., 2001) presented a new deriva-

tion of the renormalization method of Kanatani. We

use this method in our robust total least squares

framework as follows. Starting from the standard TLS

estimation p

T LS

= p

we iteratively solve the general-

ized eigenvalue problem

ξ = λ N

ξ (6)

with p

k+1

= ξ

min

the smallest eigenvector to the

smallest eigenvalue λ

min

. The matrices are given by

∑

i=1

(7)

and

∑

i=1

, (8)

where A is the unsmoothed structure tensor, i.e. J with

= 1 ∀i ∈

{

1,...,N

}

, and C is the covariance matrix

of the image gradients. The iteration is stopped, when

− p

k−1

| < ε where ε is a small ﬁxed threshold. In

case of a translational motion model the covariance

matrix C

is constant in a neighbourhood, so the de-

nominator can be excluded from the sum, which sim-

pliﬁes computation. For further details we refer to

(Chojnacki et al., 2001).

For estimation of large distances a coarse-to-ﬁne

framework based on Gaussian pyramids is used. In

order to propagate the result through the scales we use

recursive TLS following (Boley et al., 1996). Then

the structure tensor on the current scale l is achieved

+ β

l−1

(9)

with recursive weighting factor β.

is the standard

structure tensor of the current scale, but with warped

temporal gradients g

. These temporal gradients are

computed following (Simoncelli, 1999), i.e. the ﬁlters

are sheared in the direction of the ﬂow. The temporal

derivatives g

then include the motion estimate from

the previous scale and therefore

l−1

contains only

the 2× 2 spatial structure tensor of the previous scale.

The last row and column of

l−1

are set to zeros.

3 COVARIANCE ESTIMATION

Following (Nestares et al., 2000) and (Schuchert

et al., 2010) we approximate the error covariance ma-

trices of the optical ﬂow estimates by the inverse of

the Hessian, given by

H =

||p||



S −

||p||







(10)

where S is given by the 2 × 2 spatial structure tensor,

I is an identity matrix of the same size as S. The pa-

rameter γ is proportional to the signal to noise ratio.

We set this parameter for all our experiments to γ = 1.

The Covariance matrix is propagated through the

different scales analogue to the structure tensor using

the recursive technique proposed in (9).

4 EXPERIMENTS

We evaluated our algorithm (RRTLS) on synthetic

sinusoidal sequences and on the Middlebury bench-

mark (Baker et al., 2011). We used 3 × 3 Scharr

ﬁlters (Scharr, 2005) for spatial gradient estimation.

The image pyramid starts by sizes of 20 pixel on the

smaller side and we used a downsampling factor of

0.85 for all sequences. The recursive weighting factor

is set to β = 0.4 for both, the structure tensor and the

covariance matrix.

We compare our results with the Anisotropic-Huber-

L1 method proposed in (Werlberger et al., 2009). This

algorithm is among the best and fastest algorithms

on the Middlebury benchmark and is named AHL1

in the remainder of this paper. AHL1 is based on a

variational framework using a smoothness term in or-

der to estimate motion also in regions where no or

not much data is available. The method is public-

ity available and offers the possibility to calculate the

geometric conﬁdence measure (cmp. Sec. 1). We use

the proposed high accuracy parameters from (Werl-

berger et al., 2009) and leave all other parameters in

the standard conﬁguration for all sequences. We use

two frames for experiments with AHL1 and three im-

ages for RRTLS wherever possible. (Werlberger et al.,

2009) stated that the two frames version of their esti-

mator yields better results, on the Middlebury bench-

mark. However RRTLS produced better results, es-

OPTICAL FLOW ESTIMATION WITH CONFIDENCE MEASURES FOR SUPER-RESOLUTION BASED ON

RECURSIVE ROBUST TOTAL LEAST SQUARES

465

pecially more reliable covariance estimations, if three

frames are used.

4.1 Sinusoidal Sequences

In order to evaluate the performance of our RRTLS

and its covariance estimation, we used two sinusoidal

patterns which move contrary to each other with a non

structured background and a non structured rectan-

gle in the middle (cmp. Fig. 1a). The colour coded

ground truth motion is displayed in Fig. 1b and the

corresponding colour code is shown in Fig. 1g. The

magnitude of the motion is below one pixel. Nev-

ertheless both algorithms are using the coarse-to-ﬁne

framework. The estimates of AHL1 and of RRTLS are

shown in Fig. 1c and e, respectively.

The effect of the smoothness constraint in AHL1 can

be clearly seen as motion is also estimated in regions,

where neither structure nor motion is present. RRTLS

estimates no or only small motion in regions where

no motion is present, however at borders the estimates

are heavily degraded. Figure 1h shows the estimated

trace of the covariances from RRTLS coded with a

grey scale. White means low and black high conﬁ-

dence.

The regions where estimates are highly unreliable are

clearly visible, i.e. white and grey. Also the AHL1

yields a geometric conﬁdence measure. Figures 1d

and f show 50 percent of the best motion estimates

masked by the corresponding conﬁdence measures of

AHL1 and RRTLS. The geometric conﬁdence measure

of AHL1 masked some regions, where no structure is

available, but not all. Moreover It removes regions on

the patterns, where estimates should be reliable. Our

proposed method masks unreliable estimates much

better and keeps most of the regions on the patterns.

We also tested other conﬁdence measures for

structure tensor methods, proposed in (Haussecker

and Spies, 1999), where the corner measures c

yields

best results. The corner measure c

is the difference

between the total coherence measure and the spatial

coherence measure and is computed from the eigen-

values of the structure tensor method.

Table 1 summarizes average endpoint errors

(cmp. (Baker et al., 2011)) for 50% masks by the trace

of the propagated covariance, the trace of the covari-

ance only on the ﬁnest grid, i.e. not propagated, and

the corner measure. Errors are calculated for different

noise standard deviations σ

. The propagated covari-

ance shows to be clearly the most reliable conﬁdence

measure.

a b

c d

e f

g h

Figure 1: Sinusoidal pattern data with noise and estimated

ﬂow and conﬁdence measure. a: one image of the input se-

quence, b: is the true ﬂow (colour coded), c: ﬂow estimates

of AHL1 and d: masked ﬂow estimates of AHL1 using their

conﬁdence mask. e and f are ﬂow estimates of RRTLSand

the masked results with the RRTLSconﬁdence measure. g

and h show the colour code and the estimated covariance

grey scale coded of our algorithm, respectively.

Table 1: Comparison of different conﬁdence measures. Av-

erage endpoint error for the 50% most reliable estimates of

RRTLS on the sinusoidal pattern sequence.

Conﬁdence measure σ

= 0 σ

= 1 σ

= 5

propagated covariance 0.008 0.024 0.075

covariance 0.012 0.179 0.212

corner measure (c

) 0.032 0.161 0.208

4.2 Middlebury Benchmark

We evaluated our algorithm on the well known Mid-

dlebury benchmark (Baker et al., 2011). We are not

interested in achieving overall best results on this

benchmark, but to have best results in regions where

the conﬁdence of our measurement is high, i.e. where

the trace of the estimated covariance matrix is low.

Figure 2 shows the input image of the Urban3 se-

quence (Fig. 2a), the colour coded ground truth ﬂow

(Fig. 2d) and the estimated ﬂows of AHL1 (Fig. 2c)

and RRTLS(Fig. 2e). In Fig. 2d and f only the best

50% of the estimates according to their conﬁdence

ICPRAM 2012 - International Conference on Pattern Recognition Applications and Methods

466

Table 2: Average Endpoint Error on the Middlebury data set using standard masks (disc and untext) and masks based on

conﬁdence measures. Results in brackets are for the 2 frame RRTLS, because only 2 frames are available.

Algorithm Mask Dim Grove2 Grove3 Hyd RW Urban2 Urban3 Venus

AHL1 all 0.175 0.161 0.641 0.202 0.155 0.389 0.721 0.332

RRTLS (sym) all (0.375) 0.207 0.751 0.449 0.271 1.159 0.707 (0.931)

AHL1 disc 0.269 0.473 1.222 0.502 0.528 1.734 2.298 1.098

RRTLS (sym) disc (0.537) 0.562 1.494 1.053 0.758 2.358 2.797 (2.132)

AHL1 untext 0.200 0.138 0.631 0.129 0.137 0.402 0.778 0.350

RRTLS (sym) untext (0.438) 0.201 0.637 0.293 0.257 1.210 0.704 (1.215)

AHL1 95% 0.174 0.123 0.537 0.176 0.126 0.246 0.612 0.278

RRTLS (sym) 95% (0.352) 0.168 0.647 0.405 0.223 0.977 0.660 (0.855)

AHL1 75% 0.176 0.088 0.316 0.115 0.093 0.185 0.403 0.243

RRTLS (sym) 75% (0.262) 0.124 0.337 0.268 0.157 0.643 0.360 (0.599)

AHL1 50% 0.175 0.068 0.171 0.066 0.084 0.138 0.244 0.237

RRTLS (sym) 50% (0.209) 0.105 0.193 0.147 0.125 0.594 0.168 (0.339)

measures are displayed. The same effects as on

the sinusoidal sequence can be seen here. AHL1

over-smooths motion borders, whereas RRTLS yields

highly erroneous estimates in some parts of the im-

age, e.g. lower left border. Nevertheless the covari-

ance of these estimates is high and can be masked out,

whereas wrong estimates of AHL1 are not recognized,

e.g. the over-smoothing in the lower bottom left part

of the image.

a b

c d

e f

Figure 2: One sequence of the Middlebury benchmark. a:

one input image, b: true ﬂow (colour coded), c and e: ﬂow

estimates AHL1 and RRTLS respectively. d and f: masked

estimates using the corresponding conﬁdence masks.

Table 2 shows results on the other Middlebury

benchmarks for both estimators. Additional to the

masks from Middlebury, i.e. disc (discontinuities) and

untext (untextured regions), the masks ﬁlter 90%,

75% and 50% of the best estimates according to the

conﬁdence measure. It can be clearly seen, that AHL1

outperforms our method in almost all cases. Using

conﬁdence masks, our method achieves on almost all

sequences considerable better results and the errors

get near or even below the results of AHL1. This

demonstrates the reliability of our covariance mea-

sure and that our estimator can compete with AHL1

in accuracy, if only 50% of the best estimates are con-

sidered. Moreover our method is more than 3 times

faster than AHL1 on images of size 640 × 480 pixel.

5 APPLICATION

A reliable conﬁdence measure of the motion estima-

tion may be used to improve further processing steps.

Super-resolution from multiple images needs reliable

motion estimation. Most approaches assume con-

stant or afﬁne motions over the whole image. These

assumptions do not hold for deformable surfaces,

i.e. faces. Moreover we have the problem, that not all

facial images in a sequences are achieved from front

view. Some super-resolution techniques therefore ﬁl-

ter incoming images before computation in order to

reject images with large motions or achieved from dif-

ferent view points.

In order to cope with these problems, we integrated

the RRTLS-algorithm into a super-resolution frame-

work, based on the multi-frame super-resolution tech-

nique proposed by (Farsiu et al., 2003). There atmo-

spheric blur is neglected and a blur-warp-model

Farsiu starts with a warp-blur-model but translates it into a

blur-warp-model because of a translational motion assumption.

This is not the case for deformable surfaces like faces. However

(Wang and Qi, 2004) showed, that when motion is estimated on

low-resolution images, the warp-blur-model has a systematic error

and the blur-warp-model may be used.

OPTICAL FLOW ESTIMATION WITH CONFIDENCE MEASURES FOR SUPER-RESOLUTION BASED ON

RECURSIVE ROBUST TOTAL LEAST SQUARES

467

used

= DF

Hx + v

k ∈ [1, N] , (11)

where x is the high resolution image of size

[rM

× rM

], y

are the k low resolution images

of size [M

× M

] and v

is the additive system-

atic noise. The Blurring operator H is the Point-

Spread-Function of the camera (here a 5x5 Gaus-

sian ﬁlter), the warping-operator F

is modelled by a



× r



matrix and the downsampling-

factor D by a



× r



matrix. Using a L

norm we have to ﬁnd the minimum of

x = argmin

∑

k=1

kDF

Hx − y

. (12)

Farsiu estimates the minimum solution in two steps.

First the low-pass ﬁltered high resolution image

z = Hx is determined by data fusion of the low-

resolution images based on motion estimates. The

low-resolution images are registered on the high reso-

lution grid and then ﬁltered by a median for each high

resolution pixel. In the second step the image z is de-

blurred. We use the estimated covariance of RRTLS in

both steps. In step one the data is fused based on the

estimated motion. The median ﬁlter is realized as a

weighted median based on the trace of the covariance

estimates. In the second step the trace of the covari-

ance also inﬂuences the deblurring of the high resolu-

tion image when the single high resolution pixels are

weighted. For further details on the super-resolution

algorithm we refer to (Farsiu et al., 2003).

Figure 3 shows estimation results for a se-

quence of facial images. The low resolution images

(e.g. Fig. 3a) are computed from the high resolution

images (Fig. 3b) by downsampling, smoothing and

addition of noise. The sequence consists of 9 facial

images with small head movements and motion of

the lips (speaking) and 58 images showing parts of

the head or background only. All low-resolution im-

ages are referenced on the low-resolution image of

Fig. 3e. The estimated ﬂow for one low resolution

image with RRTLS is depicted in Fig. 3c and the trace

of the corresponding covariance estimates in Fig. 3d.

Again white deﬁnes high and black low covariances.

Figures 3e and f show the estimated high resolution

images with and without using the covariance infor-

mation. In Fig. 3g and h the difference error between

the estimated super-resolution images and the original

high resolution image is shown for both algorithms.

The difference error is scaled by a factor of 6. Differ-

ences appear mostly in regions where larger motions

are present, i.e. at the sides of the head and in the re-

gion around the mouth. Using covariance informa-

tion for super-resolution highly improves the estima-

tion (cmp. Fig. 3g).

a b

c d

e f

g h

Figure 3: Results for super-resolution. a: One low res-

olution input image, b: high resolution image, c: esti-

mated ﬂow with RRTLS, d: estimated covariance grey

value coded, e and f: high resolution images estimated by

our algorithm with and without using the covariance, re-

spectively, and g and h: difference images between com-

puted super-resolution and high resolution image.

6 SUMMARY

In this paper we presented a novel method for motion

estimation with conﬁdence measures implemented

on GPU. The method is based on robust total-least

squares, embedded in a coarse-to-ﬁne framework us-

ing a recursive algorithm. We have shown that the

estimated covariance matrices which are propagated

through the whole pyramid are well suited as conﬁ-

ICPRAM 2012 - International Conference on Pattern Recognition Applications and Methods

468

dence measure (e.g. the trace of the covariance ma-

trix). Experiments on synthetic sequences showed the

good performance of this approach.

We used the estimated motion and conﬁdence mea-

sures to improve a super-resolution method. How-

ever, motion estimates are still not as accurate as best

estimators in the Middlebury benchmark. In future we

plan for a improvements for large motions, more com-

plex motion models, e.g. afﬁne motion models, and

motions under changing illumination, e.g. by consid-

ering gradient information or estimation of illumina-

tion changes.

REFERENCES

Bab-Hadiashar, A. and Suter, D. (1998). Robust optic ﬂow

computation. International Journal of Computer Vi-

sion, 29(1):59–77.

Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.,

and Szeliski, R. (2011). A database and evaluation

methodology for optical ﬂow. International Journal of

Computer Vision, pages 1–31. 10.1007/s11263-010-

0390-2.

Barron, J. L., Fleet, D. J., and Beauchemin, S. S. (1994).

Performance of optical ﬂow techniques. Int. Journal

of Computer Vision, 12:43–77.

Black, M. and Anandan, P. (1996). The robust estimation of

multiple motions: parametric and piecewise-smooth

ﬂow ﬁelds. In Computer Vision and Image Under-

standing, volume 63(1), pages 75–104.

Boley, D., Steinmetz, E., and Sutherland, K. (1996). Recur-

sive total least squares: An alternative to using the dis-

crete kalman ﬁlter in robot navigation. In Reasoning

with Uncertainty in Robotics, volume 1093 of Lecture

Notes in Computer Science, pages 221–234. Springer

Berlin / Heidelberg. 10.1007/BFb0013963.

Bruhn, A. and Weickert, J. (2006). A conﬁdence measure

for variational optic ﬂow methods. In Properties for

Incomplete data, pages 283–298. Springer.

Chojnacki, W., Brooks, M. J., and Hengel, A. V. D. (2001).

Rationalising the renormalisation method of kanatani.

Journal of Mathematical Imaging and Vision, 14:21–

38.

Farsiu, S., Robinson, D., Elad, M., and Milanfar, P. (2003).

Fast and robust multi-frame super-resolution. IEEE

Transactions on Image Processing, 13:1327–1344.

Foerstner, W. and Guelch, E. (1987). A fast operator for de-

tection and precise location of distinct points, corners

and centers of circular features. In Proceedings of the

ISPRS Intercommission Workshop on Fast Processing

of Photogrammetric Data, pages 281–305.

Haussecker, H. and Spies, H. (1999). Motion. In Handbook

of Computer Vision and Applications, volume 2, chap-

ter 13, pages 309–396. Academic Press, 1 edition.

Horn, B. and Schunck, B. (1981). Determining optical ﬂow.

Artiﬁcial Intelligence, 17:185–204.

Kanatani, K. (1996). Statistical Optimization for Geometric

Computation: Theory and Practice. Elsevier.

Kondermann, C. (2009). Postprocessing and Restoration of

Optical Flows. PhD thesis, Heidelberg University.

Lucas, B. D. and Kanade, T. (1981). An iterative image

registration technique with an application to stereo vi-

sion. In Proceedings of the 7th international joint

conference on Artiﬁcial intelligence - Volume 2, pages

674–679, San Francisco, CA, USA.

Nestares, O., Fleet, D. J., and Heeger, D. J. (2000). Likeli-

hood functions and conﬁdence bounds for Total Least

Squares estimation. In Proc. IEEE Conf. on Computer

Vision and Pattern Recognition (CVPR’2000).

Scharr, H. (2005). Optimal ﬁlters for extended optical ﬂow.

In Complex Motion, 1. Int. Workshop, G

unzburg, Oct.

2004, volume 3417 of Lecture Notes in Computer Sci-

ence, Berlin. Springer Verlag.

Schuchert, T. (2010). Plant leaf motion estimation using

a 5D afﬁne optical ﬂow model. PhD thesis, RWTH

Aachen University.

Schuchert, T., Aach, T., and Scharr, H. (2010). Range ﬂow

in varying illumination: Algorithms and comparisons.

IEEE Transactions on Pattern Analysis and Machine

Intelligence, 32:1646–1658.

Senst, T., Eiselein, V., Evangelio, R. H., and Sikora, T.

(2011). Robust modiﬁed l2 local optical ﬂow esti-

mation and feature tracking. In IEEE Workshop on

Motion and Video Computing, pages 685–690, Kona,

USA.

Simoncelli, E. P. (1999). Bayesian multi-scale differential

optical ﬂow. In Handbook of Computer Vision and

Applications, volume 2, chapter 14, pages 397–422.

Academic Press, San Diego.

Wang, Z. and Qi, F. (2004). On ambiguities in super-

resolution modeling. Signal Processing Letters, IEEE,

11(8):678–681.

Weickert, J., Bruhn, A., Brox, T., and Papenberg, N. (2006).

A survey on variational optic ﬂow methods for small

displacements. In Mathematical Models for Registra-

tion and Applications to Medical Imaging, volume 10

of Mathematics in Industry, pages 103–136. Springer

Berlin Heidelberg.

Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers,

D., and Bischof, H. (2009). Anisotropic huber-l1 opti-

cal ﬂow. In Proceedings of the British Machine Vision

Conference (BMVC), London, UK.

OPTICAL FLOW ESTIMATION WITH CONFIDENCE MEASURES FOR SUPER-RESOLUTION BASED ON

RECURSIVE ROBUST TOTAL LEAST SQUARES

469