Quality of Experience in 3D Multimedia

Antoine Lavignotte, Christophe Gravier, Julien Subercaze and Jacques Fayolle

Universit´e de Lyon, F-42023, Saint-

Etienne, France

Universit´e de Saint-

Etienne, Jean Monnet, F-42000, Saint-

Etienne, France

T´el´ecom Saint-

Etienne, ´ecole associ´ee de l’Institut T´el´ecom, F-42000, Saint-

Etienne, France

Laboratoire T´el´ecom Claude Chappe (LT2C), F-42000, Saint-

Etienne, France

Keywords:

Quality of Experience, QoE, 3D Video, Subjective Tests, Quality Metrics.

Abstract:

Quality of Experience (QoE) is a key element in the search for rapid growth of three-dimensional video

technology. Research concerning the study of QoE in a 3D environment is on the move. This paper proposes a

review of 3D QoE speciﬁcities. We present the actual ﬁndings in this area as well as a national and international

leading projects. Thus, any researcher wishing to know more about this domain will be able to start with the

foundations necessary to understand the issues related to 3D.

1 INTRODUCTION

A lot of technologies and services which contain HCI

(Human Computer Interface) can be deployed due to

their ready-to-use technology. Most of them struggle

to emerge because they fall short of users’ expecta-

tions.

The users’ perception (QoE) is a key element to

take into account during the development of a new

technology or service, especially in Multimedia (Jain,

2005). ITU-T (International Telecommunication

Union) deﬁnes QoE by:

-“The overall acceptability of an application or

service, as perceived subjectively by the end-user”

(ITU-T.StudyGroup12, 2009).

This deﬁnition contains two additional notes:

- “Quality of Experience includes the complete

end-to-end system effects (client, terminal, network,

services infrastructure, etc)”.

- “Overall acceptability may be inﬂuenced by

users’ expectations and context”.

A more general notion of QoE is proposed by

Alben to deﬁne all the different interaction aspects

between a product and a person (Alben, 1996).

Unlike Quality of Services (QoS), QoE is a vector of

subjectivity, expressed or bound to a given context.

QoE has been studied for many years in 2D mul-

timedia. Recent research is focusing on aspects of

QoE dedicated to 3D multimedia. This is due to the

relatively moderate user adoption of 3D technologies,

although more and more TV manufacturer offers this

feature in their product range. An increased adoption

will strongly depend on its impact on the user. 3D will

has therefore to signiﬁcantly enhance the user experi-

ence in a multimedia broadcast. Taking into account

QoE notion during 3D service development will be a

crucial step to its durability.

The rest of this paper is organized as follows. Sec-

tion 2 describes the different approaches to improve

the QoE measured by a user during a movie diffusion.

In Section 3, a review of the existing works on QoE

associated with 3D is provided. Section 4 presents

the various national or international projects whose

works concerns the study of the QoE associated with

3D. Finally, Section 5 concludes.

2 QoE APPROACHES

Several studies have been carried out on QoE. We can

actually identify two principal approaches to evaluate

QoE:

The objective methods and the subjective methods.

2.1 The Objective Evaluation

The objective evaluation is based on objective metrics

associated with the ﬁnal service. There are two main

119

Lavignotte A., Gravier C., Subercaze J. and Fayolle J..

Quality of Experience in 3D Multimedia.

DOI: 10.5220/0004448001190125

In Proceedings of the 15th International Conference on Enterprise Information Systems (ICEIS-2013), pages 119-125

ISBN: 978-989-8565-61-7

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

families of studies: Feature-based and Model-based

evaluations.

The Feature-based method consist of studying the

impact of video’s degradation by signal processing

methods. The most often used metrics are MSE

(Mean Squared Error) and PSNR (Peak Signal to

Noise Ratio) (Zepernick, 2007), (Winkler and Mo-

handas, 2008), (Khan et al., 2009). These metrics

are widely used due to their ease of computation.

However, (Sheikh, 2006) and (Winkler, 2007) demon-

strate a moderate correlation between PSNR and hu-

man perception.

Unlike Feature based method, Model based family

recreates the human perception and study the impact

of artifacts and degradations on a subject.

Another type of model-based method consists in

focusing on altering effects produced during the con-

tent transport. This approach traditionally focuses

on QoS parameters (Varga et al., 2006), (T Rahrer,

2006), (Claypool and Tanner, 1999). This view ig-

nores the most important part of QoE: the end user.

2.2 Subjective Evaluation

Subjective evaluation is usually based on tests with a

group of users who need to grade different conﬁgu-

rations of a service, pictures or movies. In scientiﬁc

and industrial ﬁelds, subjective evaluation is the most

direct way to evaluate human perception of a service

and allows one to optimize 3DTV systems. Several

tests using this method have been established during

the last few decades (Chen et al., 2009), (Lee et al.,

2010), (Lee et al., 2011).

This kind of evaluation can be divided into two

groups: psycho-perceptual and user-centered ap-

proaches.

The psycho-perceptual approach needs a con-

trolled laboratory environment and study the relation

between physical stimuli and sensorial experience.

Concerning 2D subjective video quality assessment

methodologies, the ITU-R BT.500 (ITU-R, 2002) rec-

ommendation is widely used for assessing television

pictures quality. Another very popular method in 2D

is subjective assessment methodology for video qual-

ity (SAMVIQ) (Brotherton et al., 2006).

The user-centered approach is quite different due

to the perspective chosen to conduct this kind of stud-

ies. We need to put ourselves in the user’s place rather

than that of the system. This is achievedby taking into

account typical users, required system characteristics,

context of potential usage scenario and goal-related

evaluation tasks.

The ﬁgure 1 depicts the hierarchy of QoE as-

sessment methods. A more complete description of

QoE

Methods

Subjective

tests

Objective

tests

Psycho

Perceptual

User

Centered

Model

Based

Feature

Based

Methods:

ITU-R.500

SAMVIQ

Methods:

PSNR

MSE

QoS parameters

Figure 1: QoE assessment methods.

the different tests can be found in (Goldmann and

Ebrahimi, 2010).

In the following part, we report ﬁndings on QoE

3D that subjective evaluations have generated.

3 FINDINGS IN 3D QOE

One may assume that 3D is just a sum of a 2D tech-

nology and the depth added to the pictures. If this

were the case, one could simply extends existing 2D

QoE assessment methods to take depth into account.

Unfortunately, as Goldman states in (Goldmann and

Ebrahimi, 2010), it is not so simple.

3.1 The 3D Content Visualization

Criteria

The third dimension tries to imitate the human visual

system. Unfortunately, each person’s 3D perception

system is unique. Unlike other technological leaps,

such as the transition from black and white to color

where everybody was in agreement on the improve-

ment provided by this new technology, for 3D, it is

completely different. Some viewers complain about

visual discomfort, fatigue, eye strain, problems of vi-

sual quality and distortions in 3DTV broadcasting, . ..

In order to understand the cause of these problems, we

must deﬁne the different criteria that play a role in the

perceived quality by a user.

As stated in (Bulat et al., 2010), we can ﬁnd four

main criteria in the literature: image quality, natural-

ness, viewing experience, and depth perception. The

ICEIS2013-15thInternationalConferenceonEnterpriseInformationSystems

120

ﬁrst three criteria are traditional: the same than in 2D

QoE. However,depth perceptionis the new criteria in-

troduced by 3D. It includes several important charac-

teristics that are described in the following paragraph.

3.1.1 Binocular Disparity and Stereopsis (Woo

and Sillanpaa, 1979), (Howard and J

Rogers, 1995)

This refers to the difference seen by the left and the

right eye for the same object. The brain uses the

binocular disparity to extract the depth information

from images perceived by the two eyes.

3.1.2 Accomodation and Vergeance (Ukai and

Howarth, 2008), (Yano et al., 2002)

The accommodation is the adaptive optical modiﬁca-

tion that permits a clear image for different vision dis-

tances. The vergence allows for the measurement of

the focusing properties of the users.

3.1.3 Asymmetrical Binocular Combination and

Individual Differences

One must take into account the speciﬁcities of 3D per-

ception of each user. The last aspect has a large re-

search ﬁling which contains:

Stereoblindness. Richards (Richards, 1970) shows

that a part of the population was unable to see in

3D using stereo vision due to the inability to per-

ceive depth information by combining and com-

paring images from their two eyes. A study by

Eyecaretrust, an English health organism, showed

that 12% of the english population has a visual

impairment and can not enjoy 3D ﬁlms in all their

glory

Strabismus. A part of the population is inﬂicted with

a non alignment of the optical axes of the two

eyes. The gaze of each eye is not perfectly aligned

on the point in the space and affects the depth per-

ception.

Interpupillary Distance. Dodgson pointed out in

(Dodgson, 2004) and (BAHN, 2002), that the

mean interpupillarydistance (IPD) is an important

and oft-quoted measure in stereoscopic works. He

proved that the mean IPD is around 63mm, but a

range 40-80mm has to be taken into account to

cover the whole population (adults and children).

Age. Age is also an important variable. Norman

(Norman et al., 2000) & Hayslip (Hayslip and

http://www.eyecaretrust.org.uk/view.php?item

id=566

Panek, 1989) demonstrated that older adults can

extract depth and shape from optical patterns con-

taining differential motion or binocular disparities

but these abilities are often manifested with lower

levels of performance.

Display Duration. The test display duration has to

be deﬁned. (Tam and Stelmach, 1998) and (Patter-

son and Fox, 1984) investigated the role of display

duration in stereoscopic depth perception. Re-

sults conﬁrmed large individual differences in the

display duration required for a stereoscopic depth

perception.

3.2 Speciﬁcity of 3D Assessments

As explained in 2.0.2, ITU-R BT.500 (ITU-R, 2002)

is a recommendation widely used to assess the quality

of 2D videos. The previously exposed singularities of

3D over 2D make that recommendation inappropriate

for 3D assessments. Chen and al propose an extend

of the ITU recommendation greater completeness and

to take into account the speciﬁcity of 3D which is next

presented. In this part, we will present the particulari-

ties of 3D assessment exposed by (Chen et al., 2010).

3.2.1 General Viewing Conditions

We can list several new important conditions for a

good experience in 3D:

Problems of Luminance and Contrast Ratio.

They appear due to the use of additional optical

instruments for 3D viewing (glasses and ﬁlters).

Greater Importance of the Display Position. The

display position has to be taken into account

compared to 2D tests. The perception of real

background and the perceived display depth

depends on it.

Room Illumination. It has to be speciﬁed precisely

to avoid conﬂicts and perturbations with the me-

dia broadcasted to the user.

Monitor Resolution. The monitor resolution is also

an important parameter.

Recommendations. Minimum values for spatial and

temporal view resolution and stereoscopic resolu-

tion must be addressed (Holliman, 2010).

Preferred Viewed Distance. The PVD recom-

mended in BT-500 does not take into account the

depth perception. It should certainly be added in

the recommendation (Patterson, 2007).

QualityofExperiencein3DMultimedia

121

Viewing Position. is an important parameter which

can cause luminance reduction depending of the

viewing angle or motion parallax on multiview

autostereoscopic displays.

Depth Rendering. Finally, the depth rendering has a

signiﬁcant inﬂuence on the QoE for autostereo-

scopic displays (Barkowsky et al., 2009)

3.2.2 Source Signals

Actually, several video formats are available for 3D

videos as ”video plus depth” (Fehn, 2003), ”Multi-

View video plus depth representation and Cod-

ing(MVC)” (Merkle et al., 2007) and ”Layer Depth

Video (LDV)” (S. Jolly, 2009), etc. In 2011, the

MPEG 3D committee issued a Call for Proposal (CfP)

on 3D video coding technology with the objective

to “deﬁne a data format and associated compression

technology to enable the high-quality reconstruction

of synthesized views for 3D displays”. Unlike many

studies that have focused on the stereoscopic tech-

nology, this CfP also contained the auto-stereocopic

multi-view display technology. The results are not ac-

tually known but a decision will be taken during the

year 2012.

3.2.3 Selection of Test Materials

Main elements of 2D video complexity deﬁned by

ITU-T P910 (ITU, 1996) are the Spatial perceptual

Information (SI) and the Temporal perceptual Infor-

mation (TI). In 3D, new elements as Depth perceptual

information (DI) should also be added (Chen et al.,

2010).

3.2.4 Test Methods

A new element is revealed during 3D assessments:

the visual discomfort. It is an important point. It can

be measured by optometric tests, ERP (Event-Related

Potential) (Li et al., 2008), eye tracking or question-

naires.

3.2.5 Observers

The number of 15 participants deﬁned in ITU-BT.500

can be discussed du to the inter-individual differences

(Ukai and Howarth, 2008). Moreover, about 10-15%

of the population cannot perceive the depth in 3D due

to eyes malfunctions. We are therefore not capable of

addressing all the population with only one test.

3.2.6 Test Duration

During a 3D diffusion, an accommodation time is

needed to the viewer. The 10 seconds proposed by

the recommendation BT.500 seems too short for good

results.

3.3 Standard and Dataset for QoE 3D

Evaluation

We present the efforts made towards standardization

and dataset creation for future experiments.

3.3.1 A standard for Evaluation Tests

An European COST Action named QUALINET

(European Network on Quality of Experience in

Multimedia Systems and Services) has been invited

by MPEG for the CfP to participate during the

evaluation part of the different 3D video coding

algorithms.

The main scientiﬁc objective of QUALINET is the

development of methodologies for subjective and

objective quality metrics by taking into account

current and new trends in multimedia communication

systems as witnessed by the appearance of new types

of contents and interactions.

The ability to compare the different video qual-

ity assessment methods is crucial, therefore the

development of a standard certiﬁcation mechanism

is critical. To demonstrate the importance of a

certiﬁcation procedure, a group of four institutions

(e.g. NTNU, EPFL, UBC and Acreo) conducted

a cross-laboratory analysis to estimate the correla-

tion of quality scores obtained by each laboratory

for an identical test (Perkis et al., 2012). Results

demonstrated that different laboratories employing

different subjects can still produce highly correlated

results, as they follow similar guidelines to carry out

assessments.

That is an interesting basis for future QoE tests that

will be able to be carried out by many laboratories in

the same conditions for greater impartiality.

3.3.2 3D Databases

To unify the future research in the domain, EPFL

(Goldmann et al., 2010) proposed a comprehensive

stereoscopic video database that contains a large vari-

ety of scenes and different captured parameters. The

database also provides subjective quality scores. It

can be used to evaluate the performance of visual

quality metrics as well as for the comparison and de-

sign of new metrics. More information can be found

at http://mmspg.epﬂ.ch/3dvqa.

ICEIS2013-15thInternationalConferenceonEnterpriseInformationSystems

122

4 CURRENT PROJECTS &

REFLEXIONS FOR QoE 3D

Since 2010, national and international projects on 3D

have been launched around the world. The QoE is a

key element in the production chain of a 3D movie.

So, it is often an important part of these projects.

In this chapter, we will scan all the past and actual

projects over the world that address the QoE notion in

3D.

4.1 3D Live

This french project was composed of 9 academic and

industrial partners namely Orange, Institut Telecom,

Technicolor, INRIA, AMP, Binocle 3D, Thales

Angenieux, Thomson and Grass Valley. The project

covered all the 3D diffusion chain from coding to the

quality perceived by the user. Their main objective

was to improve the user experience by working on all

the 3D creation chain. Their research covered:

- speciﬁc equipments as cameras or optics

- software development for disparity correction,

content adaptation (upscaling, downscaling) and data

protection

- metadata management (depth, camera type)

- text and graphical insertions without conﬂicts in the

scene depth (subtitles, logos)

The program ﬁnished on march 2012. Informations

and publications can be found at www.3dlive-

project.com.

4.2 Jedi 3D

The project Jedi 3D is a consortium of 21 part-

ners in 3 countries (Belgium, France and Spain). It

aims to study and develop a high quality end to end

3DTV for consumer, with a user centric approach,

with stereoscopic as basis, toward multi-view. Two

work-packages are interesting for the QoE commu-

nity: WP3 ”3D User experience” and WP4: ”Demon-

strations & User assessment” which are led by the

societies NXP and Barco. More information can be

found at http://www.jedi-itea2.org/

4.3 Future Internet Engineering

The ”Future internet Engineering” is a Polish national

project ﬁnanced by the European Union. The project

covers the development and testing of infrastructure

and services for future Internet generation. One part

of the project consist of developing a measurement

methodology for 3D video and services. Several

interesting publications has been produced on QoE

(Stankiewicz and Jajszczyk, 2011), (Janowski et al.,

2012). The project just ended on december 2012.

4.4 3D4YOU

3D4YOU is an European project funded by ICT (In-

formation and Communication Technologies), a the-

matic priority for research and development under

the speciﬁc program ”Cooperation” of the Seventh

Framework Program 2007-2013. Industrial and aca-

demic partners that work on multimedia joined to-

gether on this project (Philips NL, BBC UK, Fraun-

hofer Heinrich Hertz Institut DE, Thomson R&D FR,

Orange France Telecom FR, Christian Albrechts Uni-

versitat DE and KUK Filmproduktion DE). Their ob-

jective is to develop the key elements of a practical

3D television system. A ﬁrst approach of the con-

tent requirements for subjective testing is presented

in (3D4YOU, 2008) A second document (3D4YOU,

2011), much more complete, has been produced

in march 2011 to list the consortium proposals to

achieve subjectives quality tests for 3D multimedia.

4.5 VQEG & 3DTV

VQEG (Video Quality Experts Group) is an ex-

pert group born in 1997 to deal with the ﬁeld of

video quality assessment by investigating new and ad-

vanced subjective and objective techniques (subjec-

tive assessment methods and objective quality met-

rics). Several projects are actually in progress. One

of them is called 3DTV. It’s goal is to investigate how

to assess 3DTV subjective video quality. This activity

is linked to the ITU-R Question 128/6. A ﬁrst draft is

actually available

. The ITU’s schedule provides a

project completion in 2015.

4.6 QUALINET

The European COST named Qualinet is an expert

group which want to create a network for multidisci-

plinary QoE research in Europe: Their objectives are:

- the development of methodologies to assess the

perceived QoE for multimedia presentations,

- the identiﬁcation of measurable features which are

relevant for the subjectively perceived QoE,

- the development of metrics to measure the QoE

perceived by a human observer,

- the development of tools to optimize existing

systems,

http://www.its.bldrdoc.gov/vqeg/project-pages/3dtv/

3dtv.aspx

QualityofExperiencein3DMultimedia

123

- and the design of new interaction models between

users and content multimedia.

It is actually composed of 24 COST countries

and 6 non-european. Every year, they orga-

nize workshops on QoE as QoMEX (Quality of

Multimedia Experience). A white paper writ-

ten on QoE’s deﬁnition is available at http://

www.qualinet.eu/images/stories/whitepaper

v1.1 dag

stuhl

output corrected.pdf. More details can be found

at http://www.qualinet.eu/

5 CONCLUSIONS

This paper presented a State Of The Art (SOTA) on

QoE 3D. It describes the QoE notion and explain why

it is a matter of importance for a rapid user adoption of

3D technologies. An explanation of different evalua-

tion is described to help readers to make a good choice

in case of a new multimedia experience. A collection

of most important 3D speciﬁcities is listed to better

comprehend the parameters that inﬂuence the user’s

experience. We also describe the method to conduct

a 3D assessment to maximize the results consistency.

A brief description of national, european and inter-

national projects is done to assist the reader research

on this domain. We hope these elements will enable

each researcher to better understand the concept and

the importance of QoE in 3D multimedia.

REFERENCES

3D4YOU (2008). Requirements on post-production and

formats conversion. ICT- 215075, Deliverable D2.1.2.

3D4YOU (2011). 3D video formats and conversion. ICT-

215075, Deliverable 4.2.2.

Alben, L. (1996). Deﬁning the criteria for effective interac-

tion design. Interactions.

BAHN, J. K. (2002). Effects of Interpupillary Distance and

AC/A Ratio on Binocular Fusion and Depth Percep-

tion. Proc Int Disp Workshops.

Barkowsky, M., Cousseau, R., and Le Callet, P. (2009). In-

ﬂuence of depth rendering on the quality of experi-

ence for an autostereoscopic display. In 2009 Inter-

national Workshop on Quality of Multimedia Experi-

ence, pages 192–197. IEEE.

Brotherton, M. D., Huynh-thu, Q., and Hands, D. S.

(2006). Subjective Multimedia Quality Assess-

ment. IEEE Signal Processing Magazine, E89-

A(November):2920–2932.

Bulat, J., Grega, M., Janowski, L., Juszka, D., Leszczuk,

M., Papir, Z., and Romaniak, P. (2010). Emerging Re-

search Directions on 3D Video Quality Assessment.

UCMedia 2010 2nd International ICST Conference

on User Centric Media Palma de Mallorca September

13 2010.

Chen, K., Wu, C., Chang, Y., and Lei, C. (2009). A crowd-

sourceable qoe evaluation framework for multimedia

content. Proceedings of the seventeen ACM interna-

tional conference on Multimedia, pages 491–500.

Chen, W., Fournier, J., Barkowsky, M., and Le Callet, P.

(2010). New requirements of subjective video quality

assessment methodologies for 3DTV. In Proceedings

of VPQM, volume 2010.

Claypool, M. and Tanner, J. (1999). The effects of jitter on

the peceptual quality of video. pages 115–118.

Dodgson, N. A. (2004). Variation and extrema of hu-

man interpupillary distance. Proceedings of SPIE,

5291(January):36–46.

Fehn, C. (2003). A 3D-TV system based on video plus

depth information. The ThritySeventh Asilomar Con-

ference on Signals Systems Computers 2003, 2:1529–

1533.

Goldmann, L., De Simone, F., and Ebrahimi, T. (2010).

A comprehensive database and subjective evaluation

methodology for quality of experience in stereoscopic

video. Database, 7526(1):75260S–75260S–11.

Goldmann, L. and Ebrahimi, T. (2010). 3d quality is more

than just the sum of 2d and depth. Signal Processing,

pages 2–3.

Hayslip, B. and Panek, P. (1989). Adult Development and

Aging. Harper and Row.

Holliman, N. S. (2010). 3D display systems. In Science,

volume 38, pages 31–36. IOP Press.

Howard, I. P. and J Rogers, B. (1995). Binocular vision and

stereopsis, volume 75. Oxford University Press.

ITU (1996). ”subjective video quality assessment meth-

ods for multimedia applications” in recommendation

p.910.

ITU-R (2002). Methodology for the Subjective Assessment

of the Quality of Television Pictures.

ITU-T.StudyGroup12 (2009). Itu-t rec. p.10/g.100 amend-

ment 2 (07/2008) vocabulary for performance and

quality of service amendment 2: New deﬁnitions for

inclusion in recommendation itu-t p.10/g.100. pages

1–10.

Jain, R. (2005). Quality of experience. Multimedia, IEEE,

11(1):96.

Janowski, L., Romaniak, P., and Papir, Z. (2012). Content

driven qoe assessment for video frame rate and frame

resolution reduction. Multimedia Tools and Applica-

tions, 61:769–786.

Khan, A., Sun, L., Jammeh, E., and Ifeachor, E. (2009).

Content classiﬁcation-based and qoe-driven video

send bitrate adaptation scheme. Proc. of the 5th Inter-

national Conference ICST Mobile Multimedia Com-

munications Conference, London, united Kingdom,

(16):1–4.

Lee, J.-S., De Simone, F., and Ebrahimi, T. (2011). Subjec-

tive Quality Evaluation via Paired Comparison: Ap-

plication to Scalable Video Coding. Multimedia, IEEE

Transactions on, 13(5):882–893.

Lee, J.-S., De Simone, F., Ramzan, N., Zhao, Z., Ku-

rutepe, E., Sikora, T., Ostermann, J., Izquierdo, E.,

and Ebrahimi, T. (2010). Subjective evaluation of

scalable video coding for content distribution. pages

65–72.

ICEIS2013-15thInternationalConferenceonEnterpriseInformationSystems

124

Li, H.-c. O., Seo, J., Kham, K., and Lee, S. (2008).

Measurement of 3D visual fatigue using event-related

potential (ERP): 3D oddball paradigm, Kwangwoon

University , 447-1 Nowon-Gu , Wolgae-Dong , Seoul

, Korea Kangwon National University , 192-1 Hoja-

Dong , Chuncheon , Korea. Measurement, pages 213–

216.

Merkle, P., Smolic, A., Muller, K., and Wiegand, T. (2007).

Multi-view video plus depth representation and cod-

ing. Image Processing 2007 ICIP 2007 IEEE Interna-

tional Conference on, 1:201–204.

Norman, J. F., Dawson, T. E., and Butler, A. K. (2000). The

effects of age upon the perception of depth and 3-D

shape from differential motion and binocular dispar-

ity. Perception, 29(11):1335–1359.

Patterson, R. (2007). Human factors of 3-D displays.

Journal of the Society for Information Display,

15(11):861.

Patterson, R. and Fox, R. (1984). The effect of test-

ing method on stereoanomaly. Vision Research,

24(5):403–408.

Perkis, A., You, J., Xing, L., Ebrahimi, T., De Simone, F.,

Rerabek, M., Nasiopoulos, P., Mai, Z., Pourazad, M.,

Brunnstrom, K., Wang, K., and Andren, B. (2012).

Towards certiﬁcation of 3D video quality assessment.

In Proceedings of the 6th International Workshop on

Video Processing and Quality Metrics for Consumer

Electronics - VPQM 2012.

Richards, W. (1970). Stereopsis and stereoblindness. Ex-

perimental Brain Research, 10(4):380–388.

S. Jolly, e. a. (2009). 3D Content Requirements and Initial

Acquisition Work. 3D4YOU project.

Sheikh, H. (2006). Image information and visual quality.

Image Processing.

Stankiewicz, R. and Jajszczyk, A. (2011). A survey of

QoE assurance in converged networks. Computer Net-

works, 55(7):1459–1473.

T Rahrer, R Fiandra, S. W. (2006). Triple-play service qual-

ity of experience mechanismwt-126v0.5. DSL Forum:

Working text, pages 1–60.

Tam, W. J. and Stelmach, L. B. (1998). Display dura-

tion and stereoscopic depth discrimination. Technical

Report 1, Communications Research Center, Ottawa,

Ont., Canada.

Ukai, K. and Howarth, P. (2008). Visual fatigue caused

by viewing stereoscopic motion images: Background,

theories, and observations. Displays, 29(2):106–116.

Varga, P., Kn, G., Sey, G., Moldovn, I., and Gelencsr,

P. (2006). Correlating user perception and measur-

able network properties: Experimenting with qoe.

4268:218–221.

Winkler, S. (2007). Video quality and beyond. Proc. Euro-

pean Signal Processing Conference.

Winkler, S. and Mohandas, P. (2008). The evolution of

video quality measurement: From psnr to hybrid met-

rics. Broadcasting, IEEE Transactions on broadcast-

ing, 54(3):660–668.

Woo, G. C. and Sillanpaa, V. (1979). Absolute stereoscopic

thresholds as measured by crossed and uncrossed dis-

parities. American journal of optometry and physio-

logical optics, 56(6):350–355.

Yano, S., Ide, S., Mitsuhashi, T., and Thwaites, H. (2002).

A study of visual fatigue and visual comfort for 3D

HDTV/HDTV images. Displays, 23(4):191–201.

Zepernick, H. (2007). Perceptual-based quality metrics for

image and video services: A survey. Next Generation

Internet Networks.

QualityofExperiencein3DMultimedia

125