TRANSMISSION OF LOW-MOTION JPEG2000 IMAGE

SEQUENCES USING CLIENT-DRIVEN CONDITIONAL

REPLENISHMENT

∗

J. J. S´anchez-Hern´andez

, J. P. Garc´ıa-Ortiz

, V. Gonz´alez-Ruiz

, I. Garc´ıa

and D. M¨uller

University of Almer´ıa, Comp. Architecture and Electronics Dept., Almer´ıa, Spain

European Space Agency, ESTEC, Noordwijk, Netherlands

Keywords:

JPEG 2000, Conditional replenishment, Video transmission.

Abstract:

This work proposes a strategy for browsing interactively sequences of high resolution JPEG 2000 remote im-

ages. These sequences can be displayed in any order (forward and backward) and following any play/timing

pattern. In order to increase the quality of the reconstructions where the retrieved images are only known at the

moment of the visualization, this work has proposed and evaluated a novel technique based on conditional re-

plenishment. This solution proﬁts from the SNR/Spatial scalability of JPEG 2000 to determine which regions

of the next image should be transmitted and what regions should be reused from the previously reconstructed

image. Experimental results demonstrate that, even without motion compensation and with a transmission

exclusively controlled by the client, the reconstructions are consistently better, both visually and from a rate-

distortion point of view, than those that only remove the spatial redundancy (such as Motion JPEG 2000).

Other advantages of our approach are that no data overhead is generated, the computational complexity is very

small compared to similar techniques, and the fact that it can be used with any JPIP server.

1 INTRODUCTION

Some of the powerful features of the new JPEG

2000 multi-part standard (International Organization

for Standardization, 2004) are very efﬁcient loss-

less/lossy compression, random access to the com-

pressed data streams, incremental decoding and high

scalability. These characteristics make JPEG 2000 a

state-of-the-art solution for remote browsing of high-

resolution images. Using the JPIP protocol, deﬁned

in Part 9 (International Organization for Standardiza-

tion, 2005) of the JPEG 2000 standard, clients can

interactively explore remote image data by specifying

a window of interest (WOI). This data exchange uses

the available bandwidth efﬁciently and does not re-

quire any recoding or additional processes. The server

extracts only the required data from the images and

transmits it to the clients.

JPEG 2000 has already been successfully used in

many scientiﬁc areas; e.g., tele-microscopy or tele-

∗

This work has been funded by grants from the Spanish

Ministry of Science and Innovation (TIN2008-01117) and

Junta de Andaluc´ıa (P08-TIC-3518), in part ﬁnanced by the

European Regional Development Fund (ERDF).

medicine. A promising application in space sciences

is the JHelioviewer project (M¨uller et al., 2009), de-

veloped by the European Space Agency (ESA) in col-

laboration with the National Aeronautics and Space

Administration (NASA). Its main goal is to provide

an interactive data browsing, visualization and access

platform to accommodate the staggering data volume

of 1.4 TB of images per day that are returned by the

Solar Dynamics Observatory (Pesnell, 2008). Among

other data products, SDO is providing full-disk im-

ages of the Sun taken every 12 seconds in ten dif-

ferent ultraviolet spectral bands with a resolution of

4096 × 4096 pixels. As of today, the combination of

JPIP and JPEG 2000 seems to offer the best solution

in order to efﬁciently browse image data sets of this

magnitude.

The basic functionality of JHelioviewer allows

users to explore the available data for a given point

in time. An interesting extension of this functional-

ity is that it enables users to move smoothly through

a sequence of time-coded solar images given a spe-

ciﬁc time range while displaying the changes in the

WOI. This work therefore focuses on JPIP applica-

tions, such as JHelioviewer, that are designed to in-

J. Sánchez-Hernández J., P. García-Ortiz J., González-Ruiz V., García I. and Müller D..

TRANSMISSION OF LOW-MOTION JPEG2000 IMAGE SEQUENCES USING CLIENT-DRIVEN CONDITIONAL REPLENISHMENT.

DOI: 10.5220/0003518600110016

In Proceedings of the International Conference on Signal Processing and Multimedia Applications (SIGMAP-2011), pages 11-16

ISBN: 978-989-8425-72-0

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

Figure 1: An example of different time instances from a remote browsing session.

teractively explore remote image sequences. Fig. 1

shows an example of ﬁve sequential time instances

during a remote browsing session using JHelioviewer.

Once the user has selected a time range, the server

builds a virtual JPEG 2000 ﬁle which only contains

links to those solar images whose date/time stamp

belongs to that time range. The client starts a JPIP

session for that ﬁle and requests the ﬁrst image, dis-

played at time t

. The user can then watch the se-

quence of images belonging to the time range, one

by one, jumping to any image, and changing direc-

tion at any moment. In this example, at time t

the

user zooms in on a certain region, thus changing the

current WOI. From this new WOI, the user continues

moving forward through the sequence, to instances t

and t

. In the “video mode”, where the images are vi-

sualized without pause for a given frame-rate that the

user can control, the amount of data that it is retrieved

from the JPIP server depends on the available band-

width and the frame-rate. In any case, images are

transmitted by quality in order to increase the qual-

ity of the reconstructions as much as possible.

The fact that SDO takes images of the entire Sun

at high temporal resolution results in a high degree of

spatial correlation between consecutiveimages (about

33dB of PSNR on average between images that have

been taken with a cadence of 432 seconds

) and there-

fore, they show a certain degree of temporal redun-

dancy. For this reason, we focus this paper on the

removal of this redundancy by means of Conditional

Replenishment (CR).

2 RELATED WORK

CR has been previously used with success to encode

low-motion image sequences. In (Mounts, 1969),

Mounts proposed a method for the lossy (8:1) com-

pression of head-and-shoulder digital TV signals. In

(McCanne et al., 1997), McCanne et al. explored the

possibility of CR for the MBone (multicast backbone)

This result has been determined by computing the av-

erage PSNR of one image respect to the next (in time) one.

SDO offers the possibility of retrieve images at a higher ca-

dence, up to 12 seconds.

thanks to its reduced complexity, the option to gener-

ate scalable video and its data loss resilience. More

recently, three major solutions that offer good com-

pression results and decoding ﬂexibility have been

studied. In (Cheung and Ortega, 2007), Cheung and

Ortega have investigated the performance of a I14P

motion-compensated predictive video codec that is

based on Distributed Source Coding and support for-

ward and backward playback. In (Devaux et al.,

2009), Devaux et al. have proposed a solution based

on CR to exploit the temporal redundancy of video

surveillance sequences that have previously been en-

coded with JPEG 2000. Although this approach

does not rely on motion compensation, the sender

(or server) has to determine, and to transmit to the

receiver, the optimal points at which it is better to

use the JPEG 2000 data than the prediction (gener-

ated with the last reconstructed frame and an estima-

tion of the background). Finally, the work of Na-

man et al. is similar in essence, but they propose the

use of CR with multiple reference frames and motion

compensation to encode those image sequences with

a large amount of movement (Naman and Taubman,

2009) and the same architecture but without motion

compensation for more static sequences (Naman and

Taubman, 2010).

After an analysis of all these approaches our work

is motivated by the following main reasons: (1) all the

proposed approaches need to process the sequence of

images on the server-side in some way and therefore

the sequence (and in some cases even the order of the

images) must be known a priori, (2) the computational

resources required by the sender to perform real-time

transmission are extremely high due to the amount of

data to be managed (4Kx4K images, and at least 20

images/second) and (3) none of them are fully JPIP

compliant because additional data has to be transmit-

ted from the sender to the receiver. Therefore, none of

the existing solutions can be easily applied to an inter-

active browsing system like JHelioviewer. Moreover,

in (Ortiz et al., 2010) a prefetching strategy has been

proposed that would further enhance the user experi-

ence and that is fully compatible with the proposal of

this paper, but would be difﬁcult to implement with

the approaches described above.

The remainder of this paper is structured as fol-

SIGMAP 2011 - International Conference on Signal Processing and Multimedia Applications

Figure 2: Example with the ﬁrst four iterations of our algorithm.

Figure 3: A comparison between our proposal (“with CR”)

and the standard method (“without CR”). On the top for the

SDO/AIA sequence. Below, the Akiyo sequence.

lows. Section 3 introduces our client-driven retriev-

ing algorithm. Section 4 provides experimental re-

sults and comments about the visual increment of the

quality of our reconstructions. Section 5 summarizes

our work and describes the main future research lines

that could extend it.

3 PROPOSAL

In large-scale solar image sequences, such as in many

other videos where signiﬁcant portions of the image

do not change much for a long period of time, there

is room for the use of CR, even though the predic-

tions are not motion compensated. In this case, it

is possible to perform a real-time reconstruction of a

4Kx4K-resolution video using a frame-rate of 25 im-

ages/second in machines with modest computational

resources and limited band-widths.

In our discussion it will be supposed that the

band-width of the communication channel between

the JPIP server and a client as well as the frame-rate

selected by the user remain constant over time. This

fact does not affect the description nor the efﬁciency

of our approach.

We denote the image sequence with I and the n-

th image of this sequence with I

, where 0 ≤ n < N.

The images that are ﬁnally displayed with CR are de-

noted as I

′

. Because in the context of JPEG 2000

the images can be decompressed from the same code-

stream with several spatial resolutions, we deﬁne that

= I

is the image at the highest resolution and I

is the spatially scaled version of I

, that has a reso-

lution of S/2

, where S is the size of the images and

0 ≤ r < R+1. The encoding parameters for the image

sequence are P, Q and R, where P denotes precinct

dimensions, Q the number of quality layers and R the

number of wavelet decomposition levels, or stages.

Basically, Our algorithm works as follows: the

ﬁrst image of the sequence is progressively transmit-

ted for a given period of time. This is the image that is

visualized. After, a thumbnail of the second image is

transmitted with the aim of knowing which precincts

of the second image should be updated by the CR.

Notice that the JPEG 2000 packets that are necessary

to reconstruct this thumbnail are also needed to re-

construct the second image at high resolution. This

process is repeated with the following images. It

also takes care to compute the distortion between im-

ages with the last update of each precinct that are not

neccesary in the previous image. With these ideas in

mind, we propose the following client-driven trans-

mission scheme:

1. Let q ← 1 be the quality layer used for the current

TRANSMISSION OF LOW-MOTION JPEG2000 IMAGE SEQUENCES USING CLIENT-DRIVEN CONDITIONAL

REPLENISHMENT

Figure 4: Two SDO/AIA reconstructed images, left with and right without CR. The refreshed precincts have been highlighted

with a white square.

Figure 5: Two Akiyo reconstructed images, left with and right without CR. The refreshed precincts have been highlighted with

a white square.

reconstructions.

2. Let n ← 1 be the index of the reconstructed image.

3. Retrieve the next quality layer of I

n−1

and initial-

ize I

′

n−1

with this value.

4. Retrieve only the next quality layer of I

R+1

(the

thumbnail image).

5. Store in list L and sort in descending order, the

precincts whose MSE (Mean Square Error) be-

tween I

R+1

n−1

and I

R+1

are larger than a given

threshold λ.

6. I

′

← I

(L) ∪ (I

′

n−1

∩ I

(L)), i.e., copy from I

′

n−1

to I

′

those precincts that remain “constant” and

update from I

those precincts that are in list L

with quality q.

7. n ← n+ 1.

8. If n < N − 1, go to step 4.

9. q ← q+ 1.

10. If q ≤ Q, go to step 2.

With the objective of describing the operation of

the algorithm, Figure 2 shows a schematic example

of the transmission of the ﬁrst ﬁve images of a se-

quence where the signiﬁcant changes only happen in

SIGMAP 2011 - International Conference on Signal Processing and Multimedia Applications

the center of the pictures. The data that are ﬁnally

displayed are labeled with “Displayed with CR”. As

it can be seen, with CR only the center of the im-

ages is retrieved with a higher quality (in this case,

with 1 quality layer) and therefore, for a given bit-

rate, CR reconstructions should be better than without

CR from a SNR and from a visual point of view.

4 EVALUATION

This proposal has been tested with two different low-

motion image sequences. The ﬁrst one is composed

by a set of 140 4Kx4K-resolution SDO/AIA images

taken in the 17.1 nm channel (to show the ﬁnest struc-

ture and the one that is most dominated by Fe IX/X

emission from 10

K hot plasma), which has been

log10-valued-scaled and corrected for exposure time

variations. The second sequence

is composed by the

CIF (352x288 pixels) color (although only the luma

component has been used in our experiments) images

of the Akiyo sequence. This sequence includes a static

background and foreground with very little motion,

only a head-and-shoulder video of an almost static

announcer. The encoding parameters for SDO/AIA

sequence has been: P = 128, Q = 8 and R = 8; and

P = 32, Q = 8 and R = 3 for the Akiyo sequence. Ex-

perimentally, a good value for the threshold param-

eter λ was found to be 1. The reconstruction of the

SDO images has been performed with a resolution

of 1024x1024 pixels because this is the largest size

that most actual displays can show of a power-of-two-

resolution-scaled 4Kx4K image. In the case of Akiyo,

no scaling has been applied.

The transmission of the SDO/AIA sequence has

been simulated using a bit-rate of 27×10

bits/second

(see the top plot of Figure 3). Although lower

band-widths have been tested, the refreshing-rate (25

frames/second) of the precincts generated by the CR

procedure is too slow to represent all the motion that

the rotation movement of the Sun generates in the im-

ages. Nevertheless, a bit-rate of 8 × 10

bits/second

has been enough to generate good reconstructions for

the Akiyo sequence (see the bottom plot of Figure 3).

A visual comparison can be done using Figure 4

and 5 that shows the reconstruction of the second im-

age of the SDO/AIA sequence and the fourth image

of the Akiyo sequence, respectively. In these exper-

iments, much smaller bit-rates (1163400 bits/second

for SDO/AIA and 343800 bits/second for Akiyo)

have been used in order to generate large visual dif-

ferences between using CR and not. A frame-rate of

Downloadable from http://trace.eas.asu.edu/yuv/akiyo/

akiyo qcif.7z.

25 images/seconds has been considered for both se-

quences.

5 CONCLUSIONS

This work shows how to improve the remote visu-

alization of JPEG 2000 image sequences by means

of client-driven conditional replenishment. The pro-

posed system is fully compatible with the JPIP stan-

dard because we only havechangedthe order in which

the packets are retrieved from the server. With the

idea of improving the quality of the reconstruction of

sequences with a higher degree of movement, motion

compensated predictions could be generated by the

clients. In any respect, this modiﬁcation of the tested

algorithm does not affect the applicability of our pro-

posal.

REFERENCES

Cheung, N.-M. and Ortega, A. (2007). Flexible video de-

coding: A distributed source coding approach. In

IEEE Workshop on Multimedia Signal Processing,

pages 103 – 106.

Devaux, F.-O., Meessen, J., Parisot, C., Delaigle, J.-F.,

Macq, B., and Vleeschouwer, C. D. (2009). Re-

mote interactive browsing of video surveillance con-

tent based on jpeg 2000. IEEE Transactionf on Cir-

cuits and Systems for Video Technology, 19(8):1143 –

1157.

International Organization for Standardization (2004). In-

formation Technology - JPEG 2000 Image Coding

System - Core Coding System. ISO/IEC 15444-

1:2004.

International Organization for Standardization (2005). In-

formation Technology - JPEG 2000 Image Coding

System - Interactivity Tools, APIs and Protocols.

ISO/IEC 15444-9:2005.

McCanne, S., Vetterli, M., and Jacobson, V. (1997). Low-

complexity video coding for receiver-driven layered

multicast. IEEE Journal on Selected Areas in Com-

munications, 15(6):983 – 1001.

Mounts, F. (1969). A video encoding system with con-

ditional picture-element replenishment. Bell System

Technical Journal, 48:2545 – 2554.

M¨uller, D., Fleck, B., Dimitoglou, G., Caplins, B. W.,

Amadigwe, D. E., Ortiz, J. P. G., B. Wamsler,

A. A., Hughitt, V. K., and Ireland, J. (2009). JHe-

lioviewer: Visualizing large sets of solar images using

JPEG 2000. Computing in Science and Engineering,

11(5):38–47.

Naman, A. and Taubman, D. (2009). Rate-distortion opti-

mized jpeg2000-based scalable interactive video (jsiv)

with motion and quantization bin side-information. In

IEEE International Conference on Image Processing,

page 3081 3084.

TRANSMISSION OF LOW-MOTION JPEG2000 IMAGE SEQUENCES USING CLIENT-DRIVEN CONDITIONAL

REPLENISHMENT

Naman, A. and Taubman, D. (2010). Predictor selection us-

ing quantization intervals in jpeg2000-based scalable

interactive video (jsiv). In IEEE International Confer-

ence on Image Processing, pages 2897 – 2900.

Ortiz, J. G., Ruiz, V. G., Garc´ıa, I., M¨uller, D., and Dimi-

toglou, G. (2010). Interactive browsing of remote jpeg

2000 image sequences. In IEEE International Con-

ference on Pattern Recognition, pages 3179 – 3182,

Istambul, Turkey. IEEE press.

Pesnell, W. (2008). The Solar Dynamics Observatory: Your

eye on the Sun. In 37th COSPAR Scientiﬁc Assem-

bly, volume 37 of COSPAR, Plenary Meeting, pages

2412–+.

SIGMAP 2011 - International Conference on Signal Processing and Multimedia Applications