Image Presentation with Smell for Digital Signage and the Effect on

Eye Catching

Keisuke Tomono

, Hajime Katsuyama

, Shuhei Yamamoto

and Akira Tomono

Department of Information Media Technology, School of Information and Telecommunication Engineering,

Tokai University, 2-3-23 Takanawa, Minato-ku, Tokyo 108-8619, Japan

Graduate school of Engineering, Tokai University, 4-1-1 Kitakaname, Hiratsuka-shi, Kanagawa, 259-1292, Japan

Keywords: Image, Smell, Gaze Detection, Inhalation, Tactile.

Abstract: This paper describes the effect on eye catching by digital signage that releases smell from screen and the

method of smell presentation to human olfactory receptor. The effect on eye catching is investigated by

analyzing movements of eye with and without smell using a detector. Visual image of foods is presented to

a viewer, and his or her gazing time on a food object is detected. This experiment reveals advertisement

accompanied with smell is more attracted to a viewer. The proposed method for smell presentation is to

induce a person inhalation by tactile sensation caused by airflow and released smell at a time of inhalation.

This experiment discusses the possibility of inducing subjects to receive smell efficiently.

1 INTRODUCTION

Recently, large digital signage has been prevailing

(Burke, 2009). It is far more flexible than the

conventional billboard which uses photograph, and it

is possible to install it in a passage underground in a

city or at an entrance of a store. Various advertised

commodities by digital signage have been discussed,

and it is believed that cooked foods and other food

items can be most effectively displayed by it. As is

well known, olfactory sensation has a strong

influence on human appetite so that the provision of

smells is also an important factor. We often

experience to be lured by the smells of food when

entering a restaurant. Therefore, if it is possible to

provide the smells of food displayed on digital

signage, a high advertising effect can be expected

(Nakamoto, 2008).

Regarding a display of image with smell, it is

common to place an olfactory display device next to

an image display device (Sakaino, 2008). However,

in the case of a large display like digital signage, the

positions of generating images and generating smells

are distant away from each other, so it is difficult to

express a realistic sensation as if the image of a

product is generating the smell, as these two devices

are placed side by side as mentioned above.

Requirement of a space for an olfactory display

device is also a problem.

In order to overcome these problems, we propose

the “KANSEI Multi-Media Display” (the term of

KANSEI here means sensation, emotion, and mood,

in Japanese), a device that can present image with

smell at the same time by combining an image

display device and an olfactory display device

(Tomono, 2011). Holes are created on a thin display

panel through which gases can pass, and an airflow

discharge device is installed on the back side of the

display so that it can present smells to viewers. We

create KMMD by using LED and a projector to

confirm the basic principles of this device. However,

the followings are pointed out for practical uses: (1)

Clarification of the effect of discharging smells from

the vicinity of the targeted image; (2) Necessity of

developing a method where the discharged smells do

not linger long because smells tend to spread out

into an area and remain for a long period of time.

2 KMMD AND EXPERIMENTAL

ENVIRONMENT

KMMD applying a projector shown in Fig. 1 was

used for the purpose of discussing the eye catching

property of digital signage of several food images

through discharging smells. The size of the screen

was 1,800mm (width) x 1,200mm (length), on which

157

Tomono K., Katsuyama H., Yamamoto S. and Tomono A..

Image Presentation with Smell for Digital Signage and the Effect on Eye Catching.

DOI: 10.5220/0004025001570162

In Proceedings of the International Conference on Signal Processing and Multimedia Applications and Wireless Information Networks and Systems

(SIGMAP-2012), pages 157-162

ISBN: 978-989-8565-25-9

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

nine panels (400 x 200 mm) at maximum with holes

were attached, from which smells were discharged.

These panels were 1.5mm in thickness. The diameter

of the holes was 2 mm, and the distance between the

holes was 3.5 mm.

A smell generating device was installed in the

reverse side of the panel. This device consisted of a

box to discharge airflow by increasing atmospheric

pressure on the reverse side of hole, a mechanism to

increase atmospheric pressure, and a duct

mechanism to connect them. It discharged a vortex

link of gas by instantaneously increasing the

atmospheric pressure on the reverse side of the

holes. Moreover, it discharged airflow like wind

continuously by increasing the atmospheric pressure.

If some smells were placed in the box, the smells

were carried on airflow and vortex link and

presented to the observers.

Subjects were asked to gaze at the screen sitting

on a chair at a visual distance of 200 cm. The

subjects were separated from the researchers by a

curtain so that they could concentrate on the image.

Figure 1: Experimental environment of digital signage

where image with smell can be presented.

There were various measurement instruments

installed behind the subjects, which were operated

by the researchers.

3 EYE CATCHING PROPERTY

OF A SCENE WITH SMELL

In the field of advertisement, it is important that

pedestrians are attracted to a signboard and watch it

carefully. Our research group previously simulated

that a digital signage accompanied with smell

particularly increased attractiveness to viewers

though there were many non-smelled signboards

(Tomono, 2010). In this paper, the method of

presenting the smell for the person who had stood in

front of one advertisement to gaze at the target

commodity at long time was examined. Therefore,

an experiment was conducted on how eye catching

property and psychology were affected by dis-

charging smell from the vicinity of a food image, by

arranging smells in that way. In this experiment, the

gaze movements of subjects over real dishes on a

table and their picture images of various dishes with

and without smell and on a screen were measured,

and a questionnaire for KANSEI evaluation was

implemented. Fig. 2 shows the experimental envi-

ronment of investigating the eye-catching property

using a gaze detection device and the condition of

experiment scene. Table 1 shows 20 adjective-pairs

used for KANSEI evaluation based on a semantic

differential method (Heise, 1970).

Six or ten students participated into the

experiment.

Figure 2: Experimental method of eye catching evaluation.

3.1 Scene of Rows of Food Images

(1) Experimental Method. An image simulating a

billboard where several food photos seen in a

restaurant area were arranged was projected onto a

large screen, and pedestrians’ gaze movements were

analyzed (Fig.2). The foods presented there were

four foods of noodle, curry, yakitori, and pizza, all

of which had relatively strong smell. As the gaze

tendency was considered to be influenced by the

SIGMAP2012-InternationalConferenceonSignalProcessingandMultimediaApplications

158

display position, the positions of the four food

images were rotated. The display time was 20

seconds. The gaze detection device was an NAC Eye

Mark Recorder EMR-9. A gaze point was measured

in the image captured by a view camera. The

detection accuracy was approximately one deg.

Furthermore, because gaze tendency was considered

to be influenced by personal likes or dislikes, a

questionnaire about tastes of food was conducted.

Table 1: Adjective-pairs used for sensibility evaluation.

(2) Experimental Results. Fig. 3 (A) is an analysis

of the gaze-stationary point of Subject C. The gaze-

stationary point is defined as the area where the gaze

point stays more than 0.1 second within a range of 2

degrees of the viewing angle. From this figure, the

gaze point stayed on the object on the bottom-left of

the screen (yakitori) for a long period of time, and it

almost evenly stayed on other three objects.

Referring to the result of questionnaire for the

favorite tastes, it was known that the subject was

looking at the image of yakitori for a long period of

time because he/she was interested in it.

Furthermore, as for the rotation of images, there was

a tendency that the gaze time on the food shown on

the left side was longer.

(B) shows the average gaze time of the six subjects

for each object. In the questionnaire about tastes of

food, three subjects chose yakitori, and the other

three subjects chose three different foods. The gaze

time on yakitori was the longest, and it was more or

less similar among the other objects.

objects which the subjects answered as their favorite

food and those on other objects. The result was the

same as the daily experience that the gaze-stationary

point is concentrated on one's favorite food.

(D) shows a radar chart presenting KANSEI

evaluation. As for the impression when looking at

the billboard, the scores of the adjectives of

“delicious” and “favorite” were high.

Figure 3: Experimental results of foods image without

smell.

3.2 Scene of a Row of Real Foods on a

Table

(1) Experimental Method. Real foods were

arranged on a table to produce the scene similar to

the experiment on 3.1, the blindfold was removed

from the state of the blindfold, and the subject was

made to gaze on the table for 20 seconds. The

temporal variation of the gaze-stationary point was

investigated.

(2) Experimental Results. Fig. 4(A) shows a result

of Subject D. Because the subjects stood beside the

table, the foods arranged in lower part of this figure

were nearer the subject's position, it seemed that

yakitori and pizza were seen easily. However, the

gaze time of an upper right curry and rice was long.

The subject answered that the concern was high

because the smell of the spice of the curry and rice

was strong.

(B) is average gaze time of six subjects to each food.

The gaze time of the curry and the yakitori was

ImagePresentationwithSmellforDigitalSignageandtheEffectonEyeCatching

159

longer than that of the noodle and the pizza. From

this experiment, it has been understood that the

existence of an impressive smell in addition to the

favor greatly influences the eye catching. Moreover,

it has been understood that the scores to the

adjectives such as "Strong" and "Heavy" in addition

to "Delicious" and "Favor" were high.

Figure 4: Experimental results of real foods scene.

3.3 Scene of Food Images from Which

Smells Are Being Discharged

(1) Experimental Method. A smell of food was

discharged from the place where the food image was

displayed by using the system of Fig. 1. Because it

was a purpose to examine the relation between the

smell presentation and the eye catching property in

this experiment, the smell was discharged only from

the upper right corner of the screen. The food image

positions had been gradually changed. Condition 1

of discharging the same smell as the food image and

condition 2 of discharging the smell different from

the food image were examined.

(2) Experimental Results. Fig. 5(A) shows the

result of subject E in condition 1. It is understood

that the image of the curry in the upper right corner

into which the smell was discharged was seen long.

In this experiment, we calculated the average gazing

time from ten subjects. As the result from Picture

(B) describes, the picture of curry was gazed longer

than the pictures of any other foods, regardless of

orders of the pictures. In the case of replacing the

picture of curry with pizza and teriyaki chicken, and

presenting the smell related to the picture, these

upper right pictures were gazed at long as well as the

case of the curry by subjects.

On condition 2, if the food image that related to

the smell was in the screen, the image was searched

and seen for a long time. That is, the person tends to

be induced to the olfactory stimulus, and to see the

sight object that doesn't contradict the smell.

However, at this time, the half of the subject had

noticed contradiction (irrationality) into which the

smell was discharged from a place different from the

place where food image was displayed. In the case

of that the foods images displayed on the screen

were not corresponded to emitted smell, subjects

seemed to feel weirdness because they could not

find the visual which matched with the smell. On the

questionnaire, some of them answered “the smell

was more memorized than the foods images.”

The result of the sensibility evaluation radar

chart was near the above-mentioned result when real

foods were seen.

Figure 5: Experimental results of foods image with smell.

3.4 Discussion

A large difference in the distribution of gaze-

stationary points was observed between food images

only and food images with smell. It was confirmed

that the image with smell attracted attention to the

food and gave a strong impression. In this point, it is

close to looking at real food.

Regarding the influence of the weirdness sense

given by the difference between the positions where

food image is presented and smell is discharged, the

number of subjects who realized it was not

negligible, so it seems that the place and direction of

emitting smell are important factors to improve the

reality of visual. Our previous researches suggested

that emitted smell made one to initiatively look for

the visual matched with it on the screen (Tomono,

2011). In this experiment, it was successful to emit

smell corresponded to the visual from the vicinity of

the food image because of using KMMD. Providing

smell from the displayed point enabled people to

naturally feel comfortable. This occurred because

they do not need to search for the visual and the

smell is strongly connected to corresponded visual in

his or her mind. The only matter was not to evaluate

on enough number of subject.

SIGMAP2012-InternationalConferenceonSignalProcessingandMultimediaApplications

160

4 SMELL PRESENTATION VIA

INDUCED INHALATION BY

TACTILE SENSATION OF AIR

4.1 Aim of the Experiment

Since one of the characteristics of smell is once it

emits it diffuses quickly, it is desirable to use small

amount of smell as much as possible. It was easily

guessed that if subjects know the place and direction

of emitted smell by using KMMD prior to the

emission even small amount of smell would make it

possible to increase the effectiveness of olfaction,

since they can initiatively activate their inhalations.

Since smell is delivered by airflow, in real-life

situation, we often perceive smell in the wind. For

example, when gentle breeze strokes the cheeks,

people tend to take a breath to sense a smell in the

wind. As the wind contains the information of the

surrounding environment, sniffing allows people to

effectively grasp the surrounding situation (Sawada,

2008). This suggests the possibility that people learn

to do sniffing for the purpose of comprehension of

their situation.

If this proposition is experimentally verified, as

shown in Fig.6, it will be possible to present a smell

to the user at the time of inhalation by inducing

inhalation using an air-cannon with the function to

release a vortex ring. Note that behind the screen are

two types of air cannons in the same figure. Air-

cannon 1 ejects a vortex ring for tactile sensation.

Then, air-cannon 2 ejects a vortex ring containing

smell. As is indicated in (A), human inhalation nor-

mally repeats itself at an interval of once per 5~7

sec. However, when a tactile sensation is given,

sniffing is induced, as is indicated in (B), and so if

smell is delivered at this time, it will be effectively

received.

4.2 Experimental Method

In order to demonstrate that smell is perceived more

effectively by giving tactile sensation to the face and

causing sniffing to occur, the system illustrated in

Fig.6 was used to present smell under two

conditions, shown in Fig.7, and the rate of successful

perception of smell was investigated. The first part

of this experiment was to present smell at a constant

time interval of inhalation under normal conditions.

The second part was to present smell one second

after tactile sensation based on a model predicting

such induction of sniffing. The duration of smell

presentation was 5s in both parts of the experiment.

Bergamot (PalmTree Co., Ltd.) was used in this

experiment. In both parts, a subject was asked to

hold a counter by his or her right hand and press the

counter when he or she sensed a smell. Smell was

presented 20 times during the experiments. The

experiments took 1min and 40s. The number of the

subjects was eleven (eight men and three women).

Figure 6: Smell presenting method.

4.3 Experimental Results and

Discussion

The rate of successful smell perception increased

significantly for eight subjects among eleven in the

second part of the experiment. For the remaining

three, the difference in the rate in the first part and

the one in the second part was small. Fig.8 shows

the average number of successful smell perceptions

over the subjects and the standard deviation. When a

statistical test procedure was carried out using the

null hypothesis that there was no difference in the

rate in the first and second part, the hypothesis was

rejected at the significance level of 5%. Thus, the

rate of successful smell perception was higher in the

second part.

After the experiment, the subjects were asked to

fill out a questionnaire. There are many answers that

they unintentionally sniffed because they thought

that there was a change in the surrounding

environment. In this experiment, the subjects were

blindfolded so that they need to use the senses other

than vision to learn about the environment, and this

ImagePresentationwithSmellforDigitalSignageandtheEffectonEyeCatching

161

might lead people to check what happened when

tactile sensation was given by using the olfactory

organ. Thus, there is a high possibility that

inhalation can be induced by tactile presentation. In

addition, the results also suggest the possibility that

induction of inhalation is stronger for images

containing steam such as cooking-scene of steaming

dishes with the tactile sensation.

Figure 7: Experimental method.

Figure 8: Experimental Results.

5 SUMMARY

For application to digital signage that can release

smells from several locations on its screen, we

compared the eye movement over a visual image of

foods, over real foods, and a visual image of foods

with smell. Also, we considered the method to

release subtle smells at the time of inhalation. As a

result, we revealed the following:

1) The taste of the subject and location of the

presented image on the screen has great influence on

the gaze time of the subject looking at visual images

of food. When the subject was looking at the real

food or the visual image with smell, in addition to

the aforementioned factors, the intensity of the smell

had an impact on the gaze time.

Visual accompanied by smell was tended to be

gazed longer. It seems like subjects feel

comfortable only when the point of emitting smell

on the screen is exactly matched with the one of

displaying visual, since they do not need to look

for where smell is coming from.

Sensory

evaluation when presented with scented visual

images of food is similar to the one when presented

with real foods.

3) The rate of successful scent perception of smell

increased after tactile sensation was given. This

suggests that tactile sensation induces inhalation.

Using this, it will be possible to make effective

olfactory reception.

In this experiment, the number of subject was not

enough to evaluate the data analytically. However,

by repeating the experiment with larger population

of subject, more accurate result will be obtained in

the future. We would like to improve a sense of

reality of digital signage by releasing smells from

the image of the products.

REFERENCES

Burke, R. R., 2009. Behavioral effects of digital signage.

In Journal of Advertising Research,49(2),180-185.

Nakamoto, T., 2008. Olfactory display : Multimedia tool

for presenting scents, The fragrance journal Ltd.

Sakaino, A., 2008. The concept of sensitivity communi-

cation using aroma and a plan of demonstration

experiment. In IEICE technical report. Electronic

information displays, 108(291), pp.53-57.

Tomono, K., et al., 2011. Scent-Emitting Display Panel

and Its Psychological Effects. In The Journal of ITE

65(10), 1411-1419.

Tomono, A., et al., 2010. The Eye Catching Property of

Digital-Signage with Scent and a Scent-Emitting

Video Display System. In IEEJ Transactions on Elec-

tronics, Information and Systems,130,4,pp.668-675.

Heise, D., 1970. The semantic differential and attitude

research. In Chapter 14 in Attitude Measurement. pp.

235-253. Edited by Gene F. Rand McNally.

Tomono, A., et al., 2011. Effect of Smell Presentation on

Individuals with Regard to Eye Catching and Memory.

In Electronics and Communications in Japan, 94, 3,

pp. 9-19.

Sawada, E., et al., 2008. A wind communication interface:

BYU-BYU-View. In Transactions of the Virtual

Reality Society of Japan 13(3), pp.375-383.

SIGMAP2012-InternationalConferenceonSignalProcessingandMultimediaApplications

162