![](bg8.png)
found the target object in shorter or longer timings.
For the participants who took longer times to find the
target object, the LI condition would be the timing
interval that should have induced Mode 2b instead
of Mode 2c. On the other hand, for the participants
who took shorter times to find the target object, the
SI condition would be the timing interval that should
have induced Mode 2b instead of Mode 2a. Since the
omnidirectional movie experiment resulted in smaller
variance in object finding timing compared to the 2D
movie experiment as shown by Figure 6 (a), it prob-
ably restrained such experimental trials that required
longer or shorter times in object finding. Therefore,
participants obtained the visual information at the
similar timing and for those participants, the MI con-
dition was ideal timing for presenting information ad-
dition part.
This result on viewing behavior can be further dis-
cussed by the presence of visual marker. This made
it easy for the visual system to detect the targets and
induce a quick response. It effectively decreased the
number of participants who took long times to locate
the target object. The necessity of visual guidance
part of audio guide and visual marker is to help the
participants quickly find the target object and works
for aligning the time to start the timer to begin the in-
formation addition part of audio guide. In this timer
aligned situation, the timing of the visual marker to
appear should have worked for the restraint on the
end of the timer when the information addition part
to start. This restraint should have increased the
number of participants who could synchronize audio
with vision smoothly in the MI condition. Smooth
synchronization and integration of multi-modal in-
formation work positively on memorization, as it is
stated on theorical simulation conducted in the previ-
ous study (Kitajima et al., 2019). As a result, the MI
condition marked remarkable effect on memory.
6 CONCLUSION AND FUTURE
WORK
This paper investigated the effect of the timing inter-
val of audio guide on memory. A model of modes
based on timing differences was introduced to make
rational explanations for relationship between timing
intervals and memory. The omnidirectional movie ex-
periment showed that timing intervals can be used to
make experience memorable in omnidirectional envi-
ronment.
For now, this study only focsed on the omnidirec-
tional movie viewing behavior to simulate the real en-
vironment. But, for applying it to our everyday life,
such as museum, gallery, guided tours, etc., it is im-
portant to apply and examine what this paper found
in the outside experiment situations. Also, since the
viewing behavior plays a significant role, there is need
to examine the effect in movies with different char-
acteristics like the fast moving targets or immovable
targets.
ACKNOWLEDGEMENTS
We would like to express our appreciation to Abiko
City Office for letting us use the dome theater for this
experiment. This work was partly supported by JSPS
KAKENHI Grant Number 19K12246.
REFERENCES
Hirabayashi, R., Motoki, S., Nakahira, K, T., and Kitajima,
M. (2019). How auditory timing affect memory when
watching movie with audio guide. FIT2019, 3:225–
228.
Kanai, A. (2000). Rhetoric and viewpoint in film cognition.
Cognitive Studies, 7(2):172–180.
Kitajima, M., Dinet, J., and Toyota, M. (2019). Multi-
modal interactions viewed as dual process on multi-
dimensional memory frames under weak synchroniza-
tion. In COGNITIVE 2019 : The Eleventh Interna-
tional Conference on Advanced Cognitive Technolo-
gies and Applications, pages 44–51.
Kitajima, M., Shimizu, S., and Nakahira, K. T. (2017).
Creating memorable experiences in virtual reality:
Theory of its processes and preliminary eye-tracking
study using omnidirectional movies with audio-guide.
In 2017 3rd IEEE International Conference on Cyber-
netics (CYBCONF), pages 1–8.
Kitajima, M. and Toyota, M. (2015). Multi-dimensional
memory frames and action generation in the mhp/rt
cognitive architecture. Procedia Computer Science,
71:202–207.
Mayer, R. E. (2002). Multimedia learning. The Annual
Report of Educational Psychology in Japan, 41:85 –
139.
Moreno, R. and Mayer, R. (2007). Interactive multimodal
learning environments. Educational Psychology Re-
view, 19:309–326.
Ohtani, A. (1971). An analysis of eye movements during a
visual task. Ergonomics, 14(1):167–174.
Tabbers, H. K., Martens, R. L., and Van Merrienboer, J.
J. G. (2004). Multimedia instructions and cognitive
load theory: Effects of modality and cueing. British
Journal of Educational Psychology, 74(1):71–81.
How Auditory Information Presentation Timings Affect Memory When Watching Omnidirectional Movie with Audio Guide
169