Reducing the Split-Attention Effect in Assembly based Instruction by

Merging Physical Parts with Holograms in Mixed Reality

David Dixon

, Uwe Terton

and Ruth Greenaway

Faculty of Arts Business and Law, University of the Sunshine Coast, Sippy Downs Dr, Sippy Downs, Australia

Centre for Support and Advancement of Learning and Teaching, University of the Sunshine Coast,

Sippy Downs Dr, Sippy Downs, Australia

Keywords: Instructional Design, Memory, Cognitive Load, Human Computer Interaction, Assembly, Scaffolding.

Abstract: Split attention in instructional materials is a recognised problem known to cause an increase in cognitive load.

Instructional designers often try to resolve this by using a variety of methods that do not account for the spatial

disconnect between diagrams being matched up with physical parts during an assembly task. The emergence

of Mixed Reality offers a solution, using “holograms” which can project 3d images into the physical

environment around the user. This paper reports on a project that proposes the creation of a software prototype

that simultaneously enables part identification and tracking of parts for assembly. It conceives a new way of

providing instructions when assembling flat pack furniture by endeavouring to facilitate working memory

constrains. The software prototype will assist the user by showing where parts should be placed and by

providing real-time feedback based on interaction.

1 INTRODUCTION AND

BACKGROUND

Split attention in instructional materials is a

recognised problem known to cause an increase in

cognitive load. Instructional designers often resolve

this by placing text and diagrams close together to

reduce working memory constraints. While this

method is effective in solving split attention within

the diagram itself it does not account for the spatial

disconnect between looking at the diagram and

matching up the physical parts required during

assembly. Flat pack instruction manuals often show

diagrams from an optimal viewing angle to make it

easy to understand but this representation is still only

two dimensional leaving the user to interpret it into

three dimensions or from a different angle, which is

not ideal. The emergence of Mixed Reality (MR)

offers a solution, using “holograms” which can

project 3d images into the physical environment

around the user. These 3d images can act as an

animated guide which can be looked at from all

angles at any scale. While using “holograms” as a

reference may improve user perception, it does not

solve the split attention effect to aid the user.

Holographic computing is an emerging

technology that allows “holograms” to be perceived

within the users’ physical environment and treated as

if they are there. Holographic displays are referred to

as “mixed reality” which incorporates the best parts

of virtual and augmented reality (a technology that

superimposes a computer-generated image on a

users’ view of the real world, therefore providing a

composite view) using a clear head mounted display.

In assembly and instructional design for example

computer based manuals or augmented reality

experiences via mobile devices provide little to no

positive or negative feedback. This is due to training

systems having inadequate built in perception based

artificial intelligence (computer vision) or no

“intelligent tutoring” within the software

(Westerfield et al., 2015, p. 169).

It is up to the user or external observer to make

sure a task is being done correctly. While the full

capabilities of mixed reality technology are yet to be

explored, the author intends to investigate the

possibilities of computer vision integration on a

headset such as the HoloLens™ as a tool to help

assembling a piece of flat pack furniture.

The problem that has been identified is a visual

disconnect between instructional assembly diagrams

and the physical parts of the assembly such as in flat

Dixon, D., Terton, U. and Greenaway, R.

Reducing the Split-Attention Effect in Assembly based Instruction by Merging Physical Parts with Holograms in Mixed Reality.

DOI: 10.5220/0006691202350244

In Proceedings of the 10th International Conference on Computer Supported Education (CSEDU 2018), pages 235-244

ISBN: 978-989-758-291-2

235

pack furniture, this is known to cause split attention.

Ayres and Cierniak, (2012, p. 3172) explain that

“Split-attention occurs when learners are required to

split their attention between two or more mutually

dependent sources of information (e.g. text and

diagram), which have been separated either spatially

or temporally”.

Split attention is known to negatively impact on

the capacity and duration of working memory,

(Cooper, 1998) leading to reduced learning efficiency

and comprehension (Kalyuga et al., 1999, p. 351).

A prototype software application could be devised

that provides users with immediate feedback on their

correct and incorrect actions during assembly tasks

and progress. The design would help to integrate the

instruction into the users’ physical environment by

overlaying diagrams onto the physical parts via the

head mounted display instead of a separate diagram

as a point of reference. Integration is based on the

split attention design principle in multi-media

instruction. This principle stipulates that instruction

should be designed so that users do not have to

mentally integrate multiple sources of information as

it requires processing to be decided between the

sources leading to split-attention (Mayer & Moreno,

1998, pp. 318-319).

A gap in knowledge was identified as there is no

research on overlaying “holograms” on physical

objects to substitute the use of separate 3d model

reference or a diagram on an augmented display.

When using existing technology such as augmented

reality or a head mounted display attention could be

split in different ways depending on the solution.

When using an augmented device, attention is split

between identifying the physical parts needed for the

assembly and the device itself. This attention split

also occurs when a 3d model is displayed through the

headset as a reference only, this is because it still

requires comparison to be made with the parts. While

this method does integrate the 3d model into the

users’ environment it is still spatially a separate

diagram.

A review of the literature raised the questions:

Can a prototype application be designed that uses a

holographic guide to lessen the impact of split

attention in assembly based tasks by reducing the

amount of working memory used?

In what ways is the use of computer vision to

identify, track and place physical parts using

holograms different to using a holographic

instructional reference guide?

This paper outlines the proposed creation of a

software prototype that simultaneously enables part

identification and tracking of parts for assembly.

Endeavouring to reduce working memory the

software will show where parts should be placed and

provide real-time feedback based on interaction when

assembling flat pack furniture.

2 INSTRUCTIONAL DESIGN

According to Smith and Ragan, (1999, p. 2), the term

instructional design refers to “the systematic and

reflective process of translating principles of learning

and instruction into plans for instructional materials,

activities, information resources and evaluation”. It is

important for instructional designers to be able to

explain the philosophical foundations and theory of

their field because they are a key source for design

principles. Contrary to this many designers have a

limited theoretical background which has

implications when constructing instructional material

and selecting a suitable learning position (Ertmer &

Newby, 2013, p. 43).

This research draws upon the cognitivist learning

theory approach because it emphasises how internal

mental processes (memory) can affect problem

solving and attention (Spector, 2014, p. 5). Cognitive

teaching strategies can be applied to the features of

the software prototype to appraise its effectiveness in

instruction and subsequent performance.

While cognitivism has many facets, the following

concepts mentioned below were the most appropriate

for this study seeing it is taking place outside of the

classroom environment with no assistance from a

teacher.

Cognitive theorists contend that environmental

(external) influences and instructional components

alone cannot account for all the learning that results

from an instructional task (Ertmer & Newby, 2013, p.

51). Memory and internal mental processes needs to

be equally weighted with instructional materials as it

has consequences as to how the visual design and

information layout is deployed. If instruction is

poorly designed it will have a negative memory

function leading to poor comprehension.

Part of the cognitivist learning approach is the

implementation of feedback to give awareness of

results that guide and support accurate mental

connections (Thompson et al., 1992 cited in Ertmer

and Newby, 2013, p. 53).When designing, instruction

emphasis is placed on structuring, organising, and

sequencing information to facilitate optimal

processing. The use of instructional explanations,

demonstrations and illustrative examples are all

considered useful tools to aid understanding (Ertmer

& Newby, 2013, p. 52). A software application that

CSEDU 2018 - 10th International Conference on Computer Supported Education

236

gives users feedback will help to chaperone them

through the task in a supportive manner and provide

guidance to move forward.

Cognitivism takes the stance that individuals

acquire and store new information that is combined

with old information in order to learn. Preceding

changes in “behaviour” are identified to show what is

happening in the learners’ mind (Petri & Mishkin,

1994, p. 30).

Learning is said to happen when information is

stored in memory in a structured and meaningful way.

The transfer of information to long term memory is

said to occur when a learner understands how to apply

knowledge in different contexts. Prior knowledge is

used to identify the similarities and differences of

new information. Not only must the knowledge itself

be stored but also how to use it (Ertmer & Newby,

2013).

The aim of the proposed software is to make sure

the user knows the assembly process by keeping it

similar to an instructional manual. In this way, the

working concepts and functionality will have an air

of familiarly, to form correct correlations in the users’

mind. Once the assembly process is learned it could

be used to assemble a multitude of items in the same

manner.

Based on the concepts mentioned cognitive

architecture will be further explored using cognitive

load theory, to investigate the influence of

instructional materials on memory to make accurate

considerations in the design process.

3 COGNITIVE LOAD THEORY

Cognitive load can be defined as a multidimensional

concept representing the load that performing a task

inflicts on a learners’ cognitive system (Paas & Van

Merriënboer, 1994, p. 122). Cognitive load is

amplified when learning requires the mental

integration of multiple sources of information (Ayres

& Sweller, 2014, p. 206).

Cognitive load is divided into three main

frameworks namely Intrinsic, Extraneous and

Germane.

Intrinsic cognitive load is the level of difficulty

associated with an instructional topic. It is determined

by an interaction between the characteristics of the

material being learned and the ability of the learners.

It cannot be modified by instructional designers

because it comes from an individual’s existing

knowledge (Paas et al., 2003b, p. 65).

Extraneous or ineffective cognitive load is

produced by the way information is presented to

learners and is under the control of instructional

designers (Chandler & Sweller, 1991). It is defined as

the extra load beyond the intrinsic cognitive load and

its main cause is poorly designed instructional

materials (Paas et al., 2003a).

Germane or effective cognitive load is the amount

of mental effort devoted to handling the construction

and automation of schemas (Sweller, 1988). This

theory is part of cognitivism and prescribes that

knowledge is stored in long-term memory in the form

of schemas. A schema categorises elements of

information according to how they associate with

existing information to provide solutions to potential

problems (Chi et al., 1981). By reducing extraneous

load, germane load can be upheld which positively

facilitates the creation of schemas.

3.1 Memory

How memory works is central to this research

because it can impact a users’ ability to interact with

instructional material effectively. It is widely

established that the mind uses three types of memory

set apart by the Atkinson–Shiffrin memory model

(1968). These memory components are referred to as

sensory memory, working memory and long term

memory (Atkinson & Shiffrin, 1968).

Sensory memory processes stimuli that is

received from the senses including touch, smell, taste,

sight and sound. A separate division of sensory

memory exists for each one of these senses. Sensory

memories are not retained for long, nearly half a

second for visual and three seconds for auditory

information (Cooper, 1998). Within the small amount

of elapsed time the brain must allocate meaning to the

new information to be able to recall it and use it at a

later date.

Working memory is the portion of the mind that

provides consciousness and allows both logical and

creative thinking. Working memory is associated

with “where and how” attention is directed to think or

to process information. Working memory can be

increased through the presentation of information

visually and auditorily in comparison to just using a

single sense (Baddeley, 1992).The main limitation of

working memory is its capacity to process no more

than approximately seven elements of information

consecutively (Miller, 1956). Working memory is

used for problem solving, this makes it the most

important out of the three types of memory for the

purposes of this research.

Studies have observed how differences in

Working Memory Capacity impact the efficiency of

design principles in multimedia learning, including

Reducing the Split-Attention Effect in Assembly based Instruction by Merging Physical Parts with Holograms in Mixed Reality

237

the Modality Principle, which recommends using

information in both visual and auditory presentations

instead of using text and images (Seufert et al., 2009).

The approach of combining these two senses will be

applied to the software design in an attempt to aid the

working memory of the user. WMC is a limited

capacity cognitive paradigm that differs between

people and plays a key role in learning. It is shown to

reveals individual differences in the capability to

actively sustain information relevant to the task and

relate it to information from long-term memory when

faced with diversions (Cowan, 2005; Engle & Kane,

2003).

Using the proposed software, the amount of

extraneous cognitive load and the need for substantial

amounts of information to process in memory may

decrease. Therefore, this should lead to an increased

completion speed because the task will require less

problem solving leading to lower cognitive load and

expenditure of working memory.

The assembly task given to the participants as part

of this study does not require mastery therefore there

is little reason for it to be stored in long term memory.

Long term memory denotes the vast collection of

information made up of schemas and expertise that is

held in a close to permanently accessible form

(Albers & Mazur, 2014, p. 106). This assumes that it

is unlikely that the assembly of a piece of furniture

(flat pack) will need to be repeated in a real-world

context without the assistance of the software. With

the software aiding the user there is no need to spend

time or effort memorising the intricacies of the entire

task in order to assembly it.

Many people to which this software is targeted

will have used an instructional manual at some point

in the past and possess an associated schema. All the

participant needs to learn is the new process of how

the software instruction works and associate it with

their existing schema as part of their long-term

memory. The software will also reduce the number of

elements being processed at any one time by

displaying only what is needed at the time, showing

no more than seven elements of information.

3.2 Measuring Cognitive Load

There are three separate ways of measuring cognitive

load; physiological responses, subjective techniques

and task based. The intensity of mental effort can be

considered as an index of mental workload (Paas,

1992). Mental effort may be defined as the total

amount of controlled cognitive processing in which a

subject is engaged. Measures of mental effort can

provide evidence on performance and the costs

associated with cognitive learning (Paas &

Merriënboer, 1993, p. 738). Performance is perceived

by the success in undertaking a task; regularly

measured by speed, accuracy or scores. This study

will employ task based and subjective techniques

which are described below along with physiological

responses.

3.2.1 Physiological Response

The use of cardiological (heart rate) measurements

have been established as part of an empirical study

into measurement techniques within the cognitive

load framework (Paas et al., 1994). They measured

heart rate variability to estimate the level of cognitive

load. It was found to be un-intrusive but also invalid

because it could not detect minor changes in cognitive

load. Pupillary responses were measured over time in

multiplication tasks that ranged in different levels of

difficulty to measure cognitive load during a study by

Hossain and Yeasin, (2014). The findings reported

that the pupil diameter increased in relation to the

difficulty of the equation presented. Change in pupil

size during a cognitive task interaction can suggest

that the subject is in cognitive control, attentive or

potentially overloaded.

3.2.2 Subjective Response

When using the subjective method participants are

asked to self-report their invested mental effort by

translating their perceived amount of mental effort

into a linear numerical value. It is a modified form of

a rating scale for measuring perceived task difficulty

established by Bratfisch et al., (1972). This method

works under the assumption that people can report

their own mental effort. Values are reported on a

visual analogue scale from very, very low mental

effort (1) to very, very high mental effort (9). This

value needs to be combined with an effectiveness

measure, as it does not indicate performance on its

own. Depending on the task the given efficiency

measure could be a percentage of correct answers in

a data set or how many tasks were completed within

a set time.

While self-assessments to measure cognitive load

are subjective they are well established. The rating

scale has been tested through empirical studies (Paas,

1992; Paas & Van Merriënboer, 1994) for sensitivity

and reliability to construct validity and intrusiveness.

This evaluation indicated that the subjective rating

scale is sensitive to relatively minor differences in

cognitive load, valid and nonintrusive (Sweller et al.,

1998). While there are several measures available the

method using rating scales continues to be the most

CSEDU 2018 - 10th International Conference on Computer Supported Education

238

popular because of its easy implementation and

accuracy.

Having participants rate their invested mental

effort completing a task versus how difficult they

perceived a task to be should be separated into

different questions to avoid potential ambiguity of

just using one or the other. Hypothetically, invested

mental effort may contain more interpretations than

only the task, such as individual enthusiasm while

perceived task difficulty concerns mainly to the task

itself (Van Gog & Paas, 2008). Applying both

measures may provide some motivational indicators

rather than just experienced load. While the

interpretation of invested mental effort and task

difficulty ratings are still highly subjective they

provide a clear advantage over just using one or the

other. Providing set definitions to the participants

prior to the commencement of the task will help

minimise ambiguity.

3.2.3 Task based Response

Task based techniques include two subclasses:

primary task measurement and secondary task

methodology. The primary task is based on

performance while the secondary task is conducted

simultaneously (Paas et al., 2003b). Typical

performance variables are reaction time, accuracy and

error rate. A secondary task was deemed as

inappropriate for this study as it would not represent

a realistic setting and scenario in which the assembly

is trying to emulate. All the measurements will be

focused on the primary task.

4 IMPLEMENTATION

Using the Vuforia™ SDK markerless object tracking,

the dimensions of a 3d model can be specified and

stored in the database. An image of any design can

then be authored in an external application to be

printed out and wrapped around the physical object.

Currently the technology is limited to cuboid and

cylindrical shapes but this is sufficient for testing the

assembly task concept. The textures used must be the

same aspect ratio to the corresponding 3d surface. In

this instance, the pattern chosen will be a texture that

could be made unique for each surface. For aesthetic

purposes, the pattern could easily be changed to

replicate different finishes commonly found for

furniture. Custom software can be developed inside

the Unity 3d game engine to incorporate the target

model data into a Universal Windows Platform

(UWP) application.

The Time of Flight camera sensors in the

HoloLens™ headset facilitate “spatial mapping”

allows for placement of virtual objects, occlusion,

physics and navigation to provide real-world

behaviours (Microsoft, 2017). This research is

focusing on the placement aspect to enable virtual

objects to interact with the physical environment,

keeping their position even when out of the camera

frame.

Due to the limited Field of View (FOV) on the

HoloLens it is possible to track where the user is

looking at any time in relation to the holograms and

the users’ relative position in the room. The HoloLens

FOV is approximately 30 x 17.5 a 16:9 aspect ratio

(Doc-Ok.org, 2015) which means the user must look

around to view any holograms outside. That ability

can be taken as an advantage for recording and

recreating the environment inside unity 3d as another

method for observation. This observation aims to

uncover split attention, based on where the user is

gazing to see if it correlates with their cognitive load.

4.1 Human Computer Interaction

Multimodal interfaces are interactive systems that are

designed to enhance natural human capabilities to

communicate though speech, gestures, touch and

facial expression. (Turk, 2014, p. 189). Multimodal

interfaces are growing in importance due to advances

in hardware and software. These interfaces are

unlikely to replace traditional desktop computing and

graphical user interface but offer an alternative.

Multimodal interfaces are a more natural way of

communicating. Combining several ways of

interacting is well suited to the saturation of the

mobile computing platforms (Cutugno et al., 2012, p.

197). Given multiple options users can pick an

interaction method that is more suited to them, a

particular task or to reduce repetitive physical

motions. Obrenovic and Starcevic, (2004, p. 65)

express that people naturally communicate with the

world multimodally through consecutive use of

multiple senses. In other words, people use all their

senses to interact with their environments and that has

shifted to technology. In this case of this research

users will be able to interact with the HoloLens™

using the software by using gestures, speech

recognition and by moving physical objects in the real

world through marker less tracking.

Reducing the Split-Attention Effect in Assembly based Instruction by Merging Physical Parts with Holograms in Mixed Reality

239

4.2 Assembly

The underlying concept for the experimental software

prototype is demonstrated in Figure 1. The diagram

shows a physical part being placed inside the

holographic template which can be set up anywhere

in the users’ room for convenience. Ideally the

hologram would be positioned in the location the

furniture item would situate upon completion of the

assembly to make sure it can fit before

commencement. To indicate the current step of the

assembly the physical parts required will be

highlighted when the user looks at the set of parts to

distinguish them from those that are not needed for

the present step.

Figure: 1 Holographic assembly workflow concept.

To further assist with assembly, the corresponding

holographic part will pulse a blue colour on the

template to match with the physical part, this is

intended to capture the users’ attention. When the

physical part is placed into the holographic template

the part will stop pulsing blue and stay solid, showing

that the part is in the correct position and orientation.

These visual cues will be complemented with text and

audio prompts. Jeung et al., (1997, p. 342) support

this notion of including graphic indicators such as

flashing, colour changes or simple animation. They

agree that it is necessary for audio-visual instruction

to be an effective instructional technique in situations

where visual searching is prevalent.

The use of holograms negates redundancy found

in step by step diagrams by combining all the steps

into an interactive diagram. This is because there is

no need to look at several diagrams for comparison

over multiple steps. Interactivity also makes the

assembly system more coherent to users because

there is no irrelevant information displayed, this is

known as the coherence principle (Fenesi et al., 2016,

p. 691). That is, only showing what is needed to make

the task comprehensible at any one time.

4.3 Scaffolding

Educational instructional scaffolding refers to the

process in which teachers show how to solve a

problem and then offer support as needed, which

traditionally takes place within a classroom

environment. Jerome Bruner a Psychologist and

instructional designer first used the term 'scaffolding'

in the field of pedagogy between teachers and

children as cited in Wood et al., (1976, p. 90).

McLoughlin, (2004) argues that “the concept of

scaffolding needs to be redefined ... into contexts

where the teacher is not present”.

With technology now commonplace in both the

workplace and at home it makes sense that the use of

scaffolding concepts be extended outside of the

classroom (Jumaat & Tasir, 2014, p. 74). Interest for

such integration has been expanding in teaching

(Reiser, 2004, p. 275) and further into

commercialised applications.

When demonstrating instruction in the context of

assembling flat pack furniture for example it typically

would be constructed in the home or office.

Scaffolding needs to be structured in such a way that

is both self-explanatory and self-directing. The

benefit of computer based scaffolding is that it

provides infinite patience and is highly scalable. A

clear downside of computer based scaffolding is

flexibility. In comparison to one on one interactions

the software has to base its response on pre-defined

behaviours

(Belland, 2017, p. 24).

In place of the teacher the following scaffolding

examples will be used in the prototype application to

facilitate the user in a way that is responsive and

supports their needs: There is no general consensus

on the underling concepts of scaffolding although

Van de Pol et al., (2010) review revealed some

commonly used strategies such as fading,

contingency and transfer.

4.3.1 Fading

The idea behind this concept is to provide support to

a novice learner which is gradually removed over

time to develop independent learning strategies

(Collins, 1987, p. 19). This is employed by having the

software display a tutorial that explains how the

assembly system works on the first couple of steps of

the assembly. By instructing the user through a

worked example it will provide a frame of reference

for the future steps.

CSEDU 2018 - 10th International Conference on Computer Supported Education

240

4.3.2 Contingency

Contingency is an aid mechanism that is responsive

to the users’ needs through tailored, differentiated, or

calibrated support (Van de Pol et al., 2014, p. 604). It

can be thought to be similar to a backup plan when a

user gets stuck on a task. It will be designed in a way

that offers support throughout the entire assembly

task. If users reach an impasse and need to be

reminded on the way then the software steps in and

the tutorial section at the start can be viewed again at

any time. Hints will be presented by highlighting

where the part should be placed and what parts are

currently needed to avoid any potential confusion.

4.3.3 Transfer

In a teaching situation, the notion of transfer was

traditionally a “relationship” that shifts between

teacher and student when a student takes increasing

learner control (Van de Pol et al., 2010, p. 275). Based

on the context of this study, transfer can be thought to

be the changing of the responsibility from the

software developer to the user once they have learned

how to use it (usually after fading has taken place).

Transfer is never fully shifted to the user in this

instance because the software is always fully active.

Instead the relationship is one of constant back and

forward as the user is prompted to make their next

move. When a part is placed in the right place the user

is presented with visual or audio cues (response) to

keep them on track and moving forward. At any time

during a task the shift of responsibility may need to

change between the user and the software which will

fall back to the contingency responses.

5 FUTURE DIRECTIONS

The next step in the research process is to prototype a

simple version of the assembly application in Unity

for deployment of the HoloLens. 3d printed parts will

be devised to make up a scaled down cabinet using

Computer Aided Design and Drafting (CADD)

software. This needs to be done because the

dimensions of the printed parts have to match those

entered into the object recognition data base in order

to be tracked. Further investigation needs to take

place into the method of wrapping the 3d printed parts

with patterned paper to see if making holes in the

parts has a negative effect on the tracking.

Testing will take place using the purpose built

prototype as there is no off the shelf solutions

available that offer suitable feature sets for control

groups. Three control groups will be asked to

complete the same assembly task using the HoloLens.

Every iteration will be facilitated by an animated

guide to act like an instruction manual. The amount

of assistance provided will vary with feature sets

building on previous iterations.

The first prototype iteration provides a baseline

and will use a holographic reference positioned next

to the assembly location. This iteration provides no

feedback to the user and they are left to make their

own decisions.

The second iteration offers feedback in the form

of part identification and tracking for each step of the

assembly. This aims to reduce split attention by

assisting the user with finding parts.

The third iteration has the most comprehensive

scaffolding and feedback. It uses the part

identification and tracking found in test two with one

change, physical parts can be placed inside a

holographic template. This approach removes the

separate reference hologram and shifts the task into a

single point of focus. This is in contrast to the

previous two tests which rely on the user to use a

hologram as a diagrammatic reference and carry out

the assembly next to it.

The Visual Analogue Scale will be employed to

measure task difficulty and perceived cognitive load

at set times during the assembly task. Following this

a survey will be administered to gather feedback on

what could be improved based on user experience.

Performance measures will correspondingly be

captured based on completion times of each step, in

order to calculate the relative condition efficiency of

each prototype iteration.

Data analysis will take place using Analysis of

Variance (ANOVA) to examine if there is any

statistical significance between the test groups.

Consulting with a statistician will provide advice on

the correct classes of model to be employed before

any testing takes place. This will uncover correlations

between the prototype design iterations and their

efficiency when compared to each other. The use of

instantaneous load over time shown in in

Paas et al.,

(2003b) will also be explored to identify and define

trends in the pattern of cognitive workload.

Comparing the efficiency of the prototype

iterations will conclude if user performance has

increased when cognitive load has reduced and how

it is impacted by memory. By doing this analysis the

proposed research questions will be addressed.

Reducing the Split-Attention Effect in Assembly based Instruction by Merging Physical Parts with Holograms in Mixed Reality

241

6 CONCLUSIONS

A review of the literature informs the proposal of a

software prototype that seeks to use the cognitive

memory architecture to assist users in processing

information. The prototype uses holographic

templates to address split attention and redundancy by

eliminating the use of a reference manual or separate

diagram. Merging the physical parts with holograms

aims to free up users to focus their attention on the

assembly task instead of constantly referring to a

separate 3d diagram or a visually disconnected

augmented device. Furthermore, the part

identification will avoid constantly searching for the

set of parts needed to complete a particular step by

highlighting them when in the users view. Reducing

visual searches has been shown as an effective way to

eliminate split attention and facilitate learning (Cerpa

et al., 1996, p. 2).

The software will employ scaffolding that fades

out over time while still offering contingency if the

user gets stuck or forgets where they are up to. The

scaffolding provisions first introducing how the

assembly system works with a demonstration

consisting of step by step worked example through

the initial stage of the assembly. While the

scaffolding may be pre-defined leveraging the

software with a form of built-in intelligence removes

the frustration commonly found in static instructional

materials.

The responsibility of how to progress through a

task is never transferred entirely to the user making

the experience feel balanced and much more

enjoyable. Users have no need to learn the intricacies

of the assembly task, but rather understand how to

interact with the elements on the holographic system

to guide them. In this way working memory is freed

and the assembly system associations are stored in

long term memory as a schema.

This proposal of merging holograms and physical

parts in mixed reality has emphasised the critical

importance of considering the characteristics of

human cognition and cognitive load when devising

instructional materials.

REFERENCES

Albers, M. J. & Mazur, M. B. 2014, 'Information design in

technical communication', In Content and complexity,

Routledge,

Atkinson, R. C. & Shiffrin, R. M. 1968, 'Human memory:

A proposed system and its control processes',

Psychology of learning and motivation, [Online]. 2, pp.

89-195. [Accessed 28 January 2017]. Available from:

http://www.sciencedirect.com/science/article/pii/S007

9742108604223.

Ayres, P. & Cierniak, G. 2012, 'Split-Attention Effect', In

N. M. Seel (ed.), Encyclopedia of the Sciences of

Learning, Springer US, Boston, MA, pp. 3172-3175,

[Accessed 1 August 2017], https://link.springer.com/

referenceworkentry/10.1007%2F978-1-4419-1428-

6_19.

Ayres, P. & Sweller, J. 2014, The split attention principle

in multimedia learning, Available from:

https://www.researchgate.net/publication/285315734_

The_split_attention_principle_in_multimedia_learning

Baddeley, A. 1992, 'Working Memory', Science,[Online].

255 (5044), pp. 556-559. [Accessed 25 April 2017].

Available from: http://www.jstor.org.ezproxy.usc.edu.

au:2048/stable/2876819.

Belland, B. R. 2017, 'Instructional Scaffolding:

Foundations and Evolving Definition', In Instructional

Scaffolding in STEM Education: Strategies and

Efficacy Evidence, Springer International Publishing,

Cham, pp. 17-53, [Accessed 1 September 2017],

https://doi.org/10.1007/978-3-319-02565-0_2.

Bratfisch, O., Borg, G. & Dorna, S. 1972, 'Perceived Item-

Difficulty in Three Tests of Intellectual Performance

Capacity'. [Accessed 30 April 2017]. Available from:

https://eric.ed.gov/?id=ED080552.

Cerpa, N., Chandler, P. & Sweller, J. 1996, 'Some

conditions under which integrated computer-based

training software can facilitate learning', Journal of

Educational Computing Research,[Online]. 15 (4), pp.

345-367. [Accessed 27 Septemeber 2017]. Available

from:

http://journals.sagepub.com/doi/abs/10.2190/MG7X-

4J8N-CKYR-P06T.

Chandler, P. & Sweller, J. 1991, 'Cognitive Load Theory

and the Format of Instruction', Cognition and

Instruction,[Online]. 8 (4), pp. 293-332. [Accessed 24

April 2017]. Available from: http://www.jstor

.org.ezproxy.usc.edu.au:2048/stable/3233596.

Chi, M. T., Glaser, R. & Rees, E. 1981, Expertise in

problem solving, Pittsburgh Univ., PA. Learning

Research and Development Center., viewed 14 August

2017, https://eric.ed.gov/?id=ED215899.

Collins, A. 1987, Cognitive Apprenticeship: Teaching the

Craft of Reading, Writing, and Mathematics. Technical

Report No. 403,

Cooper, G. 1998. Research into cognitive load theory and

instructional design at UNSW. Citeseer,[Online].

[Accessed 24 April 2017]. Avaliable from:

http://dwb4.unl.edu/Diss/Cooper/UNSW.htm.

Cowan, N. 2005, Working Memory Capacity, Hove, East

Sussex, UK: Psychology Press.

Cutugno, F., Leano, V. A., Rinaldi, R. & Mignini, G. 2012,

'Multimodal framework for mobile interaction', In:

Proceedings of the International Working Conference

on Advanced Visual Interfaces, ACM, pp. 197-203.

[Accessed 6 September 2017].Available from:

https://dl.acm.org/citation.cfm?id=2254592.

CSEDU 2018 - 10th International Conference on Computer Supported Education

242

Doc-Ok.org. 2015. Hololens and Field of View in

Augmented Reality,[Online]. [Accessed 15 January

2017]. Avaliable from: http://doc-ok.org/?p=1274.

Engle, R. W. & Kane, M. J. 2003, 'Executive Attention,

Working Memory Capacity, and a Two-Factor Theory

of Cognitive Control', In Psychology of learning and

motivation, Academic Press, pp. 145-199.

Ertmer, P. A. & Newby, T. J. 2013, 'Behaviorism,

cognitivism, constructivism: Comparing critical

features from an instructional design perspective',

Performance Improvement Quarterly,[Online]. 26 (2),

pp. 43-71. [Accessed 18 February 2017]. Available

from:

http://onlinelibrary.wiley.com/doi/10.1111/j.1937-

8327.1993.tb00605.x/abstract.

Fenesi, B., Kramer, E. & Kim, J. A. 2016, 'Split-Attention

and Coherence Principles in Multimedia Instruction

Can Rescue Performance for Learners with Lower

Working Memory Capacity', Applied Cognitive

Psychology,[Online]. 30 (5), pp. 691-699. [Accessed 9

July 2017]. Available from: http://onlinelibrary.wiley.

com/doi/10.1002/acp.3244/abstract.

Hossain, G. & Yeasin, M. 2014, 'Understanding effects of

cognitive load from pupillary responses using Hilbert

analytic phase', In: Proceedings of the IEEE

Conference on Computer Vision and Pattern

Recognition Workshops, pp. 375-380. [Accessed 28

April 2017]. Available from: https://dl.acm.org/

citation.cfm?id=2679711.

Jeung, H. J., Chandler, P. & Sweller, J. 1997, 'The role of

visual indicators in dual sensory mode instruction',

Educational Psychology,[Online]. 17 (3), pp. 329-345.

[Accessed 22 May 2017]. Available from:

http://www.tandfonline.com/doi/abs/10.1080/0144341

970170307.

Jumaat, N. F. & Tasir, Z. 2014, 'Instructional Scaffolding in

Online Learning Environment: A Meta-analysis', In:

2014 International Conference on Teaching and

Learning in Computing and Engineering, 11-13 April

2014], pp. 74-77. [Accessed 27 July 2017].Available

from: http://ieeexplore.ieee.org/document/ 6821832/

?reload=true.

Kalyuga, S., Chandler, P. & Sweller, J. 1999, 'Managing

split-attention and redundancy in multimedia

instruction', Applied Cognitive Psychology, [Online].

13 (4), pp. 351-371. [Accessed 18 May 2017].

Available from: https://www.researchgate.net

/publication/238680916_Managing_SplitAttention_an

d_Redundancy_in_Multimedia_Instruction.

Mayer, R. E. & Moreno, R. 1998, 'A split-attention effect

in multimedia learning: Evidence for dual processing

systems in working memory', Journal of Educational

Psychology,[Online]. 90 (2), p. 312. [Accessed 2 April

2017]. Available from:http://psycnet.apa.org/record

/1998-02710-011.

McLoughlin, C. 2004, 'Achieving excellence in teaching

through scaffolding learner competence', In: Teaching

Learning Forum, Feburary 9-10], Perth Murdoch

University. [Accessed 7 July 2017].Available from:

https://clt.curtin.edu.au/events/conferences/tlf/tlf2004/

mcloughlin.html.

Microsoft. 2017. Spatial mapping, [Online]. [Accessed 4

March 2017]. Available from:

https://developer.microsoft.com/enus/windows/mixed-

reality/spatial_mapping.

Miller, G. A. 1956, 'The magical number seven, plus or

minus two: some limits on our capacity for processing

information', Psychological Review,[Online]. 63 (2),

pp. 81-97. [Accessed 29 July 2017]. Available from:

http://psycnet.apa.org/record/1957-02914-001.

Obrenovic, Z. & Starcevic, D. 2004, 'Modeling multimodal

human-computer interaction', Computer,[Online]. 37

(9), pp. 65-72. [Accessed 7 July 2017]. Available from:

https://www.researchgate.net/publication/2956202.

Paas, F. 1992, 'Training strategies for attaining transfer of

problem-solving skill in statistics: A cognitive-load

approach', Journal of Educational

Psychology,[Online]. 84 (4), p. 429. [Accessed 24 April

2017]. Available from: https://www.researchgate.net/

publication/232523259_Training_Strategies_for_Attai

ning_Transfer_of_ProblemSolving_Skill_in_Statistics

_A_CognitiveLoad_Approach.

Paas, F., Renkl, A. & Sweller, J. 2003a, 'Cognitive load

theory and instructional design: Recent developments',

Educational psychologist,[Online]. 38 (1), pp. 1-4.

[Accessed 15 August 2017]. Available from:

http://www.tandfonline.com/doi/abs/10.1207/S153269

85EP3801_1?journalCode=hedp20.

Paas, F., Tuovinen, J. E., Tabbers, H. & Van Gerven, P. W.

M. 2003b, 'Cognitive Load Measurement as a Means to

Advance Cognitive Load Theory', Educational

psychologist,[Online]. 38 (1), pp. 63 - 71. [Accessed 24

July 2017]. Available from: http://www.tandfonline.

com/doi/abs/10.1207/S15326985EP3801_8.

Paas, F. & Van Merriënboer, J. 1994, 'Variability of worked

examples and transfer of geometrical problem-solving

skills: A cognitive-load approach', Journal of

Educational Psychology, [Online]. 86 (1), pp. 122-133.

[Accessed 30 April 2017]. Available from:

http://psycnet.apa.org.ezproxy.usc.edu.au:2048/journal

s/edu/86/1/122.

Paas, F., Van Merriënboer, J. & Adam, J. 1994,

'Measurement of cognitive load in instructional

research', Perceptual and motor skills,[Online]. 79 (1),

pp. 419-430. [Accessed 26 April 2017]. Available

from: https://www.researchgate.net/publication

/15390085_Measurement_of_cognitive_load_in_instr

uctional_research.

Paas, F. G. W. C. & Merriënboer, J. J. G. V. 1993, 'The

Efficiency of Instructional Conditions: An Approach to

Combine Mental Effort and Performance Measures',

Human Factors, [Online]. 35 (4), pp. 737-743.

[Accessed 22 April 2017]. Available from:

http://journals.sagepub.com/doi/abs/10.1177/0018720

89303500412.

Petri, H. L. & Mishkin, M. 1994, 'Behaviorism,

Cognitivism and the Neuropsychology of Memory',

American Scientist,[Online]. 82 (1), pp. 30-37.

Reducing the Split-Attention Effect in Assembly based Instruction by Merging Physical Parts with Holograms in Mixed Reality

243

[Accessed 26 September 2017]. Available from:

http://www.jstor.org/stable/29775098.

Reiser, B. J. 2004, 'Scaffolding Complex Learning: The

Mechanisms of Structuring and Problematizing Student

Work', The Journal of the Learning Sciences, [Online].

13 (3), pp. 273-304. [Accessed 24 July 2017]. Available

from:http://www.jstor.org.ezproxy.usc.edu.au:2048/sta

ble/1466939.

Seufert, T., Schütze, M. & Brünken, R. 2009, 'Memory

characteristics and modality in multimedia learning: An

aptitude–treatment–interaction study', Learning and

Instruction, [Online]. 19 (1), pp. 28-42. [Accessed 10

September 2017]. Available from:

http://www.sciencedirect.com/science/article/pii/S095

9475208000121.

Smith, P. L. & Ragan, T. J. 1999, Instructional design, 2nd

ed, New York: Wiley.

Spector, J. M. 2014, 'Conceptualizing the emerging field of

smart learning environments', Smart learning

environments,[Online]. 1 (1), p. 2. [Accessed 11 July

2017]. Available from: https://slejournal.springeropen.

com/articles/10.1186/s40561-014-0002-7.

Sweller, J. 1988, 'Cognitive Load During Problem Solving:

Effects on Learning', Cognitive Science,[Online]. 12

(2), pp. 257-285. [Accessed 24 April 2017]. Available

from: http://dx.doi.org/10.1207/s15516709cog1202_4.

Sweller, J., van Merrienboer, J. J. G. & Paas, F. G. W. C.

1998, 'Cognitive Architecture and Instructional

Design', Educational Psychology Review,[Online]. 10

(3), pp. 251-296. [Accessed 25 April 2017]. Available

from: http://dx.doi.org/10.1023/A:1022193728205.

Thompson, A. D., Simonson, M. R. & Hargrave, C. P.

1992, Educational technology: A review of the

research: JSTOR.

Turk, M. 2014, 'Multimodal interaction: A review', Pattern

Recognition Letters,[Online]. 36, pp. 189-195.

[Accessed 5 June 2017]. Available from:

www.sciencedirect.com/science/article/pii/S01678655

13002584.

Van de Pol, J., Volman, M. & Beishuizen, J. 2010,

'Scaffolding in teacher–student interaction: A decade of

research', Educational Psychology Review,[Online]. 22

(3), pp. 271-296. [Accessed 9 Septemeber 2017].

Available from: https://link.springer.com/article/

10.1007/s10648-010-9127-6.

Van de Pol, J., Volman, M., Oort, F. & Beishuizen, J. 2014,

'Teacher Scaffolding in Small-Group Work: An

Intervention Study', Journal of the Learning

Sciences,[Online]. 23 (4), pp. 600-650. [Accessed 6

Septemeber 2017]. Available from: http://dx.doi.org/

10.1080/10508406.2013.805300.

Van Gog, T. & Paas, F. 2008, 'Instructional efficiency:

Revisiting the original construct in educational

research', Educational psychologist, [Online]. 43 (1),

pp. 16-26. [Accessed 21 May 2017]. Available from:

http://www.tandfonline.com/doi/abs/10.1080/0046152

0701756248.

Westerfield, G., Mitrovic, A. & Billinghurst, M. 2015,

'Intelligent Augmented Reality Training for

Motherboard Assembly', International Journal of

Artificial Intelligence in Education,[Online]. 25 (1), pp.

157-172. [Accessed 9 July 2017]. Available from:

https://link.springer.com/article/10.1007%2Fs40593-

014-0032-x.

Wood, D., Bruner, J. S. & Ross, G. 1976, 'The role of

tutoring in problem solving', Journal of child

psychology and psychiatry, [Online]. 17 (2), pp. 89-

100. [Accessed 12 July 2017]. Available from:

http://onlinelibrary.wiley.com/doi/10.1111/j.1469-

7610.1976.tb00381.x/full.

CSEDU 2018 - 10th International Conference on Computer Supported Education

244