Information Assistance for Smart Assembly Stations

Mario Aehnelt

and Sebastian Bader

Fraunhofer IGD, Joachim-Jungius-Str. 11, 18059 Rostock, Germany

MMIS, University of Rostock, Albert-Einstein-Str. 22, 18059 Rostock, Germany

Keywords:

Smart Manufacturing, Information Assistance, Cognitive Architectures.

Abstract:

Information assistance helps in many application domains to structure, guide and control human work pro-

cesses. However, it lacks a formalisation and automated processing of background knowledge which vice

versa is required to provide ad-hoc assistance. In this paper, we describe our conceptual and technical work to

include contextual background knowledge in raising awareness, guiding, and monitoring the assembly worker.

We present cognitive architectures as missing link between highly sophisticated manufacturing data systems

and implicitly available contextual knowledge on work procedures and concepts of the work domain. Our

work is illustrated with examples in SWI-Prolog and the Soar cognitive architecture.

1 INTRODUCTION

Evaluations show that people with a detailed work

plan complete their tasks faster than without it, even

if they did not carry out the planning themselves

(Kokkalis et al., 2013). This is a key motivation for

intelligent systems which assist the worker by creat-

ing work plans autonomously and guide him through

single work procedures aiming to improve both ef-

ﬁciency and effectiveness of his work. Such intelli-

gent systems will help in manufacturing to ensure a

high product quality even when working with insuf-

ﬁciently qualiﬁed personnel. They inform the worker

about current and upcoming tasks, provide him with

detailed knowledge on assembly procedures, or mon-

itor correct work order execution.

Although manufacturing industries already use

powerful data management systems that support the

planning, execution and monitoring of production

processes, there is still a lack of methods and tech-

nologies which bring intelligent assistance to the shop

ﬂoor. Basically, it lacks an automated processing

of the lion’s share of domain dependent background

knowledge. It is hidden in work related standards,

regulations, guidelines or simply maintained by ex-

perts, thus not available for the average worker.

Our research speciﬁcally addresses information

assistance for assembly stations at the manufacturing

shop ﬂoor. Although smart factories establish digital-

isation and automation to streamline manufacturing

processes and quality, there is still the need for man-

ual assembly operations (W

urtz and K

olmel, 2012).

Here, it requires systematic information assistance in

order to manage the complexity and heterogeneity of

extremely small lot sizes.

Throughout this paper we focus on smart factories

in which individually customised products are assem-

bled by humans. In particular, we assume a lot size

of one. This implies, that basically every product is

unique and required new construction plans. In con-

trast to larger lot sizes or series production, it is not

proﬁtable to invest much into product-speciﬁc assis-

tance systems. Therefore, we need to derive useful

assistance systems from existing information sources.

This paper is organised as follows: First, we dis-

cuss different types of assistance within the focus of

manual assembly processes. Sec. 3 and 4 describe

how to detect important situations and how to pro-

vide assistance, respectively. In Sec. 5 we show how

cognitive architectures can be used to formalize and

process the missing background knowledge. We con-

clude our work with Sec. 6.

2 REQUIRED ASSISTANCE

Below we discuss use cases in which different types

of assistance are required to support manual assembly

processes. But ﬁrst we give a short introduction to

assembly work activities.

The assembly of machines and technical systems

is an essential part of production. It consumes up to

143

Aehnelt M. and Bader S..

Information Assistance for Smart Assembly Stations.

DOI: 10.5220/0005216501430150

In Proceedings of the International Conference on Agents and Artiﬁcial Intelligence (ICAART-2015), pages 143-150

ISBN: 978-989-758-074-1

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

40 percent of costs and even 70 percent of production

time. During an assembly single machine parts are

joined to ﬁrst-order assemblies (pre-assembly), then

to assemblies of higher order (intermediate assem-

bly) and ﬁnally to an end product (ﬁnal assembly).

The German VDI guideline 2860 (German Engineers’

Association, 1990) further differentiates between pri-

mary assembly, which includes the main joining oper-

ations as deﬁned with the DIN 8580 (DIN Deutsches

Institut f

ur Normung e.V., 2003), and secondary as-

sembly, including all additional assembly activities

like handling, adjustment and control of parts, mate-

rial, tools, and machines. Assembly operations such

as joining directly refer to physical activities of the

worker. A more detailed description of the manual

operations as well as of the physical work environ-

ment can , for example, be found in (Aehnelt et al.,

2014).

Traditionally, a worker is equipped with work or-

ders describing the work to be done on an abstract

level. It strongly depends on his individual knowl-

edge and experiences to interpret the given informa-

tion correctly. Information assistance, especially in

complex and continuously changing assembly work

processes, improves the quality of work by ensuring

an immediate transfer of required work information

to the workplace and back to planning systems for ex-

ample. Here, we differentiate between ﬁve general

types of information assistance:

• Raising awareness: The worker requires up-to-

the-minute knowledge about his direct and rele-

vant work environment in order to align his own

activities accordingly. Knowing early enough the

malfunction or breakdown of a machine, which

produces parts for his own assembly, inﬂuences

his situational decisions and activities. Thus, in-

formation assistance needs to make the worker

aware of relevant states, events and occurrences

within the work environment which have an in-

ﬂuence on the planning and execution of the

worker’s tasks.

• Guiding: The worker requires orientation with re-

spect to his current and upcoming assembly tasks.

This needs to be given through operational guid-

ance which ﬁlters available information for each

speciﬁc work step, in order to reduce the parallel

information load to a required minimum. Know-

ing the exact joining procedure beforehand does

not reduce the risk of failures, especially in com-

plex assembly cases. Thus, information assistance

needs to split complex procedures into smaller

but easier understandable parts, used to guide the

worker step-by-step through the assembly.

• Monitoring: In the ﬁrst place, monitoring the as-

sembly process has a practical value. It allows a

detailed comparison as well as re-calculation of

planned and real time ﬁgures. Additionally, it en-

ables the early identiﬁcation of quality issues or

interruptions. Information assistance needs to col-

lect required data from the workplace which sup-

ports the continuously production re-planning. It

also needs to control the correct execution of as-

sembly procedures to avoid reworking in case of

wrong assembly orders, skipped parts or incorrect

tool usage.

• Documenting: When it comes to quality issues or

even complaints, information assistance needs to

support tracking back these issues to their roots,

which requires a parallel documentation of assem-

bly tasks. However, this kind of documentation

can also be helpful to evaluate assembly proce-

dures ﬁnding examples of best practice or expert

knowledge inherited within individual work pro-

cesses.

• Guarding: The physical and cognitive loads at the

assembly workplace vary from situation to situ-

ation. Information assistance needs to guard the

worker from overload by balancing the load lev-

els within healthy borders or by visualising it.

Based on the use cases introduced above, we dis-

cuss the current state of the art in assistance for man-

ual assembly tasks. Smart assistance is no novelty

in manufacturing. However, we ﬁnd there a major-

ity of specialised and single task solutions focussing

on quality assurance and information transfer (Berndt

and Sauer, 2012). Other research deals with con-

cepts for smart factories which automate the plan-

ning and execution of manufacturing processes in au-

tonomously working factories. Focussing on the au-

tomation of robot cells Mayer et al. proposed the

usage of intelligent systems to resemble human de-

cision making and problem solving for complex as-

sembly tasks (Mayer et al., 2011). They introduce

the cognitive control unit (CCU) which ensures the

numerical planning of robot behaviour backed by a

cognitive architecture. Cognitive architectures can be

understood as a mean to implement intelligent and au-

tonomous behaviour in assistance applications. They

have proven capable of supporting complex problem

solving tasks. A recent example is the simulation of

mission management for unmanned aircrafts (Gunetti

et al., 2013).

Our own work contributes to the growing demand

of information assistance for manual work operations

in manufacturing which underlies human ﬂexibility

and failures as motivated by (Bader and Aehnelt,

2014). In particular, we focus on assistance that can

be generated automatically from existing knowledge

ICAART2015-InternationalConferenceonAgentsandArtificialIntelligence

144

wire-spool

operation = soldering

tools = [soldering-iron]

wires tin-solder

|=|

wire wire wire wire wire

>> >> >> >>

Figure 1: A simple task model stating that a wire-spool is

build from 5 wires and tin-solder. In addition, the top-most

node contains the joining operation (soldering) and the tools

to be used (soldering-iron).

sources. This allows to use the approach also for

small lot sizes.

3 DETECTING SITUATIONS

To actually provide assistance, we ﬁrst need to rec-

ognize situations in which this is necessary. We will

discuss the recognition process only brieﬂy, because

we simply utilize an approach presented at ICAART

2014. Therefore, we only review some ideas pre-

sented in (Bader and Aehnelt, 2014). Based on a

formalization of assembly tasks, using so-called task

models, a Hidden Markov Model (HMM) is synthe-

sized. The resulting HMM is used to ﬁlter sensor data

and compute a probability distribution over the cur-

rent state of the process. As sensory inputs only the

processed building blocks are used and the accuracy is

analysed with respect to different sensor errors. Here,

we use a similar approach, but instead of using the

single building blocks as sensory inputs, we also uti-

lize the tools used for the assembly. Figure 1 shows

such an enhanced task model, stating that a wire-spool

is built from 5 wires and some tin-solder. The parts

are joined by soldering them using a soldering-iron.

To raise awareness for important state changes in

the environment as well as for guiding the worker, the

current state of the assembly process needs to be de-

termined. For this, we rely on the results reported in

(Bader and Aehnelt, 2014), indicating that this is in-

deed feasible. Based on our annotated models, we

believe that the results should be even better, but a

detailed analysis is subject to future work.

To detect assembly errors or deviations from the

usual procedure is harder to track. In (Bader and

Aehnelt, 2014), the authors describe how the system

reacts with respect to different types of sensor errors

(missing and repeated readings).

But here, we need to detect actual assembly errors,

which has not been tackled before. As the task model

produces(’WireMachine’, wire ).

task(wire−spool, % goal

soldering , % operation

[ soldering −iron], % tools

[ tin −solder, 5∗wire ]). % parts

Listing 1: Static background information describing that a

WireMachine produces wires and that a wire-spool is as-

sembled by soldering wires using a soldering-iron. The task

corresponds to the task-model shown in Figure 1.

storage(wire , 3).

assembles(worker1, wire−spool).

broken(’WireMachine’, ’no copper available ’ ).

knows(worker1, howTo(soldering )).

Listing 2: Dynamic background information describing that

3 wires are available, worker1 assembles a wire-spool, and

that the wire machine is broken.

describes valid construction paths only, the result-

ing HMM will always compute a probability distri-

bution over valid paths. Therefore, an assembly error

is not directly observable. To track assembly errors

nonetheless, we analyse the development of the prob-

ability distribution over states. Because valid paths

result in rather crisp probability distributions (i.e., one

path has a very high probability while all others a low

one), errors result in a higher entropy of the distri-

bution. Based on this insight, we are able to detect

situations in which an error occurred.

4 PROVIDING ASSISTANCE

Information assistance at the assembly workplace

aims at the continuous information exchange between

leading manufacturing data systems (enterprise re-

source planning, manufacturing execution, etc.) and

the worker in order to support efﬁcient working and to

avoid interruptions, failures or quality related issues.

To keep argumentation and presentation simple,

we assume that the state of the world is known ex-

actly. The system described here, has been imple-

mented in SWI-Prolog

. The state of the world is de-

scribed as static and dynamic facts shown in Listing 1

and 2, respectively.

Below, we work out information which helps to

provide awareness on the assembly situation, guides

the worker through an assembly, and ﬁnally monitors

his work and results.

http://www.swi-prolog.org

InformationAssistanceforSmartAssemblyStations

145

informationDemand(InfoType, Person, Info) :−

% Person assembles a given item ..

assembles(Person, Item ),

% ... containing a Part with a given Quantity

isPartOf( Part , Item, Quantity ),

% The Machine producing the Part ...

produces(Machine, Part ),

% ... is broken for a given Reason

broken(Machine, Reason),

% The Quantity is larger than the StoredQuantity

storage( Part , SQ), Quantity > SQ,

InfoType = awareness(stateOf(Machine)),

Info = brokenMachine(Machine, Part, SQ).

Listing 3: Speciﬁcation of an information demand to raise

awareness with respect to a broken machine. The predicate

isPartOf allows to access sup-parts as deﬁned through the

task model.

4.1 Raising Awareness

Situational awareness with respect to his work envi-

ronment helps the worker to orientate within complex

processes and to align his own activities accordingly

(Gutwin and Greenberg, 2002). It can be understood

as inherent information demand of carrying out and

completing work tasks. All changes of the virtual or

physical work environment which inﬂuence the ongo-

ing or upcoming assembly tasks need to made aware

to the worker. This includes, for example:

• planned tasks which can change in time and pri-

ority,

• missing material, tool or information which are

required but not available for assembly, or

• deviations from normal procedures, orders and

qualities.

Depending on the information impact as well as ur-

gency, assistance needs to help perceiving it embed-

ded in the ongoing work ﬂow.

After evaluating the information demand as

speciﬁed in Listing 3 with respect to the cur-

rent state of the world, InfoType is uniﬁed with

awareness( stateOf (’WireMachine’)) and Info with

brokenMachine(’WireMachine’, wire, 3). In addition,

all premisses are known. This allows to generate the

output shown in Figure 2. A simple verbalisation

engine is used to translate the predicates (e.g.,

shown in bold font in Listing 3) instantiated while

computing the information demand into english

sentences. Predicates corresponding to dynamic facts

are verbalised as assumptions.

Type: awareness(stateOf(WireMachine))

User: worker1

Info: We are running low on wire, because WireMa-

chine is broken and only 3 items are left in storage.

Explanation: It is assumed that worker1 assembles

wire-spool. Item wire is needed 5 times to assem-

ble a wire-spool. WireMachine produces wire. It

is assumed that WireMachine is broken, because

no copper available. It is assumed that 3 items of

wire are left in storage. Worker1 should be aware

of the state of WireMachine.

Figure 2: Infomessage and explanation generated for the

information demand speciﬁed in Listing 3.

4.2 Guiding

Similar to formal education processes, information

assistance in form of guiding can be understood as an

informal way of mediating and learning facts (what),

procedures (how) and concepts (why) required for a

speciﬁc assembly task. The shaping and depth of

guidance varies depending on a speciﬁc assistance ob-

jective to be supported (Aehnelt and Urban, 2014). In

a ﬁrst step the worker is required to remember, under-

stand and apply the given information in order to pre-

pare and execute his assembly task correctly. Thus,

information assistance has to collect and visualise:

• bills of material which identify the material to be

used for an assembly step,

• bills of tools which lists the tools and machines to

be used for joining procedures,

• procedures which describe the correct handling,

adjusting, joining, and controlling of materials

and tools including safety relevant information,

and

• planned ﬁgures which detail the expected assem-

bly times and results.

As motivated in Section 2, it is important to split com-

plex assembly procedures into smaller instructions

(steps), reducing thus cognitive loads for the worker

and giving them a clear work structure (Kokkalis

et al., 2013). Showing then the required informa-

tion parallel to the ongoing work process for each step

only, helps to achieve the three assistance objectives.

Similar to the information demand speciﬁed in

Listing 3, it is possible to formalise guiding knowl-

edge. This includes for example the tool to be used for

the current assembly operation as shown in Listing 4.

Figure 3 shows the resulting output after verbalising

the predicates.

ICAART2015-InternationalConferenceonAgentsandArtificialIntelligence

146

informationDemand(InfoType, Person, Info) :−

% Person assembles a item

assembles(Person, Item ),

% get Join−Operation from task model

task(Item, JoinOp, Tools , Parts ),

InfoType = guiding(JoinOp),

Info = currJoinOp(Person, Item, JoinOp, Tools ).

Listing 4: Speciﬁcation of an information demand with re-

spect to the current type of join operation and the tools to

be used.

Type: guiding(JoinOp)

User: worker1

Info: You should soldering the wire-spool using

soldering-iron.

Explanation: It is assumed that worker1 assem-

bles wire-spool. Item wire-spool is assembled

via soldering, using a soldering-iron.

Figure 3: Infomessage and explanation generated for the

information demand speciﬁed in Listing 4.

Similar speciﬁcations can be used to refer to the

items to be used. For complex assembly tasks, many

different types of joining operations have to be per-

formed by the worker. Usually, not every worker has

the same in-depth training for all of them. There-

fore, more background knowledge should be provided

for unknown operations. This can be formalised as

shown in Listing 5.

informationDemand(InfoType, Person, Info) :−

% Person assembles a given part

assembles(Person, Part ),

% get information from task model

task( Part , JoinOp, Tools , Children ),

% if there is a description D of the op ...

bgInfo(joinOp(JoinOp), description (D)),

% ... and the person does not know it

not( knows(Person, howTo(JoinOp)) ),

InfoType = bgInfo(joinOp(JoinOp))

Info = bgInfo(joinOp(JoinOp), description (D)).

Listing 5: Speciﬁcation of an information demand with re-

spect to missing background knowledge.

Assuming knows(worker1, howTo(soldering)) to be

true, and false for worker2, results in different infor-

mation assistance for both.

4.3 Monitoring

Monitoring the ongoing assembly activities of the

worker establishes a feedback channel to the infor-

mation assistance system. It requires a close observa-

tion of work progress, rejects, and issues for practical

reasons. Manufacturing execution systems need this

information to allow a detailed planning of produc-

tion processes. For situation recognition we require

more detailed knowledge from monitoring: the ma-

terial taken as well as tools picked up and their con-

ﬁguration, which enables us to draw conclusions on

the current assembly step executed and possible devi-

ations in comparison to the provided instructions.

5 USING SOAR TO PROVIDE

ASSISTANCE

The previous sections showed conceptual and tech-

nical considerations with respect to acquiring knowl-

edge on actual assembly situations as well as to pro-

viding information assistance at the assembly work-

place. Although, a vast amount of required infor-

mation can already be found in manufacturing data

management systems, the major share of procedural

and conceptual knowledge is not yet formalised in

systems which allow their automated processing (see

Figure 4). We ﬁnd work instructions, standards, or

assembly guidelines normally written in natural lan-

guage within accompanying documents. However,

there has already been research to distinguish between

the semantic meaning of instructions and their vi-

sual representation (Mader and Urban, 2010) based

on controlled vocabularies which allows for automa-

tion. They still require a manual authoring of instruc-

tions for each assembly process individually. What

we require in contrast for guiding for example, is an

abstract formalisation of assembly procedures in gen-

eral which is ﬁlled at runtime with factual knowledge

about the speciﬁc product or situation.

Cognitive architectures are a mean to bridge the

gap between common assembly descriptions, that we

ﬁnd in VDI 2860 or DIN 8580, and reusable proce-

dural knowledge in information assistance systems.

Below, we summarize our approach of utilising the

cognitive architecture Soar for assisting the worker

during assembly.

5.1 Cognitive Architectures

Cognitive architectures can be traced back to Newell’s

early hypothesis that any artiﬁcial intelligence is

based on a symbol system and related rules (Newell,

1980). As of today, a cognitive architecture describes

the mental structure for human information process-

ing, the representation and organization of informa-

tion within these structures as well as the functional

InformationAssistanceforSmartAssemblyStations

147

processing required to acquire, use, and modify in-

formation (Langley et al., 2008). Hence, they allow

modelling and implementing intelligent behaviour in

smart applications and environments. We also need it

for providing assistance as described in Section 2. In

our own work we use the cognitive architecture Soar

(state, operator and result) for:

• situation detection based on observations of the

physical work environment and following reason-

ing,

• the formalisation and processing of contextual

background knowledge (e.g. procedures, expla-

nations),

• interactions with the worker as well as the phys-

ical environment to provide assistance by raising

awareness and guiding for example, and for

• additionally learning assembly related practices

from observation.

In general, processes in Soar are related to the grad-

ual alternation of information and states in working

or long-term memory (Laird, 2012). Here, a situation

is formalized as a state in working memory, which

is modiﬁed by evaluating and applying operators un-

til an intended ﬁnal state is reached. The operator

deﬁnition consists of required conditions and actions

on the working memory. It inherits procedural and

conceptual knowledge from the corresponding knowl-

edge domain. New operators can also be derived by

observation of decision making processes (chunking)

and through learning processes.

First examples on how we use Soar to provide as-

sembly assistance are illustrated below.

Assembly Information

Facts Procedures Concepts

• tasks

• material

• tools

• configuration

• instructions

• guidelines

• standards

• best practice

• explanations

• consequences

• simulations

What? How?

Why?

Enterprise Resource

Planning System

(ERP)

Manufacturing

Execution System

(MES)

Enterprise Content

Management

System (ECMS)

Expert Knowledge

Enterprise Content

Management

System (ECMS)

Expert Knowledge

Figure 4: Required information is contained in different en-

terprise resources. Partly it is individual expert knowledge

which is not externalised in any management system.

5.2 Situation Detection

In Soar we represent the individual situations of

the work environment during an assembly by states.

Each state identiﬁes a different condition of elements

within the Soar working memory, which ﬁnally holds

a virtual copy of the physical environment. Thus, we

transfer the state of real objects, e.g. tools, the mate-

rial stack, or even the workplace, into logical objects

and their attributes in working memory. Soar con-

nects then to sensors which allow the observation of

individual object states and events, e.g. the usage of a

tool, in order to update attributes of the related work-

ing memory object. The working memory is so the

basis of all reasoning on discrete states of the work

environment.

In Listing 6 we illustrate the usage of Soar’s state

operator mechanism for a speciﬁc soldering situation.

It consists of two parts, an operator proposal rule and

an application rule. The ﬁrst one deﬁnes the pre-

condition of a state described by working memory ob-

jects and their attributes. In our example it requires at

least two materials or parts and a soldering iron in or-

der to start a soldering operation. If the current state

of the working memory matches these conditions, the

operator join-part becomes candidate in following de-

cision making.

# parts can only be soldered if there are at least

# two parts taken , a soldering iron and solder to

# support the joining operation

sp {propose∗soldering−parts

( state <s> ˆname assemble)

(<s> ˆcount taken > 1)

(<s> ˆiron taken <=> yes)

(<s> ˆ solder taken <=> yes)

−−>

(<s> ˆoperator <o> + =)

(<o> ˆname soldering−parts ) }

# soldering parts reduces the taken parts to a

# single compound part and consumes solder

sp {apply∗soldering−parts

( state <s> ˆoperator.name soldering−parts)

(<s> ˆcount taken <t>)

(<s> ˆ solder taken <m>)

−−>

(<s> ˆname soldering)

(<s> ˆcount taken 1)

(<s> ˆcount taken <t> −)

(<s> ˆ solder taken <m> −) }

Listing 6: Deﬁnition of operator and production rules in

Soar for joining assembly step.

Soar evaluates the likelihood of each operator

based on the current state of objects in working mem-

ory and all candidate proposal rule deﬁnitions. It uses

ICAART2015-InternationalConferenceonAgentsandArtificialIntelligence

148

contextual knowledge (see Section 5.3) to compare

and select candidate operators during decision mak-

ing.

With applying ﬁnally the operator soldering-parts

the working memory state is changed by modifying

single memory objects, e.g. by reducing the amount

of available parts.

5.3 Contextual Knowledge

One of Soar’s strengths lays in its interaction be-

tween working and long-term memory. While the

working memory holds information about the cur-

rent condition of logical objects (see Section 5.2), the

long-term memory represents the contextual knowl-

edge which is required to select and apply an opera-

tor. Here, we ﬁnd ﬁve different knowledge types in

Soar: knowledge which qualiﬁes an operator for a sit-

uation, knowledge to compare operators, knowledge

to select a single one, knowledge to change the work-

ing memory, and knowledge to elaborate a state. All

types contain contextual knowledge of the application

work domain, in our case of assembly activities as for-

mulated in VDI 2860 and DIN 8580.

In Listing 7 we encode this assembly knowledge

to deﬁne the sequential order of single work steps and

their requirements. In this example, we require fur-

ther solder material prior to collecting the soldering

iron, which will be deﬁned by additional preference

operators, such as +,>,<, or !.

# soldering parts requires additional solder

# material which need to be taken ﬁrst

sp {propose∗take−solder

( state <s> ˆname assemble)

(<s> ˆiron taken <=> yes −ˆsolder taken)

−−>

(<s> ˆoperator <o1> + ; <o1> > <o2>)

(<o1> ˆname take−solder)

(<o2> ˆname take−iron) }

Listing 7: Contextual knowledge of the work domain is en-

coded in Soar’s production rules.

In this way, we were able to transfer the relevant as-

sembly process logic into Soar operators. They help

us guiding the worker with small sized assembly in-

structions as required for an automated information

assistance.

5.4 Interaction

We also use Soar to establish an interaction between

information assistance system and the worker as well

as vice versa. As described in Section 4 we aim to

raise the worker’s awareness with respect to relevant

information on his ongoing assembly process, guide

him step by step through the assembly, and monitor

his work activities. It ﬁnally requires the interaction

to inform him and collect data from him. This can

also be formalised by operator rules.

# provide information assistance once the worker

# is not informed on his following work step

sp {propose∗inform

( state <s> ˆtype state )

(<s> ˆname <n> − ˆis informed)

−−>

(<s> ˆoperator <o1> !)

(<o1> ˆname inform) }

Listing 8: Information assistance is modeled as operators in

Soar.

Listing 8 shows the proposal rule of an inform op-

erator which will provide the worker with instructions

for his next assembly steps. In a similar manner we

deﬁne operators for raising awareness for example.

6 CONCLUSIONS

In this paper we showed how to provide information

assistance for a smart assembly station. After deﬁn-

ing different types of information demands, we brieﬂy

discussed a possibility to detect situations in which

assistance is needed. Then we showed how assistance

can be provided using a crisp state of the world and

logical speciﬁcations of the information demand (us-

ing an implementation in SWI Prolog). Even though,

most information can already be found in manufactur-

ing data management systems, the majority of proce-

dural and conceptual knowledge is not yet formalised.

To bridge this gap we use the Soar architecture.

Although the concept of cognitive architectures

is not new to implementing systems with intelligent

behaviour, it is still rarely used to make the contex-

tual background knowledge from an application do-

main accessible for complex problem solving tasks.

Our approach shows on both, conceptual as well as

technical level, the usage of an cognitive architecture

for supporting information assistance on the manu-

facturing shop ﬂoor. It illustrates the role of logical

modelling and the transfer of implicit and barely for-

malised knowledge into predicate logic and state op-

erators.

However, one of the next required steps is to learn

new procedures and novel connections from observa-

tion of real assembly activities. Here it is promising

to start with a basic guidance skeleton and detail the

missing assembly steps by tracking, interpreting, and

learning from work procedures of assembly experts.

InformationAssistanceforSmartAssemblyStations

149

In addition, we are working on an experimental eval-

uation of the ideas. This includes the collection of

real sensor data and the formalisation of real-world

examples. And ﬁnally we will work on an automated

transfer of real existing knowledge into an assistance

system.

REFERENCES

Aehnelt, M., Gutzeit, E., and Urban, B. (2014). Using activ-

ity recognition for the tracking of assembly processes:

Challenges and requirements. In Bieber, G., Aehnelt,

M., and Urban, B., editors, WOAR 2014, pages 12–21.

Fraunhofer-Verlag, Stuttgart.

Aehnelt, M. and Urban, B. (2014). Follow-me: Smartwatch

assistance on the shop ﬂoor. In Proceedings of the

16th International Conference on Human-Computer

Interaction (HCII), Lecture Notes in Computer Sci-

ence. Springer-Verlag.

Bader, S. and Aehnelt, M. (2014). Tracking assembly pro-

cesses and providing assistance in smart factories. In

Proceedings of the 6th International Conference on

Agents and Artiﬁcial Intelligence (ICAART).

Berndt, D. and Sauer, S. (2012). Visuelle assistenzsysteme

in der montage verhindern ausf

alle. MM Maschinen-

Markt, (19):46–49.

DIN Deutsches Institut f

ur Normung e.V. (2003). Din 8580

manufacturing processes - terms and deﬁnitions, divi-

sion.

German Engineers’ Association (1990). Vdi 2860:1990-

05 assembly and handling; handling functions, han-

dling units; terminology, deﬁnitions and symbols.

Gunetti, P., Dodd, T., and Thompson, H. (2013). Simula-

tion of a soar-based autonomous mission management

system for unmanned aircraft. Journal of Aerospace

Information Systems, 10(2):53–70.

Gutwin, C. and Greenberg, S. (2002). A descriptive

framework of workspace awareness for real-time

groupware. Computer Supported Cooperative Work

(CSCW), 11(3):411–446.

Kokkalis, N., K

ohn, T., Huebner, J., Lee, M., Schulze, F.,

and Klemmer, S. R. (2013). Taskgenies: Automat-

ically providing action plans helps people complete

tasks. ACM Transactions on Computer-Human Inter-

action, 20(5):1–25.

Laird, J. E. (2012). The soar cognitive architecture. Arti-

ﬁcial Intelligence and Simulation of Behavior Quar-

terly, (134):1–4.

Langley, P., Laird, J. E., and Rogers, S. (2008). Cognitive

architectures: Research issues and challenges. Cogni-

tive Systems Research, 10(2):141–160.

Mader, S. and Urban, B. (2010). Creating instructional con-

tent for augmented reality based on controlled natu-

ral language concepts. In Proceedings of 20th Inter-

national Conference on Artiﬁcial Reality and Telexis-

tence (ICAT 2010).

Mayer, M. P., Odenthal, B., Wagels, C., Kuz, S., Kausch, B.,

and Schlick, C. M. (2011). Cognitive engineering of

automated assembly processes. In Harris, D., editor,

Engineering Psychology and Cognitive Ergonomics,

volume 6781 of Lecture Notes in Computer Science,

pages 313–321. Springer Berlin Heidelberg, Berlin,

Heidelberg.

Newell, A. (1980). Physical symbol systems*. Cognitive

Science, 4(2):135–183.

urtz, G. and K

olmel, B. (2012). Integrated engineering

– a sme-suitable model for business and information

systems engineering (bise) towards the smart factory.

In Camarinha-Matos, L., Xu, L., and Afsarmanesh,

H., editors, Collaborative Networks in the Internet of

Services, volume 380 of IFIP Advances in Informa-

tion and Communication Technology, pages 494–502.

Springer Berlin Heidelberg.

ICAART2015-InternationalConferenceonAgentsandArtificialIntelligence

150