Contextual Design in Industrial Settings:

Experiences and Recommendations

Mirjam Augstein

, Thomas Neumayr

, Sebastian Pimminger

, Christine Ebner

, Josef Altmann

and Werner Kurschl

Communication and Knowledge Media, University of Applied Sciences Upper Austria, Hagenberg, Austria

Research and Development, University of Applied Sciences Upper Austria, Hagenberg, Austria

Operations Management, University of Applied Sciences Upper Austria, Steyr, Austria

Human-Centered Computing, University of Applied Sciences Upper Austria, Hagenberg, Austria

christine.ebner@fh-steyr.at

Keywords:

Contextual Design, Contextual Inquiry, Requirements Analysis, User Involvement.

Abstract:

The Contextual Design (CD) methodology offers a framework for planning and implementing a user-centered

design process throughout all project phases. It is team-based and was designed especially for interdisciplinary

teams. The application of CD is particularly proﬁtable in projects confronted with implicit requirements and

hidden factors of inﬂuence. In contrast to many other design and evaluation methods such as focus groups or

usability tests, CD does not take users out of their everyday setting and more easily reveals important design

issues and contextual inﬂuences like users’ motivation, values, emotions or real-time interruptions. Despite

these advantages, CD is often not used due to high costs in terms of time and effort. This paper provides a

report on experiences with CD in two research projects in the industry domain. It is intended to help other

researchers to plan and implement a CD process in industrial settings and beneﬁt from our lessons learned.

1 INTRODUCTION

Requirements elicitation is a crucial task in every pro-

ject that might have strong inﬂuence on the further

procedure, methodical and implementation-related

decisions and quality of the results. Gathering requi-

rements is a non-trivial task, especially within pro-

jects of broader context and of scientiﬁc nature (where

possible solutions are usually not known a-priori).

Additionally, as e.g., discussed by (Holtzblatt et al.,

2005; Holtzblatt and Beyer, 2017) the description of

requirements is difﬁcult for humans who often fail

in uncovering so-called “hidden” factors, especially

when being personally involved or affected. The rea-

sons for this can e.g., be rooted in a certain operational

blindness (when being directly or indirectly involved)

or an (unintended) mental limitation of design free-

dom due to everyday experiences and habits. Humans

often fail in describing what they do, and how and

why they do it. They tend to perform certain tasks,

especially routine ones, subconsciously and particu-

larly fall short in the perception of contextual inﬂu-

ences. Thus, requirement analysis might fail even if

all important stakeholders are involved. The Contex-

tual Design (CD) methodology described by (Holt-

zblatt and Jones, 1993; Beyer and Holtzblatt, 1999;

Holtzblatt et al., 2005; Holtzblatt and Beyer, 2017)

offers a framework for planning and implementing a

User-Centered Design process with particular focus

on uncovering hidden information and identiﬁcation

of actual user needs. It is an extensive step-by-step

process for collecting ﬁeld data and using it to de-

sign any sort of technical product (Beyer and Holtz-

blatt, 1997) and involves techniques to collect, ana-

lyze and present user data, drive ideation from these

data, design product solutions and iterate them with

customers (Holtzblatt and Beyer, 2014).

CD is intended to accompany a project from the

ﬁrst elicitation of requirements to a successful im-

plementation. It thus comprises an interrelated se-

quence of methods that suit different project phases.

(Holtzblatt and Beyer, 2014) describe CD as a team-

based methodology designed to “take advantage of a

cross-functional team including product management,

marketing, product architects, user experience desig-

ners, developers, and service designers, each provi-

ding their unique skills and insights to help invent

the right solutions for users”. One of the main is-

Augstein, M., Neumayr, T., Pimminger, S., Ebner, C., Altmann, J. and Kurschl, W.

Contextual Design in Industrial Settings: Experiences and Recommendations.

DOI: 10.5220/0006674904290440

In Proceedings of the 20th International Conference on Enterprise Information Systems (ICEIS 2018), pages 429-440

ISBN: 978-989-758-298-1

429

sues related to other design and evaluation methods

such as usability tests, focus groups or questionnai-

res (or, in general, any technique with ﬁxed questions

and pre-deﬁned tasks) is that they take users out of

the context of their everyday settings. According to

(Holtzblatt and Beyer, 2014), the missing context le-

ads to such methods often not being able to reveal the

most important design issues like users’ motivation,

values, emotions, strategies, work-arounds, real-time

interruptions and interactions with others. Thus, it is

a main aim of CD to understand users in their own

context and to be able to apply these understandings

to design problems.

Although the application of CD involves a wide

range of advantages, it is often not used, especially

in time-critical projects (which already gave rise to

a compacted variant called rapid CD in 2005). The

main disadvantage which is often decisive, is that ap-

plying CD is costly in terms of time and effort. It re-

quires not only project personnel but also the coope-

rating clients to contribute intensively. This is cer-

tainly also the case during the requirements elicitation

phase of projects not using CD (especially those that

follow agile approaches), however the intensity incre-

ases with using CD. Further, for some domains there

are only few reported examples that provide lessons

learned and practical recommendations.

This paper describes the application of a CD ap-

proach in industrial settings using the example of two

different research projects (RP1 and RP2) that are

conducted by research organizations in cooperation

with industrial partners. It explains the different pha-

ses in the CD process and provides a practical set of

implications and recommendations related to appli-

cation of CD in industry. The following paragraphs

provide a concise overview of the two projects which

have in common that they aim at creating novel, inno-

vative solutions beyond the current state of the art.

RP1 researches novel interaction methods for ma-

nual industrial welding machines. It comprises con-

ceptualization as well as prototypic implementation

and evaluation in the ﬁeld and aims at improving the

welding process in terms of time and quality. Cur-

rently, the welding process often has to be interrupted

in case a parameter (such as current) needs to be chan-

ged on the ﬂy because every little movement of the

welder’s hands negatively affects the high-precision

welding task and its output. RP1 aims at the intro-

duction of alternative interaction methods that ﬁt the

welders’ needs and comply with industrial standards.

RP1 is a two-year project. The main reasons for em-

ploying CD methods in RP1 were that researchers did

not have domain knowledge, the speciﬁc (micro) in-

teractions with the welding machine cannot be simu-

lated easily outside the usage context, and is not com-

parable to working with other interactive systems in

regard to time pressure and precision.

RP2 aims at the conceptualization and prototy-

pic construction of a human-centered industrial work-

place. It focuses on two types of industrial work: i)

manufacturing/assembly (at the example of industrial

welding machines and industrial computers) and ii)

commissioning. The experiences described in this pa-

per are related to the ﬁrst use case (machine manufac-

turing) only. RP2 researches work context analysis

and personalized means of assistance via a contextual

feedback system. It is a four-year project and aims at

optimizing the industrial work place in terms of ergo-

nomics, efﬁciency and worker satisfaction. We have

experienced that (for the reasons mentioned earlier)

just asking workers about their needs for assistance

will not lead to success ultimately. Also, hidden kno-

wledge, which is not documented and passed on only

informally, is often necessary when assembling pro-

ducts. We employ CD methods to i) better understand

the users and ii) identify activities and work steps that

can be supported by technical assistance systems. Our

goal is to minimize the training phase and maximize

the quality of daily work.

2 RELATED WORK

This section explains in further detail the background

behind CD and summarizes selected CD use cases in

concrete industrial and other domains.

2.1 Contextual Design

According to (Holtzblatt and Beyer, 2014), CD was

ﬁrst invented back in 1988. In 1990, CD was ﬁrst

published and presented as an “emergent method for

building effective systems” at CHI (Wixon et al.,

1990). One of the objectives back then was to

challenge traditional, predictive research techniques

which “have fallen short of delivering timely and re-

levant design information”. The process explained by

Wixon et al. already involved the core characteristics

of today’s CD. For instance, they list the following

guidelines for the conduction of a contextual inter-

view (which has its roots e.g., in the work of (Wino-

grad and Flores, 1988; Ehn, 1988)): i) interview users

about their work in the real work environment, ii) be

concrete and talk about what the user is doing, just

did or talk in the context of a work product, iii) let the

user lead the conversation in order to co-interpret and

co-design, iv) expand and challenge the background

of assumptions one brings to each interview, probe all

ICEIS 2018 - 20th International Conference on Enterprise Information Systems

430

surprises and assumptions, v) summarize your under-

standing at the end of each session to determine what

to focus on next, vi) build an understanding of user

work and of the environment being designed, vii) un-

derstand, design and build a ﬁrst cut at a user environ-

ment, and viii) iterate the prototype with real users

doing real work. These guidelines are still valid to-

day, mainly for Contextual Inquiry (CI), i.e., the CD

method recommended for early project phases.

In 1993, (Holtzblatt and Jones, 1993) described

CI in further detail, afﬁrming the challenges addres-

sed by Wixon et al., e.g., that “people speak about

their work in abstractions” and pointing out the “diffe-

rence between summary information and ongoing ex-

perience”. According to Holtzblatt and Jones, when

asked about a computer system, people provide a

summary or their general opinion which is often a

strongly abstracted statement derived from everyday

experience. This might lead also to people not repor-

ting positive aspects because they are frequently not

aware of what they like about a system as it works

well and does not call attention to itself. We can af-

ﬁrm these problems as we experienced similar beha-

viours when talking to stakeholders. (Beyer and Holt-

zblatt, 1997) published the ﬁrst book describing CD

in-depth and involving an extensive guide to the pre-

paration and implementation of a CD process. They

explain all CD phases and provide practical exam-

ples of roles, artifacts, analysis methods and results.

(Holtzblatt et al., 2005) introduced Rapid Contex-

tual Design (RCD) which was our primary guide, ac-

counting for the immense effort involved with CD.

RCD provides a more dense guide for practitioners

and focuses on the core CD techniques that most ea-

sily drive customer data into the corporate design pro-

cess. Starting in 2013, CD was overhauled in or-

der to account for changes in technology and its in-

ﬂuence on humans’ lives, e.g., through “always-on,

always-connected, and always-carried devices” (Holt-

zblatt and Beyer, 2017).

2.2 Application Use Cases

(Gellatly et al., 2010) describe the ﬁrst of several CD

projects undertaken by the General Motors UX de-

sign team. The project called “Journey” focused on

gaining a deeper understanding of how drivers inte-

ract with today’s entertainment, communication, na-

vigation, and information systems in their vehicles. It

followed the CD process as described by (Beyer and

Holtzblatt, 1997; Holtzblatt et al., 2005) and speci-

ﬁcally aimed at i) documenting user intents for en-

tertainment, communication, navigation and informa-

tion system usage, ii) studying the balance between

in-car systems and carried-in devices usage, iii) de-

termining how users’ outside lives should be suppor-

ted in-car, iv) capturing how in-car tasks interact with

the driving task, v) determining effects of individual

differences in tolerance to sensory overload, mental

models of navigation and age, and vi) uncovering the

values that customers have around brand, design and

aesthetics. The process comprised CI, interpretation

sessions, afﬁnity diagram, work models and visioning

and is thus similar to ours regarding methodology.

However, while Gellatly et al. focus on the description

of their ﬁndings, our paper emphasizes more strongly

the experiences with CD and related lessons learned.

(Coble et al., 1995) describe a CI process con-

ducted to gather physicians’ requirements for a cli-

nical information system. Their CI sessions compri-

sed orientation, interview and wrap-up. Following the

CI session, data were analyzed and used to derive a

sequence, a ﬂow and a context model, and detailed

observations and a user proﬁle. The sequence mo-

del documented the sequences of activities the phy-

sician performed. The ﬂow model documented the

information and items ﬂowing between physician and

other people or places. The context model documen-

ted external inﬂuences which affected how the physi-

cian cared for patients. The observations contained

the details of what the physician did and were the

primary source for requirements. The process fol-

lowed in our research projects differed in that our

main data consolidation and analysis tool was the

afﬁnity diagram as suggested by (Holtzblatt et al.,

2005). Coble et al. describe their CI efforts as ex-

tensive and time-consuming which complies with our

experiences. E.g., they report 80 physician hours, 130

staff hours in direct participation, 90 staff hours in

post-session documentation and additional 1300 staff

hours in data analysis sessions over three months.

(Fouskas et al., 2002) applied CI for gathering

users’ requirements in the context of mobile exhibi-

tion services. They describe their process with a fo-

cus on the method itself, aiming at providing a guide

for further investigations and experiments in using the

CD methodology (with emphasis on CI). Their paper

is thus of similar nature compared to ours, however,

we believe that i) the application domain does actu-

ally inﬂuence “best practices” for the conduction of

CD/CI and ii) the characteristics of interactive soluti-

ons have changed since 2002, which is also afﬁrmed

by (Holtzblatt and Beyer, 2014). Other major diffe-

rences can be found in the project phase discussed;

Fouskas et al. do not describe the analysis of the data

gathered via CI, which is one of the main contributi-

ons of our paper. Further, they suggest the derivation

of a uniﬁed requirements speciﬁcation report from CI

Contextual Design in Industrial Settings: Experiences and Recommendations

431

data in combination with external business-strategic

and technological requirements. Our work uses CI

data as the exclusive source of user requirements.

(Viitanen, 2011) describes the application of CI

for user-centered clinical IT system design. She

brieﬂy outlines the structure and themes of her in-

quiries and lists advantages and challenges related to

the CI method. Her ﬁndings are largely rather gene-

ral (such as “might be time-consuming”) or closely

related to the domain (such as “makes it possible to

analyse clinicians’ actions with interactive systems in

environments in which numerous systems are used si-

multanously”). Some of the ﬁndings afﬁrm the issues

named by Holtzblatt et al. while others might not be

transferable to industrial settings. Here, we thus aim

at i) an in-depth description of the CD/CI process and

ii) a clear focus on lessons learned related to industry.

3 CONTEXTUAL DESIGN IN

INDUSTRIAL SETTINGS

This section describes our settings, procedure and ex-

periences with the CD methodology. Lessons learned

(speciﬁc to CD) and practical considerations (general

tips) can be found at the end of the sections, a more

exhaustive reﬂection is provided in Section 4. We fo-

cus on the phase of CI but describe other CD methods

as well. CI is especially beneﬁcial if, as in our case,

the project personnel is not familiar with the project

domain. Our CD process comprises all steps from CI

to ﬁrst functional prototypes in RP1 and all steps from

CI to a consolidated afﬁnity diagram in RP2.

In RP1, eight CIs were conducted at two com-

panies. Data were then used to construct an afﬁnity

diagram which organizes data in groupings based on

their natural relationships. The afﬁnity diagram was

revised and consolidated in several steps before being

used to create personas and derive requirements. In a

subsequent visioning process, comprehensive visions

were created that later served as a basis for the con-

ceptualization, design and implementation of functi-

onal prototypes. These phases took about a year in

total (10 weeks for CI). The companies involved were

a welding process specialist and a producer of agri-

cultural machinery. All CI participants were profes-

sional welders. Not more than one CI was conducted

per day. The core team that did the CIs and instructed

the further phases consisted of two researchers (both

involved in all CIs). During the following phases, four

additional researchers were engaged.

In RP2, four CIs and four additional interviews

were conducted at two companies in the area of auto-

mation technology and welding machine production.

Data gathered there was used to construct and conso-

lidate an afﬁnity diagram which will serve as a basis

for i) the deduction of requirements and ii) the deri-

vation of design ideas. The CD process so far took

about nine months (four weeks for CI and standalone

interviews). The CI participants were production em-

ployees, the four standalone interview partners were

a production line scheduler, a shift supervisor, a team

leader and a manager. The CI research team was led

by three core researchers but involved seven people in

total. Several CIs had to be conducted in parallel (on

three dates) due to the tight schedule at the industrial

partners. Later, nine researchers were involved. Our

CD process is sketched in Figure 1 and described in

further detail in the following sections.

3.1 Contextual Inquiry Preparation

Before the actual CIs were planned, researchers of

RP1 including the core inquiry team (all unfamiliar

with the welding process and professional welding

equipment) participated in a one-day, hands-on wel-

ding workshop. The two persons responsible for the

CIs in RP1

then underwent a half-day interview trai-

ning. Because they had no prior experience with CD,

they both researched and practiced the most impor-

tant parts of CI interviews which proofed very use-

ful. The preparation of the CIs themselves started

with the authoring of a “project focus” (Holtzblatt

et al., 2005) that contains the objectives of the rese-

arch project (e.g., novel interaction approaches in the

ﬁeld of industrial welding, or design of efﬁcient and

ergonomic work place settings), the affected stake-

holders (e.g., professional welders), typical activities,

tasks and procedures, relevant situations and locati-

ons. Moreover, numerous equipment was employed

during the CIs. To record the interview portions, the

dictation machine functionality of a smartphone was

used. The audio quality was sufﬁcient in quiet envi-

ronments; another advantage was that a smartphone

lying on the table is perceived as less invasive than

other recording mechanisms might be. Video recor-

ding was done with two different conventional ca-

meras from two different perspectives and one ca-

mera with a wide-angle lens capturing the whole set-

ting. Finally, a professional digital camera was used

to photograph important situations and artifacts (e.g.,

interaction methods, control elements or equipment).

Both interviewers took paper-based notes during the

CI. It was also important to bring pen and paper for

the participants as many of them created sketches

to explain their statements. Important organizatio-

The CI phase of RP1 was before RP2 so the general

preparation was re-used for RP2.

ICEIS 2018 - 20th International Conference on Enterprise Information Systems

432

1) Interview

2) Observation

3) Wrap-Up

Interpretation

Affinity Building

Contextual Inquiry

Work Group

Target Users

Indirect Users

Contextual Analysis

Field Interview

Text Notes Audio

Video Pictures

Affinity

Diagram

Hot Ideas

Vision

Visioning

Wall Walk Workshop

Prototyping

Questionnaire

Affinity Notes User Profile

Personas Construction of Personas

Deduction of Requirements

Requirements

Specification

Models

Hardware Prototypes Software Prototypes

Figure 1: Overall CD process. The small arrows indicate documents and artifacts used as input for the respective step. The

documents and artifacts resulting from each step are shown in green.

nal questions that should be clariﬁed with the partner

companies before the CIs dealt with adequate forms

of consent, secrecy, the time period that employees

cannot do their usual work, and a short brieﬁng each

company should do with their participating employ-

ees in advance. The brieﬁng can have a major inﬂu-

ence on the participants’ attitudes and should be care-

fully formulated. It is recommended to highlight im-

provements of peoples’ working conditions and emp-

hasize that humans are not intended to be replaced by

the technology to be introduced. The personality of

the ideal participant in a CI is curious and open to

new ideas. A good mixture of established employees

and newcomers is advantageous. Participants should

provide honest and constructive answers and be ena-

bled to work on their daily tasks at their own work

place rather than in a lab environment. Before car-

rying out the ﬁrst CI, the researchers internalized the

suggestion of (Holtzblatt et al., 2005) that intervie-

wers should act as curious apprentices. They stated

this explicitly during the CIs and it had a positive im-

pact on the conversations. E.g., they explained that

they did not know much about the circumstances in

this profession and the participants could help them

understand how they work, which problems typically

arise, or what aspects make the activities enjoyable.

Differences in the Preparation. As all CIs of RP1

took place before the ones of RP2, experiences from

RP1 could be utilized for RP2. Some differences in

the preparation and process that arose are described

as follows. In RP1, the company that provided most

participants for the CIs was part of the project consor-

tium. In planning meetings and discussions, the re-

sponsible people there already had basic knowledge

about what will happen during the CD process. The-

refore, time schedules and recruiting of suitable par-

ticipants according to criteria that were deﬁned toget-

her could be arranged with little effort. Yet, it was

necessary to clarify what should be part of the parti-

cipant brieﬁng and which details should rather not be

mentioned. RP2 worked solely with external partner

companies. Therefore, more effort was necessary to

brief them and a detailed guideline for the CIs and

Contextual Design in Industrial Settings: Experiences and Recommendations

433

further process was created that included a descrip-

tion of the CD methodology, aims, interview and ob-

servation guidelines, and a check list for the CIs. It

also identiﬁed the different roles of the requested par-

ticipants which included i) employees (assembly wor-

kers) and ii) supervisors (e.g., production line schedu-

ler).

Lesson: If you design for an unknown usage con-

text, it is tempting to familiarize yourself before the

CIs take place. However, there is a chance that you

are not unbiased anymore after such a familiariza-

tion. Researchers could make assumptions about pro-

cesses and working methods that might not be in ac-

cordance to (industrial) reality. On the other hand,

a familiarization might facilitate the observation of

certain factors that might otherwise be hard to detect,

e.g., physical effort resulting from body posture. Note

and discuss any assumptions or interpretations you

have with your interview partners during the CIs.

3.2 Pretest

Before the actual CIs, a pretest was conducted as a

trial run for the later inquiries. It is important that

a pretest and the following CIs are carried out under

the same conditions (e.g., using the same CI interview

guide and questionnaire, audio and video recording).

A pretest can help to control i) the logical sequence,

ii) timing and schedule, and iii) technical equipment.

It can show that interview topics and their order are

not harmonious and thus have to be changed. In ad-

dition, the asked questions might prove too extensive

and have to be revised due to time constraints. It is

also important to check the technical equipment. It

must be ensured that recording over the entire period

of the interview and observation is interruption-free.

Lesson: It is favorable to perform the pretest ap-

proximately 1-2 weeks before the ﬁrst interview. This

leaves enough time to apply changes to documents

and processes. Depending on the quality and outcome

of the pretest, it is justiﬁable and helpful to interpret

and use the materials of the pretest in later analysis.

3.3 Contextual Inquiry

This section describes the phases of the CI sessions as

conducted in RP1 and RP2. Part 1 (about 60 minu-

tes) comprised the following items in the order listed

here (the items marked italic were audio-recorded):

welcome, introduction of interviewer, project con-

text, agenda, introduction of participant, question-

naire (RP1), interview according to guide, question-

naire (RP2). Part 2 (about 90 minutes) comprised an

observation of work activities (video-recorded). Part

3 (about 10 minutes) comprised a wrap-up and thank

you (including a small gift), both audio-recorded.

After careful consideration, we planned the CIs

with two interviewers (and not one) both due to the

inexperience in the methodology of the persons in-

volved and the need for an exhaustive observation, alt-

hough according to (Holtzblatt et al., 2005) this yields

a number of drawbacks: “To establish an intimate re-

lationship you want the user to focus on relating to

one person. When one person drives the conversation

he follows his focus and the conversation has a co-

herent thread that the user can engage with. But if

two people talk, then the necessary jumping back and

forth makes the conversation more disconnected, cre-

ates less intimacy, and produces competing threads of

conversation.” Holtzblatt et al. consequently suggest

that if it is necessary to inquire with two persons, the

roles “interviewer” and “note taker” should be clearly

deﬁned and many switches should be avoided. Thus,

we positioned one researcher as the main interviewer

while the other one acted as a note taker but could

bring up questions on their own. For the latter it was

easier to observe the interview process, create sket-

ches and keep an eye on the big picture (e.g., whether

all desired topics had been addressed). A description

of the individual CI phases is provided in the follo-

wing subsections. We processed about 12 hours of

audio, 16 hours of video recordings, 370 photographs

and 35 pages of written notes to create 419 afﬁnity

notes after CI in RP1. The total effort for the resear-

chers was about 500 man-hours. In RP2, we proces-

sed about 13 hours of audio, 10 hours of video recor-

dings, about 150 photographs and 35 pages of written

notes to create 682 afﬁnity notes (effort: about 430

man-hours). The effort for the industrial partners was

45 to 50 hours in RP1 and RP2.

Lesson: In our experience it can be useful to inquire

with two persons if they are either not acquainted with

CD methodology or a thorough observation of details

of the context is important. Two heads may be better

than one, but avoid switching back and forth bet-

ween the roles of an interviewer and note taker.

Lesson: We see building trust and rapport a key is-

sue in a successful inquiry. Only then will your users

share sensitive information with you.

3.3.1 Introduction

The ﬁrst step was to welcome the participants and in-

troduce the researchers before participants gave infor-

med consent. After a short explanation of the project

ICEIS 2018 - 20th International Conference on Enterprise Information Systems

434

context, the researchers provided an overview of the

goals and inquiry approach and explained the inquiry

agenda. At this point, audio recording was started and

participants were asked to introduce themselves. This

included career progression and a brief description of

their current position in the company.

3.3.2 Interview

In preparation of the interview, an initial pool of ques-

tions was scripted. In total, we prepared 17 questions

(e.g., about daily routine or working instructions) in

RP1 and 26 in RP2. We found it helpful to ﬁrst ask

for typical day-to-day routines on a speciﬁc day (e.g.,

last Tuesday). This provides an easy starting point for

the participants and helps the researchers to expand

the different areas of interest from that on. We see

the prepared questions as an aid and not as a checklist

that has to be worked through. Thus, new questions

can arise through conversation and observation.

Lesson: Prepare your initial interview questions

well and start with a speciﬁc question about day-to-

day routine. This helps to ask for a speciﬁc day. The

information provides a good overview of daily activi-

ties and then can be transferred into a user proﬁle.

Lesson: To ensure the quality of the ﬁndings the

user should be open and talkative. In our experience

this can be achieved by transition from a question-

answer situation into a smooth conversation.

Lesson: Actively motivate participants to question

and reconsider aspects of their daily work.

3.3.3 Questionnaire

A written questionnaire was used to gather informa-

tion about the participants’ general situation. It was

divided into three parts: i) general information (e.g.,

socio-demographic data), ii) educational background,

occupation and work experience, and iii) experience

with new technologies. In RP1, the questionnaire

included additional, project-speciﬁc data about wel-

ding experience (e.g., welding hours per day, prefer-

red equipment). However, the questionnaire was kept

short (one page) to be answered quickly. The main

intention was not to be forced to collect this data du-

ring the interview. The questionnaire was ﬁlled in be-

fore the interview in RP1. However, after considering

that the questionnaire might interfere with the esta-

blishment of a trust base, it was handed out at the end

of CI’s ﬁrst part in RP2. However, no differences in

quality of gathered data could be identiﬁed.

Lesson: Even if a questionnaire does not seem to

be the most important part of a CI, the answers will

help you to get a more complete picture of your par-

ticipants. Also, personas are easier to author with

basic demographic data and additional information.

3.3.4 Observation

The observations should, if possible, take place at

the participant’s workplace and users should do their

daily tasks in an undisturbed way. While this was pos-

sible for RP2, not all participants in RP1 worked with

manual welding on a daily basis. For this and other

reasons, most of them could not be observed in their

usual workplace but on a recreated one. All obser-

vations were recorded with three different cameras.

In RP1, the work steps included various weld joints

with different materials and material strengths. Parts

and components for a serial production were also pro-

cessed. The observations lasted about 45 minutes on

average. In RP2, the observed work tasks involved the

assembly of a complete product. The observation was

done in a production line with several stations (but by

a single employee). One observation took place in the

context of a training where the user had not manufac-

tured the product before and got step by step instructi-

ons from an experienced employee. The observations

lasted about 75 minutes on average.

Practical Consideration: Use a wide-angle ca-

mera. Even seemingly stationary activities might sud-

denly spread. People could pass by or participants

might fetch tools from outside their workplace.

Practical Consideration: Plan enough time for

equipment setup (it took 15 to 30 minutes per CI).

Practical Consideration: It is important to provide

all devices with sufﬁcient power supplies. As ob-

servations could become lengthier than expected, you

can use power banks in between but should try to con-

nect as many devices to the mains as possible for

permanent charging. Bring your own outlet power

sockets if unsure about local equipment.

3.3.5 Wrap-Up

The ﬁnal step of our CIs was to engage the partici-

pants in a short talk about what had just been obser-

ved. The researchers explained how they understood

the different activities and interactions they observed

and asked if these interpretations were correct. Thus,

any unclear points or backlogs of questions from pre-

vious inquiries might then be clariﬁed. Finally, par-

ticipants were asked to provide a vision about fu-

Contextual Design in Industrial Settings: Experiences and Recommendations

435

ture work practices (e.g., interaction methods or work

place designs) and express own ideas.

Lesson: Keep and update a list of open questions

that can be clariﬁed during later CI’s wrap-ups. In

our projects, this list was integrated into the interview

guidelines. As CI is a qualitative method, we freely

adapted our guidelines as we saw ﬁt in order to gather

as many insights as possible in each session.

3.4 Generation of Afﬁnity Notes

Afﬁnity notes are the basic building blocks of the CD

methodology. They result directly from observation

and questioning in the actual work context and are

a description of meaningful events, problems or is-

sues. The material reviewed for this process inclu-

ded the notes taken during interview and observation

(typically two to four pages), voice recordings of the

interviews and audiovisual material of three came-

ras during the observation. We started by creating

a short user proﬁle containing the respective parti-

cipant’s name, picture, background, main activities,

and daily routines before we created the afﬁnity notes.

Example notes in RP1 are “Participant used a compo-

nent he randomly stumbled across to brace against the

working surface” or that a participant expressed their

wish that the welding torch was as light as a pen. In

RP1, generation of afﬁnity notes was done directly af-

ter the CI. Thus we could construct the most interes-

ting topics, issues and interactions and create the no-

tes from fresh memory. This was not possible in RP2,

due to its tight schedule with multiple interviews per

day. For later reference, the notes were given a unique

ID in the form P

N (participant id, consecutive num-

ber). We created 40 to 70 afﬁnity notes per session.

Lesson: As recommended by (Holtzblatt et al.,

2005), it helped keep the memory fresh to not talk

to anyone about the interview and observation bet-

ween CI and interpretation which is based on the no-

tion that the human brain prioritizes those memories

lower that have already been externalized.

Practical Consideration: If there is a longer period

of time between inquiry and interpretation, you will

have to rely on the audio recordings of the interview,

which increases time and effort (but generally works).

3.5 Generation of Afﬁnity Diagram

The aim of transforming the afﬁnity notes into an af-

ﬁnity diagram is to systematically organize the gathe-

red pieces of information. In both projects, the ge-

neration of the diagram took place in several team

workshops (see Figure 2) with in-depth discussions

about the observed phenomena. Each workshop was

directed to a systematic grouping of the afﬁnity no-

tes resulting in hierarchical arrangement. In RP1, the

ﬁrst workshop took place after all but one CIs, in RP2

after all CIs. We used a large room decorated with ﬂip

charts and prepared 20 stacks of randomly assembled

afﬁnity notes. A core team of three researchers then

started to group the afﬁnity notes on the wall. This

was done by thinking about what each afﬁnity note

really implies. E.g., when a participant used a random

component to brace against the working surface, this

might mean that they needed stabilization. Regarding

a participant’s wish for his torch to be as light as a pen,

the participant’s underlying assumptions were proba-

bly that the welding torch should be i) as small as pos-

sible (but well graspable), ii) as light as possible, iii)

as easily movable and ﬂexible as possible (e.g., wit-

hout the obstructive cable-hose assembly). According

to these aspects, the afﬁnity notes were grouped on the

wall. In RP1, up to 6 people worked on the diagram

simultaneously (up to 9 people in RP2).

On the ﬁrst full day event, we could group most of

the several hundred afﬁnity notes on the wall and add

blue labels (as suggested by Holtzblatt et al.) to most

of the emerging groups. In several subsequent works-

hops, the remaining afﬁnity notes and those gathered

from the last CI were integrated. Pink and green la-

bels were added to further aggregate the groups and

identify top-level categories. For an alternative to a

paper-based approach see (Judge et al., 2008) who

describe multiple display environments for diagram-

ming. In total, the effort during the afﬁnity building

and visioning phase was about 650 man-hours for the

researchers in RP1 and 340 hours so far in RP2. The

effort for the industrial partners was about 100 man-

hours in RP1 (in RP2 the afﬁnity diagram has not yet

been consolidated with the partners).

Lesson: It is important to abstain from a keyword-

based grouping in order to enable development of na-

tural, semantic groupings.

Lesson: The support of additional researchers that

join the afﬁnity building process was not always

helpful (people who think in solutions right from the

beginning might even hinder the process at this stage).

3.6 Review of Afﬁnity Diagram

After completing the afﬁnity building, the diagram

was reviewed together with domain experts of the in-

dustrial partners. The main objective of the review

was to examine whether CI data were correctly un-

ICEIS 2018 - 20th International Conference on Enterprise Information Systems

436

Figure 2: Three researchers building an afﬁnity diagram.

derstood and interpreted. Furthermore, some of the

questions and data holes that came up during the af-

ﬁnity building workshops could easily be ﬁxed while

reviewing the diagram. Since the afﬁnity diagram was

constructed with notes printed out and arranged on a

ﬂip chart, a digital version was created for RP1. The

digital version is much cleaner and easier to read, ad-

ditionally it conveys a more professional impression

with stakeholders (e.g., industry partners) as printed

out with a plotter. A drawback might be that the

diagram cannot be edited on the ﬂy when discussed

(which is possible otherwise through simple rearran-

gement of sticky notes). We used Microsoft Visio for

digitalization which is particularly suitable in case the

afﬁnity notes already exist in digital form (e.g., in Mi-

crosoft Excel). It is then easy to import them in Visio

and visualize the data based on shape templates. This

makes it also easier to generate different versions of

the diagram. For example, we created a version that

includes the notes’ IDs for internal use and an ano-

nymous version that could be given to the industrial

partner. In RP2, we did not create a digital version of

the afﬁnity diagram for time reasons. See the plot of

the RP1 afﬁnity diagram in Figure 3.

Lesson: For industrial partners (especially on the

executive level) that are often used to follow formal

processes, there is the danger that a CD approach

might appear too informal. Coming there with a my-

riad of ﬂipchart pages and sticky notes might see-

mingly conﬁrm this impression for them. Therefore,

a suitable visual preparation of the diagram (e.g.,

printed or plotted) can help to translate between bu-

siness language and CD. The diagram was valued

highly by all external partners in our case.

Practical Consideration: If plotting the afﬁnity di-

agram, use a font of sufﬁcient size and allow for

enough white space to later place sticky notes.

3.7 Construction of Personas

The construction of personas that are grounded in

contextual data was mainly useful for two reasons.

Firstly, they made it easier having real target group

users in mind when designing technological solutions.

Secondly, they were very supportive for introducing

new stakeholders (e.g., designers, developers) into the

project later on. In order to ensure that the personas

are composed with a real foundation in the contextual

data we gathered, we carried out the following steps:

i) determine which participants are suitable as a ba-

sis for the persona, ii) assign a realistic name to the

persona, iii) extend user proﬁles (by using notes from

the interview, audio recordings and afﬁnity notes), iv)

author the persona as a synthesis of the material, v)

reﬁne personas by listing roles, tasks, and goals, and

vi) add a portrait picture that matches the persona.

3.8 Deduction of Requirements

In many organizations it is an established procedure

to formalize and create a catalog of requirements. As

a ﬁnal result of a requirements engineering process,

stakeholders and executives are typically longing for

this piece of documentation. However, in CD-driven

projects, a formal requirements catalog is not the main

focus. The requirements are embedded in their speci-

ﬁc context directly within the afﬁnity diagram that is

the basis for the next steps. By reﬂecting on the dia-

gram, design solutions can be devised that are groun-

ded directly in the contextual data instances. It is im-

portant to keep in mind and communicate that the afﬁ-

nity diagram does not represent designs or even requi-

rements, but rather an “accurate and complete picture

of the users’ work domain, including their concerns

Contextual Design in Industrial Settings: Experiences and Recommendations

437

and descriptions of their current usage” (Hartson and

Pyla, 2012). Yet, it can be useful for a quick overview

or for strategic reasons to obtain a list of requirements.

Hence, (Hartson and Pyla, 2012) describe how it can

be extracted from an afﬁnity diagram. According to

our aims, we extracted interaction requirements (IR)

and corresponding system requirements (SR) directly

form the pink label (i.e., mid-level) groups. For exam-

ple, a pink label we introduced read “Feedback during

the welding process is important to me.”. This was

further translated to the IR “Feedback about the wel-

ding process must be available for welders.” and the

SR “The system must show welders currently relevant

feedback.”.

Lesson: Communicate early that requirements will

not necessarily be available in the way they would in

a more formal requirements engineering process and

explain why this is not necessary.

3.9 Wall Walk Workshop

The main purpose of the Wall Walk workshop was

to introduce persons from outside of the core inquiry

team to the data gathered so far and spark feasible

and practical ideas from the different professions and

experiences among all participants (about 20, inclu-

ding industrial partners, in RP1). In preparation, a

large room was decorated with the plotted afﬁnity di-

agram. Firstly, all participants silently inspected the

whole diagram (see Figure 3), read the data instan-

ces (i.e., afﬁnity notes and labels) from the top do-

wnwards, inserted sticky notes of emerging ideas or

issues and placed them at those positions in the dia-

gram where they emerged. For about two hours, par-

ticipants remained almost completely silent and fo-

cused solely on the data. This process was followed

by a break of about one hour that was used by the

core team to gather and digitalize all ideas and issues.

When the participants returned from their break, the

ideas were discussed in their particular context. The

participants’ different backgrounds were invaluable to

evaluate ideas based on their feasibility, practicabi-

lity, and corporate philosophy. Finally, a list of “Hot

Ideas” (high-potential ideas for technological soluti-

ons for the interaction with welding machines) was

prepared.

3.10 Visioning

Based on the list of Hot Ideas that was discussed and

compiled during the Wall Walk workshop, a Visio-

ning session was conducted. Six team members gat-

hered in a large room that was prepared with the Af-

Figure 3: Participants engaged in the Wall Walk workshop.

ﬁnity Diagram and personas. First, the participants

(re)familiarized themselves with the Afﬁnity Diagram

during a personal and silent wall walk (of ca. 60 mi-

nutes). Afterwards, the list of Hot Ideas was revie-

wed. To come up with starting points for the visions,

each participant could freely distribute six votes. The

highest-ranked Hot Ideas were then integrated into the

visions as initial solutions. Over about two hours, a

brainstorming and group storytelling process unfol-

ded. One team member acted as “pen”, responsible

for sketching the ideas of the stories told by other

members, and one as a “poker”, keeping an eye on

the big picture and pointing to Hot Ideas that should

also be considered. Three visions (i.e., sketches of the

envisioned work practices including new technology

to help welders) were created accordingly. Next, the

visions were evaluated and a list of positive and nega-

tive points was compiled for each vision. On a sepa-

rate date, two team members integrated all ideas that

were evaluated positively and ﬁt together well into a

consolidated vision. This ﬁnal vision was created on a

large interactive touchscreen (Microsoft Surface Hub

84”) with sketching functionality which proofed use-

ful (e.g., zooming, copy and paste, and electronic dis-

tribution were very valuable).

3.11 Prototyping and Current Status

In RP1 functional hardware and software prototypes

are currently created. The features and properties of

these prototypes are strongly informed by the contex-

tual data we gathered. Although a thorough evalu-

ation is yet to come, we can a priori say that many

subtle constraints would have stayed hidden without

the CIs (e.g., a strong focus on weight reduction of the

equipment). Apart from constraints, visioning helped

us to come up with prototypic solutions we are con-

ﬁdent will help target group users. In RP2, the next

steps are the Wall Walks with industrial partners.

ICEIS 2018 - 20th International Conference on Enterprise Information Systems

438

4 DISCUSSION

In this section we discuss our experiences with the CD

methodology in industry-related projects and derive

recommendations for researchers and practitioners.

4.1 General Remarks

To ensure a smooth CD process, initially clarify all

framing conditions (e.g., participants, declarations of

consent) with industrial partners and create a shared

understanding of the methodology with all involved

parties. We recommend to rely strongly on the com-

panies regarding the selection of participants (but give

them criteria). Inform the company that you need the

employees for at least three hours but they can get

some work done in the meantime. Some companies

might want to provide you with personnel that is ea-

sily dispensable. For the CIs themselves, plan enough

time. Some participants might need time to gain trust,

open up and then converse more extensively.

4.2 Contextual Inquiries

The CIs do take a lot of effort from you, industrial

partner companies, and the participants themselves.

It is difﬁcult for companies to remove workers form

their activities for a long time, since they cannot work

as productively as usual during this period. This

should also be communicated early. Time is a criti-

cal factor and companies try to keep interruptions for

employees as low as possible. Thus, in RP2 several

CIs were conducted by several teams on one day.

In RP2, we planned the CI with different target

groups with potentially different interests (e.g. pro-

duction line planners and workers). We found that the

collected data complemented each other well. This

could also be observed when creating the afﬁnity

where the data of two companies was combined. It

is also important to carefully select and inform the

employees that should participate in the inquiry. Par-

ticipants are not expected to deliver the “best possible

performance”. In RP2, a selected employee had to be

replaced on short notice. She had been informed that

she would be taking part in an observation and then

memorized the work instructions over the weekend.

Due to a tight schedule, resource bottlenecks can

also occur in the inquiry team. Since several surveys

had to be carried out on one day in RP2, researchers

without CD experience were involved. These were

i) briefed in advance, ii) present during the pretest or

iii) in a team with experienced researchers when pos-

sible. Yet, differences in the quality of the collected

data could be observed, e.g., inexperienced resear-

chers paid less attention to the answers in the questi-

onnaire. Although the data could be completed from

audio recordings, this information should have been

acquired together with the user. This led to a lot avoi-

dable effort. We managed to develop a sound basis

of trust with all participants, and in our experience,

most participants were open-minded and willing to

give constructive and honest answers to the questions

we asked. They knew that they were experts and their

respective ﬁelds – while the researchers were the “ap-

prentices” – and they were pleased to be involved.

To prevent loss of contextual data, we recom-

mend bringing at least two audio recording devices (in

RP2’s pretest the recording failed so we had to inter-

pret data from notes and memory). When observing

participants in productions halls, the noise level was

high which made audio recording more difﬁcult. It

is thus advisable to deal with certain questions again

in the wrap-up. Video recordings from different per-

spectives were also essential. Countless times, we

discovered important details in the interaction with

machinery or the handling of equipment only after ca-

reful analysis of all different perspectives. To facili-

tate this, we synchronized all different image tracks

to one video. Finally, it is important to plan sufﬁ-

cient time and resources for data interpretation. Ac-

cording to (Holtzblatt et al., 2005), this phase should

take place within 48 hours. This was not possible in

RP2 and the preparation of the afﬁnity notes took al-

most one month. This caused considerable additional

effort as researchers had to repeatedly view the video

recordings where memory was not instant.

Hidden Requirements. In both projects, we could

uncover numerous requirements and constraints that

would most probably have stayed hidden without CI.

E.g., we observed that a participant took a look at a fa-

raway central display A, although they were holding

a display B presenting the needed information right in

their hand. When asked why they looked for the infor-

mation on display A, they replied that they were just

standing nearby. Yet, after video analysis, we saw a

deliberate movement to the display A only for the pur-

pose of receiving the information. The reasons could

be that it was not legible in display B or the partici-

pant trusted display A more. Findings like these had

strong implications for the later design stages and can

help us to develop usable and reliable solutions.

4.3 Afﬁnity Building

In both projects, the building of the afﬁnity diagram

started with a workshop. First, the building was done

Contextual Design in Industrial Settings: Experiences and Recommendations

439

together, after a certain time in pairs and later indivi-

dually. In RP2, the transition to work independently

was done too quickly. This led to a too hasty inte-

gration of the afﬁnity notes into the diagram. There

was a strong thinking in the given categories (labels)

which we should have divided at an early stage. Furt-

hermore, we advise you to start with the pink labels

only after the blue labels are ready. Another problem

was “pattern matching”. For some researchers, it was

difﬁcult to correctly interpret the key message of af-

ﬁnity notes. Similar words on similar notes do not

necessarily indicate a similar message. We also saw

difﬁculties in ﬁnding the right form of abstraction and

some researchers were often unsure whether an infor-

mation was relevant or not. In our experience, poorly

instructed team members slow down the afﬁnity buil-

ding process or cause a lot of extra work. We recom-

mend working with a small team (3-4 people) perma-

nently involved in the building process. If people are

only available temporarily, assign them self-contained

tasks (e.g., subdivision of a blue label group).

5 CONCLUSIONS

In this paper, we presented and discussed experien-

ces and lessons learned with the CD methodology.

They are rooted in the industry domain as both CD

processes we described here were conducted in indus-

trial research projects. In summary we would like to

strongly encourage other researchers in using CD in

spite of well-known challenges such as the time and

effort involved for all participating parties. In our ex-

perience, CD did not only help to reveal hidden fac-

tors of inﬂuence, requirements and design ideas for

RP1 and RP2 but also other information not of di-

rect relevance for the projects but generally of interest

for the industrial partners. All partners concordantly

conﬁrmed that the additional effort paid off for them.

This paper provides a guideline intended to help re-

searchers to i) circumvent avoidable risks and mista-

kes and ii) more realistically assess the effort invol-

ved with each of the CD phases in beforehand. We

do not suggest deviation from the CD methodology

described by Holtzblatt et al. but highlight it from an

industry-related perspective. Future work could study

more in-depth how the time- and effort-related cost of

the (R)CD methodology could be further reduced.

ACKNOWLEDGEMENTS

Our work has been conducted within the scope of

the projects Welding Interaction in Future Industry

and Human-Centered Workplace 4 Industry, funded

through the BRIDGE and COIN programs, managed

by the Austrian Research Promotion Agency (FFG).

REFERENCES

Beyer, H. and Holtzblatt, K. (1997). Contextual Design:

Deﬁning Customer-Centered Systems. Morgan Kauf-

mann, 1 edition.

Beyer, H. and Holtzblatt, K. (1999). Contextual design. In-

teractions, pages 32–42.

Coble, J., Mafﬁtt, J. S., Orland, Matthew, J., and Kahn,

M. G. (1995). Contxtual inquiry: Discovering physi-

cians’ true needs. In Proceedings of the Annual Sym-

posium on Computer Application in Medical Care.

Ehn, P. (1988). Work-Oriented Design of Computer Arti-

facts. PhD thesis, Arbetslivscentrum, Stockholm.

Fouskas, K. G., Pateli, A. G., Spinellis, D. D., and Virloa,

H. (2002). Applying contextual inquiry for capturing

end-users behaviour requirements for mobile exhibi-

tion services. In Proceedings of the 1st International

Conference on Mobile Business.

Gellatly, A., Hansen, C., Highstrom, M., and Weiss, J. P.

(2010). Journey: General motors’ move to incorporate

contextual design into its next generation of automo-

tive hmi designs. In Proceedings of the 2nd Interna-

tional Conference on Automotive User Interfaces and

Interactive Vehicular Applications, Pittsburgh, USA.

Hartson, R. and Pyla, P. S. (2012). The UX Book: Pro-

cess and Guidelines for Ensuring a Quality User Ex-

perience. Elsevier.

Holtzblatt, K. and Beyer, H. (2014). Contextual Design

Evolved. Morgan & Claypool Publishers.

Holtzblatt, K. and Beyer, H. (2017). Contextual Design.

Design for Life. Morgan Kaufmann, 2 edition.

Holtzblatt, K., Burns Wendell, J., and Wood, S. (2005).

Rapid Contextual Design. A How-To Guide to Key

Techniques for User-Centered Design. Morgan Kauf-

mann.

Holtzblatt, K. and Jones, S. (1993). Contextual inquiry: A

participatory technique for system design. In Schu-

ler, D. and Namioka, A., editors, Participatory De-

sign. Principles and Practices, chapter 9. Lawrence

Erlbaum Associates.

Judge, T. K., Pyla, P. S., McCrickard, D. S., and Harrison, S.

(2008). Using multiple display environments for afﬁ-

nity diagramming. In Workshop on beyond the labora-

tory: supporting authentic collaboration with multiple

displays at CSCW, volume 8, pages 9–12.

Viitanen, J. (2011). Contextual inquiry method for user-

centered clinical it system design. Studies in Health

Technology and Informatics.

Winograd, T. and Flores, F. (1988). Understanding Com-

puters and Cognition. New Foundation for Design.

Ablex.

Wixon, D., Holtzblatt, K., and Knox, S. (1990). Contextual

design: An emergent view of system design. In Pro-

ceedings of the ACM SIGCHI Conference on Human

Factors in Computing Systems, CHI’90, Seattle, USA.

ICEIS 2018 - 20th International Conference on Enterprise Information Systems

440