Tactile Information Coding by Electro-tactile Feedback

Peter Schmid, Mona Bader and Thomas Maier

Institute for Engineering Design and Industrial Design, Department of Industrial Design Engineering,

University of Stuttgart, Pfaffenwaldring 9, Stuttgart, Germany

Keywords: Electro-tactile Feedback, Tactile Information Coding, Human Machine Interaction, Human Machine

Interface, Touch Interface, Tactile Perception.

Abstract: Touch user interfaces offer a wide range of interaction and manipulation possibilities. However, when

interacting with this technology, the feedback is usually only provided via the visual or audio-visual channel

of perception. Therefore, the study investigates how electro-tactile feedback can support the interaction with

touch interfaces. The aim is to use electro-tactile feedback to transmit information during an adjustment task

on a touch interface. For this purpose, five different types of electro-tactile feedback were investigated in a

user study with 15 test persons. During the execution of a main task, a simple adjustment task had to be done

in parallel on the electro-tactile touch interface. The electro-tactile feedback supports the execution of a main

task and a secondary task, but the study also shows that by concentrating on the electro-tactile feedback the

actuation time is extended.

1 INTRODUCTION

On touch-sensitive screens, information can not only

be displayed but also edited directly. It is possible to

execute as many complex functions on a small

interaction surface. When using this technology,

feedback to the user is usually audio-visual, the

feeling of a haptic feedback is completely lost.

According to Hoggan et al., the loss of haptic

feedback when operating a virtual input element leads

to a higher error rate and a lower input speed (Hoggan

et al., 2008). Studies by Harrison et al. and Koskinen

et al. show that haptic feedback makes interaction

with touchscreens more efficient and satisfying

(Harrison et al., 2009; Koskinen et al., 2008). Studies

from Schmid et al. or Winterholler show that in

situations with high audio-visual information content,

tactile information can be used to support people in

the performance of a primary and secondary task

(Schmid et al., 2019; Winterholler, 2019).

With regard to the design of haptic feedback, there

are already standards such as DIN EN ISO 9241-910

or VDI/VDE 3850-3, which focus on haptic feedback

for touch user interfaces as a supplement to auditory

or visual feedback (ISO, 2011; VDI/VDE, 2015).

According to DIN EN ISO 9241-112, information

coding can reduce the cognitive load on users by

supporting them in the performance of the task and by

providing necessary information in an unambiguous

way when interacting with a machine (ISO, 2017).

Therefore, the focus of this paper is on the electro-

tactile information coding of a virtual slider control

element concerning a medical use case. In an

operating room, there is a lot of noise and all feedback

from medical devices is visual or acoustic. This leads

to an overload of the human perception channel.

Consequently, concentrated work by doctors is not

possible. (Siegmann & Notbohm 2013)

However, there is still another perception channel,

which is completely unused in operating medical

instruments. To reduce the overload of the audio-

visual perception channel we can address the haptic

or tactile perception channel.

For the investigation of electro-tactile coding we

considered gas insufflation. Minimally invasive

surgery uses gas insufflation to fill the abdomen with

in order to increase the operating field. For this

purpose, we derived five interface modules, which are

needed to control the gas insufflation. In addition to a

start/stop module, it is necessary to control the CO

pressure, the gas flow as well as the smoke extraction

and gas consumption. Currently these functions

provide only a visual or acoustic feedback in gas

insufflation. Hence, this project will evaluate the

tactile perception channel of humans with regard to

its supporting capacity in terms of tactile feedback

design. According to Schmid & Maier, different

Schmid, P., Bader, M. and Maier, T.

Tactile Information Coding by Electro-tactile Feedback.

DOI: 10.5220/0010066300370043

In Proceedings of the 4th International Conference on Computer-Human Interaction Research and Applications (CHIRA 2020), pages 37-43

ISBN: 978-989-758-480-0

tactile coding patterns on a touch user interface can

be used for information coding by intensity design of

the electro-tactile feedback during a sliding input

gesture. Different design options are available,

depending on a discrete or continuous feedback. The

identified tactile coding patterns divide into tactile

coding features and tactile coding ranges for both

discrete (Fig. 1a) and continuous feedback (Fig. 1b).

The tactile coding feature describes a single point, a

so-called tick mark of the slider on the touch user

interface, which can indicate to the user, for example,

a middle as well as a beginning and end or maximum

and minimum values by means of a tactile intensity

change (Fig. 1c). Tactile coding progressions, on the

other hand, describe areas such as the increase or

decrease in tactile feedback in a defined range (Fig.

1b). (Schmid & Maier, 2020)

2 METHOD

The research of Winterholler demonstrates the

possibility to transfer haptic information by coding of

physical rotary control elements. (Winterholler 2019)

In contrast, this study investigates electro-tactile

information coding using virtual sliders on an electro-

tactile touch user interface (Fig. 2). The slider control

elements differ in the type of tactile feedback. For this

purpose, five different control elements are

implemented based on discrete or continuous

feedback. With discrete feedback, the user feels

individual feedback impulses with his finger during

the adjustment process, similar to a slider with tick

marks. With continuous feedback, on the other hand,

the user perceives continuous tactile feedback with

his finger during the adjustment process. In addition

to the discrete or continuous feedback, the intensity

level of the feedback also varies.

This is why this study focuses on the information

coding on the one hand of a fine adjustment task, in

particular for adjusting a defined value along a scale,

and on the other hand the coding of different areas

along a scale. Fig. 1 shows the five different

indicators. Each slider is built on a scale (x-axis)

coded by different characteristics and intensity levels

(y-axis) of tactile feedback. Fig. 1a shows the

characteristic of a discrete tactile feedback. This

means the test person feels different feedback marks

while sliding along the virtual scale. The tactile

coding of control element 2 represents a continuous

input in two ranges (“Stop”-“Start”) with different

intensities of tactile feedback (Fig. 1b). The aim of

this tactile area coding is to investigate whether a

separation of two ranges is perceptible by the

intensity level and contributes to a better performance

of the main task. Control element 3 characterises

continuous feedback with three tick marks. In

contrast to control element 1, the marking of the tick

marks is achieved by continuously increasing and

decreasing intensity along the virtual slider. When the

test person slides along the virtual slider, hill and dale

are perceived, which are generated by the variation of

the intensity level (Fig. 1c). The fourth control

element contains the same function as control element

3. A distinction is also to be made between the three

setting ranges. In contrast to control element 3,

however, the ranges are coded by three continuously

different intensity levels. (Fig. 1d). Interface element

5 has a feedback impulse to separate the ranges

“Reset” and “Start”. If the user sets the slider from

“Start” to “Reset”, he feels a short resistance at the

transition, which represents the transition between the

two ranges (Fig. 1e).

The study evaluates whether the tactile feedback

supports the operation of the five interface elements

and thus improves the concentration on a main task

performed in parallel. It should also be determined

whether the operation with electro-tactile feedback

requires less effort.

a) Interface element 1: discrete input

b) Interface element 2: continuous input with discrete

markers

c) Interface element 3: continuous input

d) Interface element 4: continuous input

Figure 1a: Schematic structure of the tactile coding of the

interface elements 1-4.

CHIRA 2020 - 4th International Conference on Computer-Human Interaction Research and Applications

e) Interface element 5: continuous input with a discrete

mar

Figure 1b: Schematic structure of the tactile coding of the

interface element 5.

Besides the schematic representation of the tactile

coding of the control elements, Fig. 2 shows the

implemented test interface with which the adjustment

tasks are carried out. Fig. 2 demonstrates the

start/stop module, the module for CO

pressure, the

gas flow module as well as the smoke extraction

module and gas consumption module.

Figure 2: Test interface of the slide control elements for the

adjustment tasks with electro-tactile feedback.

2.1 Participants

For the experiments 15 volunteers are available, 11 of

them men and 4 women aged 22 to 35 years. The

average age is 26 years (SD=3.05 years). None of the

test persons pursues a hobby or a job that puts so

much strain on the fingertips that their sensation

would be severely restricted. Test persons suffering

from skin or nerve diseases of the hands are also

excluded. All test persons are students in a technical

course of studies or academic employees of the

institute. The test persons have no experience with the

existing type of haptic feedback on touch screens. The

test duration is about 30 minutes.

2.2 Test Setup

The experimental set-up consists of a main and a

secondary task (Fig. 3). The use case for the

investigation of electro-tactile slider operation is in

the field of medical technology. In a medical

intervention the focus lies on the human being or the

operating field, within which very precise work must

be carried out. Therefore, the main task of this

experimental setup is to perform a dexterity task,

which requires concentrated work. The secondary

task in the operating theatre can, for example, be the

operation of medical equipment. Therefore, the main

task of the experiment is to hold a stylus with its

metallic tip in a 5 mm diameter hole as still as

possible without touching the edge. For this purpose,

the circuit board for the main task is placed to the

dominant hand of the test person. The main task is

therefore performed with the dominant hand, while

the arm is slightly bent but not supported. The contact

of the stylus with the edge of the hole is detected.

Similarly, in this test scenario, the electro-tactile

display is to be operated parallel to the dexterity task.

The secondary task is an adjustment task on the

electro-tactile touch interface. The electro-tactile

display with the test user interface is placed beside

the test person. The tablet is connected via a cable to

a laptop on which the test program is opened.

Figure 3: Experimental set-up.

2.3 Experimental Procedure

After the welcome, the investigator informs the test

person about the operation of the test user interface

and the test procedure. This is followed by

information on privacy, consent to the study and the

collection of demographic data. In addition to age and

gender, it is also recorded whether the test person has

a hobby or a disease that would result in a reduction

in fingertip sensitivity. In the subsequent introductory

Main Tas

Secondar

Tas

Sub

ect

Investi

ator

Tactile Information Coding by Electro-tactile Feedback

and learning phase, the test person is allowed to test

the controls, feel the feedback and become familiar

with the perception. No further explanations of the

haptic feedback are given.

During the test, 11 different adjustment tasks are

performed on the five control elements with and

without haptic feedback. The adjustment tasks were

selected on the basis of a benchmark for the operation

gas insufflation. For this purpose, the operation of gas

insufflation in everyday clinical practice was

analysed and the operation was transferred to the use

case tested in this study with regard to electro-tactile

feedback. The different adjustment tasks are

performed with the non-dominant hand, while the

main task is performed in parallel with the dominant

hand. An adjustment task must always be carried out

in a sliding movement without settling. As soon as the

correct position of the main task is assumed, the

experimenter gives a signal, whereupon the

adjustment task may be started. There are four

adjustment tasks for control element 1. Two

adjustment tasks are also carried out for control

element 2, 3 and 4. One adjustment task is provided

for control element 5. The exact adjustment tasks are

listed below in table 1.

Table 1: Adjustment tasks for experimental procedure.

Tas

Description Type of Feedbac

Set slider value from 0

to 12

control element 1

Increase slider value of

12 b

Decrease slider value of

18 by 2

Set slider value from 18

to 26

A5 Set slider to "Start

control element 2

A6 Set the slider to “Stop”

Set the slider from

“Low” to “High”

control element 3

Move the slider from

“Hi

h” to “Medium”

Move the slider from

“Low” to "Hi

h”

control element 4

A10

Set the slider from

“High” to "Medium”

A11 Set the slider to “Reset” control element 5

After each adjustment task, the task completion of

the secondary task, the setting time, the errors

concerning the primary task, the distraction caused by

the haptic feedback, as well as the effort during the

adjustment task of the test person are recorded. The

respondent using a 7-level Likert scale evaluates the

test criteria distraction through feedback and the

effort during the adjustment task.

3 RESULTS

The results of the study are presented below. First, the

comparative presentation of the results of the

measured values recorded during operation without

and with haptic feedback are presented. These

parameters include the objectively measured

variables such as task fulfilment, setting time and the

number of errors of the main task. The effort during

the adjustment task of the test persons as well as the

distraction caused by the feedback was also recorded

by a subjective survey.

3.1 Task Fulfilment of the Secondary

Task

There are no differences concerning the task

fulfilment of the adjustment task executed with and

without tactile feedback. With regard to the task

fulfilment of the adjustment task, no significant

differences between the control tasks and the design

of the electro-tactile feedback of the control elements

could be identified.

3.2 Setting Time

Fig. 4 shows the recorded setting times of all

feedback variants. For comparison, the setting times

per adjustment task for operation without and with

feedback are placed next to each other. Each

adjustment task is plotted in form of a box plot in

order to be able to draw conclusions about the

dispersion of the adjustment task. The setting times

vary between 0.14 and 5.21 seconds. For tasks A1,

A2, A3, A4, A5 and A8, the setting times change only

minimally from operation without to operation with

haptic feedback. For the other five tasks (A6, A7, A9,

A10), the setting times are slightly longer when

operating with than without feedback. For task A9,

the setting time is even increased by almost 47%. The

trend thus shows an increase in setting time due to

haptic feedback. Therefore, the control elements 1 to

5 were tested to significant differences by a Wilcoxon

Test. With the exception of A 11, no significant

differences in setting time were found between the

control elements concerning the adjustment tasks A1

to A11 with and without haptic feedback. A

significant difference (z=-2.471, p=0.015, n=15) was

found for A 11. Consequently, in this study the setting

time with haptic feedback is not significantly higher

than without feedback.

CHIRA 2020 - 4th International Conference on Computer-Human Interaction Research and Applications

3.3 Error Number of the Main Task

Fig. 5 shows the number of errors. The number of

errors are the contacts with the edge of the bore during

the execution of the main task. It can be seen that the

number of touches is lower when performing a

secondary task with haptic feedback. The greatest

improvement in the average number of contacts can

be seen with adjustment task A4 (Fig. 4). In this task,

the slider is set from the value 18 to the value 26.

However, it should also be mentioned that the

variance of the measured values is particularly high.

Consequently, the number of touches varies greatly

among the test persons. In a difference test, using the

Wilcoxon test it is also evident that the differences in

control element 1 are significant concerning the

adjustment tasks A1, A3 and A4. With regard to

adjustment task A1 (z=-2.659, p=0.008, n=15) and

A4 (z=-3.190, p=0.001, n=15), the difference in

operation with and without haptic feedback is high

significant lower. For control element 2, 3 and 5, on

the other hand, the haptic feedback has no significant

effect on the number of touches for the main task.

Further significant differences are found for task A10

(z=-2.292, p=0.022, n=15). Here, too, the haptic

feedback results in a lower number of touches of the

main task.

3.4 Effort during Adjustment Task

The box-plot diagram in Fig. 6 shows the results in

terms of the effort during the adjustment task. The

evaluation results vary between no effort and high

effort. The highest effort is in the fourth task (A4).

The averaged effort here is 4 out of 6. The effort for

task A5 is only minimally higher on average when

operating without haptic feedback than when

performing task A6. Comparing task A7 and A9, on

average, the effort involved in operating without

feedback is slightly higher for control element 3 than

for control element 4. The two control elements differ

only in the way they provide haptic feedback.

Fulfilling the task with feedback requires on average

the same stress with task 7 as without feedback. The

effort induced by task A11 is rated as the highest. To

have a closer look at the results a Wilcoxon Test was

conducted. As we can see in Fig. 5 the effort of task

A1 (z=-2.460, p=0,014, n=15), A2 (z=-2.373,

p=0,018, n=15), and A3 (z=-2.310, p=0,021, n=15) is

significant less rated concerning the adjustment task

with and without electro-tactile feedback. Also task

A4 (z=-3.671, p=0,001, n=15), A10 (z=-2.598,

p=0,009, n=15) and A11 (z=-2.739, p=0,006, n=15)

show a high significant better rating.

3.5 Distraction through Feedback

The results of the evaluation of how much attention is

paid to haptic feedback during a task are shown in

Fig. 7 as a box plot across all adjustment tasks. The

evaluation is on a scale from no distraction to high

distraction. The median values lie between three for

A1 and four for the tasks A3 and A7 to A11.

Furthermore, it is noticeable that for the tasks A5 and

A6, with an average rating below three, somewhat

less attention is paid to feedback than for the tasks at

the other control elements. In tasks A9 and A10 on

Figure 4: Setting time concerning the secondary task.

Figure 5: Number of errors concerning the main task.

Tactile Information Coding by Electro-tactile Feedback

control element 4, on average a little more attention

is paid to haptic feedback than in tasks A7 and A8.

For task A11, haptic feedback is also used relatively

heavily. The median value here is four. Significant or

high significant differences could not be found.

Figure 6: Effort of the secondary task during operating in a

main task.

Figure 7: Distraction caused by the electro-tactile feedback

operating in a main and secondary task in parallel.

4 DISCUSSION

Using control element 1, it is evident that it makes

sense to code an interface element with short tick

mark distances by means of haptic feedback pulses.

On the one hand, the error rate on a task performed in

parallel during operation is improved (Fig. 5), on the

other hand the operation is perceived as significantly

less strenuous, as can be seen in Fig. 6. The feedback

contributes greatly to orientation on the control

element. Adjustment tasks with feedback pulses of

higher intensity (tasks A2, A3 and A4) show stronger

improvement values. This leads to the conclusion that

a consistently strong feedback is more suitable. With

small changes in settings and strong feedback, the tick

marks are partly counted via the tactile feedback

pulses. In theory, even blind operation would be

conceivable here. Moreover, the feedback feels

pleasant for many test persons.

With both adjustment tasks (A5 & A6), control

element 2 shows an improvement both in errors on a

parallel activity and in the induced effort (see Fig. 5

and 6). This indicates that the haptic feedback

facilitates operation even with a short adjustment

distance and the change between only two states. The

execution of task A5 is somewhat more difficult.

However, the ability to concentrate on the parallel

task is improved more than in task A6. The induced

effort, on the other hand, is improved more by the

haptic feedback in task A5.

In comparison of control element 3 and 4, the

haptic course with the intensity jumps causes a

stronger improvement than the haptic course with the

extreme points of intensity. When performing tasks

A5 and A7, the greatest difference between the two

haptic progressions can be seen. While the effort does

not change for control element 3 and the errors of the

main task improves by only about 8%, it improves by

about 27% for control element 4 and effort by about

15% (Fig. 6). This reveals that for control elements

with three ranges and large distances between them,

a jump in feedback intensity is preferable. However,

even here the test persons cannot feel the exact

location of the haptic feature (intensity jump).

However, they recognize the different intensity levels

in the different ranges.

Control element 5 is the only one that shows a

greater improvement in effort required for operation

than for concentration on parallel activity (Fig. 5 and

6). Consequently, the haptic feedback facilitates the

task. The test persons experience a clear support by

the feedback and feel a significantly lower effort.

The haptic feedback has a positive effect on all

control elements. The accuracy of the execution of a

parallel activity is improved for all control elements.

The haptic feedback during operation also has a

positive effect on the perceived effort during the

execution of a parallel task for all control elements.

CHIRA 2020 - 4th International Conference on Computer-Human Interaction Research and Applications

5 CONCLUSION

The study reveals that the test persons can concentrate

better on their main activity if they feel a haptic

feedback when operating the touch user interface in

parallel. The reduction in the number of touches

across all test subjects and all five control elements is

37% overall. The greatest improvement in errors

concerning the main task on parallel activity is found

at control element 1. The individual feedback

impulses contribute to orientation, are partially

counted and improve the concentration on the parallel

task the most. The subjectively induced effort is

experienced as less when operating the user interface

in parallel with haptic feedback than operating

without haptic feedback. The setting time, however,

shows an increase over all tasks and test persons. In

addition, the feedback used for the control elements

feels too weak. The information coding behind the

haptic feedback is also partly not intuitively

understandable. This should be further improved.

After a short explanation at the end of the test, the

coding could usually be understood quickly. It is

therefore interesting to see how much the errors of the

main task improves when the test persons are briefly

introduced to the coding logic of the feedback before

the start. Exercise also leads to a better use of the

feedback and further contributes to an increase in the

ability to concentrate and a reduction in effort. The

setting time, which tends to increase with feedback

also shortened through practice.

In conclusion, the haptic feedback has a high

potential with touch screens. There is a great level of

freedom in design and a wide variety of technical

approaches to implementation. In addition, this study

is able to show that haptic feedback at the user interface

both improves the concentration on a parallel activity

and reduces the effort during operation.

ACKNOWLEDGEMENTS

This work is founded by the German Research

Foundation (DFG) within the scope of the research

project “Ageing-appropriate adaptive electro-tactile

touch user interfaces in a translatory application”,

grant MA 4210/6-3.

REFERENCES

Harrison, C., Hudson, S. (2009). Providing dynamically

changeable physical buttons on a visual display. In

Proceedings of the 27th International Conference on

Human Factors in Computing Systems. Boston (pp.

299-308). ACM Press.

Hoggan, E., Brewster, S. A. & Johnston, J. (2008).

Investigating the Effectiveness of Tactile Feedback for

Mobile Touchscreens. In Burnett, M., Constabile, M.

F., Catarci, T., Ruyter, B.,,Tan, D., Czerwinski, M. &

Lund, A. (Hrsg.) Proceedings of the 26th Annual

Conference on Human Factors in Computing Systems.

New York (pp.1573-1582). ACM Press.

International Organization for Standardization. (2011).

Ergonomie der Mensch-System-Interaktion. Teil 910-

Rahmen für die taktile und haptische Interaktion (EN

ISO 9241-910).

International Organization for Standardization. (2017).

Ergonomie der Mensch-System-Interaktion. Teil 112-

Grundsätze der Informationsdarstellung (EN ISO 9241-

112).

Koskinen, E., Kaaresoja, T., Laitinen, P. (2008). Feel-good

touch: Finding the most pleasant tactile feedback for a

mobile touch screen button. In Proceedings of the 10th

International Conference on Multimodal Interfaces.

Chania (pp. 531-538). ACM Press.

Schmid, P., Winterholler, J., Maier, T. (2019).

Untersuchung zum Entlastungspotential des visuellen

Informationskanals durch das nutzerzentrierte Design

eines Drehbedienelements. Stuttgarter Symposium für

Produktentwicklung 2019.

Schmid, P., Maier, T., Wizani, M. (2019). Electro-tactile

feedback as support for an adjustment task on touch

control elements?. In: eProceedings Human Systems

Integration (HSI). Biarritz.

Schmid, P., & Maier, T. (2020). Adaptiv elektrotaktile

Feedbackgestaltung zur Kompensation altersbedingter

Verluste bei der Bedienung von Touch-Bedien-

oberflächen. GfA Frühjahrskongress 2020: Digitaler

Wandel, digitale Arbeit, digitaler Mensch? GfA

Frühjahrskongress.

Siegmann, S. & Notbohm, G. (2013). Noise in hospitals as

a strain for the medical staff. In: The Journal of the

Acoustical Society of America. 133(5). 3453.

VDI/VDE, Gesellschaft Mess- und Automatisierungs-

technik. (2015). Gebrauchstaugliche Gestaltung von

Benutzungsschnittstellen für technische Anlagen. Blatt

3: Merkmale, Gestaltung und Einsatzmöglichkeiten von

Benutzungsschnittstellen mit Touchscreens. VDI/VDE

3850.

Winterholler, J. (2019). Haptische Informationsüber-

tragung von Drehmomentverläufen im Kontext einer

Haupt- und Nebenaufgabe. Universität Stuttgart,

Stuttgart. Institut für Konstruktionstechnik und

Technisches Design, Forschungs- und Lehrgebiet

Technisches Design, Dissertation.

Tactile Information Coding by Electro-tactile Feedback