Wireless User-computer Interface Platform for Mental

Health Improvement through Social Inclusion

Ana Londral

, Neuza Nunes

, Hugo Silva

and Luís Azevedo

PLUX, Wireless Biosignals, Lisbon, Portugal

Instituto de Telecomunicações, Lisbon, Portugal

ANDITEC, Tecnologias de Reabilitação, Lisbon, Portugal

Abstract. Loss of communication due to long-term neurological conditions

leads to isolation and loneliness. Individuals in these conditions raise the risk of

depression, since they can’t convey their needs and wants and loose their social

networks. This paper describes the development of a wireless platform for user-

computer interface, targeted at mental health improvement through communi-

cation and social inclusion. We describe the proposed approach in the context

of an input device based on a single channel electromyographic signal, al-

though the use of other biosignals is discussed to expand users’ possibilities.

Users’ needs in terms of user interface implementation are considered, concern-

ing severe speech and physical impairments.

1 Introduction

There are several neurological conditions in which affected individuals dramatically

loose generalized motor control (e.g. brainstem stroke, motor neurodegenerative

diseases, cerebral palsy, traumatic brain injuries or spinal cord injuries) [1]; [2]; [3].

Severe motor limitations often cause motor speech disorders and consequently se-

vere difficulties in communicating. Communication loss will raise depression factors

in user, due to isolation and loneliness [4]; [5]. Moreover it will difficult clinical

support, since individuals cannot express symptoms and needs. In these conditions, if

there is no effective technology for enabling the user to communicate, communication

will be restricted to yes/no answers, which represent a great limitation to several im-

portant aspects in a person’s life, as expressing needs and wants, decision making and

social closeness[6]; [7]. Recognizing and addressing communication disorders is then

of utmost importance.

Concerning communication disorders, access to Information and Communication

Technologies (ICT) is important, since it offers means for providing social and emo-

tional support, beyond space and time constraints caused by severe speech and physi-

cal impairments [7]; [8].

In this context, the great challenge is finding a user-computer interface that the

user can control with autonomy, efficiently and consistently [1]; [9], in spite physical

Londral A., Nunes N., Silva H. and Azevedo L..

Wireless User-computer Interface Platform for Mental Health Improvement through Social Inclusion.

DOI: 10.5220/0003892301140118

In Proceedings of the 2nd International Workshop on Computing Paradigms for Mental Health (MindCare-2012), pages 114-118

ISBN: 978-989-8425-92-8

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

impairments. Efficient control of a user-computer interface will make possible for the

user to break isolation and total dependence through access to the computer (e.g. user

could independently control a virtual keyboard to access to the Internet).

Our work presents the development of a wireless platform targeted at mental

health improvement through social inclusion. We describe the application of the pro-

posed platform to a single channel user-computer interface, based on electromyo-

graphic (EMG) signal control. Important aspects concerning users’ needs and tech-

nology development are presented. New possibilities to connect other types of sen-

sors are discussed. The rest of the paper is organized as follows: Section 2 outlines

the needs faced by the target user groups; Section 3 describes the proposed approach;

and Sections 4 and 5 present the discussion and main conclusions.

2 User Needs

Concerning severely speech and physically impaired (SSPI) users, the following

characteristics were considered in implementation of the presented user-computer

interface:

(a) Mobility – wireless sensors allow user to be in a confortable position or move

independent from the place where the computer is.

(b) Biofeedback – developed software includes the possibility to go through a training

period, where the user can watch and learn to control its own signal (EMG) using

biofeedback. In case of individuals in rehabilitation process, this tool is useful for the

clinicians to evaluate the best place for the sensors and to explore new muscles.

computer and choose the software that will receive the events from this interface.

(d) Communication – it was developed a specific connection between our platform

and a specific software for Augmentative and Alternative Communication (AAC) and

Computer Access (©The Grid 2, from Sensory Software), aiming to provide specia-

lized features for user communication.

3 Proposed Platform

3.1 EMG Signal Processing

The electromyographic signal is a record of the electrical activity generated by mus-

cle cells when they are electrically or neurologically activated. For the detection of

the EMG natural activation, the signal is usually rectified and filtered to obtain its

envelope [10] or processed using statistical based methods [11].

In this work, the EMG data is collected and processed at real time using Python

[12] with the NumPy [13] and PyWin [14] packages. The processing procedure con-

sists in removing the sensor offset value and rectifying the result. To extract the sig-

nal's envelope, a smoothing filter is applied to the rectified signal.

115

Fig. 1. Representation of the communication process.

When the muscle is activated, the amplitude of the EMG envelope increases and if

it surpasses a defined threshold, the initial instant of muscular activity is accepted

after validating it for sporadic activations - by checking if the end of the activation

was at least 100ms after the start of the activation. If accepted, the voluntary activa-

tion is converted into a triggering event to control an external predefined software

(e.g. a virtual keyboard software). This voluntary activation can be configured as a

Windows keyboard event or a software specific event (implemented for ©The Grid 2,

Sensory Software), and is sent through Windows Messages [15]. A block diagram

representative of the overall communication flow is presented in Figure 1.

3.2 User Interface

A web-based application was developed in order to provide a user-friendly tool to

visualize the signals in real time, store the specifications of the software and provide

feedback concerning the muscle activations.

The application is independent of the external software to which the signal activa-

tion is sent. In the application configurations it is possible to choose the specific soft-

ware to which events will be sent – keyboard events (e.g. enter or space key) or soft-

ware-specific events previously registered with Windows Messages.

As the application starts, the signal is graphically displayed on the screen. The

threshold that sets the activation level necessary to accept the event is defined after

signal calibration. After pressing the calibration button, the subject has to perform a

few muscle activations and the threshold is defined as a third part of the maximum

contractions' mean value. As the amplitude of the EMG signal may decrease with

muscle fatigue for long acquisitions, a slider bar provided in the web interface

enables the manual adjustment of the threshold value.

When the muscle is active and surpasses the defined threshold, the event is sent to

the external software. To indicate that activation, a visual feedback is shown in the

interface. A screenshot of the web interface with the functions here described is pre-

sented in Figure 2. This single event will activate a scanning process [1] by which the

user can select a virtual key (containning a character, a text message or other types of

commands to access to computer) in a virtual keyboard.

4 Discussion

Our work was focused on creating a wireless user-interface platform based on a sin-

gle-channel biosignal onset detection, which can operate signals from a variety of

electrophysiological or biomechanical sources. We presented an embodiment of this

116

platform that produces the necessary events for manipulating a virtual keyboard. We

worked with surface EMG sensors, which can be placed anywhere in the body.

Fig. 2. Web interface and its functionalities.

An alternative to the presented EMG sensor, and for detection of specific eye

movements, would be an electrooculography (EOG) sensor, which has a higher gain

than the EMG sensor and can be suitable for low muscle electrical activity.

A relevant approach would be to combine different miniaturized and unobtrusive

sensors to capture and send different types of events. An electroencefalography

(EEG) sensor could be used in the occipital region of the head to capture EEG alpha

rhythm, which appears after eye closure [16]. The event created through this mechan-

ism could be used for other functions like starting or shutting down the application or

switch between menus.

Tests are being made in real scenarios with SSPI users. This will be important to

solve real problems arising from user-computer interaction.

5 Conclusions

Accessibility to ICT is important to SSPI individuals since it breaks isolation and

restores the ability to communicate, which has a major impact on mental health of the

users. Access to a computer may allow these individuals, not just to express needs

and wants, but also to restore social roles and access to assisted living services.

A wireless platform and user-computer interface was developed taking into ac-

count users needs, in the context of severe motor limitations. We described an appli-

cation of the proposed approach to the use of EMG sensors, which allow users to

control the computer using minimal muscle movements. Connection to a specific

software for AAC and Computer Access was considered in this work, fulfilling users’

specific needs for Communication. Future work will focus on real-world validation of

our system both with EMG, and other biosignals, which can be seamlessly introduced

in our platform to expand users’ possibilities.

117

By opening a way to ICT, our work can be helpful in sustaining and expanding

social networks, reducing isolation then challenging multiple mental health factors

affecting the target user groups.

References

1. Cook, A. and Hussey, S., (2001) Assistive Technologies: Principles and Practice (2nd ed).

Mosby.

2. Beukelman, D. R. and Garrett, K., (1988). Augmentative communication for adults with

acquired severe communication disorders. Augmentative and Alternative Communication,

4, 104-121.

3. Beukelman, D. R., Yorkston, K. M., Reichle, J., (eds). 2000. Augmentative and Alternative

Communication for Adults with Acquired Neurologic Disorders. Paul H Brookes Publish-

ing.

4. Dietrich M., Verdolini Abbott, K., Gartner-Schmidt, J. and Rosen, C. A., (2008). The

frequency of perceived stress, anxiety, and depression in patients with common pathologies

affecting voice. J Voice. 22(4):472-88.

5. Clark, L. W., (1994). Communication disorders: what to look for, and when to refer. Geria-

trics. 49(6):51-5; quiz 56-7.

6. Bryen, D. N., Chung, Y., and Segalman, B., (2009). Depression, social isolation, and

ACOLUG. In P. Formica, R. Conti and S. Osgood (Eds.) Proceedings of the Biennial Pitts-

burgh Employment Conference for Augmented Communicators, Pittsburgh: SHOUT

PRESS, 13-20.

7. Light, J. C., e Gulens, M., (2000). Rebuilding Communicative Competence and Self- De-

termination. In: Beukelman, D. R., Yorkston, K. M., Reichle, J. (eds). Augmentative and

Alternative Communication for Adults with Acquired Neurologic Disorders. Brookes Pub-

lishing: 137-179.

8. Balilin and Balandin, (2007). An exploration of loneliness: Communication and the social

networks of older people with cerebral palsy. Journal of Intellectual and Developmental

Disabilities, 32 (4), 315-327

9. Bailey, R. W., (1996). Human Performance Engineering: Designing High Quality Profes-

sional User Interfaces for Computer Products, Applications and Systems (3rd Ed.). Prentice

Hall.

10. D'Alessio, T. and Conforto, S., (2002). Extraction of the Envelope from Surface EMG Sig-

nals. Engineering in Medicine and Biology Magazine, IEEE 20(6), 55-61.

11. Bonato, P., D'Alessio, T., and Knaflitz, M., (1998). A Statistical method for the meas-

urement of muscle activation intervals from surface myoelectric signal during gait. IEEE

Trans. Biomed. Eng. 45, 287-298.

13. Rossum, G., (2003) The Python Language Reference Manual. Network Theory Ltd.

14. Oliphant, T., (2006) Guide to Numpy. Tregol Publishing.

15. Hammond, M., (2000) Python Programming on Win32: Help for Windows Programmers

(1st ed). O'Reilly Media.

16. Araújo, T., Nunes, N., Palma, S., Gamboa, H., (2010) Alpha rhythm onset detector based

on localized EEG sensor. In Proceedings of 7th ESBME - European Symposium on Bio-

medical Engineering, Chalkidiki, Greece, 2010.

118