Health@Home Scenario: Creating a New Support

System for Home Telerehabilitation

Ant

onio Teixeira, Carlos Pereira, Miguel Oliveira e Silva, Nuno Almeida,

Joaquim Sousa Pinto, Cl

audio Teixeira, Fl

avio Ferreira and Andr

e Mota

DETI/IEETA, University of Aveiro, Campus Universit

ario de Santiago, Aveiro, Portugal

Abstract. The creation of innovative methods and technologies for elderly is the

main purpose for Ambient Assisted Living. This paper provides a description on

all the associated stages and development questions required for the establish-

ment of a new telerehabilitation service. The service intends to provide elderly

people with the possibility of performing rehabilitation sessions in their houses,

with constant medical supervision via video surveillance. Following the princi-

ples of a new conceptual architecture for services, and developed according to

user-centric paradigms such as multimodality and high usability criteria, early

evaluation results point the service as an asset for remote rehabilitation.

1 Introduction

The enhancement of domestic environments with technology is nowadays a reality.

Technology creates a positive impact on quality of life especially on older generations

since technological solutions can facilitate the daily life of the elderly, by ﬁghting iso-

lation and exclusion, and by increasing their pro-activity and autonomy.

The Living Usability Lab (LUL [1]) project is a collaborative effort for R&D be-

tween academia (University of Aveiro/IEETA and FEUP/INESC Porto) and portuguese

industry (Microsoft, Micro IO and Plux). Fueled by technologies such as distributed

computing, next generation networks, natural interfaces and universal design while fo-

cused on usability rates, the project aims at having impact at general population, espe-

cially on elder and special need citizens, by envisioning the creation of a Living Lab

capable of providing support for the creation of innovative applications, services and

technologies for them.

To enable the existence of a geographically distributed lab for new AAL services

creation, evolution and evaluation - our Living Lab -, suitable architectures for the Liv-

ing Lab and its middleware - supporting creation and deployment of new services - were

needed. To conduct research and test associated ideas, a number of scenarios based on

real necessities were conceptualized.

This paper presents the associated research and development for Health@Home.

The Health@Home scenario envisions the creation of a home telerehabilitation ses-

sion with remote medical supervision. Telerehabilitation has been introduced in several

ﬁelds, from neuropsychology to occupational therapy and physical therapy. It allows

Teixeira A., Pereira C., Oliveira e Silva M., Almeida N., Sousa Pinto J., Teixeira C., Ferreira F. and Mota A..

Health@Home Scenario: Creating a New Support System for Home Telerehabilitation.

DOI: 10.5220/0003879000370047

In Proceedings of the 2nd International Living Usability Lab Workshop on AAL Latest Solutions, Trends and Applications (AAL-2012), pages 37-47

ISBN: 978-989-8425-93-5

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

for remote populations to improve their quality of life by decreasing the constant need

to travel to the healthcare centers while permitting for health professionals to be more

aware of their patients by facilitating their interaction [8]. At a technical level, a tel-

erehabilitation service poses many demands being a distributed application with a high

focus on components such as video, speech, user modeling, environment properties and

real-time communication [7].

To provide support for such components, an infrastructure was built in addition

to a conceptual architecture for service provisioning. These became the basis for the

telerehabilitation application which was later developed and evaluated. Also, given the

special necessities of AAL applications, a variety of services were also introduced to

fasten development.

This paper is structured as follows. In Section 2, we provide an overview on the

conceptual architecture followed by LUL. In Section 3 we describe the infrastructural

deﬁnitions required for the completion of the Health@Home scenario. Section 4 gives

insight into speciﬁc services that were required for the scenario. Section 5 provides

an overview on the telerehabilitation applications. In Section 6 we provide some early

evaluation results and point out future work possibilities.

2 Conceptual Architecture

The Service Oriented Architecture created on the scope of LUL adopts a less centered

view allowing for important gains in both modularity and availability. The choice for

a service oriented approach derives from its ability to achieve loose-coupling abilities

without much effort. All together, our intention was to provide developers with better

conditions to be able to create innovative AAL applications and services using the pro-

posed architecture as a basis and have at their disposition a number of services that may

assist them in creating their intended business logic.

Fig. 1. Conceptual Ambient Assisted Living architecture for LUL.

Infrastructural Layer. The architecture is composed by four main layers as seen in

Figure 1. AAL applications often use devices such as sensors, adapters, mobile devices,

desktop computers, among others. These kind of devices represent the Infrastructural

Layer. Given that devices may be added or removed from the system dynamically, we

impose that devices included in this layer must be made accessible via Web Services,

to be encapsulated and accessed from a service level layer. For this, devices should

be introduced via a “Device as a Service” paradigm [2], which allows devices to be

accessible via a well deﬁned interface.

Common Services Layer. The Common Services Layer is responsible for providing

services such as access between nodes, monitoring, security and user management. We

intended for services in this layer to include third-party services which may be needed

by certain applications. In order to facilitate the location of certain business logics, we

established that any service within the architecture must advertise itself in the service

registry. Technologically, the registry, running at a central server accessible to all nodes,

functions like a WSDL [9] repository.

Living Lab Services Layer. The Living Lab Services layer aims to involve all pos-

sible services that may arise, either from the implementation of certain devices, (for

instances, a video capture service), either from the development of new services (like

a sensor service), or either from the inclusion of already existing exterior services (a

monitoring service) that may be needed in the development of certain applications. To

comply with the user-centric paradigm associated to the project, it is expected that built

applications follow user-friendly and user-adaptable paradigms. To help them, this layer

includes services like context and user modeling providing applications with means to

better know the end user and achieve user adaptation.

Applicational Layer. In the top layer, the architecture includes an Applicational layer

where developed applications should be “placed”. An important requirement that ex-

isting SOA proposals didn’t meet was the inclusion of multimodal capabilities. In our

view, to establish maximum usability, multimodality must be also included. The telere-

habilitation application which will be later explained is one such example.

3 Infrastructure

In the conceptual architecture, a layer of services was established exclusively focused

on integrating specialized services to be developed as part of the LUL project - Liv-

ing Lab Service Layer. These services require a physical location where they can be

deployed. In this sense, two options came up: to use a central server accessible by all

project nodes (scalable to a large increase in a more realistic AAL scenario) or to use

a smaller home based server. We selected a subset from both due to the differences in

the envisioned services. While some are generic and must be made available at a well

known address, others are more “house oriented”.

Our option for LUL is divided into two parts, to have a central server (named LUL

server) where generic services can be deployed and published being complemented by

at least a small server at each house (designated as home server). This separation en-

ables local concerns to be treated only in the home server simplifying communications

between both servers, increasing their modularity.

Due to the heterogeneity inherent to Living Lab, it was established that Next Gen-

eration Networks (NGN) would be adopted into the infrastructure given its open in-

terfaces support to a wide range of services, applications, and mechanisms based on

service building blocks. For communication and in order to assure high interoperabil-

ity and integration rates, all communications within the proposed architecture use the

Internet Protocol, both UDP (for RTP [5] transmissions) and TCP.

To provide support for the rehabilitation applications, additional devices such as a

pan and tilt camera as well as a sensor from Plux [4] capable of obtaining measures

from several electrodes on the patient and communicating them to an application via

Bluetooth, were introduced.

4 Support Services

In order to help developers create new applications associated with the objectives of

LUL, a set of services were made available so that they can rapidly integrate business

logics in their applications without much effort. These services were deployed within

the Living Lab infrastructure and their interfaces made available on the service registry.

An important aspect common to all deployed services is tolerance to failures. Services

cannot be fully dependent on others and not continue to function in case of a depen-

dency issue.

4.1 Application Registry

Many new AAL applications will need to created following a distributed logic (i.e., ap-

plications divided in a number of locations but in constant communication). Because of

the critical aspects associated with some of these applications it is necessary to ensure

that connectivity is not lost, and in such a case, automatically restore it as soon as pos-

sible. As such, and to accomplish a highly dynamic distributed architecture, a speciﬁc

service must be provided to applications allowing them to easily ﬁnd, connect and know

the current status of others.

Due to these issues, a service for Application Registration was implemented and

deployed in the LUL server. The service is similar to a broker. Applications must reg-

ister themselves at the startup on the registry and also inform the registry of any status

change. With this information, client applications obtain the necessary information to

connect and to regain connectivity taking in consideration the status of its partners. In

case of a failure, the service will help in the reestablishment of the communication in

an easier and faster manner.

Fault Tolerance. The application registry service as presented provides a solution for

connectivity losses. But to achieve a truly fault tolerance environment, the effort must

be extended to the applications as well. Applications need to able to continue to operate

in cases where services that they require become unavailable. They must be able to

self-adapt to current conditions, which may implicate simply to cease a functionality or

even shutdown graciously.

To obtain this, we’ve established a set of rules to be performed within the architec-

ture, with a special focus on the application registry as its core:

1. Applications notiﬁes the broker for which services should they be alerted in case of

failures. The broker stores this information in a registry.

2. The broker isn’t 100% fail proof. As such, applications need to frequently “ping”

the broker to know that it remains fully functional. In case it fails, applications must

possess safeguards so they can maintain their autonomous execution.

3. The broker needs to guarantee that all services within the system are functional. To

do this, the broker performs pooling routines on the services.

4. Based on the registry, the broker notiﬁes all interested parties in case of a service

failure. These must be able to adopt mechanisms to guarantee their functioning. It

will be the broker’s responsibility to try and regain connectivity with the service

alerting the interested parties when it occurs.

With this set of rules adding to the application registry, services and applications

achieve better fault tolerance rates. Connections to the broker however are critical to-

wards the functionality and robustness of the overall system. To minimize effects of

possible broker failure, redundancy must be applied to the broker itself.

4.2 Video Streaming and Camera Control Service

Many potential AAL applications make use of video for surveillance and detection of

events or simply for communication. Additionally, it is also often important to allow

remote real-time control of the camera, enabling the retrieval of images on a moving

subject.

The video service is based on a producer-consumer approach. The service handles

all connections/sessions and video is transmitted using RTP. After the session is estab-

lished, the client can also use service functionalities directly regarding camera control,

such as pan, tilt and zoom.

4.3 Service for Actuators and Sensors

Information from sensors and control actuators are fundamental aspects for a typical

AAL application. Usually this information is used by the application’s own business

logic to infer with conditions relating with the user. In the developed service, all sensors

and actuators are available through web services based on a “Device as a Service”

paradigm.

The Figure 2 represents how the service was deployed and subsequently used by

a remote client application. Each generic sensor service has an interface, monitoring

capabilities and a set of devices.

Fig. 2. Conceptual view of Sensor and Actuator Service.

4.4 Multimodal Support Service

Multimodality represents an important aspect in the Living Lab ideals since it can be

used as a paradigm for achieving higher rates of usability and accessibility [3]. Cur-

rently, and in order to facilitate the integration of multimodal logics within applications,

a multimodal service is in development. This service will allow for complex algorithms

such as fusion and ﬁssion to be accessible via a web service.

Fusion will be made available to clients by requesting that initially applications

invoke the service so that a speciﬁc instance is created for them. Communication is

established between the created instance and the application. Then, by providing a con-

ﬁguration ﬁle where it declares all possible input options, the service will wait for events

which it can fuse.

Fission on the other hand requires information of all available output modalities

which includes knowing their characteristics and current availability. With it, and in-

spired by a algorithm called Adapto [6], it uses context data such as distance to a screen

or the user’s hearing capabilities, to decide on what modalities should be used or which

are indicated at the time.

An important aspect regarding the multimodal service is its almost full local auton-

omy capability. The service uses local information for its processing with the exception

of user aspects and characteristics which are centralized. This becomes important in the

sense that communications between the server and the application are reduced resulting

in increased responsiveness on the interaction.

5 Telerehabilitation Application

A telerehabilitation application allows for the execution of rehabilitation services to re-

mote and underserved populations, improving quality of life and preventing secondary

complications. With such a service, the need and frequency of the patient having to

travel to the healthcare centers is decreased allowing medical staff to not only interact

with patients on a more regular basis but also be able to stay in touch with them after

discharge [8].

5.1 Requirements

The ﬁrst step on constructing the application was to be able to specify what its require-

ments were. Given the goal of a telerehabilitation system, the ﬁrst requirement is that

the system must be used simultaneously at two different and possibly distant places:

one being the health professional current location, the other the elder home.

The second requirement is related to operating services. In a telerehabilitation ses-

sion, information like sensor data, video and feedback communication become critical

to a health professional for successful monitorization. In addition, exercise information,

instructions and user adaptation are indispensable on a patients perspective.

Most of these requirements are already answered by the development architecture,

particularly the Living Lab service layer. As such, development focused on the human-

computer interfaces in the applications. Given the different logics associated with the

two participants, the need for the two different interfaces became apparent - one for the

patient and another for the health professional.

5.2 Multiple Patient Sessions

Ideally, in order to provide maximum attention to a patient, medical supervision should

be focused on a single patient at a time. This however can only be applied in theory.

In practice, it becomes impossible to devise constant rehabilitation sessions following

a one to one basis since sessions can consume considerable time and the number of

patients are vastly superior to the number of available supervisors. Figure 3 presents a

mockup screen for the health professional application ideals.

Fig. 3. Conceptual screen on a multi-patient application for the health professional.

We’ve established a need to built the applications following a set of properties that

allows for two very speciﬁc functionalities to be achieved. First, each professional ap-

plication must be able to connect to several patient applications concurrently, that is, the

supervisor should be able to perform several sessions simultaneously, shifting his atten-

tion to one in speciﬁc for a particular reason (right bar on Figure 3). In truth, depending

on the type of rehabilitation session, some times the full attention of a supervisor is

simply not required. Second, in such cases, what is required is a “safeguard” mecha-

nism, that is, a method that alerts the supervisor in case of need. When an event such

as this happens, the medical application simply needs to shift its attention to the proper

one (alert button on Figure 3). Additionally, Adapto may be used to maximize the alert

notiﬁcation (by using several interaction modalities).

At this time, current implementation doesn’t yet fully reﬂect these characteristics,

but they will be considered for future scenarios.

5.3 Health Professional Application

The health professional application has the goal of providing information regarding an

ongoing rehabilitation session while allowing for feedback to be given if necessary. As

such, the developed application allows the health professional to:

– remotely monitor the elderly using video and biosensors information.

– plan, apply and control an exercise program.

– provide the elderly with feedback regarding their performance

Fig. 4. Health Professional Application Interface.

The interface shown in Figure 4 responds to these necessities by being composed by

ﬁve components. The ﬁrst component, session planning (number 1 in Figure 4), allows

the health professional to monitor the exercise session by choosing the exercises to be

performed.

In the second component (marked with number 2 in the ﬁgure), sensor information

is provided to the health professional so that he can visualize and analyze biological

data such as heart rate throughout the session.

In component three, the health professional can communicate and provide feedback

to the patient via textual messages. The video component, four in the ﬁgure, is an im-

portant tool for monitoring a session, as it enables the health professional to visualize

the performance of the user and check the completion of the exercises, allowing him to

correct errors and give different indications to the patient.

Finally, in component ﬁve, the professional is allowed to track an ongoing session,

such as details about the connection and the status of the current exercise.

5.4 Patient Application

Creating an application for elders is more demanding due to certain limitations as-

sociated with them. Their average expected physical limitations, different capabilities

(hearing and vision acuity, for example), context conditions (such as light and noise

levels, or distance from the devices), and even the freedom of movements intrinsic to

physiotherapy must all be taken into account.

With such aspects in mind, the main user interface was deployed into a large size

computer monitor (acting as a large size TV) given the need for exercises to be executed

a couple of meters away from the screen. A set of biosensors were introduced to provide

the health professional with additional information and input and output devices such

as microphones, speakers and video cameras, required for the interaction between the

users and the platform, and sensors to detect environment factors were included.

Fig. 5. Patient Application Interface

Figure 5 demonstrates the overall aspect of the patient’s application. The user inter-

face of the application is composed by seven visual components, which can be divided

into three blocks:

– A monitoring block placed on top corresponding to components 1, 2 and 3;

– A reception information block, placed in between. It is composed by components

4 and 5;

– And a user input block, at the bottom, constituted by components 6 and 7.

The monitoring block presents a summary of the current session’s state. Its goal is

to provide information to the user of what is happening by including the description of

the session state (indication of whether a session with the professional is in progress or

not), time and date given by component 1; a logging area describing the latest actions

taken by the health professional and the elder (component 2); and showing a list of

exercises planned for the current session, highlighting the current one (component 3).

The reception information block shows all information provided from the health

professional or from the service to the elder. This includes two components: an animated

presentation illustrating the current exercise (component 4) and video screening of the

user, allowing the patient to view in real-time his/her current performance and perform

self-correcting aspects on the exercises (component 5).

Finally, the user input block presents some interaction options for the patient by

providing a conversation area (component 6), where the user can directly communicate

with the health professional via messages; and a command list area (component 7)

where the user can issue commands to the system.

6 Early Evaluation Results and Future Work

The new service was recently tested in a simulated environment by a small set of end

users in order to assess their acceptance rates and gain feedback for future improve-

ments. Evaluation aimed aspects such as the subjects’ participation, activities pace, use

of resources and identiﬁcation of missing functionalities. Data collection on these sub-

jects was achieved by recording critical incidents (in loco) and, at the end of the session,

by answering a questionnaire (post-evaluation). The service assessment questionnaire

assesses: graphical user interface (layout), usability and satisfaction.

Participants were satisﬁed with their session, mainly due to the fact of having suc-

ceeded in accomplishing the indications of the physiotherapist and feeling comfortable

using the service. Participants reported being receptive to the use of such a service at

home. Some improvements were suggested by the introduction of speech capabilities

and minor adjustments to the interface (especially when away from screen).

In the next iterations for the service, we intend to give answer to these aspects and

provide new features and adaptability capabilities by introducing already underdevel-

opment services such as user and context modules and new multimodal paradigms. We

expect for the next iteration of the service to be fully tested with real users in a real

environment (either in an elderly institution or a hospital) for a longer period of time.

Additionally, we also envision further tests being performed to the support architecture

by the creation of other scenarios within the Living Lab scope.

References

1. Microsoft Corporation, Universidade de Aveiro, IEETA, FEUP, INESCPorto, MicroIO, Plux,

and Assoc. Salvador. Living usability lab. http://www.livinglab.pt/.

2. Scott de Deugd, Randy Carroll, Kevin Kelly, Bill Millett, and Jeffrey Ricker. SODA: Service

oriented device architecture. IEEE Pervasive Computing, 5(3):94–96, c3, July 2006.

3. Bruno Dumas, Denis Lalanne, and Sharon Oviatt. Human machine interaction. chapter Mul-

timodal Interfaces: A Survey of Principles, Models and Frameworks, pages 3–26. Springer-

Verlag, Berlin, Heidelberg, 2009.

4. Plux. Plux wireless biosignals. http://www.plux.info/, 2011.

5. The Internet Society. Rtp: A transport protocol for real-time applications. http://www.ietf.org/

rfc/rfc1889.txt, 1996.

6. A. Teixeira, C. Pereira, M. Oliveira e Silva, O. Pacheco, A. Neves, and J. Casimiro. Adapto.

adaptive multimodal output. In Proceddings of the International Conference on Pervasive and

Embedded Computing and Communication Systems (PECCS) 2011, 2011.

7. Ant

onio Teixeira, Nelson Rocha, Carlos Pereira, Jorge Sousa Pinto, et al. The living usabil-

ity lab architecture: Support for the development and evaluation of new aal services for the

elderly. In Ambient Assisted Living Book. Taylor and Francis, 2011.

8. Deborah G. Theodoros. Telerehabilitation for service delivery in speech-language pathology.

Journal of Telemedicine and Telecare, 14(5):221–224, July 2008.

9. W3C. Web services description language (wsdl). http://www.w3.org/TR/wsdl, 2001.