Insights from a Long-Term in-the-Wild Study with Post-Stroke

Patients using a Socially Assistive Robot

Ronit Feingold Polak

and Shelly Levy-Tzedek

1,2,3 b

Recanati School for Community Health Professions, Department of Physical Therapy Ben-Gurion University of the Negev,

Beer Sheva, Israel

Zlotowski Center for Neuroscience, Ben-Gurion University of the Negev, Beer Sheva, Israel

Freiburg Institute for Advanced Studies (FRIAS), University of Freiburg, Germany

Keywords: Neurorehabilitation, Stroke, SAR, HRI, in-the-Wild.

Abstract: The growing care gap in rehabilitation calls for ways to help patients perform their exercises in a safe

environment, while receiving feedback on their progress. Socially assistive robots have been suggested as

potential agents in helping patients in their rehabilitation regimen. Here, we present a set of guidelines that

we developed, based on our experience with running a 2-year in-clinic study with 20 stroke patients who used

a platform we developed for post-stroke training over a 5-7-week period; 10 of those trained with a socially

assistive robot, and 10 with a computer-based system. The guidelines we provide here are aimed to assist

researchers who wish to implement a long-term technological intervention program with patients in the wild.

1 INTRODUCTION

Effective, scalable rehabilitation strategies are

expected to be in higher demand in the coming

decades, with increased patient survival after diseases

with severe functional deficits, such as stroke

(Kellmeyer et al. 2018). In recent years, many

research works studied the applicability of using

socially assistive robots (SARs) in different domains

such as health, education, and elderly care.

Since March 2020, when the World Health

Organization declared COVID-19 a pandemic, the

rehabilitation world has been facing new challenges

because of the requirement for social distancing,

especially in at-risk populations. Khan and Amatya

(2020) noted the following two challenges that the

realm of rehabilitation faces in light of COVID-19:

(1) Providing safe physical environments within

rehabilitation wards that comply with social

distancing and hygiene; (2) mitigating risk (as able)

for a potential COVID-19 exposure to patients and

staff. The requirement to keep a social distance and

reduce physical contact stresses the need for

alternative rehabilitative tools, such as SARs, to

enable patients to have an uninterrupted (even if

https://orcid.org/0000-0002-6244-9095

https://orcid.org/0000-0002-5853-3235

modified) rehabilitation regime. We thus argue that it

is now more crucial than ever to develop SARs for

healthcare.

We developed a novel gamified system for post-

stroke long-term rehabilitation, using the humanoid

robot Pepper (SoftBank, Aldebaran). We used the

participatory-design approach, and implemented the

robotic training platform in a rehabilitation clinic with

10 patients over a 2-year period; another group of 10

patients used the platform we developed using a

configuration that does not include the robot, but uses

the exact same rehabilitation exercises (Feingold

Polak, Barzel, and Levy-Tzedek 2021). We thus

conducted the first study (to the best of our

knowledge) to evaluate a long-term intervention

using a SAR with post-stroke patients in a

rehabilitation center, as part of their conventional

rehabilitation program.

Though social robots have a great potential to

assist patients in the domains of health care and

therapy [for examples see (Pulido et al. 2019;

Broadbent et al. 2018; Bundea, Bader, and Forbrig

2021), there is still a limited number of works

describing longitudinal studies within this domain

(Leite, Martinho, and Paiva 2013). Leite, Martinho,

Feingold Polak, R. and Levy-Tzedek, S.

Insights from a Long-Term in-the-Wild Study with Post-Stroke Patients using a Socially Assistive Robot.

DOI: 10.5220/0010719400003060

In Proceedings of the 5th International Conference on Computer-Human Interaction Research and Applications (CHIRA 2021), pages 319-323

ISBN: 978-989-758-538-8; ISSN: 2184-3244

319

and Paiva (2013) noted several reasons for this: (1)

longitudinal studies are much more laborious and

time consuming than short-term studies, especially in

ecological environments and in the wild; (2) only in

the last few years technology has become robust

enough to allow for some degree of autonomy when

users interact with robots for extended periods of

time. There are thus very limited resourced upon

which to draw, when researchers enter this “rough

terrain” of longitudinal patient studies. For this

reason, based on our experience in this 2-year in-

clinic study, as well as on the experience reported by

fellow researchers, we constructed a set of guidelines

to be used by researchers who wish to run long-term

studies with patient populations in the wild.

2 METHODOLOGY

We present the methodology of this study in brief, as

it is detailed elsewhere (Feingold Polak, Barzel, and

Levy-Tzedek 2021), and is not the focus of the

current paper.

Twenty patients post-stroke participated in a long-

term study, in which they used a platform we

developed for post-stroke rehabilitation. The platform

included seven gamified exercise sets, which

corresponded to exercise sets that patient need to

perform as part of their rehabilitation routine. In those

exercise sets, we used everyday objects (such as

kitchen items, keys, cards), which were all equipped

with RFID tags, so we can track their location and

provide feedback to the patients based on it.

Ten of these patients received the instructions for the

exercise sets and the feedback on their performance

from the Pepper robot (SoftBank Robotics); The other

10 patients used the same platform, and received the

instructions for the exercise sets and the feedback on

their performance via a standard computer screen.

Each patient came in for 15 sessions with the

platform, each lasting 45-60 min, over a 5-7 week

period. We thus report here the guidelines we drafted

based on a total of 306 sessions.

3 GUIDELINES

3.1 Intensity, Specificity &

Engagement

Matarić et al. (2009) suggested that the design of

interactions with social robots for rehabilitation

should follow two guiding principles: (ⅰ) high

intensity of task-specific training and (ii) a system

that will be engaging and user-friendly.

3.2 Task Variety

For a system to be applicable to a wide variety of

patients and different levels of impairments, and in

order for it to engage patients in the long-term, there

should be a variety of tasks, with different levels of

complexity, which can be executed by both low-

functioning and high-functioning patients. Users

should be able to progress in the task according to

their ability and motor performance. The participants

in our study highlighted the variety of the tasks this

platform offered practice on, which they did not get

the opportunity to practice in other therapy sessions

they received as part of their standard rehabilitation

program.

3.3 Integration with Patients’

Rehabilitation Program

In our experiment (Feingold Polak and Levy-Tzedek

2020; Feingold Polak, Barzel, and Levy-Tzedek

2021), we placed the SAR in a rehabilitation center as

part of patients' scheduled rehabilitation program,

which we believe was a facilitator to the success of

the implementation of the system. We recommend

that, if possible, the SAR will be situated in a familiar

location, which the patient visits on a regular basis, so

that travelling to train with the robot is not an added

hurdle – for the patients or for the family members

who drive them. In addition, scheduling sessions with

the robot in the same day as other rehabilitative

activities and having a single point-of-contact for

both can facilitate the maintenance of a regular

schedule for training.

3.4 Communication

The instructions given to the user should be simple,

gradually increasing in difficulty, and spoken slowly

and clearly. However, the response time of the robot

should be as fast as in human-human interaction

(Feingold Polak et al. 2018). From the experience

from the current study and from previous ones

(Feingold Polak et al. 2018; Feingold Polak and

Levy-Tzedek 2020), when the response time of the

system is longer than 4-5 sec participants experience

it as slow, which causes frustration. We added to the

robot a reaction of "I'm checking" if it took it longer

than four seconds to examine whether the order

matched the displayed image, so the participant will

not experience the robots' response time as too long

Humanoid 2021 - Special Session on Interaction with Humanoid Robots

320

(for more on the effect of timing on users’ perception

of HRI, see (Langer and Levy-Tzedek 2020).

3.5 Fatigue Management

Since stroke patients experience frequent fatigue

(Acciarresi, Bogousslavsky, and Paciaroni 2014;

Cumming et al. 2016) and muscle weakness, patients

should have the ability to rest when needed. When the

patient is fatigued and cannot complete the task

without using undesirable compensatory movements

(Kashi et al. 2020), either the patient should rest, or

the session should end. In our system, in addition to

enabling the participant to rest or to pause when

desired, we also added built-in stretching breaks.

3.6 Safety without Direct Contact

In our study, a clinician was always present in the

room, to offer assistance if needed. Future studies on

SAR interaction should strive to use a room with one-

way mirror, so that the participant will be able to

interact safely with the system without the presence

of a research assistance, who will be sitting on the

other side of the glass and will be able to see the

participant and to intervene in case of a technical

failure or if other assistance is required.

3.7 Multidisciplinarity and

Participatory Design

Our multidisciplinary lab team built and developed

the rehabilitation platform we used. It included a

physical therapist who specialized in post-stroke

rehabilitation, and engineering students. During the

process we interviewed clinicians (Feingold Polak et

al. 2019; Feingold Polak and Levy-Tzedek 2020;

Feingold Polak, Barzel, and Levy-Tzedek 2021),

patients and their family members (submitted for

publication). We believe that a multidisciplinary

team, and the participatory-design approach are

central components in the success of this platform.

3.8 Feedback and Reward

Users need to receive feedback on their performance

and on their results, as this is an essential component

of their motor learning (Cirstea and Levin 2007).

However, as the participants in our study noted, the

feedback should be given in a manner and at a

frequency that will not negatively affect their

compliance to keep on training. Some of the

participants in our study, especially the younger ones

(<45 yo) mentioned they do not wish to receive verbal

feedback on their performance after each trial, but

would rather receive verbal feedback after several

trials and visual feedback (like the sign of raised

thumb for "like") following the other trials. In

addition, they mentioned they would like to receive

feedback on their motor performance. That is, they

sought feedback on their body movements as they

performed the task, whether they involved any

compensatory movements, in addition to their task

performance. We are currently in the process of

developing this capability (Kashi et al. 2020).

3.9 Real-Time Technical Support

During the 2-year in-clinic study we faced several

technical challenges; software and hardware

malfunctions, which, at times, were not solved on the

spot, and led to frustration on the part of the

experimenters and patients alike. In the context of

rehabilitation, it is advantageous to have clinicians

run the study; the clinicians may not have a strong

technical background, and thus their ability to resolve

technical problems that arise may be limited.

Technical problems are part of any technology

implementation, specifically of novel prototype

devices. Therefore, having quick-responding

technical support, and training the clinician to solve

basic technical problems which may occur, is

essential for the success of technology

implementation in a rehabilitation setting.

3.10 Personalization

The value in adapting the rehabilitation program to

the personal needs of the patient was also stressed by

the participants in our study, who mentioned the

importance of personalizing the design of HRI and

Human-Computer Interaction (HCI) and tailoring it

to the specific task and patient needs. They mentioned

they would like the system to be able to adapt to their

personal performance, e.g. by adapting the feedback

to their movement patterns, and by automatically

progressing through the exercise game levels based

on their success rates.

Importantly, personalization of human-robot

interactions in the context of rehabilitation is multi-

layered, and needs to be frequently updated, as

opposed to a single setting that might suffice in other

context. For example, personalization should include

adaptive responses to the patient’s motor ability, or

physiological state (Feingold-Polak and Levy-Tzedek

2021) In rehabilitation, personalization is not only

important in order to establish engagement, but it is

an essential component for the recovery of motor and

Insights from a Long-Term in-the-Wild Study with Post-Stroke Patients using a Socially Assistive Robot

321

cognitive abilities over a long-term interaction, and is

an essential part of establishing trust between the

patient and the SAR.

4 CONCLUSIONS

During our experience in running an in-the-wild long-

term study with patients, we identified several factors

that can support the success of such an

implementation of novel technology. The factors are

related to the system itself (e.g., task variety), to the

technical aspects of running the experiment (e.g.,

technical support), and to user-related factors (e.g.,

personalization). We anticipate that the insights we

collected will be useful to researchers who wish to run

a study with patients using novel technology, and

most particularly to those who wish to run a long-term

study in the wild.

ACKNOWLEDGEMENTS

The research was partially supported by the Helmsley

Charitable Trust through the Agricultural, Biological

and Cognitive Robotics Initiative and by the Marcus

Endowment Fund, both at the Ben-Gurion University

of the Negev. Financial support was provided by the

Rosetrees Trust, the Borten Family Foundation, the

Robert Bergida bequest, and the Consolidated Anti-

Aging Foundation. This research was also supported

by the Israel Science Foundation (grants No. 535/16

and 2166/16), by the Israeli Ministry of Health, by the

National Insurance Institute of Israel, the Negev Lab

in Adi-Negev, and received funding from the

European Union’s Horizon 2020 research and

innovation programme under the Marie Skłodowska-

Curie grant agreement No 754340.

REFERENCES

Acciarresi, Monica, Julien Bogousslavsky, and Maurizio

Paciaroni. 2014. 'Post-stroke fatigue: epidemiology,

clinical characteristics and treatment', J European

neurology, 72: 255-61.

Broadbent, Elizabeth, Jeff Garrett, Nicola Jepsen, Vickie Li

Ogilvie, Ho Seok Ahn, Hayley Robinson, Kathryn Peri,

Ngaire Kerse, Paul Rouse, and Avinesh Pillai. 2018.

'Using robots at home to support patients with chronic

obstructive pulmonary disease: pilot randomized

controlled trial', Journal of medical Internet research,

20: e45.

Bundea, Alexandru, Sebastian Bader, and Peter Forbrig.

2021. "Interaction and Dialogue Design of a Humanoid

Social Robot in an Analogue Neurorehabilitation

Application." In International Conference on Human-

Centered Intelligent Systems, 76-85. Springer.

Cirstea, MC, and Mindy F Levin. 2007. 'Improvement of

arm movement patterns and endpoint control depends

on type of feedback during practice in stroke survivors',

Neurorehabilitation neural repair, 21: 398-411.

Cumming, Toby B, Marcie Packer, Sharon F Kramer, and

Coralie English. 2016. 'The prevalence of fatigue after

stroke: a systematic review and meta-analysis',

International Journal of stroke, 11: 968-77.

Feingold-Polak, Ronit, and Shelly Levy-Tzedek. 2021.

"Personalized Human Robot Interaction in the Unique

Context of Rehabilitation." In Adjunct Proceedings of

the 29th ACM Conference on User Modeling,

Adaptation and Personalization, 126-27.

Feingold Polak, Ronit , Ariel Bistritsky, Yair Gozlan, and

Shelly Levy-Tzedek. 2019. "Novel gamified system for

post-stroke upper-limb rehabilitation using a social

robot: focus groups of expert clinicians." In 2019

International Conference on Virtual Rehabilitation

(ICVR), 1-7. IEEE.

Feingold Polak, Ronit, Oren Barzel, and Shelly Levy-

Tzedek. 2021. 'A robot goes to rehab: a novel gamified

system for long-term stroke rehabilitation using a

socially assistive robot-methodology and usability

testing', Journal of neuroengineering rehabilitation, 18.

Feingold Polak, Ronit, Avital Elishay, Yonat Shahar,

Maayan Stein, Yael Edan, and Shelly Levy-Tzedek.

2018. 'Differences between young and old users when

interacting with a humanoid robot: A qualitative

usability study', Paladyn, Journal of Behavioral

Robotics, 9: 183-92.

Feingold Polak, Ronit, and Shelly Levy-Tzedek. 2020.

"Social Robot for Rehabilitation: Expert Clinicians and

Post-Stroke Patients' Evaluation Following a Long-

Term Intervention." In Proceedings of the 2020

ACM/IEEE International Conference on Human-Robot

Interaction, 151-60.

Kashi, Shir, Ronit Feingold Polak, Boaz Lerner, Lior

Rokach, and Shelly Levy-Tzedek. 2020. 'A machine-

learning model for automatic detection of movement

compensations in stroke patients', IEEE Transactions

on Emerging Topics in Computing.

Kellmeyer, Philipp, Oliver Mueller, Ronit Feingold Polak,

and Shelly Levy-Tzedek. 2018. 'Social robots in

rehabilitation: A question of trust', Science Robotics, 3:

eaat1587.

Khan, Fary, and Bhasker Amatya. 2020. 'Medical

rehabilitation in pandemics: towards a new perspective',

Journal of Rehabilitation Medicine, 52: 5-8.

Langer, Allison, and Shelly Levy-Tzedek. 2020. 'Priming

and Timing in Human-Robot Interactions.' in,

Modelling Human Motion (Springer).

Leite, Iolanda, Carlos Martinho, and Ana Paiva. 2013.

'Social robots for long-term interaction: a survey',

International Journal of Social Robotics, 5: 291-308.

Humanoid 2021 - Special Session on Interaction with Humanoid Robots

322

Matarić, Maja, Adriana Tapus, Carolee Winstein, and Jon

Eriksson. 2009. 'Socially assistive robotics for stroke

and mild TBI rehabilitation', Advanced technologies in

rehabilitation, 145: 249-62.

Pulido, Jose Carlos, Cristina Suarez-Mejias, Jose Carlos

Gonzalez, Alvaro Duenas Ruiz, Patricia Ferrand Ferri,

Maria Encarnacion Martinez Sahuquillo, Carmen

Echevarria Ruiz De Vargas, Pedro Infante-Cossio,

Carlos Luis Parra Calderon, and Fernando Fernandez.

2019. 'A socially assistive robotic platform for upper-

limb rehabilitation: a longitudinal study with pediatric

patients', IEEE Robotics Automation Magazine, 26: 24-

39.

Insights from a Long-Term in-the-Wild Study with Post-Stroke Patients using a Socially Assistive Robot

323