Reliability of a Screening Tool to Prevent Running Injuries

The RunningSmart Tool

Justien Cornelis

, Rudi Frankinouille

1,2

, Filip Stremersch

, Stig Vaerewijck

and Dirk Vissers

Department of Rehabilitation Sciences and Physiotherapy, University of Antwerp, Universiteitsplein 1, Wilrijk, Belgium

Department of S.P.O.R.T.S, Antwerp University Hospital, Wilrijkstraat 10, Edegem, Belgium

Keywords: Screening Tool, Running Injuries, Prevention, Reliability.

Abstract: The goal of this research was to assess inter- and intra-rater reliability of the RunningSmart screening tool,

developed to determine weak-links at baseline in novice, recreational and athlete runners. The tool provides

a selection of exercises targeting the identified weak-links. Overall, the tool had a fair to good inter- and

intra-rater reliability for the separate clinical tests based on the kappa value. The RunningSmart Tool

focusses mainly on recreational runners or people that initiate running. Therefore the tool is designed in a

practical and easy to apply way that can be used in a clinical setting. These data indicate that the

RunningSmart Screening tool can be confidently applied by trained individuals and used to assess the

movement patterns of recreational start-to-runners in order to make decisions related to interventions to

decline the injury risk and enhance physical activity.

1 OBJECTIVES

In the last decade, running gained popularity as a

readily accessible recreational sport and leisure. It is

estimated that 10%-20% of Americans run regularly

as it is considered to be the most efficient way to

achieve fitness (Fields et al., 2010).

In Belgium, many novice untrained runners

participate in a start-to-run program in an attempt to

resume physical activity. It is a 10 weeks

programme that should be continued after. However,

a drop-out of 31,5% was reported after 10 weeks.

Moreover, 40% of the participants reported an injury

(Cloes and Pétré, 2012).

An injury due to running is a capital reason to

renounce a running program. In literature, incidence

rates of 19,4% to 79,3% were reported (van Gent et

al., 2007).

In a healthy active adult population the most

frequently reported lower extremity injuries were

hamstring strain, anterior cruciate ligament injury,

Achilles tendon pathology and ankle sprain (Zazulak

et al., 2007). Authors state that a previous injury is

the greatest risk factor for future injury caused by

changes in proprioception, decreased mobility,

increased flexibility, etc.(Fulton et al., 2014) These

biomechanical factors are modifiable in order to

decrease the initial risk on an injury.

General biomechanical screening tools such as

the Functional Movement Screen (Minick et al.,

2010, Teyhen DS, 2012) and the Nine-test Screening

Battery (Frohm et al., 2012) were suggested in

literature and were stated to be reliable. However,

these tools were developed to screen athletes.

RunningSmart is a new clinical and

biomechanical screening tool developed to

determine individual weak-links at baseline in

novice, recreational and athlete runners. Based on

the outcome of the screening, the tool provides a

selection of exercises targeting those identified

weak-links in order to reduce the drop-out ratio. The

aim of this study was to assess the inter- and intra-

rater reliability of the RunningSmart tool.

2 METHODS

2.1 Description of the Tool

A questionnaire was implemented in order to screen

for exercise limiting pathologies.

The screening tests for running injuries described

Cornelis, J., Frankinouille, R., Stremersch, F., Vaerewijck, S. and Vissers, D..

Reliability of a Screening Tool to Prevent Running Injuries - The RunningSmart Tool.

in the tool are scientifically grounded universal

clinical tests to investigate biomechanical and

musculoskeletal disorders. An overview of these test

is given in Table 1.

Table 1: Overview of the implemented clinical tests.

Test 1 Inspection and Palpation

Test 2 Single Leg Squat

Test 3 Single Leg Jump

Test 4 Single Leg Heel Raise

Test 5 Foot Mobility

Test 6 M. Gluteus Medius Strength

Test 7 Mm. Hamstrings Length

Test 8 Abdominal Strength

Test 9 Mm. Hamstrings or Mm. Glutei

Dominance

Test 10 Mm. Quadriceps Length

Test 11 Bridging

Test 12 Iliotibial Tractus Length

Test 13 Hip Mobility

2.2 Protocol

Eligibility criteria for the subjects were (1) 18-25

years old, (2) healthy, (3) BMI 18,5-24,9 kg/m²

(male) or 17,5-23,9 kg/m² (female), (4) no

biomechanical physical injury in the past (5)

performed no severe physical activity 48 hours

before measurement, (6) not pregnant.

Both raters (A and B) were physiotherapists and

had the same clinical experience and education.

At first, participants were tested randomly by

assessor A or B, immediately followed by a retest

(the other assessor) to compare inter-rater reliability.

After seven days, all participants were tested

again by assessor B to compare intra-rater reliability.

The outcome of the movement evaluation

criterion for each test was evaluated by three

categorical observational possibilities e.g. “yes”,

“no” or “more or less”.

The ethics committee of the University Hospital

in Antwerp approved the trial, and a written

informed consent was obtained for each participant.

2.3 Statistical Analysis

The reliability of the overall final score was

evaluated using the Interclass Correlation

Coefficient (ICC) in SPSS 22.0 (Statistical Package

for the Social Sciences, IBM Corporation, NY,

USA).

The reliability of each clinical test was analysed

by the Cohen’s Kappa with a 95% Confidence

Interval (95%CI). Given the paradox of high

agreement and low Kappa values (Cicchetti and

Feinstein, 1990), the positive Proportion of

Agreement (PoA) and 95%CI was also calculated in

the Excel spreadsheet.

3 RESULTS

Twenty-nine participants met the afore mentioned

inclusion criteria (male (n=15; 22,8 kg/m²) and

female (n=14; 20,3 kg/m²)).

Overall ICC scores for inter- and intra-rater

reliability were 0.652 and 0.686 respectively.

The inter-rater and intra-rater reliability results

for the individual clinical tests, are listed in Table 2.

Table 2: Inter-rater and intra-rater reliability for the

different tests described by the Proportion of Agreement

(PoA) and Cohens’ Kappa.

Inter-rater Intra-rater

Test PoA Kappa PoA Kappa

1 0,741 0,381 0,862 0,585

2 0,483 0,193 0,776 0,293

3 0,707 0,266 0,776 0,267

4 0,759 0,425 0,707 0,239

5 0,793 0,491 0,793 0,555

6 0,759 0,100 0,690 0,385

7 0,845 0,691 0,879 0,753

8 0,931 0,164 0,828 0,328

9 0,672 0,367 0,690 0,204

10 0,793 0,430 0,776 0,510

11 0,828 0,253 0,810 0,230

12 0,931 0,558 0,879 0,294

13 0,879 0,734 0,810 0,460

4 DISCUSSION

The purpose of this study was to investigate the

inter- and intra-rater reliability of the RunningSmart

testing protocol, a new screening tool to assess weak

biomechanical links that could induce running

injuries in novice runners.

Overall, the tool had a

fair to good inter- and

intra-rater reliability (Rosner, 2010). The inter-rater

and intra-rater reliability for the separate clinical

tests based on the kappa value was average to good

(Altman, 1990). Moreover, the PoA was moderate to

excellent for inter-rater reliability and good to

excellent for intra-rater reliability.

Since the applied algorithm to obtain the final

score remains unclear e.g. different scores on

individual tests, it is assumed possible that a

different score on the subtests was obtained between

two assessors although the final score was the same.

This is the same limitation as reported in other

screening protocols (Teyhen et al., 2012). Creators

of these screening tools could provide researchers

more information about the score implementation.

The majority of the afore mentioned screening

protocols focus on athletes while the RunningSmart

Tool focusses mainly on recreational runners or

people that initiate running. Therefore the tool is

designed in a practical and easy to apply way that

can be used in a clinical setting.

The researchers in this study had the same

experience and educational level. Moreover, they

were trained as a RunningSmart Coach (two days

practical education), however their practical

experience as a therapist in a clinical setting was

limited. The clinicians tested individually and in

separate rooms. No video recordings were made.

The screening battery is easy to use for familiarized

professionals and requires minimal equipment.

The possible learning effect of the participants

was limited as the researcher did not give feedback

about the outcome.

Further research towards reliability and validity

of screening protocols to prevent running injuries is

mandatory. Moreover, randomized long-time

follow-up trials could be conducted to estimate the

incidence of running injuries in participants that

initiate running as the effect of the screening

protocol and proposed exercises could be evaluated.

These data indicate that the RunningSmart

Screening tool can be confidently applied by trained

individuals and used to assess the movement

patterns of recreational start-to-runners in order to

make decisions related to interventions to decline the

injury risk and enhance physical activity.

REFERENCES

Fields KB, Sykes JC, Walker KM, Jackson JC (2010)

Prevention of running injuries. Current sports

medicine reports 9 (3):176-182

Cloes M, Pétré B Characteristics Of Participants To'Start

To Run'Programme. Comparison Between

Maintenance And Drop Out Groups. In: Congrès

ICSEMIS 2012 «Sport… Inspiring a Learning

Legacy»(congrès scientifique Pré-Olympique), 2012.

van Gent BR, Siem DD, van Middelkoop M, van Os TA,

Bierma-Zeinstra SS, Koes BB (2007) Incidence and

determinants of lower extremity running injuries in

long distance runners: a systematic review. British

journal of sports medicine

Zazulak BT, Hewett TE, Reeves NP, Goldberg B,

Cholewicki J (2007) Deficits in neuromuscular control

of the trunk predict knee injury risk a prospective

biomechanical-epidemiologic study. The American

journal of sports medicine 35 (7):1123-1130

Fulton J, Wright K, Kelly M, Zebrosky B, Zanis M, Drvol

C, Butler R (2014) Injury risk is altered by previous

injury: a systematic review of the literature and

presentation of causative neuromuscular factors.

International journal of sports physical therapy 9

(5):583

Minick KI, Kiesel KB, Burton L, Taylor A, Plisky P,

Butler RJ (2010) Interrater reliability of the functional

movement screen. The Journal of Strength &

Conditioning Research 24 (2):479-486

Teyhen DS SS, Lorenson CL, Halfpap JP, Donofry DF,

Walker MJ, Dugan JL, Childs JD. (2012) The

Functional Movement Screen: a reliability study. J

Orthop Sports Phys Ther Jun;42(6):530-540

Frohm A, Heijne A, Kowalski J, Svensson P, Myklebust G

(2012) A ninetest screening battery for athletes: a

reliability study. Scandinavian journal of medicine &

science in sports 22 (3):306-315

Cicchetti DV, Feinstein AR (1990) High agreement but

low kappa: II. Resolving the paradoxes. Journal of

clinical epidemiology 43 (6):551-558

Rosner B (2010) Fundamentals of biostatistics. Cengage

Learning,

Altman DG (1990) Practical statistics for medical

research. CRC press,

Teyhen DS, Shaffer SW, Lorenson CL, Halfpap JP,

Donofry DF, Walker MJ, Dugan JL, Childs JD (2012)

The functional movement screen: A reliability study.

journal of orthopaedic & sports physical therapy 42

(6):530-540