Digital Media for Pragmatic-based Pre-TOEFL Listening

Arifuddin

, I Made Sujana

, Kamaludin

Master’s Degree in English Education FKIP Universitas Mataram, Jl. Majapahit, Mataram, Indonesia

Undergraduate Degree in Language Education Universitas Mataram, Jl. Majapahit, Mataram, Indonesia

Keywords: CALL, pragmatic, Pre-TOEFL, listening, colloquies, digital.

Abstract: Based on a multi-year Research and Development, the study shows that there are five primary causes of

difficulty in understanding pragmatic meanings in Pre-TOEFL listening, namely, Speech Rate Delivery,

Voice, Sentence Complexity, Mishearing and Colloquies. As a follow-up, it is essential that the draft of the

listening materials and supplementary digital media be developed. Two types of digital media are

supplemented in the draft: the researcher-made media and the adopted digital audio and video media. The

content of the media is based on the characteristics of each cause of the pragmatic difficulty. As the basis for

the development (Development phase), the research findings obtained from the research activities (Research

phase) and the Rubric of the Digital Media for Pragmatic-based Pre-TOEFL Listening are presented in Results

and Discussion section of this paper. This listening book is prospective CALL to boost the English language

proficiency of Indonesian learners of English, particularly the master’s degree students.

1 INTRODUCTION

Generally, daily communications imply pragmatic

meanings. Lack of pragmatic competence affects

English language proficiency (Sirikhan & Prapphal,

2011). In line with it, inferring pragmatic meaning is

difficult for the Indonesian learners of English which

leads to pragmatic failure and low proficiency

(Arifuddin & Susanto, 2012).

There are a number of studies focusing on the

relationship between English language proficiency

and pragmatic failure (Sujana, et al., 2003; Arifuddin

& Sujana, 2004; ETS Researcher, 2008; Saukah,

2010; ETS, 2012). However, none of those studies

focused on pragmatic understanding of short

conversations implied in Pre-TOEFL and the causes

of pragmatic failure faced by the postgraduate

students from diverse disciplines in the masters

degree programs. This is the ‘novelty’ or

‘authenticity’ of the present study which contributes

to the development of pragmatic competence and

English language proficiency.

It shows that there are five primary causes of

difficulty in understanding pragmatic meanings in

Pre-TOEFL listening, namely, Speech Rate Delivery,

Voice, Sentence Complexity (SC), Mishearing

(MisH) and Colloquies (Col) (Arifuddin, et al., 2017).

As a follow-up, it is essential that the draft of the

listening materials and supplementary digital media

be developed. Two types of digital media are

supplemented in the draft: the researcher-made media

and the adopted digital audio and video media. The

content of the media is based on the characteristics of

each cause of the pragmatic difficulty.

So far, a listening book focusing on pragmatic

understanding has not been developed. Hopefully,

this book is prospective to boost the English language

proficiency of the EFL master’s degree students.

2 RESEARCH METHOD

2.1 Research Design

This is a multi-year Research and Development (R &

D). This study employed Mixed-methods approach.

2.2 Participants

The present study employed total sampling drawn

from 85 students from the four masters degree

programs, English Language Education, Education

Administration, Science Education and Management

of the University of Mataram who have attended the

TOEFL training in the orientation program held for

the freshmen.

Arifuddin, ., Sujana, I. and Kamaludin, .

Digital Media for Pragmatic-based Pre-TOEFL Listening.

DOI: 10.5220/0009984500770082

In Proceedings of the 7th International Conference on English Language Teaching, Linguistics and Literature (ELITE 2019) - Promoting Global Diversity, Partnership and Prosperity through

English Development, pages 77-82

ISBN: 978-989-758-459-6

2.3 Data Collection Procedures

The present study was conducted as follows: 1) Doing

a survey in order to identify and select the appropriate

departments and participants. Of the seven master’s

degree programs of the University of Mataram, only

four departments introduced TOEFL during the

orientation program for the freshmen; 2) Selection of

the participants; 3) Preparing and validating the

research instruments; 4) Testing the pragmatic

competence of the students from the four departments

with Listening Part A Pre-TOEFL; 5) Data collection

with the listening test

2.4 Data Analysis

Data about pragmatic competence were analyzed with

Two-way Anova, while the data collected with

questionnaire and interview were analyzed with

‘iterative qualitative Analysis of Yin (2011). The

results of the analyses were displayed, described,

explained, discussed and inferred. Based on the

research findings, the final phase of the multi-year

study is the development of materials and digital

media for listening book (Development phase).

3 RESULTS AND DISCUSSION

3.1 Result

The development of the teaching materials and media

is the third phase of the multi-year Research and

Development. The presentation of the findings of the

study, five primary causes of difficulty, namely,

‘Speech Rate Delivery’ (SRD), ‘Voice’ (Voice), ‘SC,

‘MisH’ and ‘Col’ aims at providing the basis for the

development of the listening materials and digital

media. The following are the primary causes of

difficulty of pragmatic understanding of the students

of the four master’s degrees of the University of

Mataram.

Science Education

To the masters degree students of Educational

Science, the primary causes of difficulty in

understanding auditory pragmatic meanings are

‘SRD’ (21.06%), ‘Synonyms’ (Syn) (16.63%),

‘Voice’ (15.86%), ‘SC’ (15.86%), ‘MisH’ (15.86%),

‘Sound Clarity’ (SCl) (10.61%), and ‘Noise’ (Noise)

(7.59%). The least cause is ‘Redundancy’ (Red)

(0.76%).

Graph 1. Science Education

English Language Education

To masters degree students of English Language

Education, the primary causes of difficulty in

pragmatic understanding include ‘SC’ (22.1%),

‘Voice’ (17.1), ‘Col’ (8.27), ‘MisH’ (13.85), ‘SRD’

(15.77), ‘SCl’ (15.77), ‘Setting’ (Sett) (13.27), and

‘Noise’(11.35). Two causes of difficulty do not

appear, namely, ‘Pause’ (0%) and ‘Intonation’ (Int)

(0%). Besides, ‘Context’ (Cont)(3.85), ‘Reference’

(1.92), ‘Memory’ (1.92), ‘Red’ (1.92), ‘Rhetoric

Markers’ (MarkRh) (3.85) and ‘Type of Questions’

(3.85).

Graph 2. English Language Education

Educational Administration

The primary causes of Causes of Difficulty of

Pragmatic Understanding the Masters Degree

Students of Educational Administration include

‘SRD’ (19.28), ‘Voice’ (20.5), ‘Cultural Value’ (CV)

(10.74), ‘Col’ (11.98), ‘MisH’ (8.39), and ‘Noise’

(19.83). Surprisingly, four causes of difficulty do not

appear, namely, ‘Red’, ‘SCl’, ‘Sett’ and ‘Type of

Questions’. In addition, ‘Pause’ (3.57%), ‘Sex’

(2.38%) and ‘MarkRh’ (2.44%).

21,06

16,63

15,8615,8615,86

10,61

7,59

0,76

Percentage (%)

Causes of Difficulty

17,1

8,27

13,85

15,7715,77

13,27

11,35

3,85

1,921,921,92

3,853,85

Percentage (%)

Cause of Difficulty

ELITE 2019 - English Linguistics, Literature, and Education Conference

Graph 3. Educational Administration

Management

The primary casues of difficulty of pragmatic

understanding the Masters Degree Students of

Management include ‘SRD’ (32%), ‘Voice’

(20.49%), ‘Cultural Value’ (CV) (10.74%), ‘Col’

(11.98%), ‘Context’ (Cont) (8.35%), ‘SC’ (SC)

(9.32%), MisH (9.26%), and ‘Noise’(10.85%).

Surprisingly, four causes of difficulty do not appear,

namely, ‘Redundancy’ (Red) (0%), ‘Sound Clarity’

(SCl) (0%), ‘Setting’ (Sett) (0%), ‘Type of

Questions’(0%), ‘Rhetoric Markers’ (MarkRh)

(2.44%), ‘Pause’(3.57%) and ‘Sex’ (2.38 %).

Graph 4. Management

4 DISCUSSION

To the Masters Degree Students of English Language

Education, one of the primary causes of difficulty is

‘Col’ (Graph 2). One of the obstacles to listening is

‘idiomatic expressions’ (Goh, 1997), a type of

colloquies.

In the masters degree of Educational

Administration, there are four causes of difficulty in

pragmatic understanding that the students did not

face, namely, ‘Redundancy’, ‘Sound Clarity’,

‘Setting’, and ‘Type of Questions’, and ‘Rhetoric

Markers’ (Graph 3). Of the four masters degree

programs, there are three programs or departments

which place ‘SRD’ in the first rank of cause of

difficulty in understanding pragmatic meaning tested

in Listening Part A Pre-TOEFL. This a ‘novel’

finding. The detail of the causes of difficulty in

understanding pragmatic meaning is displayed in

Graphs 1 to 4.

The top cause of their listening difficulty is

‘SRD’. This is in line with Nemati, et al.’s (2016)

research finding indicating that the main listening

problems of the students involves difficulty in

identifying words when they listened to an audio file

due to fast speech and weak at understanding

vocabulary and colloquia terms.

Based on the rankings of causes of difficulty

based on gender (table 1), it indicates the listening

needs understanding of sentence structure. It is

relevant to Nadig’s (2013) point of view stating that

listening comprehension of spoken language needs

understanding the syntax of sentences.

Regardless of the ranks and the obtained scores,

the masters’ degree students of Language education

also experienced pragmatic failure. Based on the data

displayed in Graph 2, the masters’ degree students of

English Language Education achieved the highest

score, and the masters’ degree students of Educational

Administration got the lowest mean score. Pragmatic

failure frequently makes communication problems.

Some researchers have demonstrated that acquiring

the rules of appropriate language behavior can be

difficult even for fairly advanced learners and often

leading them to pragmatic failure (Beebe, Takahashi

and Uliss-Weltz, 1990; Bouton, 1994; Kasper &

Schmidt, 1996). It means that students from all

departments need extended pragmatic competence in

order to avoid pragmatic failure.

One of the causes of pragmatic failure is that

pragmatic issues receive relatively little attention in

the language classrooms (Kreutel, 2007). Besides,

grammatical competence does not facilitate them to

understand pragmatic meaning due to disparities

between learners’ grammatical development and

pragmatic development (Bardovi-Harlig & Hartford,

1993; Takahashi & Beebe, 1987). In line with it,

Bardovi-Harlig & Doernyei (1998) reported that even

learners who exhibit high levels of grammatical

competence may exhibit a wide range of pragmatic

competence when compared with native speakers in

conversations and elicited conditions. He and Yan

(1986) investigated the pragmatic failure by Chinese

learners of English as a foreign language and found

that the learners’ pragmatic development was not

proportional to their grammatical development. In

19,28

20,5

10,74

11,98

8,39

19,83

5,37

2,38

2,44

SRD

Voice

Col

MisH

Noise

Pause

Sex

MarkRh

Percentage (%)

20,49

0,74

11,98

8,35

9,32

9,26

10,85

0000

2,44

3,57

Digital Media for Pragmatic-based Pre-TOEFL Listening

other words, pragmatic failure is also experienced by

foreign language learners with high competence.

Those findings seem inconsistent with other

research findings which show that there is a

correlation between lexico-grammatical competence

and pragmatic competence (Khamyod dan

Aksornjarung, 2011). In relation to language

proficiency, low pragmatic competence affects

foreign language proficiency. This statement is

reasonable because based on some studies on foreign

language learners’ pragmatic development (Yoshimi,

2001), it proved that learners receiving instruction in

pragmatics outperformed those who did not. In the

present study, the masters’ degree students of English

language Education who were enriched with

pragmatic-related knowledge achieved the highest

pragmatic understanding.

A study conducted by Sirikhan (2011) shows that

English proficiency is a variable which has a great

effect on pragmatic ability. This agrees with Taguchi

(2007) who supports that language background and

English proficiency have influenced L2 pragmatic

processing. The findings of this study also confirm

the studies of Bardovi-Harling and Dornyei (1998) in

that EFL/ESL learning content, and proficiency

levels, affect the ability in pragmatic and grammatical

awareness. Besides, the findings of this study

correspond with some previous studies (Roever,

2005) in that the high language proficient participants

had better performance in pragmatics tests than the

low ones. This is similar to the findings of Matsumura

(2003) who reveals that the overall level of

proficiency in the target language plays an important

role in the acquisition of pragmatic awareness. Other

studies (Hill, 1997; Roever, 2005; Yamashita, 1996)

indicate that the high proficiency participants show

higher pragmatic competence than those with low

pragmatic competence.

A closely related study was conducted by Li

(2007) which examined the relationship between the

two kinds of competence of 42 Chinese English

learners with different levels of proficiency (high-

and low-levels) in BeiHang University. The study

indicates that there is a positive relationship between

linguistic proficiency and pragmatic ability. The

participants with high linguistic competence have

high pragmatic ability and vice versa. She argues that

pragmatics can be taught, and thus it is necessary to

teach students pragmalinguistics as well as

sociopragmatics.

It has been widely accepted that high proficiency

L2 learners are generally more competent in

interpreting implied meaning than low proficiency L2

learners (Lee, 2010). As categorised as low

pragmatic EFL learners, the subjects’ difficulty in

understanding pragmatic meaning, including speech

acts/language functions recognition, is reasonable. It

has been widely accepted that high proficiency L2

learners are generally more competent in interpreting

implied meaning than low proficiency L2 learners.

However, it is inconsistent with Liu’s (2006)

study indicating that the scores from other large-scale

proficiency tests, like TOEFL and International

English Language Testing System (IELTS), do not

correlate with pragmatic ability. Those who have

higher scores do not seem to have correspondingly

high pragmatic ability. A number of studies also point

out that learners of English as a foreign language, who

have excellent grammatical and lexical competence

of the target language, still fail to convey their

messages effectively due to, for instance, the lack of

social appropriateness rules and pragmatic

competence (Wolfson et al, 1989). This fact could be

an important input for the orientation and strategies

for the improvement of English language proficiency.

Of the four masters degree programs, there are

three programs or departments which place ‘RD’ in

the first rank of cause of difficulty in understanding

pragmatic meaning tested in Listening Part A Pre-

TOEFL. This a ‘novel’ finding. The detail of the

causes of difficulty in understanding pragmatic

meaning is displayed in Table 1 and Table 2 and

Graphs 1 to 4.

Generally, pragmatic failure relates to speech rate

known as ‘SRD’. Liu (2009) reports that factors that

affect listening comprehension of TOEFL are, for

instance, tone, intonation, pronunciation, word

recognition, background knowledge and speed of

delivery. To foreign language learners, understanding

messages or pragmatic meanings from high speed

spoken language is difficult. This is relevant with

Ur’s (1989:254) statement that virtually every

language learner initially thinks that native speakers

speak too fast. Learners will nevertheless eventually

need to be able to comprehend language delivered at

varying rates of speed and, at times, delivered with

few pauses.

As described earlier, ‘Voice’, which involves

intonation, stress, rhythm and the like, determine

pragmatic understanding. If a foreign language learner

is not accustomed to hearing the target language

voices or sounds, it generally causes listening

difficulty, especially if the utterances imply pragmatic

meanings. For example, understanding the pragmatic

meaning from two blended sounds is difficult.

According to Black (2006:17), paralinguistic features,

such as, intonation, stress, and the like determine the

understanding of pragmatic meaning.

ELITE 2019 - English Linguistics, Literature, and Education Conference

Understanding pragmatic meaning relies heavily

on the understanding of sentence structure or

grammar, for instance, ‘SC’. The finding of the

present study is consistent with Arifuddin’s (2013)

research finding. Another revealing point of the study

was there is much stronger correlation between the

lexico-grammatical and pragmatic competences of

the female EFL learners than the male participants

(Bulut, 2009).

The fourth rank of the causes of difficulty in

pragmatic understanding is ‘Mishearing’, apart of

sound or voice. ‘Mishearing’ is listening difficulty

which may result from unfamiliarity with the features

of particular sounds. Kostin (2004:3) argues that the

unfamilarity with the phonological aspects and

limited exposure to the target language may influence

listeners’ ability to comprehend conversations.

Besides ‘Mishearing’, ‘Sound Clarity’ and ‘Col’ have

been the primary causes of understanding pragmatic

meaning from auditory language. Another cause of

difficulty of the pragmatic understanding is limited

knowledge of synonyms or expressions. The finding

of the present study is relevant with Mei-Xia’s (2005)

study indicating that unfamiliarity with synonyms and

lack of vocabulary lead to pragmatic failure.

In addition, pragmatic understanding also relates

to context familiarity. The finding of the present study

is consistent with the statements of some scholars that

familiarity with or knowledge of contextual language

use contributes to foreign language learners’

pragmatic understanding (Eslami-Rasekh, 2005).

Similarly, Crandall & Basturkmen (2004) also found

that the use of language appropriate with context is

one of the efforts to develop pragmatic competence.

Based on the research findings just discussed, the

researchers designed The Rubric of Digital Media

for Pragmatic-based Pre-TOEFL Listening Book

below.

Some of the audios or videos mentioned in the

rubric are online YouTube and TOEFL Listening

Practice Tests. Due to the copy right regulations, the

media are not reproduced, but the learners are

suggested to practice listening on their own via online

YouTube and TOEFL listening practice tests. In

addition, this listening book is written in pdf. format,

so that the audios and videos are not directly

accessible. The presentation of those learning media

in this book is only as a guide. The learners have to

access those media from the internet (YouTube)

online when practice listening. The ‘Rubric of the

Digital Media for Pragmatic-based Pre-TOEFL Listening’

also containing t

he URLs of all audios and videos are

presented.

5 CONCLUSIONS

Based on a multi-year Research and Development, it

shows that there are five primary causes of difficulty

in understanding pragmatic meanings in Pre-TOEFL

listening, namely, Speech Rate Delivery, Voice,

Sentence Complexity, Mishearing and Colloquies. As

a follow-up, it is essential that the draft of the listening

materials and supplementary digital media be

developed. Two types of digital media are

supplemented in the draft: the researcher-made media

and the adopted digital audio and video media. The

content of the media is based on the characteristics of

each cause of the pragmatic difficulty. The research

findings obtained from the research activities

(Research phase) and the Rubric of the Digital Media

for Pragmatic-based Pre-TOEFL Listening are the

basis of the development of the book.

ACKNOWLEDGMENTS

Many thanks go to the DPRM Ristek Dikti the

Republic of Indonesia which sponsored the present

study. The researchers do appreciate those who

participated in the completion of the present study.

REFERENCES

Arifuddin, A. & Sujana, I.M. (2004). Kecakapan berbahasan

Inggris mahasiswa senior prorgram jurusan bahasa

Inggris LPTK di NTB [English proficiency of the senior

students of English teachers in West Nusa Tenggara

Province]. Jurnal Penelitian Universitas Mataram,

Februari 2004.

Arifuddin, A. & Susanto. (2012). Gender-based failure to

infer implicatures from Pre-TOEFL listening.

International Journal of Learning and Development,

2(6), 62-72.

Arifuddin, A. Sujana, I.M., & Kamaludin. (2017).

Indonesian masters degrees students’ difficulties in

pragmatic understanding based on fields of study and

gender. The ASIAN EFL Journal Volume 7, March

2017, 25-52.

Arifuddin. (2013). Inferring implicatures from short

conversations in Pre-TOEFL: Gender-specific and

rankings of causes of failure. Unpublished Doctor

Dissertation. Surabaya: Universitas Negeri Surabaya.

Bardovi-Harlig, K. & Hartford, B.S. (1993). Learning the

rules of academic talk: A longitudinal study of

pragmatic development. Studies in Second Language

Acquisition, 15, 279–304.

Bardovi-Harlig, K., & Dörnyei, Z. (1998). Do language

learners recognize pragmatic violations? Pragmatic vs.

grammatical awareness in instructed L2 learning.

TESOL Quarterly, 32, 233-259.

Digital Media for Pragmatic-based Pre-TOEFL Listening

Black, E. (2006). Pragmatic stylistics. Edinburgh:

Edinburgh University Press.

Bulut, D. (2009). Pragmatic awareness of foreign language

in a gender segregated society. Sosyal Bilimler

Enstitusu Dergisi Says, 26(1), 123-139.

Crandall, E. & Basturkman, H. (2004). Evaluating

pragmatics-focused materials. ELT Journal,

58/1.Oxford University Press.

Eslami-Rasekh, A. (2005). Raising the pragmatic

awareness of language learners. ELT Journal, 5, 199-

208.

ETS Researchers. (2008). Validity evidence supporting the

interpretation and use of TOEFL iBT scores. Princeton,

New Jersey: ETS.

ETS (2012). 2011-2012 Information and registration

bulletin for Paper-based Testing (PBT). Princeton, New

Jersey: ETS.

Goh, C. (1997). Metacognitive awareness and second

language listeners. ELT Journal Volume 51(4), 1997.

361-369.

He, Z. & Yan, Z. (1986). Pragmatic failure by Chinese EFL

learners. Foreign Language Teaching and Research, 3,

52-57.

Kasper, G. & Schmidt, R. (1996). Developmental issues in

interlanguage pragmatics. Studies in Second Language

Acquisition 18, 149-169.

Khamyod, T. & Aksornjarung, P. (2011). A comparative

study of the pragmatic competence of learners with high

and low English proficiency). The 3rd International

Conference on Humanities and Social Sciences April 2,

2011, Faculty of Liberal Arts, Prince of Songkla

University Proceedings-English Studies in Various

Contexts.

Kostin, I. (2004). Exploring items characteristics that are

related to the difficulty of TOEFL dialogues items.

Princeton, NJ: ETS.

Kreutel, K. (2007). I'm not agree with you. ESL learners'

expression of disagreement. TESL-EJ, 11(3), 1-35.

Lee, C. (2010). An exploratory study of the interlanguage

pragmatic comprehension of young learners of English.

Pragmatics, 20(3), 343-373.

Li, R. (2007). The relationship between linguistic

proficiency and pragmatic ability. US-China Foreign

Language, 5(1), 13-17. Retrieved January 2010, from

http://www.linguist.org.cn/doc/uc200701/uc20070104.

pdf.

Liu, S. (2006). Measuring interlanguage pragmatic

knowledge of EFL learners. Frankfurt: Peter Lang.

Liu, Yi-Chun. (2009). The utilization of listening strategies

in the development of listening comprehension among

skilled and less skilled non-native English speakers at

the college level. Unpublished Ph.D. Dissertation.

Texas: Texas A&M University.

Mei-Xia, L. (2005). Pragmatic failure in interculural

communication and English teaching in China.

http:///www.chinamediareserach.net/vol4No3/06Mei-

Xiao%20Lin.final.pdf.)

Nadig, A. (2013). Listening Comprehension. Encyclopedia

of Autism Spectrum Disorders, 1743.

Nemati, A; Qareqani, K., & Fumani, M.R.F.Q. (2016). The

Investigation of listening comprehension problems of

American accents for Iranian B.A. students of

Translation Studies. Online Journal of Humanities,

Issue I, June, 2016.

Saukah, A. (2010). Visiting Lecture at Language and

Literature Education Postgraduate Program Unesa on

20th of November 2010 at AK9 Postgraduate Program

Building Unesa Surabaya. Surabaya: Indonesia.

Sirikhan S., & Prapphal, K. (2011). Assessing pragmatic

ability of Thai hotel management and tourism students

in the context of hotel front office department. Asian

EFL Journal Professional Teaching Articles Vol. 53,

72-94.

Sujana, I.M., Syahrial, E., & Fitriana, E. (2003). Profisiensi

bahasa Inggris mahasiswa S1 bahasa Inggris FKIP

Universitas Mataram dalam Test of English as a Foreign

Language (TOEFL). Jurnal Penelitian Universitas

Mataram, 2(3), 14-26.

Taguchi, N. (2007). Development of speed and accuracy in

pragmatic comprehension in English as a foreign

language. TESOL Quarterly, 41(2), 313-338.

Takahashi, T. & Beebe, L.M. (1987). The development of

pragmatic competence by Japanese learners of English.

JALT Journal, 8, 131-155.

Ur, P. (1989). Teaching listening comprehension.

Cambridge: Cambridge University Press.

Wolfson, N., Marmor T., & Jones, S. (1989). Problems in

the Comparison of Speech Acts Across Cultures. In

Blum-Kuka S, House J and Kasper G (eds.), Cross-

Cultural Pragmatics: Requests and Apologies (pp. 174-

194). Norwood, New Jersey: Ablex Publishing

Corporations.

Yin, R. K. (2011). Qualitative research from start to finish.

New York & London: The Guilford Press.

Yoshimi, D.R. (2001). Explicit instruction and JFL

learners’ use of interactional discourse markers (pp.

223-244.). Cambridge: Cambridge University Press.

ELITE 2019 - English Linguistics, Literature, and Education Conference