Evaluation Instrument Development for Translation Learning
Anam Sutopo and Titis Setyabudi
English Department, Universitas Muhammadiyah Surakarta, Surakarta, Indonesia
Keywords: Accuracy, Readability and Translation Evaluation.
Abstract: The purpose of this study is to develop the evaluation instrument for translation learning process. This study
applies a qualitative research approach. The sources of data are informants and documents. The techniques
for collecting data used in this study are in depth interviewing and content analysis. To check the validity of
data, the writer uses expert judgment. To analyze the collected data, the writer applies interactive analysis.
The result of the study shows that there are two aspects used in developing the evaluation instrument for
translation learning process; namely accuracy and readability. The Evaluation should be made based on the
learning objective. The aspect of accuracy is in line with the similar message between source language and
target language while the readability aspect is reflected whether the target language is easy to read and to
understand or not. The scoring of the two aspects is formulated by four scoring range of 1 to 4 scale.
Finally, it could be concluded that the instrument is effective enough for evaluating the translation works
based on learning objective.
1 INTRODUCTION
Translation activities have a great role in developing
science and technology from abroad. The
development of developing science and technology
depend no more on the natural resources that from
time to time has been reduced, but will be more
influenced by the quality of human resources. There
are many ways to improve the quality of human
resources. One of them is by increasing their ability
in understanding of the recent knowledge and
technology. Meanwhile, it cannot be denied that
most of the available books, relevant with
knowledge and technology are written in English.
Sadtono in Sutopo (2014) says that there are 75 % of
the available books, which are collected by
Indonesian library are written in English but only 5
% of all readers understand them. It means that the
coming of translation is one of the alternative
solutions. In other words, we may say that
translation can be a bridge in transferring the
message from English to Indonesian. By translating
books, readers who do not understand English can
read and get the knowledge they want.
Translation is not an easy task. It needs a serious
attention and concentration. It is due to the fact that
the core of translation is a transfer of ‘meaning’
(message). The translator must be careful in
replacing meaning from source to target language
(Sutopo, 2015). It is also hard forbidden for
translator to gain or loss the messages. The translator
should be able to look for the closest meaning from
source to target language (Pym, 2007). The
translator must also keep the excess and mission in
the source language then transfer them to target one.
If the translator can translate well semantically but
cannot move the closest mission, the result of
translation is useless (Nida, 2001). The translator
spends his time and energy carelessly. Therefore, to
avoid this useless activity and to have a good work,
the translator should master not only linguistics and
materials that will be translated but also the theory
of translation.
There are many translation works that have been
translated into Indonesian, like novels, books, and
legal documents. These works are sometimes not
only done by professional translator but also by the
amateur ones. Translating many kinds of works have
different strategies. It is not easy to do translation
works; therefore, it needs an instrument for doing
evaluation. Even, it may happen on the teaching
leaning process of translation courses.
At least, there are four reasons why this
evaluation instrument is important to do. Firstly,
there many translation works read by people which
have not evaluated yet. Secondly, the accuracy of
290
Sutopo, A. and Setyabudi, T.
Evaluation Instrument Development for Translation Learning.
In Proceedings of the 1st International Conference on Educational Sciences (ICES 2017) - Volume 1, pages 290-295
ISBN: 978-989-758-314-8
Copyright © 2018 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved

the translation result must be check. Sometimes, the
clients never know whether the result of translation
is accurate or not. Thirdly, evaluation is done also to
know the readability of the translation result on the
teaching learning process. Usually the translation is
not easy to read and to understand. Fourthly, there is
no evaluation instrument on translation at present
time. Therefore, it is needed to develop the
translation evaluation instrument.
Nababan et al. (2010) mentions that translation
is made possible by an equivalence of thought that
lies behind its different verbal expressions. The
other linguist, Nida (2001), explains that translation
consists of reproducing in the receptor language the
closest natural equivalent of the source language
message, first in terms of meaning and secondly in
terms of style. It means that translation is the
process of transferring meaning from source
language into target one, for example, from English
into Indonesian. The translator must be careful in
transferring the meaning (Hui-Juan, 2007). It is due
to the fact that meaning is very important in
translation activity. If the translator cannot get the
right meaning from the source language, the result of
the translation will be dangerous.
Process of translation cannot be away from
discussing meaning. Newmark (2008) says that
meaning will reflect the message in the text. There
are many linguists who have different perceptions
about translation process. The followings are some
opinions about translation process. Sutopo (2014)
says that process of translation is some steps that
should be done by the translator before beginning
his work on translation field. Those steps are (1)
analyzing, (2) transferring, and (3) restructuring. It
means that before translating a certain text, the
translator follows some steps. The other linguist,
Nababan et al. (2010) defines that process of
translation is as the series activities which done for
transferring the message from the source language to
the target language. Nababan et al. (2010). For this,
he also has the same opinion as Soemarno’s. They
are analyzing, transferring, and restructuring. Larson
(1999) says that the focus of translation process is on
‘meaning’. It means that he processes of translation
is some phases that must be followed by the
translator in doing his task. The phases consist of
analyzing, transferring, reconstructing and
restructuring. The task here, of course, is translating
a certain text or book from source language to target
one.
Catford (2005) divides translation into three
distinctive types, namely: (1) full translation v.s
partial translation, (2) total v.s restricted translation,
and (3) rank translation. The distinction between full
and partial translation relates to the extent of source
language text, which is submitted to the translation
process. In a full translation the entire text is
submitted to the translation process, that is, every
part of the source language text is replaced by target
language text material, for examples;
Indonesian: Aku cinta Indonesia
Dia akan pergi ke sana
Mereka tidak membeli apa-apa
English: I love Indonesia
He will go there
They didn’t buy anything.
In a partial translation, however, some parts of
the source language text are left untranslated. They
are simply translated to and incorporated in the
target language text, either because they are
regarded as ‘untranslate’ or for the deliberate
purpose of introducing “local color” into the target
language text or even because they are so common
and frequently used that translation is not needed
(Weissbrod, 2009).
The distinction between total and restricted
translation relates to the levels of language involved
in translation. In total translation, Miyanda (2007)
argues that source language grammar and lexis are
replaced by equivalent of the target language
grammar and lexis. The third type of differentiation
in translation is related to the rank in grammatical
hierarchy at which translation equivalent is
established. This rank translation can be in the form
of: word for word, group for group, sentence for
sentence etc. In this type of translation, lexical and
grammatical adjustment should be applied to
achieve equivalent in terms of meaning. It means
that mainly there are three types of translation. They
are (1) word for word translation, (2) literary
translation and (3) free translation (Ordudary, 2007).
Translator never uses only one of them. Usually he
uses all of them depending on the text that will be
translated.
It is not a new issue to discuss the existence of
translation. Is translation as a science, a skill or an
art? This problem has been existed a long time ago.
For example, Nababan et al. (2010) says that
translation is an art. Perhaps, this statement is
influenced by his specialization in translating
literary books. Besides, his competence in
translating literary books is skilfully. The similar
argument stated by Meschonnic (2008). He says that
translating poetry is an art and poetry is a product of
art. He also explains that the problem of meaning is
Evaluation Instrument Development for Translation Learning
291

a part of the core in translation studies and semantics
is a part of semiotic, which needs linguistic analyses.
Meanwhile, Sang and Zhang (2008) says that
designing an effective translation learning program
involves needs’ analysis, translation syllabus,
learning methods, the role of teachers and learners,
and evaluation. Needs analysis is a discrepancy
between the actual phenomenon and desired
conditions. Needs analysis is beneficial to identify
qualification of learners’ groups expected to learn
the target language. Learners’ groups were, then,
placed at the appropriate level of competence during
the learning process. Such procedure requires: (1)
interviewing with translation learners to gain
perceptions about the main difficulties in the
language being studied, (2) interviewing with
another teacher, and (3) observing about the
learners’ skills level in understanding the material
and their learning difficulties. It is also used to base
curriculum and/or syllabus designing that can be
used to identify: learners’ learning needs,
availability requirements of the program, learners’
learning outcomes and weaknesses, information on
the needs of progress, and determine the existing
deficiencies in achieving the learning’s goals.
Syllabus is an outline of topics to be covered in a
single course or graded. It is also defined as a
specification of the work to be covered over a period
of time with a starting point and a final goal, the
specific plan of tasks for one period of time that
begins and ends appropriately to the purpose
Syllabus is specified into content-based, structural
and functional, situational and topic-based. Content-
based syllabus (ideally used at secondary schools)
combines all of the above factors within the learning
materials that refer to the users’ needs. Syllabus
content must be designed by the teachers and
actually does not need the help of others but in fact,
syllabus is often made by experts or syllabus
developers because of their inability to design it. It is
explicitly related to language courses will be taught
(not about the program obtained by the learners) and
should include a series of the expected outcomes.
2 RESEARCH METHODS
This study applied a qualitative research approach.
The data in this study were all information given by
informants and all statements and utterances taken
from documents. This study was done in Universitas
Muhammadiyah Surakarta. The sources of data were
informants and documents. The collected data were
taken from an open-questionnaire, in depth
interviewing, and content analysis. In this study, the
writer had 7 informants. The informant consisted of
3 translation experts and 4 English lecturers. To
check the validity of data, the writer used expert’s
judgment, data and method triangulations. To
analyze the collected data, the writer used interactive
analysis.
3 RESULTS AND DISCUSSION
Sutopo (2015) says that the quality of translation
belongs to good translation if 1) the translated text is
accurate, seen from the message. It means that the
meaning or message found in the target language
should be similar as the meaning or message in the
source one, 2) The translated text uses the language
norms of the target language, 3) the translator
considers the cultural understanding, and 4) the
translated text can be read and understood by the
readers easily.
Based on the data found from documents
informant AI, TS, and DC through in-dept.
interviewing in this study, there are two instruments
of the assessments for literary translation learning,
they are accuracy and readability. Besides, the
instruments should be made based on the learning
objective.
The learning objective of translation influences
the assessment of literary translation learning
because before starting the works, the teacher
considers it seriously (Perez, 2005). It will have
different consideration from the activity of
translation both based on the academic or business
purposes. In this case, the teacher becomes the
translator. He may have different focus in the two
different considerations above. It means that the
teacher influences the process of assessing literary
translation in the class. Assessment is measuring the
quality of translation although it is as the process of
teaching learning of translation. It needs relevant
between the translated materials to be evaluated.
The first instrument is in line with accuracy.
Assessing on the accuracy means checking whether
or not the message between source language and
target one is similar. Transferring the message is not
an essay duty. Principally, the translator cannot
deny, add, omit the message freely because it can
destroy the meaning. It also causes the message
found in the source language is different from the
target one. Therefore, the accuracy becomes the
main point in translation and it must be reflected in
the instrument of evaluation on literary translation.
See the following example:
ICES 2017 - 1st International Conference on Educational Sciences
292

SL: Bila seorang gadis berkenan di hati seorang
pemuda, maka ia memberitahu orang-tuanya untuk
melamar pujaan hatinya itu. Orang tua si jejaka
kemudian mengadakan lamaran kepada orang-tua si
gadis. Upacara ini disebut mepadik. …
TL1: When a young girl falls in love with a young
man, then she informs her parents about the
marriage proposal to the idol of her heart. This
ceremony is called mepadik …
TL2: When a youth has his heart set on a girl, he
then informs his parents to express the intentions of
his heart. The bachelor’s parents then deliver a
proposal to the girl’s parents. This ceremony is
called mepadik.
This data is belonging to accurate translation. It
can be seen that from the two translated result the
accuracy become the focus. Translation is the
rendering of a source language text into the target
language so as to ensure the surface meaning of the
two will be approximately similar and the structure
of the source language will be preserved as closely
as possible to the target. The meaning from the data
above is similar between meanings found in SL and
TL. Machali (2009) argues that translation focuses
on the attempt to replace a written message in one
language by the same message in different language
accurately.
The instrument is developed into the following
table:
Table 1: Accuracy instrument of evaluation.
Score
Meaning
4
The message found in the source language
has been transferred into the target language
accurately. The sentences used in the
translated material can be read and
understood easily.
3
The message found in the source language
has been transferred into the target language
accurately. The sentences used in the
translated material cannot be read and
understood easily so they need to be
rearranged again.
2
The message found in the source language
cannot be transferred into the target language
accurately. The sentences used in the
translated material can be read and
understood easily. The sentences used in the
translated material cannot be understood.
There are some errors in the sentences so they
need to be revised.
1
The message found in the source language is
quietly different from the target language.
Based on this accurate translation, it is obvious
that the translation belongs to quite accurate if the
message found in the source language has been
transferred into the target language accurately. The
sentences used in the translated material can be read
and understood easily. Then, it is stated as accurate
if the translation result can be stated as less accurate
when the message found in the source language has
been transferred into the target language accurately.
The sentences used in the translated material cannot
be read and understood easily so they need to be
rearranged again. the message found in the source
language cannot be transferred into the target
language accurately. The sentences used in the
translated material can be read and understood
easily. The sentences used in the translated material
cannot be understood. There are some errors in the
sentences so they need to be revised. Then, it is
categorized into not accurate if the message found in
the source language is quietly different from the
target language
The second instrument is dealing with is
readability. Readability is line with the result of
translation. It shows that the result of translation is
difficult or easy to read. If the translated text is easy
to read, it means it has high level of readability.
Meanwhile, if the result of the translation is not easy
to read, it has low level of readability. The result of
the translation is readable. It will influence the
readers. So, the readability must be obtained from
the targeted readers. See, the following example, is it
readable?
SL: Ia tidak baik memiliki maupun memerlukan
sebuah kamus.
TL: He neither had nor cared for a dictionary.
This translation is belonging to readable
translation. The target language is easy to read and
to understand. The result of translation “He neither
had nor cared for a dictionary” is easy for the
informant to read and to understand. All informants
have the same argument when they were
interviewed. The instrument is developed into the
following table:
Evaluation Instrument Development for Translation Learning
293

Table 2: Readability instrument of evaluation.
Score
Meaning
4
The translated sentences are very easy to
understand. The translated sentences have
been relevant with the target language rules.
The choice of words, phrases, clauses and
sentences are very easy to read by the
readers.
3
The translated sentences are very easy to
understand. The translated sentences have
been relevant with the target language rules
but the choice of words, phrases, clauses and
sentences are not easy to read by the readers.
2
The translated sentences are not easy to
understand. The translated sentences have
been relevant with the target language rules
but the choice of words, phrases, clauses and
sentences are not easy to read by the readers.
1
The translated sentences are difficult to
understand. The translated sentences are not
relevant with the target language rules but the
choice of words, phrases, clauses and
sentences are difficult to read by the readers.
Based on this readability instrument, it is clear
that the translation can be stated as quite readable
when the translated sentences are very easy to
understand. The translated sentences have been
relevant with the target language rules. The choice
of words, phrases, clauses and sentences are very
easy to read by the readers. The translation result
belongs to readable if the translated sentences are
very easy to understand. The translated sentences
have been relevant with the target language rules but
the choice of words, phrases, clauses and sentences
are not easy to read by the readers. Then the
translation is said less readable when the translated
sentences are not easy to understand. The translated
sentences have been relevant with the target
language rules but the choice of words, phrases,
clauses and sentences are not easy to read by the
readers. Finally, it can be stated not readable when
the translated sentences are difficult to understand.
The translated sentences are not relevant with the
target language rules but the choice of words,
phrases, clauses and sentences are difficult to read
by the readers.
4 CONCLUSIONS
From the discussion above it can be concluded that
the role of lecturer in the assessing on translation is
very important. The lecturer has authority and
creativity in giving the assessment on translation
subject. The assessment done by the lecturer must be
based on the learning objective. There are two
aspects used in developing the evaluation instrument
for translation learning process; namely accuracy
and readability aspects. The evaluation should be
built based on the learning objective. The aspect of
accuracy is made based on whether there is similar
message between source language and target
language or not while the readability aspect could be
reflected whether the target language is easy to read
and to understand or not. The scoring of the two
aspects is formulated in the form of four criteria with
the score range of 1 to 4 scale. They cannot be
separated one from another, although accuracy is the
dominant one.
REFERENCES
Catford, J. C., 2005. A Linguistic Theory on Translation,
Shanghai Foreign Language Education Press.
Shanghai.
Hui-juan, M., 2007. Exploring the differences between Jin
Di’s translation theory and Eugene A. Nida’s
translation theory. The Journal of Babel 53: 2 (2007),
98–111. issn 0521–9744 / e-issn 1569–9668
Larson, M. L., 1999. Penerjemahan Berdasar Makna,
Arcan. Jakarta.
Meschonnic, H. 2008. The Europe of translation. Journal
of Translation Studies. Vol. 1, No. 1, 2008, 34_40.
ISSN 1478-1700 print/ISSN 1751-2921.
Machali, R., 2009. Pedoman bagi Penerjemah, Penerbit
Kaifa. Jakarta.
Miyanda, F. 2007. Total Meaning and Equivalence in
Translation. NAWA Journal of Language and
Communication. University of Bostwana.
Nababan, N., Sumardiono, 2010. Pengembangan Model
Penilaian Kualitas Terjemahan, Universitas Sebelas
Maret. Surakarta.
Newmark, P., 2008. Approaches to Translation, Shanghai
Foreign Language Education Press. Shanghai.
Nida, E. A., 2001. Language and Culture: Contexts in
Translating, Shanghai Foreign Language Education
Press. Shanghai.
Ordudary, M., 2007. Translation Procedures, Strategy and
Methods. Translation Journal. Volume 11, No 3, July
2007.
Perez, M. C., 2005. Applying Translation Theory in
Teaching. Journal of New Voices in Translation
Studies 1. page 1-11. Universitat Jaume I, Castellón de
la Plana, Spain.
Pym, A. 2007. Natural and directional equivalence in
theories of Translation. Target, International Journal
of Translation Studies. Volume 19 Issue 2 (2007),
271–294. issn 0924–1884 / e-issn 1569–9986 © John
Benjamins Publishing Company.
Sang, J., Zhang, G., 2008. Communication across
languages and cultures: A perspective of brand name
translation from English to Chinese. Journal of Asian
ICES 2017 - 1st International Conference on Educational Sciences
294

Pacific Communication. 18:2 (2008), 225–246. doi
10.1075/japc.18.2.07san ISSN 0957–6851 / E-ISSN
1569–9838.
Sutopo, A., 2014. Terjemahan dan Penerjemahan dalam
Lintas Pemahaman, C.V Jasmine. Solo.
Sutopo, A., 2015. Penerjemahan Naskah Resmi: Telaah
Holistik Pidato Kenegaraan, MUP Press UMS. Solo.
Weissbrod, R., 2009. Philosophy of Translation Meets
Translation Studies: Three Hebrew Translations of
Kipling’s “If” in Light of Paul Ricoeur’s “Third Text”
and Gideon Toury’s “Adequate Translation”. The
Journal of Target, 21:1 (2009), page: 58–73. doi
10.1075/target.21.1.03wei ISSN 0924–1884 / E-ISSN
1569–9986.
Evaluation Instrument Development for Translation Learning
295