THAI VISUALLY IMPAIRED ’S REQUIREMENTS TO ACCESS

MATHEMATICS VIA AN AUTOMATIC MATH READER

Wararat Wongkia

Institute for Innovative Learning, Mahidol University, Bangkok, Thailand

Kanlaya Naruedonkul

Department of Mathematics, Faculty of Science, Mahidol University, Bangkok, Thailand

Nick Cercone

Department of Computer Science and Engineering, Faculty of Science and Engineering, York University, Toronto, Canada

Keywords: Math Expression, Text-to-Speech, Blind and Visually Impaired Learning.

Abstract: We present the requirements to access mathematics for the Thai visually impaired students. Blind and

visually impaired (VI) students agreed to participate in an interview to determine the requirements for using

a text–to–speech (TTS) system with the capability to automatically read math expressions. The interview

consisted of two parts: the students’ background characteristics and the students’ difficulties in and

requirement for using TTS systems to read math expressions. Our results show that the students who are

blind and visually impaired had difficulties to access mathematics. The students have basic computer skills

with standard software e.g., Microsoft Word. Therefore, the findings suggest that the students desire

assistant technology to better access to mathematics.

1 INTRODUCTION

Presently, blind and visually impaired (VI) students

can access math documents with the help of a human

reader. Unfortunately, having a human reader at

their side at all times is impractical. Braille display

is a more convenient way for blind and VI students

to access such documents. However, not many math

documents are available in Braille because of the

difficulty of production. Moreover, reading

mathematics in Braille requires special knowledge.

Video technology is another choice. Nonetheless,

blind and VI students can have difficulty obtain the

main point of the story even if they listen for a very

long continuous period. Audio devices e.g., Daisy

books and talking books are an alternative material

to provide for the blind and VI students; however

The author wishes to acknowledge the support of the Project for

Promotion of Science and Mathematics Talented Teacher

(PMST).

these books are less prepared for math and scientific

expressions. Screen readers with text–to–speech

(TTS) technology can read the documents aloud.

Then again, most TTS engines including Thai TTS

can read only plain text from a screen. They cannot

read math and scientific electronic books (e–books).

All existing technologies are inadequate for assisting

VI students to access math documents.

Although many math word problems are

available either in printed materials or electronic

forms, blind and VI students derive limited benefit

from them. Blind and VI students lose an

opportunity to develop their math skills and to

practice on their own. Limitations in writing and

reading formulas restrict blind and VI students for

an efficient and complete study of mathematics.

Therefore, we are looking for a solution to help

blind and VI students with as fair a chance to

studying as with the sighted students. Compared to

other forms of assistance, TTS systems with the

capability to automatically read math expressions

would serve as an educational tool for blind and VI

239

Wongkia W., Naruedomkul K. and Cercone N. (2010).

THAI VISUALLY IMPAIRED ’S REQUIREMENTS TO ACCESS MATHEMATICS VIA AN AUTOMATIC MATH READER.

In Proceedings of the 2nd International Conference on Computer Supported Education, pages 239-244

DOI: 10.5220/0002781002390244

 SciTePress

students to access math documents written in Thai

their convenience.

We determine the blind and VI students’

background, difficulties and requirements to access

mathematics by the verbal interview. With using

such interviewed information from the blind and VI

students, we design an automatic math expressions

reader system, called MathReader, for blind and VI

students to access math documents in Thai.

MathReader is designed to incorporate reading Thai

text and math expressions as well.

2 EDUCATIONAL

TECHNOLOGIES FOR

VISUALLY IMPAIRED

STUDENTS

Current technologies allow blind and VI students to

access to printed text via various means e.g., Braille

documents (e.g., Moço and Archambault, 2004;

Niyomphol, Tandayya, Nantachaipitak, and Intasoi,

2008), digital talking books (DAISY Consortium,

2008) and TTS systems. Moreover, Freitas and

Kouroupetroglou (2008) reviewed speech technolo-

gies for blind and VI students to access the printed,

written and visual information. With these devices,

blind and VI students can maintain their indepen-

dence while they are studying and practicing.

A text–to–speech (TTS) engine is computer

software that can read aloud text within electronic

text formats e.g., Internet Explorer, Microsoft Word,

and Adobe Reader. Often this engine works with the

screen reader software to have a more comfortable

employment for blind and VI people. TTS system is

widely used by blind and VI people to read

electronic text through a computer since the sound–

based representation is an important channel to

access information for blind and VI people.

In Thailand, groups of researchers have

researched and developed Thai TTS systems since

the 1980s. Until recently, most Thai TTS systems

generate Thai speech from Thai text with high

accuracy of pronunciation and high quality of

natural sound (Wutiwiwatchai and Furui, 2007).

However, the important problems in developing

Thai TTS system are the ambiguity of segmentation

and pronunciation, and unnatural sound of conti-

nuous synthetic speech.

At present, to enhance the access to educational

math materials, a few TTS based systems were

developed to read math expressions in many

languages, e.g., A

R, MathTalk, and MathPlayer

in English (Raman and Gries, 1995; Stevens and

Edwards, 1994; Soiffer, 2005); AudioMath in Euro-

pean Portuguese (Fereira and Freitas, 2005); and

Math Genie in English, French, and German

(Karshmer, Bledsoe and Stanley, 2004; Gillan,

Barraza, Karshmer, and Pazuchanics, 2004). In

addition, some systems presented output in both

audio format and Braille e.g., TRIANGLE, REMath-

Ex and LAMBDA (Gardner, Lundquist and Sahyun,

1998; Pavel, 2002; Schweikhardt, Bernareggi,

Jessel, Encelle, and Gut, 2006). Also, the mutimoda-

lity accessibility of math expressions was proposed

to support individual preferences. This mutimodal

provided output in audio, tactile and visual forms

(e.g., Tsonos, Kaccori, and Kouroupetroglou, 2009).

The TTS system with the capability to read math

expressions has continuously been developed and

researched to assist blind and VI people in accessing

math documents the same as for sighted people.

With the potential of TTS system, blind and VI stu-

dents have valuable tools to help them learn science

and mathematics. However, none of Thai TTS

systems has the capability to read math expressions.

3 METHODOLOGY

The requirements for using a TTS system with the

capability to automatically read math expressions

were determined. The analyzed results will con-

tribute in designing TTS system with the capability

to automatically read math expressions in Thai.

3.1 Participants

The participants are students in secondary schools

(Grade 7–9) for the blind and VI students. The study

is developed with 20 blind and VI students ranged in

age from 14 to 20 (mean = 16.8, sd = 1.6). The

participants consisted of ten blind students (seven

male and three female) and ten visually impaired

students (five males and females) from two schools

for blind and VI students in Thailand. All of them

agreed to participate in the verbal interview.

3.2 Interview

The Thai semi–structured interview was prepared

for surveying the blind and VI students’ difficulties

and requirements of TTS with the capability to read

math expressions. The interview consisted of two

parts (see Appendix). Part 1 requested the students’

background characteristics: personal information

and their ability to use the computer including their

CSEDU 2010 - 2nd International Conference on Computer Supported Education

240

experiences with computer software and aspects of

using the Internet. Part 2 covered the students’

difficulties in using existing TTS systems to read

math expressions, and the students’ requirements for

using TTS systems with the capability to read math

expressions in their study of mathematics.

3.3 Procedure

The participants were interviewed in the schools

from February to March 2009. Each student’s voices

were recorded during the interview for 10–15

minutes.

4 RESULTS

4.1 Students’ Experiences of using a

Computer

Most blind and VI students had extensive experience

using the computer, while two students who are

blind had slight experience and joined their partners

to access the computer. All students had used Micro-

soft Windows. All blind students and most VI

students also used a screen reader with a Thai speech

synthesizer to work with the computer. Some VI

students mentioned using other devices and assistive

tools such as a magnifying glass, Braille display, and

an enlarged visual display to work with a computer.

The majority of the blind and VI students had

used the Internet for more than a year. Repeatedly,

two students who joined their partners to access the

computer had accessed the Internet with their

partners. Most students used the Internet for

retrieving information and entertaining themselves.

Some students had used the Internet to obtain

knowledge; exchange ideas, news and information;

and communicate with other people.

For software, all blind and VI students were

given a list of software that could be used on a

computer. All students were quite familiar with

Microsoft Word. The students also had experience

using other software as shown in Figure 1. None of

the students who participated had used software for

writing or editing math expressions e.g., Microsoft

Word Equation Editor or Math Type Editor. Most

students had used PPA TATIP (Thai and English

TTS system works together with Microsoft Speech

and Speech Synthesizer and available for Thai blind

and VI people). None of the students used VAJA

(another Thai TTS system developed by NECTEC).

Some sound–player software e.g., Winam and Win-

dow Media Player, were useful for providing infor-

mation. Furthermore, a few of the students stated

that had used TAB Player (free software player

supports DAISY books for Thai and English).

Exc

wer

oint

Equ ation Edito r

PDF

ina

Media Playe r

ers

Number o f s tu d en t

Figure 1: Students’ experience of using computer

software.

4.2 Students’ Difficulties in using TTS

to Read Math Expressions

All blind and VI students stated that they had used a

Thai TTS system when they used the computer.

Some students had used a TTS system to read

numbers, simple math operations e.g., ‘+’ plus sign,

‘–’ minus sign and ‘%’ percent sign. Some students

also reported that the TTS system that they used read

simple math notations with an English accent. It was

difficult for the students to understand such

information. Only a few VI students noted that the

synthesizer did not work with simple math

expressions e.g., simple fractions. Most students had

never used a TTS system with the capability to read

math expressions. Only one student heard synthetic

sound of math expressions in Spanish.

Thirteen students attended inclusive schools (a

school that encourages disabled students to transi-

tion into society). The students had problems with

reading printed materials or documents from their

classroom because those documents were written in

a normal character system. Most students said that

they needed an assistant at their side for practicing

or doing homework, especially mathematics. Fur-

thermore, a few students preferred to read such

documents in Braille. However, Braille printing

devices and Braille reading displays are not provided

and available to individually blind and VI students.

The students stated that the production of such

documents in Braille took a long time because the

documents were sent to another organization and

then those documents were sent back to the students.

They also noted that, recently, the translation from

math notations into Thai Braille was practically

incomplete. Moreover, it was more difficult for the

students to read the math Braille system because

unusual coding was used to represent numbers and

special characters. The students also stated that most

THAI VISUALLY IMPAIRED 'S REQUIREMENTS TO ACCESS MATHEMATICS VIA AN AUTOMATIC MATH

READER

241

math concepts composed of graphs, tables, figures

and notations; so it was difficult for the blind and VI

to study mathematics.

Similar to the students in the inclusive schools,

the other seven students who attended a mainstream

school (a school in which all students have special

needs) faced problems reading the math Braille

system as well. All students usually read documents

and books, took short notes, and performed

exercises, homework and exams in Braille. The

students were required to remember unusual coding

for performing mathematics.

4.3 Students’ Needs for using TTS with

the Capability to Read Math

Expressions in Thai

The blind and VI students were asked to comment

on needs for using a TTS system with the capability

to read math expressions in Thai. Many students

stated that if math documents could be read aloud in

Thai, then they could do mathematics better than

before. The students also said that they could

practice exercises, do homework and perform exams

by themselves anywhere and any time that they

wanted. The students’ comments included: “It is

easy and convenient for me to do math by myself”

and “It is quick to prepare and study by themselves

before classroom”. Some VI students reported that

they had problems because their eyes were

exhausted when reading the documents for a very

long continuous period. The VI students also said

that it was more comfortable if they had computer

software read the documents aloud. Furthermore, a

few students stated that they had a problem using the

computer with documents. Such documents that the

students had obtained from the classroom as in

printed documents were not electronic files.

Most students said that they desired to use a TTS

system with the capability to read math expressions

in Thai. They wanted to use this TTS system help

them study mathematics if it was available. They

also commented that this TTS system could possibly

to enhance their mathematics learning. However,

one student was not interested in this TTS system

because he could not catch the main idea when he

listened for a very long time. He preferred to read

math documents in Braille.

The students were asked for design suggestions.

They recommended that when developing such a

system should produce correct pronunciations and

more natural speech. The system should read math

expressions aloud in all levels of mathematics

including scientific notations. The volume could be

adjustable. The students requested that this system

could be used to solve math problems, guide

methods that can be used to solve the problems. In

addition, the system should be available for free,

could work alone, and did not need other software to

work well.

5 THAI TEXT AND MATH

EXPRESSIONS

Math expressions differ from Thai plain text in

alphabet, writing and reading listed as follows

(Wongkia, Naruedomkul and Cercone, 2009):

 Alphabet and Symbols: The Thai alphabet

system includes 44 consonants, 15 vowels and

four tone markers. In math expressions, English

letters in upper case and lower case, numbers,

Greek letters and math notations are used.

 Displays: Thai text and math expressions are

displayed non-linearly. Thai text is represented

in a 4–level system. Each term is written in

linear form from left to right without spaces

between. A math expression is a multi–level

system. In general, subscripts and superscripts

can be writ-ten to an unlimited–level or non–

linear. Special math symbols including

horizontal line, summation, integral and root can

be written in combination of special symbols

which is complicated to read aloud.

 Locations and Orders: a different location and

different order of appearances of characters

maps onto different audios. The expression “

”

in Figure 2 can be read “

เอ็กซ์ ยกก

ํ

าลัง สาม”. The

word “

ยกก

ํ

าลัง /yok3-kam0-lang0/ {the power

of}

” is required to complete the reading.

xxxx

Figure 2: Different locations of “3” and “x”

 Mapping characters into their sound: a single

character in a math notation can be mapped onto

one or more sounds, e.g., “π” is mapped onto one

sound /phai0/ while the notation “<” is mapped

onto two sounds /noi3-kwa1/. In Thai text, one

sound corresponds to two or more alphabet sym-

bols, e.g., a three–alphabet symbol is mapped

onto one sound /mak2/ while “

รถ” a two–alpha-

bet symbol is mapped onto one sound /rot3/.

For each word, its pronunciation is shown in / / and its meaning is

shown in { }. The numbers 0-4 are represented five tones. Each

syllable is separated be hyphen e.g., “ปราการ” {wall} /pra0-kan0/.

CSEDU 2010 - 2nd International Conference on Computer Supported Education

242

 Homograph: the Thai word “

สระ” can be pro-

nounced /sa1-ra1/ {vowel} or /sa1/ {pool}.

There is a kind of homograph in mathematics as

well. For example, “1 – 12” means either “one to

twelve” /nueng1 teng4 sip1-song4/ or “one

minus twelve” /nueng1 lop3 sip1-song4/ depend-

ed on where it appears.

 Adding words: A word is added to indicate the

expression boundary. For example,

)( ba + is

read as /e0 buak1 bi0 tang3–mod1 yok3-kam0-

lang0-song4/, the word “/tang3–mod1/{all}” is

added to indicate that the 2nd power is for a + b.

6 PRELIMINARY DESIGN OF

MATHREADER

We present the design of MathReader–a TTS system

with the capability to generate an accurate Thai

speech output from Thai math problems with math

expressions. The students’ responses and the

differences between Thai text and math expressions

were carefully considered in designing the system

architecture.

Figure 3: MathReader architecture.

MathReader was designed to perform speech

generation in four modules: Phrase Identification,

Thai Text Reader, MathEx Reader, and Math Reader

shown in Figure 3. An overview of how MathReader

system performs was presented in Wongkia et al.

(2009). The Phrase Identification module segments

the input string into Thai text and math expression

parts since reading text is different from reading

math expressions in their nature and identifies which

phrase is Thai text or a math expression. The Thai

Text Reader module generates the speech for the text

part while the MathEx Reader module generates the

speech for the math expression part. The Math

Reader module combines all separated parts to form

the corresponding speech of the original input.

MathReader generates syllable sequences for

math word problems containing both plain text and

math equations. MathReader should prove to be a

practical educational tool for the blind and VI

students to access math documents written in Thai

their convenience.

7 CONCLUSIONS

Although Thai TTS systems are available for all

blind and VI students, these systems are unable to

generate speech from math expressions. The results

of this survey suggest that the students who are blind

and VI had difficulties to access mathematics. When

asked about an ability to use a computer that would

provide them with better access to mathematics,

most students admitted having a basic skill to use a

computer with standard software. Their responses

suggest that the design of the MathReader system

should work with that software.

Therefore, the finding from this survey confirms

that the blind and VI students desire assistant tech-

nology. It is important to enables VI students to have

the same opportunities as sighted students in study-

ing, especially in mathematics and science.

REFERENCES

DAISY Consortium (2008). Technology Overview: What

is a DTB? Retrieved December, 2008, from http://

www.daisy.org

Fereira, H. & Freitas, D. (July, 2005). AudioMath:

Towards automatic reading of mathematical

expressions. Proceedings of the 11th International

Conference on Human-Computer Interaction, USA.

Freitas, D. & Kouroupetroglou, G. (2008). Speech

technologies for blind and low vision persons.

Technology and Disability, 20, 135-156.

Gardner, J. A., Lundquist, R. & Sahyun, S. (1998).

TRIANGLE: A tri-modal access program for reading,

writing, and doing math. Proceedings of the 1998

CSUN International Conference on Technology and

Persons with Disabilities, USA.

Gillan, D. J., Barraza, P., Karshmer, A. I. & Pazuchanics,

S. (July, 2004). Cognitive analysis of equation

reading: application to the development of the Math

Genie. Proceedings of the 9th International

THAI VISUALLY IMPAIRED 'S REQUIREMENTS TO ACCESS MATHEMATICS VIA AN AUTOMATIC MATH

READER

243

Conference on Computers Helping People with

Special Needs, LNCS 3118, France, 630-637.

Karshmer, A. I., Bledsoe, C. & Stanley, P. B. (July, 2004).

The architecture of a comprehensive equation browser

for print impaired. Proceedings of the 9th Internation-

al Conference on Computers Helping People with

Special Needs, LNCS 3118, France, 614-619.

Moço V. & Archambault D. (July, 2004). Automatic

conversions of mathematical Braille: A survey of main

difficulties in different languages. Proceedings of the

9th International Conference on Computers Helping

People with Special Needs, LNCS 3118, France, 638-

643.

Niyomphol, N., Tandayya, P., Nantachaipitak, W. &

Intasoi, C. (May, 2008). A simple text-based method

of producing math/science symbols or musical notes

for use in a Thai/English Braille translation software.

Proceedings of the 2nd International Convention on

Rehabilitation Engineering & Assistive Technology,

Thailand, 0915-0930.

Pavel, G. (July, 2002). REMathEx: Reader and editor of

the mathematical expressions for blind students.

Proceedings of the 8th International Conference on

Computers Helping People with Special Needs, LNCS

2398, Austria, 486-493.

Petkleang, S. & Tandayya, P. (March, 2005). Thai Braille

translation on Nfbtrans [in Thai]. Proceedings of

NSTDA Annual Conference “S&T in Thailand:

Towards the Molecular Economy”, Thailand.

Raman, T. V. & Gries, D. (1995). Audio formatting–

making spoken text and math comprehensible.

International Journal of Speech Technology, 1, 21–31.

Schweikhardt, W., Bernareggi, C., Jessel, N., Encelle, B.

& Gut, M. (July, 2006). LAMBDA: A european

system to access mathematics with Braille and audio

synthesis. Proceedings of the 10th International

Conference Computers Helping People with Special

Needs, LNCS 4061, Austria, 1223–1230.

Soiffer, N. (October, 2005). MathPlayer: Web-based Math

Accessibility. Proceedings of the 7th International

ACM SIGACCESS Conference on Computer and

Accessibility, USA.

Stevens, R. D. & Edwards, A. D. N. (1994). MathTalk:

Usable access to mathematics. Information

Technology and Disabilities, 1(4). Retrieved June,

2008, from http://people.rit.edu/easi/itd/itdv01n4/

article5.htm

Tsonos, D., Kaccori, H. & Kouroupetroglou, G. (2009). A

design-for-all approach towards multimodal

accessibility of mathematics. P.L. Emiliani et al.

(Eds.) Assistive Technology from Adapted Equipment

to Inclusive Environments, Assistive Technology

Research Series, 25, 393-397, IOS Press, Amsterdam.

Wongkia, W., Naruedomkul, K. & Cercone, N. (Septem-

ber, 2009). Better access to mathematics for visually

impaired. Proceedings of the 2009 IEEE Toronto

International Conference-Science and Technology for

Humanity, Canada, 43-48.

Wutiwiwatchai, C. & Furui, S. (2007). Thai speech pro-

cessing technology: A review. Speech Communica-

tion, 49, 8-27.

APPENDIX

This interview is used to determine the requirements for

using a TTS system with the capability to automatically

read math expressions for blind and visually impaired

students

Part 1 students’ background characteristics:

1. Personal information (sex, age, educational level and

visual impairment)

2. Do you have a personal computer to use at home?

3. What platform do you use?

4. What computer software and supporting devices do

you use with the computer?

5. What programs can you use? (Word, PowerPoint,

Excel, Equation, PDF, IE, PPA TATIP, VAJA,

Winam, Media Player, and Others)

6. Do you have experience using the Internet? If you

have

6.1 How long have you used the Internet?

6.2 For what purposes do you use the Internet?

Part 2 students’ opinions toward TTS systems with the

capability to read math notations:

7. Do you ever use a screen reader or text-to-speech

(TTS) system? If yes, what software do you use?

8. Do you ever use a TTS system with the capability to

automatically read math expressions? If yes, what

software do you use?

9. What languages of that system do you ever use to read

math expressions?

10. What do you think of the contributions of the system

that read math expressions in Thai?

11. What do you think of the contributions of such system

for you?

12. In studying mathematics, what do you want from the

instructional media?

13. How dose such system help you to study mathe-

matics?

14. Do you need such system in your study of mathe-

matics? If yes, why?

15. What do you think of using such system in your study

of mathematics?

16. If MathReader is developed, do you want to use such

system?

17. What suggestions have you to produce and pro-mote

such a system?

CSEDU 2010 - 2nd International Conference on Computer Supported Education

244