Web based Application Quality from End User Perspective:

Case Study - Assignment Letter LPPM ITS

Henning Titi Ciptaningtyas, Muchammad Husni, Fuad Dary Rosyadi, and Rohana Qudus

Departement of Informatics, Faculty of Information and Communication Technology,

Institut Teknologi Sepuluh Nopember

Keywords: Assignment letter; End user perspective; Web based application quality.

Abstract: Lembaga Penelitian dan Pengabdian Masyarakat (LPPM) is the unit to manage research and community

service activities in Institut Teknologi Sepuluh Nopember (ITS) institution. The researcher must request an

assignment letter to the LPPM for official travel needs (such as data collection, seminars, and surveys).

LPPM staff creates an assignment letter file manually and store it in the computer folder. LPPM staff should

check the latest identity number of assignment letter in the Excel file, and write down the number on the

document. After the document has been printed and approved by the Chair of LPPM, the document is

scanned and archived in the computer folder. But it also not properly archived since different staff has

different computer folder. We create a web-based application using Laravel and MySQL database to

manage the assignment letter in LPPM ITS. The application quality then measured to ensure it has good

performance and to ensure it meets user expectation. Application is measured by checking application

functionality, calculating response time for database access and time to upload, download, or export files.

Functional evaluation shows that all the application features have no error. Performance evaluation shows

that the application could serves 100 user requirements in less than 1.5 second. It can download, upload, and

export files in less than 1 second. The quality of the web based application is measured by comparing the

independent variables of reliability, responsiveness, assurance, emphaty, and tangible as well as user aspects

divided into a variety of factors, such as age, work experience, and education. The survey analysis shows

that the application meets respondent expectation and the user quite satisfied with it (the average gap

between expectation and reality is -0.096).

1 INTRODUCTION

Lembaga Penelitian dan Pengabdian Masyarakat

(LPPM) is the unit for managing research and

community service activities in the Institut

Teknologi Sepuluh Nopember (ITS) institution

based on ITS Chancellor Regulation (Peraturan

Rektor ITS) Number 10 Year 2016 about ITS

Organization and Governance (Organisasi dan Tata

Kelola ITS or OTK ITS) Chapter X Article 106

(Perek ITS Nomor 10, 2016). One of its tasks is

organizing and monitoring research programs and

community service conducted by ITS lecturers.

In conducting out research and community

service activities, a lecturer sometimes has to leave

their job in ITS for a few days e.g. to collecting data,

training, attending conference, presenting article.

For administrative purpose (the lecturer doesn’t

considered as absent), the lecturer must have an

Assignment Letter issued by LPPM.

The researcher must request an assignment letter

to the LPPM by sending cover letter from the

department. LPPM staff creates an assignment letter

file manually and store it in the computer folder.

LPPM staff should check the latest identity number

of assignment letter in the Excel file, then write

down the number on the document. After the

document has been printed and approved by the

Chair of LPPM, the document is scanned and

archived in the computer folder. But it also not

properly archived since different staff has different

computer folder. To monitor the assignment letter

process, lecturer should contact LPPM staff by

phone, then the LPPM staff tracking the assignment

letter document manually in Window Explorer using

search box. Those processes are inefficient. So we

354

Ciptaningtyas, H., Husni, M., Rosyadi, F. and Qudus, R.

Web based Application Quality from End User Perspective: Case Study - Assignment Letter LPPM ITS.

DOI: 10.5220/0009011303540362

In Proceedings of the 7th Engineering International Conference on Education, Concept and Application on Green Technology (EIC 2018), pages 354-362

ISBN: 978-989-758-411-4

create a secure web-based application to manage the

assignment letter in LPPM ITS and calculate the

quality of the application based on end-user

perspective.

The rest of the paper is organized as follows. In

Section 2, we discuss the theoretical background and

related work. Section 3 describes our web based

application developed for LPPM ITS and method to

calculate web application quality. Section 4 shows

the experimental result of the web based application

and the questionnaire result. Finally in Section 5,

conclusions and possible future work are discussed.

2 THEORITICAL BACKGROUND

2.1 Assignment Letter – LPPM ITS

Assignment Letter is a document with which an

assignor assigns a specific portion of his/her rights

to an assignee (USLegal, 2018) LPPM-ITS creates

the assignment letter for academia (lecturer, student,

or department staff) based on lecturer request as the

lead researcher or lead of community service. The

assignment letter should follow ISO 15489-1:2016

standard. ISO 15489-1:2016 applies to the creation,

capture and management of records regardless of

structure or form, in all types of business and

technological environments (ISO 15489-1:2016,

2016). But the standard has not been followed yet

since the management of assignment letter was done

semi-manually. To meet the standard, the

assignment letter document needs to be managed in

an electronic management system. Electronic

Document Management System (EDMS) is a

management system to prepare documents in

computers, to save the document header

information, to make changes on the document, to

share with other user, and to archive the document

(M. Başıbüyük and A. Ergüzen, 2015). Examples of

common EDMS are Napersoft (Napersoft, 2016),

Message Point (Letters and Correspondence,2018),

OpenDocMan(Document Management Software,

2018 ), OpenKM (Document Management System,

2018). Example of custom EDMS for university is

Electronic Document Management System for

Kırıkkale University (M. Başıbüyük and A. Ergüzen,

2015).

In this research, we develop a custom EDMS for

managing assignment letter which is generated by

LPPM-ITS. Example of an assignment letter LPPM

ITS can be seen in Figure 1.

Figure 1: Assignment Letter LPPM ITS.

2.2 Web Based Application

Web based application is a computer program

that allows you to go to a web address in order to

submit and retrieve data to or from a database over

the internet which use HTTP as its primary

communication protocol, developed using web

technologies (e.g. HTML, CSS, JS), and can be

accessed using web browser (e.g. Google Chrome,

Firefox) (WebFinance Inc, 2018) (Evergreen, 2018).

The advantages of web based application are

improved efficiency, accessibility, higher levels of

security, easy customization and scalable, easy

installation and maintenance [10]. The assignment

letter management system in LPPM ITS is built as

web based application.

MySQL is an open-source relational database

management system (Why MySQL, 2018). Laravel

is a web application framework with elegant syntax,

easing common tasks used in the majority of web

projects (e.g. authentication, routing, sessions,

caching), and providing powerful tools for

developing robust applications. It has control

container, migration system, and tightly integrated

unit testing support (T. Otwell, 2018). Laravel use

Apache web server (Apache Software Foundation,

2018), to serve the files that form web pages to users

Web based Application Quality from End User Perspective: Case Study - Assignment Letter LPPM ITS

355

in response to their requests. Laravel has better

performance for business since data abstraction and

other inbuilt libraries help reduce development time

and also performance (execution time, response time

and throughput) is enhanced at larger loads (U.

Ibrahim, et. al. , 2018)( R. Das and L. P. Saikia,

2016).

In this research, to build assignment letter

management system we use Laravel Framework and

Apache web server, to store the assignment letter

data we use My SQL database server.

2.3 Web Based Application Quality

Usability is one of web based applications

quality requirement. Usability can be measured from

end-user viewpoint (P. Lew, et. al., 2009). The

quantitative methods focus on the performance

measurement of the website whereas the qualitative

methods estimate the user’s opinion of a website (H.

Banati, et. al., 2006).

Web based application performance is measured

by checking application functionality, calculating

response time for database access and time to

upload, download, or export files.

User opinion on satisfaction level can be

measured based on content, speed, interactivity and

appearance variable (N. K. Sumiari and I. N. Y. A.

Wijaya, 2014). User satisfaction can also be

measured by tangible (user will use the sense of

sight for assess a quality of service), reliability

(company's ability to provide services as promised

and how far a company is able to provide accurate or

no error service), responsiveness (speed of service),

assurance (the ability of employees to knowledge of

the product appropriately), and emphaty (the service

providers know them personally) which can be

detailed into 14 aspects (A. Gui, 2009).

In this research, we do the functionality

evaluation using black box test, performance

evaluation using JMeter, and user opinion using

questionnaire method.

2.4 System Design

General system design can be seen in Figure 2. The

application users are LPPM staffs who are in charge

of managing Assignment Letters and academica

(lecturer, department staff, or student) who request

the assignment letter. Printer is used to print the

draft of the assignment letter. Scanner is used to

digitize the assignment letter that has been approved

by the Chairman of LPPM. File server is to store the

scanned assignment letter. Database server is to

store the assignment letter data. Web server is to

serve the user request.

This system has features as follows:

1. Create, update and delete data on assignment

letter request as long as it has not been approved

by the head of LPPM yet.

2. Download draft of the assignment letter in

Microsoft Word (.doc) format.

3. Print the assignment letter draft using the printer.

4. Change the status of the assignment letter that

has been legalized to a "valid" status.

5. Digitize the assignment letter that has been

approved by LPPM management using a

scanner.

6. Upload assignment letter that has been

authorized to the File Server.

The activity diagram of assignment letter applicant

can be seen in Figure 3, while activity of LPPM staff

can be seen in Figure 4.

Figure 2: General System Design.

EIC 2018 - The 7th Engineering International Conference (EIC), Engineering International Conference on Education, Concept and

Application on Green Technology

356

Figure 3: Applicant Activity Diagram: (a) Create (b) Search Assignment Letter.

(a) (b)

Figure 4: LPPM Staff Activity Diagram: (a) Assignment Letter Management (b) Data Management

Web based Application Quality from End User Perspective: Case Study - Assignment Letter LPPM ITS

357

3 EXPERIMENTAL RESULT

The experiment consists of 3 evaluations:

Functionality evaluation (using black box test),

Performance evaluation (using JMeter), User

opinion (using questionnaire method).

3.1 Functional Evaluation

Functional evaluation is using black box text for

each feature and each user. Functional evaluation

result can be seen in Table 1. Succeed means the

application feature works well with no error in 20

trials. NA means the feature is Not Authorized for

the user. Web application can be seen in Figure 5.

Figure 5: Assignment Letter Website.

Table 1: Functional Evaluation.

Application Features

User

Lecturer /

Staff / Student

LPPM Staff

LPPM

Management

Create an Assignment Letter Succeed Succeed NA

Upload Assignment File, Support File Type: Pdf Succeed NA NA

Edit Assignment Letter NA Succeed NA

Correcting Assignment Letter NA Succeed NA

Display list of assignment letters Succeed Succeed Succeed

Search for Assignment Letter Data Succeed Succeed Succeed

Edit Assignment Letter Status NA Succeed NA

Monitor Assignment Letter Status Succeed Succeed Succeed

Export Assignment Data, File Type: Xls NA Succeed NA

Upload SPK Research and Community Service Data, File Type: Pdf NA Succeed NA

Download SPK Research and Community Service Data, File Type: Pdf Succeed NA NA

Management of Research and Community Service Fund Sources NA Succeed NA

Management of Research and Community Service Schemes NA Succeed NA

User Management NA Succeed NA

Download the Assignment Letter Draft File, File Type: Doc NA Succeed NA

Upload a signed Assignment Letter, File Type: Pdf NA Succeed NA

EIC 2018 - The 7th Engineering International Conference (EIC), Engineering International Conference on Education, Concept and

Application on Green Technology

358

3.2 Performance Evaluation

The performance evaluation of Create, Read,

Update, Delete (CRUD) in MySQL database uses

JMeter tools in Intranet and Internet network. Create

data or insert query is when academia submitting

“create the assignment letter” request into database.

This research calculates the average response time of

“insert query” for concurrent 100 requests per

second. Update is when LPPM staff updates the

assignment letter. This research calculates average

response time of “insert query” using single request

for 100 times. Delete is when LPPM staff deleting

the scheme or the fund source. This research

calculates average response time of “delete query”

using single request for 100 times. Read data or

select query is when user searching the data of

assignment letter. This research calculates average

response time of “select query” for concurrent 100

requests per second. The result can be seen in Table

2. Average time response is 969.75 ms on the

intranet and 1256.25 ms on the internet.

Table 2: Performance Evaluation of CRUD Query.

Query Intranet (ms) Internet (ms)

Insert

1,969.00 2,339.00

Update

21.00 305.00

Delete

2.00 90.00

Select

1,887.00 2,291.00

The performance evaluation of uploading,

downloading, and exporting file can be seen in Table

3. Upload Pdf, Download Pdf, Export Xls action are

using random file size between 500 kb until 5 MB,

while Download Doc action is using random file

size between 500 kb until 2 MB. This research

calculates average response time using single

request for 100 times. It can download, upload, and

export files in less than 1 second.

Table 3: Performance Evaluation of Upload,

Download, and Export File.

Action Filetype Intranet

(ms)

Internet

(ms)

Upload

Pdf 294.00 335.33

Download

Pdf 220.50 947.50

Download

Doc 44.00 189.00

Export

Xls 119.00 512.00

3.3 End User Perspective

The quality of the web based application is

measured by comparing the independent variables of

reliability, responsiveness, assurance, emphaty, and

tangible as well as user aspects divided into a variety

of factors, such as age, work experience, and

education. The values of user aspect are shown in

Figure 6. This experiment is using questionnaire to

collect end user perspective. The questionnaire was

distributed from July to August 2018 to LPPM staff,

management and department staff. The population of

this study is 6 respondents and the entire population

was studied. The user aspect can be seen in Figure6.

The questionnaire was tested in previous studies

(Gui, 2009) so that the questionnaire was valid and

reliable for research. Questionnaires were distributed

to all respondents. From the questionnaires

distributed, all are valid and complete. The

questionnaire distributed to respondents consisted of

2 types: the questionnaire state the respondent's

expectations for application and questionnaire

regarding the performance of the application

(reality).

Gap analysis method will produce 2 results:

satisfied and dissatisfied. Those results provide wide

area of decision making. To improve application

performance, it has to be narrowed and create the

priority aspects (aspect which has to be fixed first).

The priority can be seen from the level of perceived

satisfaction by user. The level of satisfaction can be

obtained by dividing 14 aspect of satisfaction

measurement becomes 5 classes: very dissatisfied,

dissatisfied, quite satisfied, satisfied and very

satisfied. Based on gap analysis theory, the biggest

gap value is 4 and the smallest gap value is -4. By

using the formula to find the class range, the value

of the class range is 1.6 then we classify it into 5

classes as shown in Table 4.

The survey result for each aspect can be seen in

Table 5 or Figure 7, while for each dimension can be

seen in Table 6 or Figure 8. The survey analysis

shows that the application meets respondent

expectation and the user quite satisfied with it (the

average gap between expectation and reality is -

0.096).

Web based Application Quality from End User Perspective: Case Study - Assignment Letter LPPM ITS

359

(a) (b) (c)

Figure 6: User Aspect (a) Age (b) Work Experience (c) Education Background.

Table 4: Class Distribution Level of Satisfaction.

Class Range Explanation

1 (-4) - (-2.5) Very dissatisfied

2 (-2.4) - (-0.9) Not satisfied

3 (-0.8) - 0.7 Quite satisfied

4 0.8 - 2.3 Satisfied

5 2.4 - 4 Very satisfied

17%

33%33%

17%

UserAge

<30

30‐40

41‐50

>50

67%

16%

17%

WorkExperience(years)

<10

10‐20

>20

33%

17%

EducationBackground

High

School

Bachelor

Master

Doctor

Table 5: Gap Analysis For Each Aspect.

Question Aspect Expected Reality Gap Result

1 Fast application access to database 5.00 5.00 0.00 Quite satisfied

2 System protection 4.83 4.83 0.00 Quite satisfied

3 Fast creating assignment letter 5.00 5.00 0.00 Quite satisfied

4 Online data backup 5.00 5.00 0.00 Quite satisfied

5 Feature easy to understand 5.00 4.67 -0.30 Quite satisfied

6 Fast response to input 5.00 4.50 -0.50 Quite satisfied

7 Feature easy to use 5.00 4.83 -0.20 Quite satisfied

8 Accurate assignment letter 5.00 4.83 -0.20 Quite satisfied

9 Provide clear information when an error occurs 5.00 4.83 -0.20 Quite satisfied

10 Provide complete usage instructions 5.00 5.00 0.00 Quite satisfied

11 Easy to adapt to user needs 4.83 4.83 0.00 Quite satisfied

12 Creating assignment letters is easier, faster resolved,

and easier monitored

4.67 5.00 0.30 Quite satisfied

13 Provide complete documentation 5.00 4.83 -0.20 Quite satisfied

14 Application documentation is easy to understand 4.67 4.67 0.00 Quite satisfied

Table 6: Gap Analysis for Each Dimension.

No Dimension Expected Reality Gap Result

1 Reliability (Question 1-4) 4.96 4.96 0.00 Quite satisfied

2 Responsiveness (Question 5-7) 5.00 4.67 -0.33 Quite satisfied

3 Assurance (Question 8-9) 5.00 4.83 -0.17 Quite satisfied

4 Empathy (Question 10-12) 4.83 4.94 0.11 Quite satisfied

5 Tangible (Question 13-14) 4.84 4.75 -0.09 Quite satisfied

EIC 2018 - The 7th Engineering International Conference (EIC), Engineering International Conference on Education, Concept and

Application on Green Technology

360

Figure 7: Gap for Each Satisfaction Aspect.

Figure 8: Gap for Each Satisfaction Dimension.

4 CONCLUSIONS

From the experimental result, we can conclude as

follows: The web application can provide good

assignment letter management. All application

features can work well and have no error; the

application is reliable enough. It could serve 100

user requirements in less than 1.5 second and can

serve download, upload, and export files request in

less than 1 second; the quality of reliability,

responsiveness, assurance, emphaty, and tangible

from user’s perspective is good enough. The survey

analysis shows that the application meets respondent

expectation and the user quite satisfied with it (the

average gap between expectation and reality is -

0.096).

For further research, the number of respondents

should be multiplied by various variations of user

aspects. The quality aspect of web-based

applications can also be sharpened by adding more

questions to each dimension of user satisfaction.

ACKNOWLEDGEMENTS

This research is funded by Institut Teknologi

Sepuluh Nopember (ITS), Kementerian Riset,

Teknologi, dan Pendidikan Tinggi

(Kemenristekdikti) under Penelitian Kebijakan

scheme in accordance with the Contract of Research

Number: 1235/PKS/ITS/2018.

REFERENCES

A. Gui, E. Erwin, N. V. Amanda and L. S. Phangestu,

"Tingkat Kepuasan Pengguna Aplikasi Intersystem

Business Solution (IBS) pada PT Citajaya Infinite

System," CommIT, vol. 3, no. 1, pp. 53-58, May 2009.

Apache Software Foundation, "Apache HTTP Server

Project," Apache Software Foundation, 2018.

[Online]. Available: http://httpd.apache.org/.

[Accessed 1 September 2018].

Evergreen, "5 Key Benefits of Web Applications for

Business," Evergreen, 6 August 2018. [Online].

Available: https://evergreencomputing.com/blog/5-

key-benefits-of-web-applications-for-business/.

[Accessed 10 September 2018].

Free Document Management Software, "OpenDocMan -

Open Source Document Management System," 2018.

[Online]. Available: http://www.opendocman.com/.

[Accessed 10 September 2018].

H. Banati, P. Bedi and P. S. Grover, "Evaluating Web

Usability from the User’s Perspective," Journal of

Computer Science, vol. 2, no. 4, pp. 314-317, 2006.

International Organization for Standardization, "ISO

15489-1:2016 Information and documentation -

Records management," 2016. [Online]. Available:

https://www.iso.org/standard/62542.html. [Accessed

10 September 2018].

ITS, Peraturan Rektor ITS Nomor 10 Tahun 2016 Tentang

Organisasi dan Tata Kerja ITS, Surabaya: ITS, 2016.

M. Başıbüyük and A. Ergüzen, 2015. "Electronic

Document Management System for Kırıkkale

University," Unified Journal of Computer Science

Research, vol. 1, no. 2, pp. 008-015

Message Point, "Letters and Correspondence," 2018.

[Online].Available:

4.2

4.4

4.6

4.8

5.2

1234567891011121314

SatisfactionSurvey

Question

SatisfactionAspect

Expected Reality

4.4

4.6

4.8

5.2

SatisfactionSurvey

Dimension

Satisfaction

Dimension

Expected Reality

Web based Application Quality from End User Perspective: Case Study - Assignment Letter LPPM ITS

361

https://www.messagepoint.com/solutions/letters/.

[Accessed 1 September 2018].

MySQL, "Why MySQL," MySQL, 2018. [Online].

Available: https://www.mysql.com/why-mysql/.

[Accessed 4 September 2018].

N. K. Sumiari and I. N. Y. A. Wijaya, "Pengukuran

Tingkat Kepuasan Pengguna Sistem Infromasi

Akademik Online (Studi Kasus STMIK Stikom Bali),"

in Seminar Nasional Informatika, 2014.

Napersoft, "Napersoft Software," 2016. [Online].

Available: http://www.napersoft.com/. [Accessed 9

September 2018].

Open Document Management System S.L., "Document

Management System," 2018. [Online]. Available:

https://www.openkm.com/. [Accessed 10 September

2018].

P. Lew, L. Zhang and S. Wang, "Model and Measurement

for Web Application Usability from an End User

Perspective," in Workshop on Quality Assessment in

Web, Spain, 2009.

R. Das and L. P. Saikia, "Comparison of Procedural PHP

with Codeigniter and Laravel Framework,"

International Journal of Current Trends in Engineering

& Research (IJCTER), vol. 2, no. 6, p. 42 – 48, June

2016.

T. Otwell, "Introduction," Laravel, 2018. [Online].

Available: https://laravel.com/docs/4.2/introduction.

[Accessed 5 September 2018].

U. Ibrahim, J. B. Hayfron-Acquah and F. Twum,

"Comparative Analysis of Codeigniter and Laravel in

Relation to Object-Relational Mapping, Load Testing

and Stress Testing," International Research Journal of

Engineering and Technology (IRJET), vol. 5, no. 2,

February 2018.

USLegal, "Letter of Assignment Law and Legal

Definition," 2016. [Online]. Available:

https://definitions.uslegal.com/l/letter-of-assignment/.

[Accessed 8 September 2018].

WebFinance Inc., "Web based application," 2018.

[Online]. Available:

http://www.businessdictionary.com/definition/web-

based-application.html. [Accessed 10 September

2018].

EIC 2018 - The 7th Engineering International Conference (EIC), Engineering International Conference on Education, Concept and

Application on Green Technology

362