Mining Developer Questions about Major Web Frameworks

Zakaria Mehrab, Raquib Bin Yousuf, Ibrahim Asadullah Tahmid and Rifat Shahriyar

Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology, Bangladesh

Keywords:

Mining, Web Framework, Stack Overﬂow.

Abstract:

Web frameworks are the de facto way to build web-enabled applications. Stack Overﬂow, being one of the

leading question answering sites available, has become a helpful resource in numerous software engineering

research. In this paper, we present a study of common challenges and issues among developers of two major

web frameworks namely Laravel and Django by mining questions asked on Stack Overﬂow. We extracted

the issues that the developers are most concerned about. We sorted these issues by popularity and difﬁculty

metrics and observed the contrasting nature of difﬁculty and popularity. We also noted an exception that

installation is a popular issue over both the frameworks and simultaneously it is also difﬁcult to resolve.

Besides, we found that about 50% issues are common over both the frameworks. Our ﬁndings would help the

framework developers to understand better the need of the framework users by focusing most difﬁcult and the

most popular issues.

1 INTRODUCTION

Web development is a term comprised of develop-

ing websites, web services, and web application. In

recent years, web applications require aesthetic rep-

resentation as well as an efﬁcient interaction be-

tween users and websites, leading to more complex-

ity. According to Hevner et al. (Hevner et al., 2007),

currently web engineers face three intractable prob-

lems: domain/system complexity, increased develop-

ment time and cost. Web developers feel the need

for modularization of concerns to accommodate big

and complex applications (Ginige, 1998). Web frame-

works help developers in this task by bringing a stan-

dard way to develop and implement web applica-

tions. These frameworks often help to decouple the

logic portion from the view portion, namely imple-

menting the Model-View-Controller(MVC) pattern.

Moreover, frameworks enable developers to reuse de-

sign and implementation by combining the proce-

dures of necessary tasks. According to J. Carlos

et al. (Fern

andez-Conde and Gonz

alez-Calero, 2002)

these frameworks have a positive effect on the project

with shortened development time, reduced complex-

ity, increased productivity, extensibility, and reliabil-

ity.

Since the ﬁrst appearance in the late 1990s, more

than 5,000 frameworks have been released (Github,

2018d). While using these frameworks for creating

web applications, developers often ﬁnd themselves

grounded with various problems. Developers often

post questions in Stack Overﬂow (Exchange, 2017a),

part of the Stack Exchange platform, to seek help and

guidance. According to Mamykina et al. (Mamykina

et al., 2011) Stack Overﬂow is larger than any other

social Q&A forum or programming forum. There-

fore, a thorough analysis of its posts can help us un-

derstand the problems faced by the web framework

developers.

Created in 2008 by Jeff Atwood and Joel Spolsky,

Stack Overﬂow is the ﬂagship site of the Stack Ex-

change Network. It features questions and answers

on a wide range of topics regarding computer pro-

gramming and related technologies. The huge amount

of web framework related posts in Stack Overﬂow

shows the importance of studies related to the prob-

lems faced by web framework developers. In Fig-

ure 1, we have presented a graph with the percentage

of posts related to eleven popular frameworks. To-

gether these eleven frameworks measure up to more

than 15% of the total questions on Stack Overﬂow

each month in 2017.

We used Stack Overﬂow data dump (Exchange,

2017a) to explore the issues of web frameworks that

developers face.

We picked two web frameworks, Laravel, and

Django for our analysis. The reason behind choos-

ing Laravel and Django is that both are very popular

Mehrab, Z., Yousuf, R., Tahmid, I. and Shahriyar, R.

Mining Developer Questions about Major Web Frameworks.

DOI: 10.5220/0006929501910198

In Proceedings of the 14th International Conference on Web Information Systems and Technologies (WEBIST 2018), pages 191-198

ISBN: 978-989-758-324-7

191

Figure 1: Stack Overﬂow trends of framework related post.

but not well studied, unlike Javascript frameworks.

Whereas Laravel and Django together have a little

over 45,000 stars on their GitHub repository (Github,

2018b) (Github, 2018a), one Javascript framework,

vuejs, alone has bagged over 95,000 stars on its

GitHub repository (Github, 2018c). There is also

the fact that searching with keyword ”javascript” in

(DBLP, 1993) shows 784 matches whereas ”django”

keyword produces 34 results (24 of which belongs to

author ”Django Armstrong”) and ”laravel” keyword

produces two results.

For studying the two frameworks, we extracted

contents from all the related questions. We used

them to train our model in MALLET topic modeling

tool (MALLET, 2017). We extracted the appropriate,

relevant topics of each question by Latent Dirichlet

Allocation(LDA). Once we were able to extract the

individual topic of each question, we started to an-

swer our research questions.

RQ1 What are the issues users of these two frame-

work face?

RQ2 What are the most popular issues asked by de-

velopers of these two frameworks?

RQ3 Which issues of these two frameworks are the

most difﬁcult?

RQ4 Are the issues of these two frameworks differ-

ent?

From our experiment, we found 18 issues for Lar-

avel and 20 issues of Django; faced by the develop-

ers. We sorted these issues by different metrics and

ranked them based on their popularity and difﬁculty.

We observed the contrasting nature of difﬁculty and

popularity. We also noted an exception that installa-

tion is a popular issue over both the frameworks and

simultaneously, it is also difﬁcult to resolve. Also, we

found that about 50% issues are common over both

the frameworks. Our ﬁndings can help the develop-

ers of the frameworks to better understand the need

of the users. They can categorize the needs of users

to focus on the most difﬁcult and the most popular

issues. They can also ﬁnd similar issues in multiple

frameworks those require major improvement.

The rest of this paper is organized as follows. The

next section reviews the related works. The methods

for data collection, operationalization and analysis are

presented next. This is followed by our ﬁndings and

related discussions. The paper concludes by high-

lighting possible future extensions of our work.

2 BACKGROUND AND RELATED

WORKS

Laravel is an MVC framework for PHP web lan-

guage, released in 2011. Olanrewaju et al. (Olanre-

waju et al., 2015) discussed the most famous MVC

based PHP frameworks, evaluated their performance

and found that Laravel outperforms other MVC

frameworks, dubbed Laravel as the most suitable PHP

framework for future web technology. Django is an

open source web framework built on Python program-

ming language. It was released in 2005 and quickly

gained popularity due to re-usability of components;

rapid development leads to the development of com-

plex web applications in shorter time.

2.1 Related Work

We looked into works which discuss the issues that

the developers face while using web frameworks. We

also went through several types of research for better

understanding the issues and trends among developers

from the empirical analysis of Stack Overﬂow data.

The empirical studies using Stack Overﬂow data

is now one of the growing trends of empirical soft-

ware engineering. Several papers using Stack Over-

ﬂow data has been presented. Some of them use the

data to present behavioral research question about the

Stack Overﬂow users. Others use topic modeling to

categorize the discussions in the posts.

Allamanis et al. (Allamanis and Sutton, 2013) ap-

plied topic modeling on Stack Overﬂow questions.

After associating them with programming concepts,

they found that certain types of questions are asso-

ciated with speciﬁc programming concepts. Treude

et al. (Treude et al., 2011) analyzed data from Stack

Overﬂow to categorize the kinds of questions that are

asked and found that Q&A websites are particularly

effective in code reviews and conceptual questions.

Mamykina et al. (Mamykina et al., 2011) took the de-

sign lessons from Stack Overﬂow and found that the

success of Stack Overﬂow is because of the quickness

of a question getting answered. They also found that

WEBIST 2018 - 14th International Conference on Web Information Systems and Technologies

192

the high visibility and daily involvement of the design

team of Stack Overﬂow help increase Stack Over-

ﬂow’s popularity. Barua et al. (Barua et al., 2014)

aimed to analyze the actual textual content of Stack

Overﬂow to help the software engineering commu-

nity to better understand the thoughts and needs of

developers. They found that the topics of interest to

developers range widely from jobs to version control

systems to C# syntax. They dictated that the topics

gaining the most popularity over time are web de-

velopment, mobile applications, Git and MySQL. Li

et al. (Li et al., 2013) performed an empirical study

with 24 developers to understand the needs and chal-

lenges developers face during the development phase.

Beyer et al. (Beyer and Pinzger, 2014) manually an-

alyzed 450 Android related posts to determine com-

mon problems developers face. Rosen et al. (Rosen

and Shihab, 2016) analyzed Stack Overﬂow data to

determine what mobile developers ask about by us-

ing LDA-based topic models. They also determined

what popular mobile-related issues are the most dif-

ﬁcult, explore platform-speciﬁc issues, and investi-

gated the types of questions mobile developers ask.

Bajaj et al. (Bajaj et al., 2014) presented a study of

common challenges and misconceptions among web

developers by mining-related questions about client-

side code, written in JavaScript, HTML, and CSS

asked on Stack Overﬂow. They used unsupervised

learning to categorize the mined questions and de-

ﬁned a ranking algorithm to rank all the Stack Over-

ﬂow questions based on their importance.

The related works for this topic revolve around the

empirical use of Stack Overﬂow data to determine the

issues developers face, trends of asked topics and be-

havior by the users. These tasks require textual con-

tent analyzing which is done mostly with topic mod-

eling studies. Though several empirical studies have

been done with the Stack Overﬂow data, none have

tried to determine the speciﬁc issues faced by web

framework users. We aim to ﬁll that void by look-

ing into the framework related posts and bringing out

the issues developers face.

3 METHODOLOGY

3.1 Data Extraction

The data dump of Stack Overﬂow contains all the data

of the website in XML form. We obtained the data

dump of August 2016. Among the XML ﬁles residing

in the dump, we used only the ﬁles relevant with our

purpose, namely posts.xml, tags.xml and users.xml.

We ran a script to extract these XML ﬁles to cor-

responding MySQL database tables. The database

schema of stack exchange ﬁles is available in this

link (Meta, 2017).

3.2 Data Processing

Having built our database, we focused on retriev-

ing the Laravel and Django related questions from

the Post table. The Post table contained a whopping

32,209,817 entries. Among these entries, 12,350,818

are questions. From these questions, we used a tag-

based search to extract 59,360 Laravel related ques-

tions and 130,588 Django related questions.

After that, we targeted on extracting the body of

each question and preparing them for textual analysis

using topic modeling. The bodies can contain HTML

tags and several unwanted strings. We trimmed out

the HTML tags and unwanted strings from the bodies.

Then, we created separate ﬁles for each body and kept

them in a directory. These ﬁles will represent a corpus

of documents to be used in topic modeling.

3.3 Topic Modeling using LDA

Inference of topics from a corpus of documents is

performed by Latent Dirichlet Allocation (LDA). Ac-

cording to the LDA model, each document contains

a mixture of topics. Topics are also allowed to exist

across several documents. So it is easy to discover

themes and ideas that represents all the documents as

a whole (Rosen and Shihab, 2016).

To perform topic modeling, we used MALLET

version 2.0. It is a Java-based package for statis-

tical natural language processing, document classi-

ﬁcation, clustering, topic modeling, information ex-

traction, and other machine learning applications to

text (MALLET, 2017). The MALLET topic model-

ing tool uses LDA.

Running MALLET on a corpus requires two hy-

perparameters; namely: topic numbers and words per

topic. After assembling our data, we ran the MAL-

LET tool with different combinations of these hyper-

parameters. We performed the experiment with 20

topics and 10 words per topic, 20 topic and 20 words

per topic, 50 topics and 20 words per topic. After

each run, We manually checked the words in each

topic and through consensus, we concluded that the

result found with 50 topics and 20 words per topic

captures the theme of data better than the others. So

we retained 50 topics and 20 words per topic as our ﬁ-

nal hyperparameters. The output is distributed among

two ﬁles. One, namely “key.txt”, contains word sets

of each topic along with their Dirichlet parameters.

The second ﬁle, namely “composition.txt”, contains

Mining Developer Questions about Major Web Frameworks

193

Table 1: Set of words, their merging and given name of Laravel topics.

Merged given name of set

of words

Set of words

authentication

email send password mail address message emails user laravel sending reset

code messages users function account hash link username sends

user users login admin laravel authentication logged auth password check

access username create proﬁle role permissions system account log roles

date time date event time events format day dates month carbon days year current

database datetime timestamp set code timezone start ﬁre

database

table user users model tables pivot relationship models user id company rela-

tion column eloquent list data relationships roles columns group role

table key migration error database foreign column migrations tables create

primary sql created add run constraint laravel seed migrate sqlstate

model models eloquent relationship relationships method relation related ob-

ject eager relations table collection property loading set tables attribute load

attributes

table database update data record insert row delete model ﬁeld records column

code rows save laravel deleted method ﬁelds values

query eloquent laravel table sql result column results queries select builder

mysql join count order rows raw tables return database

database data laravel connection save mysql create store stored set mongodb

document created application databases default saved project settings con-

tacts

the probability of each topic for each document of the

corpus.

Having analyzed the words for each topic, we no-

ticed that some of the word sets are similar in mean-

ing. So we merged word sets conveying similar mean-

ing and named the topics manually. Through this

merging process, we obtained 18 topics for Laravel

and 20 topics for Django. Thus we obtain the answer

to our ﬁrst research question, the issues users of these

two frameworks face.

The naming of the topics was performed through

consensus among the authors and the peers. A few

samples of the naming and merging process have been

shown in 1 and 2. The full list could not be tabulated

here due to space constraints. The complete list can

be viewed in (Authors, 2018b) and (Authors, 2018a).

In both of the frameworks, we named a topic as ”Gen-

eral Issue”, which contains words which could not be

classiﬁed to any speciﬁc topic. However, the human

consensus is error prone. The naming and merging

process of the topics may vary depending on the per-

spectives of developers. We are not claiming it to be

anything concrete.

3.4 Finding Popular and Difﬁcult Issues

With our topics identiﬁed by human understandable

names, we focused on answering our research ques-

tions. To answer our ﬁrst research questions, we cal-

culated the most dominant topics of each post with the

help of the composition ﬁle generated by MALLET.

Next, We calculated total questions and total views

for each topics using these posts where that particu-

lar topic is dominant. Then we sorted the topics by

view per question ratio. Naturally, the topic that has

the most view per question is more popular.

To answer our third research questions, we deter-

mined the difﬁculty level of each topic by associating

with it the mean and median duration in minutes of

getting an accepted answer after a related question is

posted. We ranked the topics by median time as ac-

cording to Rosen et al. (Rosen and Shihab, 2016) the

mean is likely to be skewed by long latency responses.

3.5 Finding Common Issues

For answering our ﬁnal research questions, we man-

ually checked the two sets of topics: The ﬁrst set

being the topics generated from Laravel (L) related

posts and the second set being the topics generated

from Django related posts (D). Using these sets, we

obtained three other sets as follows:

• L − D (Issues only faced by users using Laravel

Framework)

• L ∩ D (Issues those are common for both users of

Laravel and Django)

• D − L (Issues only faced by users using Django

Framework)

WEBIST 2018 - 14th International Conference on Web Information Systems and Technologies

194

Table 2: Set of words, their merging and given name of Django topics.

Merged given name of set

of words

Set of words

ﬁle upload ﬁle image upload images ﬁles uploaded django save download path video

user photo uploading picture code media thumbnail imageﬁeld uploads

form

form forms data ﬁelds ﬁeld view user validation model forms.py submit save

views.py formset post template input modelform django create

page button ajax html view click code jquery javascript function form user

django template submit views.py link load display call

form ﬁeld select ﬁelds model widget list choices selected user values choice

option input dropdown add set forms django box

date time event day events model datetime month format dates year ﬁeld days

current calendar hours django set number start

migration database django migration migrations run table south tables error created

model migrate data app create models syncdb command manage.py running

authentication

ﬁle line return usr/local/lib/python c:python usr/lib/python lib/python er-

ror traceback exception init pid call kwargs import recent request response

python tid

email user password login facebook authentication username django users

send account app token registration auth emails address oauth social google

user users model proﬁle create permissions group add created django permis-

sion custom access userproﬁle username logged groups view user’s app

user login page session logged view django redirect site users log cookie set

authentication middleware url redirected sessions cookies access

4 FINDINGS

4.1 Evolution of Framework Related

Posts

We looked into the trend of Laravel and Django re-

lated question in Stack Overﬂow quantitatively. First,

we measured the percentage of Laravel related post

to total post in each year. We measured the same

for Django related post. The public data dump we

used for topic modeling had was updated until August

2016. But for the trend of the Laravel and Django

related post, we used the Stack Exchange Data Ex-

plorer (Exchange, 2017b) which enabled us to run

queries online with the latest Stack Overﬂow data.

The trend is presented in Figure 2.

4.2 RQ1

To answer our ﬁrst research questions, we analyzed

the word sets generated by mallets and manually

provided understandable human names for the top-

ics. The naming was done with consensus among

peers proﬁcient in these two frameworks. The top-

ics for Laravel framework were named as Authenti-

cation, API, Date, Database, Documentation, Form,

Image, Installation, General Issue, Language, MVC,

Figure 2: Percentage of related post per total posts per year.

Application, Session, Syntax, Transaction, Unit Test-

ing, Elixir, and View. Similarly, the topics for

Django framework were identiﬁed by the following

names: Admin, API, App, Authentication, Cross-Site

Request Forgery, Documentation, Encoding, File Up-

load, Form, General Issue, Installation, Localization,

Migration, Model, Pep, Query, Server, Template, Unit

Testing and View. Collectively, these are the issues

that developers of these web frameworks face.

4.3 RQ2

Having named the topics, we attempted to rank them

by popularity. For that purpose, we calculated the to-

tal number of questions and total views for each topic.

Mining Developer Questions about Major Web Frameworks

195

Table 3: Popularity rank of Laravel and Django Issues.

Rank

Laravel Django

Topic Question View V:Q Topic Question View V:Q

1 Installation 5459 5814400 1065.1 Encoding 1962 3356472 1710.74

2 Documentation 1329 1302098 979.758 Installation 6610 9888933 1496.06

3 General-Issue 10337 9619519 930.591 Server 3633 4856344 1336.73

4 MVC basic 10766 9371661 870.487 Admin 5693 7372228 1294.96

5 Database 9638 8236913 854.629 CSRF

∗

3117 3830184 1228.8

6 Session 2207 1837996 832.803 Migration 3062 3729092 1217.86

7 Syntax 3842 3195133 831.633 General-Issue 10207 12225460 1197.75

8 date time 680 531020 780.912 App 12187 14481413 1188.27

9 Form 3189 2451902 768.862 View 6922 8009187 1157.06

10 Image 652 482527 740.072 ﬁle-upload 3119 3566093 1143.34

11 Authentication 2838 2009346 708.015 pep 3343 3799548 1136.57

12 API 1118 772301 690.788 Template 11401 12633372 1108.09

13 View 4305 2817349 654.436 Documentation 1791 1971313 1100.68

14 Application 1140 743720 652.386 Query 6488 6638810 1023.24

15 language-translate 181 112114 619.414 Form 15615 15826055 1013.52

16 Elixir 430 240794 559.986 Model 20068 19028206 948.186

17 unit testing 448 242270 540.781 Authentication 9284 8680880 935.037

18 Transaction 801 342016 426.986 Localization 775 690861 891.434

19 unit testing 1235 1042258 843.934

20 API 4076 3003249 736.813

CSRF

∗

= Cross Site Request Forgery

Table 4: Difﬁculty rank of Laravel and Django Issues on basis of time to get accepted answer.

Rank

Laravel Django

Topic Mean Median Topic Mean Median

1 unit testing 16034.7 373 API 18763.1 133

2 Elixir 7812.91 171 Installation 18599.5 114

3 Installation 13118.8 105 unit testing 53229.6 114

4 API 15978.9 94 Localization 13841.3 110

5 Session 11696.4 61 Server 18771.6 101

6 Transaction 4502.43 55 Migration 17148.1 61

7 Application 10116.3 51 CSRF

∗

13895.6 58

8 Authentication 8466.89 50 ﬁle-upload 11193.4 57

9 View 6251.58 45 Admin 17932.7 50

10 language-translate 5954.43 44 Authentication 12119.6 49

11 Database 5504.18 40 App 15416.5 47

12 date time 3156.09 39 Documentation 14462.9 46

13 Image 4565.55 39 General-Issue 14890 42

14 General-Issue 7311.93 38 Model 13844.1 41

15 Form 3314.9 37 Form 8715.9 39

16 MVC basic 6001.73 35 pep 10295.4 34

17 Documentation 4898.51 32 Template 13135.9 32

18 Syntax 6876.79 31 Query 11216.7 31

19 Encoding 7906.35 26

20 View 9458.15 20

CSRF

∗

= Cross Site Request Forgery

The ratio of view count by the number of question for

each topic has been taken as the ranking metric for

popularity. Table 3 presents the popular topics for

each framework, sorted from most popular to least

popular. From this table, it is evident that installation

is a major issue over both the frameworks. Installation

is the ﬁrst step of adopting a framework. Laravel has

several installation alternatives which may get con-

fusing for beginners. Django users also face trouble

with installation. So it is no wonder that it is one of

WEBIST 2018 - 14th International Conference on Web Information Systems and Technologies

196

Table 5: Common Issues between two frameworks.

Laravel Only Common Issues Django Only

Elixir Api Localization

Session Unit testing Server

Transaction View CSRF

Application Installation File Upload

Language Authentication Admin

Date time Documentaion App

Image DB/Model/Query/Migration Pep

MVC basic Form Template

Syntax General-Issues Encoding

the most popular issues for both the frameworks.

Among other issues of Laravel, we can infer from

the table that the framework is lacking in quality doc-

umentation or tutorial for new users. Moreover, the

changes between the version which are continuously

rolling for the betterment of the framework also create

confusion among the users. The beginners often ﬁnd

Laravel framework as their ﬁrst MVC pattern frame-

work and need to understand its underlying meaning

and implementation. So it is one of the most popular

topics with the highest number of question.

Similarly, we can infer that topics related to JSON

encoding, deployment and proper installment of a

server along with its possible errors and bugs are the

most popular topics related to Django framework.

4.4 RQ3

To answer our next research question, we attempted to

determine the difﬁculty level of the topics by associ-

ating with it the mean and median duration in minutes

of getting an accepted answer and ranking the topics

by median time. The topics for each of the frame-

work are presented in Table 4, sorted by their difﬁ-

culty measurements.

An interesting observation from the difﬁculty

ranking of the frameworks is the contrasting nature of

difﬁculty and popularity. More formally, Issues with

less popularity are difﬁcult, which is somewhat intu-

itive. For example, the top two difﬁcult issues of Lar-

avel are Unit Testing and Elixir which are also among

the least three popular issues of Laravel. The similar

phenomenon can also be observed in case of Django.

Another important ﬁnding by observing the two

tables is that issues related to installation are popular

as well as difﬁcult to answer. This somewhat indicates

that, regardless of any framework, users face prob-

lems while installation and the problems are also difﬁ-

cult to resolve. This denotes lack of well-documented

installation guideline or confusing installation process

for both the frameworks.

4.5 RQ4

Having observed all the issues for both of the frame-

works, we separated the issues in 3 columns as shown

in Table 5. We ﬁnd that about 50% of the issues are

common in both frameworks and the other 50% spans

over the unique issues for individual frameworks.

5 CONCLUSION AND FUTURE

WORK

Stack Overﬂow is one of the largest question-answer

sites for the programmers and developers. We en-

deavored to bring insight into the web framework re-

lated question in Stack Overﬂow. We extracted bodies

from questions of Laravel and Django and used topic

modeling to ﬁnd the topics. We ranked the topic ac-

cording to popularity and difﬁculty and determined

the common topics between these two. We found that

the most popular topics in Laravel are installation,

documentation, general issues, MVC basic, database,

session, syntax, date-time, form. The popular top-

ics in Django are encoding, installation, server, ad-

min, Cross-Site-Request-Forgery, migration, general

issues, app, and view. We found that the topics in two

frameworks are about 50% similar and both frame-

works have 50% of the topics as their unique topics.

In this way, we made a comparison between these two

frameworks that would help the developers to further

better their product and the developer community will

also be helped through establishing a statistical way to

ﬁnd what web developers are talking about.

In future, we want to conduct an empirical study

involving developers from industry as a complemen-

tary of this study. Both Laravel and Django are

an open-source framework, and their source code is

available on Github for feedback, bug report, etc. We

want to analyze with Github statistic of the frame-

works’ repositories to ﬁnd more about what the com-

munity is talking and contributing to these frame-

Mining Developer Questions about Major Web Frameworks

197

works. We want to extend our research to other frame-

works. Determining these frameworks’ issues will

enable us to conduct a comparative analysis, and we

would be able to establish a list of common issues

comprising almost all of the web frameworks. We

would also like to conduct a study which will pre-

dict the activity level of the frameworks in future,

e.g., how many users will keep using a framework or

switch to others. The impact of the version change on

the users can also be a signiﬁcant study in this area.

REFERENCES

Allamanis, M. and Sutton, C. A. (2013). Why, when, and

what: analyzing stack overﬂow questions by topic,

type, and code. In Proceedings of the 10th Working

Conference on Mining Software Repositories, MSR

’13, San Francisco, CA, USA, May 18-19, 2013, pages

53–56.

Authors (2018a). Django Set of Words. [Online; accessed

May 2018].

Authors (2018b). Laravel Set of Words. [Online; accessed

May 2018].

Bajaj, K., Pattabiraman, K., and Mesbah, A. (2014). Mining

questions asked by web developers. In 11th Working

Conference on Mining Software Repositories, MSR

2014, Proceedings, May 31 - June 1, 2014, Hyder-

abad, India, pages 112–121.

Barua, A., Thomas, S. W., and Hassan, A. E. (2014). What

are developers talking about? an analysis of topics

and trends in stack overﬂow. Empirical Software En-

gineering, 19(3):619–654.

Beyer, S. and Pinzger, M. (2014). A manual categorization

of android app development issues on stack overﬂow.

In 30th IEEE International Conference on Software

Maintenance and Evolution, Victoria, BC, Canada,

September 29 - October 3, 2014, pages 531–535.

DBLP (1993). DBLP Computer Science Bibliography. [On-

line; accessed May 2018].

Exchange, S. (2017a). Stack Exchange Data Dump. [On-

line; accessed August 2017].

Exchange, S. (2017b). Stack Exchange Data Explorer. [On-

line; accessed August 2017].

Fern

andez-Conde, C. and Gonz

alez-Calero, P. A. (2002).

Domain analysis of object-oriented frameworks in

framedoc. In Proceedings of the 14th international

conference on Software engineering and knowledge

engineering, SEKE 2002, Ischia, Italy, July 15-19,

2002, pages 27–33.

Ginige, A. (1998). Web engineering: Methodologies for

developing large and maintainable web based infor-

mation systems. In In: Proceedings of the IEEE Inter-

national Conference on Networking the India and the

World CNIW’98, Ahmeddabad.

Github (2018a). Github Repository of Django. [Online;

accessed May 2018].

Github (2018b). Github Repository of Laravel. [Online;

accessed May 2018].

Github (2018c). Github Repository of Vue JS. [Online;

accessed May 2018].

Github (2018d). List of web frameworks. [Online; accessed

May 2018].

Hevner, A. R., Linger, R. C., and Walton, G. H. (2007).

Next-generation software engineering introduction to

minitrack. In 40th Hawaii International International

Conference on Systems Science (HICSS-40 2007),

CD-ROM / Abstracts Proceedings, 3-6 January 2007,

Waikoloa, Big Island, HI, USA, page 276.

Li, H., Xing, Z., Peng, X., and Zhao, W. (2013). What help

do developers seek, when and how? In 20th Work-

ing Conference on Reverse Engineering, WCRE 2013,

Koblenz, Germany, October 14-17, 2013, pages 142–

151.

MALLET (2017). MALLET. [Online; accessed August

2017].

Mamykina, L., Manoim, B., Mittal, M., Hripcsak, G., and

Hartmann, B. (2011). Design lessons from the fastest

q&a site in the west. In Proceedings of the Interna-

tional Conference on Human Factors in Computing

Systems, CHI 2011, Vancouver, BC, Canada, May 7-

12, 2011, pages 2857–2866.

Meta, S. E. (2017). Database Schema of Stack Overﬂow

Data. [Online; accessed August 2017].

Olanrewaju, R. F., Islam, T., and Ali, N. (2015). An Em-

pirical Study of the Evolution of PHP MVC Frame-

work, pages 399–410. Springer International Publish-

ing, Cham.

Rosen, C. and Shihab, E. (2016). What are mobile develop-

ers asking about? A large scale study using stack over-

ﬂow. Empirical Software Engineering, 21(3):1192–

1223.

Treude, C., Barzilay, O., and Storey, M. D. (2011). How do

programmers ask and answer questions on the web?

In Proceedings of the 33rd International Conference

on Software Engineering, ICSE 2011, Waikiki, Hon-

olulu , HI, USA, May 21-28, 2011, pages 804–807.

WEBIST 2018 - 14th International Conference on Web Information Systems and Technologies

198