Trends and Issues in STEM + C Research: A Bibliometric

Perspective

Hanxiang Du

, Wanli Xing

, Bo Pei

, Yifang Zeng

, Jie Lu

and Yuanlin Zhang

College of Education, University of Florida, Gainesville, Florida, U.S.A.

College of Education, Texas Tech University, Lubbock, Texas, U.S.A.

Department of Computer Science, Texas Tech University, Lubbock, Texas, U.S.A.

Keywords: STEM + C, Computational Thinking, Computing Education.

Abstract: The integration of computing education or computational thinking with STEM majors has gained substantial

research interests. A number of research papers of the topic were published. This work is to provide a

comprehensive overview of literature in the STEM + C field through both bibliometric and content analysis.

We conducted a systematic search to identify articles and utilized machine-learning-based techniques to

analyze these articles. Common bibliometric indicators were used for bibliometric analysis. Machine-

learning-based text mining techniques such as LDA topic modelling and flow analysis were used for content

analysis. Our analysis spotted STEM + C publication trends, popular topics and their dynamics over time.

This work also pinpointed commonly used methodologies for integration study for both PK, K–12 and higher

education. Meanwhile, several future research directions were identified. This work contributes to the

literature by systematically examining the existing literature and bringing machine-learning-based data

mining techniques to the analysis.

1 INTRODUCTION

STEM + C, defined as a field of “science, technology,

engineering, mathematics (STEM) and computing”,

is also interpreted as an integration of computational

thinking (CT) to STEM disciplines by National

Science Foundation (NSF, 2020). The integration

practice of CT into STEM disciplines has drawn

increasing research interests over the years. Despite

the discrepancies in the definition of and elements of

CT among various research (NRC, 2010), STEM + C

has remained an active research field in the past

decade. However, to our best knowledge, no research

has explored the existing literature on the topic using

quantitative methods. Bibliometric analysis is used

for the study of qualitative features and research

performance, especially for large quantities of

publications (Wallin, 2005). By conducting a

bibliometric and content analysis on the field of

STEM + C, this paper aims to provide valuable

references on existing literature to researchers and

potential topics for future work.

https://orcid.org/0000-0002-9081-0706

https://orcid.org/0000-0002-1446-889X

1.1 What Is STEM + C?

There is much discussion on what STEM, computing

and CT are for. Although no common agreement has

been achieved on their definition, we provide our

perceptions and rationales before diving into the field.

The goal here is not to exhaust various definitions of

the terms, but to clarify the scope of our work. What

the acronym STEM stands for is quite clear: Science,

Technology, Engineering and Math, as are its

alternative versions STEAM and STREAM, which

include the Arts and Reading respectively. However,

agreement has not yet been reached about what the

four letters mean when strung together. Among many

perspectives, adopted definitions assume STEM to be

one or more of the four isolated subjects, or an

integrated continuum of multidisciplinary elements

(Bybee, 2010; Kelley & Knowles, 2016). Even within

the integrated STEM field, the discussion on the

relationships and conceptual frameworks for learning

among science, technology, engineering and

Du, H., Xing, W., Pei, B., Zeng, Y., Lu, J. and Zhang, Y.

Trends and Issues in STEM + C Research: A Bibliometric Perspective.

DOI: 10.5220/0010998800003182

In Proceedings of the 14th International Conference on Computer Supported Education (CSEDU 2022) - Volume 1, pages 69-80

ISBN: 978-989-758-562-3; ISSN: 2184-5026

mathematics remains unresolved (English, 2016;

Kelley & Knowles, 2016).

The NSF’s definition on STEM + C injects C

with two components: computing education and CT

education. This definition explicitly indicates that

computing is a concept different from, but related to

CT. Computing represents an integrated field of

computer science, information science, computer

engineering and information technology (Wing,

2008). According to a report that explores how

computing is taught worldwide, computer science and

information technology are categorized as disciplines

of the field of computing (Jones, 2011). Similarly,

computing education is considered to be a broad term

that may include one or more of the following areas:

computer science, technology literacy or fluency, and

information technology (Krakovsky, 2010).

Meanwhile, the term “computing” is often used

interchangeably with “computation” and

“computational” in the context of programming or

calculations (Psycharis, 2018), as are “computing

education” and “computer science education”

(Garneli et al., 2015). The goal of this work is to

investigate literature of the entire STEM + C field

using bibliometric analysis methods. Hence, we will

take computing education as a broad term which

includes, but is not limited to, computing education.

Jeannette Wing (2006) argued that CT

represents “a universally applicable attitude and skill

set” that involves problem solving, system design and

understanding human behaviour, and that CT will

benefit everyone rather than solely computer

scientists. Conceptualizing CT with a focus on

problem solving has both advocates and critics (Barr

& Stephenson, 2011; Bundy, 2007; Glass, 2006; The

Royal Society, 2012). Despite the discrepancy on

CT’s definition, conceptual framework, and key

elements, most literature does not question that CT is

a skill related to computing or programming practice.

Evidently, CT does not equal program writing. In

particular, CT is generally considered to be a different

skill from programming, while programming is

commonly used to teach CT (Lye & Koh, 2014). As

discussions on CT’s definition and components

continue, this work keeps an open mind on related

publications and their adopted conceptual

frameworks.

This work considers multiple perspectives on

STEM and CT to be legitimate providing that they are

in the context of the aforementioned core disciplines.

Similarly, this work takes computing education in its

broad form which includes computing education,

information technology, information literacy and

their variations.

1.2 Research on STEM + C

Reviews of literature on CT, computing education,

and their integration with one or more isolated STEM

subjects have been conducted. Grover & Pea (2013)

went through various definitions of CT and illustrated

how they were related to the idea of “computational

literacy” or “procedural literacy” in past decades.

They also examined research and educational practice

in CT or computing education, however, STEM did

not appear as a requirement to the integration

practice. Many studies they mentioned aimed to

prompt CT through computing education, rather than

STEM. Garneli et al. (2015) examined 47 peer-

reviewed articles on K–12 computer science

education with a focus on educational contexts and

efficient instructional tools and practices. Upadhyaya

et al. (2020) collected over 500 publications on K–12

CT research in the USA and conducted a longitudinal

analysis of publications from 2012 to 2018. Their

focus was to provide a general description of the

current status of computing education, including

curriculum content, grade levels, and the way in

which computing education was delivered. Similarly,

several other studies investigated related publications

focusing on either computing education (Robins et

al., 2003), or CT from multiple perspectives like

instructional tools and practice, conceptual

frameworks, and assessments (Hsu et al., 2018;

Kalelioglu et al., 2016; Tang et al., 2020; Zhang &

Nouri, 2019).

Several review studies were conducted to

explore the relationship between CT and mathematics

and/or science. Weintrop et al. (2016) framed CT in a

science and mathematics context by identifying a

taxonomy of four categories. They reviewed

discussions on CT’s definition and its crucial

connection to science and mathematics learning, as

well as CT-promoting practices at K–12 schools. In

addition to a practical taxonomy, their work provided

a solid conceptual framework for future research.

Barcelos et al. (2018) collected 42 publications that

had an experimental design specifically aimed at

developing CT skills through mathematics learning

activities. They reported a systematic analysis on

instructional tools and materials, experimental

designs, assessments, and reported achievements.

Hickmott et al. (2018) searched 6 databases and

identified 393 peer-reviewed articles on CT in K–12

education, then classified results into five categories

based on mathematical concepts like algebra or

geometry. They found that most studies were from the

domain of computer science and focused more on

CSEDU 2022 - 14th International Conference on Computer Supported Education

programming skills rather than mathematics

concepts.

Existing works linking STEM to CT mainly

focus on conceptual frameworks or pedagogical

strategies, and most of them are empirical studies.

Jona et al. (2014) suggested an alternative strategy to

improve students’ engagement and sustention in

computer science by embedding CT activities within

their ongoing STEM coursework. Similarly, Swaid

(2015) proposed a comprehensive project to integrate

CT into STEM by enforcing CT elements in STEM

gate-keeping courses like the introductory level

courses of STEM and computer science. Leonard et

al. (2016) designed learning activities to integrate

technology with CT utilizing robotics and game

design. Psycharis (2018) outlined various research

and practices for STEAM integration. Although the

role of CT, computing education, and their integration

was discussed, the goal was to support their proposed

model: Computational STEAM Pedagogy. Their

research was more qualitative than quantitative.

1.3 Research Goals

Bibliometric analysis uses statistical analysis to

systematically extract measurable features from

publications within a field (Agarwal et al., 2016). By

utilizing various bibliometric indicators and different

methodologies, bibliometric analysis can help assess

the impact of research, measure the importance of

publications, as well as decompose the evolution of a

research topic, and identify potential research topics

(Agarwal et al., 2016; Song et al., 2019). It has been

shown as a reliable and useful tool to overview the

existing literature of a research field (Campbell et al.,

2010). However, bibliometric analysis has also been

criticized for its exclusion of content (Hung, 2012).

To address this issue, we extend this work by

enabling data-based content analysis to provide a

more comprehensive and systematic overview of

STEM + C literature. Our work addresses the

following research questions (RQ):

RQ 1. What are the current trends, popular topics

and their dynamics in STEM + C research?

RQ 2. What is the role of CT or computing

education in STEM + C research?

RQ 3. What potential research directions shall be

addressed based on current literature?

2 DATA

For the purpose of investigating the whole STEM + C

field of work, we take STEM, CT, and computing

education in their broad terms and do not exclude

articles based on discrepancies with one specific

definition. The search terms are defined as a

combination of x AND y AND z, as shown in Table

1. Both x and y are used to search article titles,

representing key terms of STEM and C, respectively.

Meanwhile, z is used to search within abstracts for

educational articles where applicable. To search

efficiently, “computer” was excluded from y: if

“computer” was included in y, then it would form a

combination “computer science” with “science” from

x. Even with restrictions on abstracts, “computer

science” would result in a dramatically large and

unnecessary number of results. The term “computer

science” or “CS” was excluded for this same reason.

Table 1: Search terms.

Field Key Terms

x Title STEM OR science OR technology

OR engineering OR math OR

biology OR chemistry OR physics

y Title computational thinking OR

programming OR computing

z Abstract learn OR course

Figure 1 presents the data retrieval that consists of

a two-round search. In the first round, we

systematically searched three academic databases:

Web of Science, IEEE Xplore, and ACM Digital

Library. Publications were considered if they were:

(1). Articles written in English, including journal

articles and conference proceedings, excluding

dissertations, books, or book chapters. (2). Included

content and topics falling into the STEM + C field

with a focus on integration practice, whether they

were empirical studies or not. Based on search terms

and restrictions, 2855 records were retrieved and

manually filtered by checking the content of titles and

abstracts. As a result, 56 records were kept after the

first-round search.

Figure 1: Data retrieval process.

Trends and Issues in STEM + C Research: A Bibliometric Perspective

Web of Science is a well-known database of high-

quality academic work and widely used to search

articles for bibliometric analysis. However, the

journals it links to are selected by humans. Google

Scholar is a scholarly search engine that connects to

the entire Internet, covering more valuable records

that cannot be found on Web of Science (Kiduk &

Meho, 2006). However, due to the computational

settings of the search engine, the search results can be

different even one uses the same search term. In

addition, search results from Google Scholar cannot

be exported systematically for further processing. To

maintain the systematic nature and consistency of the

searching process, Google Scholar was only involved

in the second-round search.

The goal of the second-round search, or

supplement search, is to maximize the retrieval of

related work which may have been neglected in the

first-round search. First, in addition to search terms,

we also used “STEM + C” in Google Scholar.

Different from prior practice, both searching and

filtering were conducted at the same time. Second,

forward and backward tracing was conducted to

supplement the search. We checked articles that cited

the filtered results, as well as those cited by them. As

a matter of fact, more records were identified at this

stage. Our speculation is that those article titles do not

necessarily meet the identified key terms

combinations. Instead, more common terms were

used like school, students, or education. However, if

we used isolated search terms, including the ones

mentioned, searched results may be too general to be

an efficient search. As a result, 202 publications were

finalized for this study.

3 METHODS

3.1 Bibliometric Indicators

Several popular bibliometric indicators were used in

this work to measure the impact of collected articles.

Publication and citation count are commonly used

indicators to assess productivity and influence. Two

thresholds of total citation count were used to

measure the influence of an author: 100 and 300

(Merigó et al., 2015). Meanwhile, the h-index which

measures the level of scientific achievement was also

included. The h-index used in this work was collected

from author’s personal page on Google Scholar.

3.2 Sleeping Beauties in Science

Citations are commonly used to evaluate scholarly

articles’ impact and research performance. Citation

dynamics and quoted papers describe the

dissemination trajectories of research articles.

Sleeping Beauties analysis is one way to examine the

citation history of papers. Sleeping Beauties in

science refer to articles that were not recognized until

years later after publication (van Raan, 2004). Li

(2014) proposed a parameter-free criterion to assess

the imbalance of citation distribution and later was

used to identify Sleeping Beauties in science. Let C

be the total number of citations, and c

(i ∈ {1, 2, …

, n}) be the number of citations received in the ith

year. Gs is an adjustment of Gini coefficient and

defined as:

𝐺𝑠 = 1 −

×[×



()×



...



]

 × 

,𝐶 > 0 (1)

, where Gs ∈ (-1, 1]. When the article receives a total

citation of 0, Gs is 1. Otherwise, the higher Gs is, the

more citations one article receives in its later years.

3.3 Textual Data Pre-Processing

Data processing is necessary as it systematically and

automatically helps trim and clean the textual data by

eliminating redundant information. As a result, the

textual data will be presented as more structured and

relevant, and its meaningful structures can be

captured. Several commonly used textual data

processing techniques are involved in this work.

Special characters and punctuation are removed.

Commonly used words across fields that carry little

information like “a”, “the”, and “of” are removed as

well using a stopwords package. Tokenization divides

a string into several substrings for future pre-

processing. Lemmatization and stemming are

commonly used techniques to reduce inflectional

forms of terms. For example, “books”, “book”,

“book’s” and “books’” will be mapped to “book”.

The lemmatizer and Porter stemmer package are used

in this process.

3.4 Keywords Flow Analysis

Using textual data mining techniques, word flow

analysis is employed to present the keyword

dynamics over time (Du et al., 2019). We define the

keywords as terms used repeatedly in abstract. Term

frequency of keywords over time are then calculated.

The results are presented in a flow chart that provides

a general overview of keyword dynamics over time.

CSEDU 2022 - 14th International Conference on Computer Supported Education

The keywords flow will help identify popular

research directions.

3.5 Latent Dirichlet Allocation (LDA)

Latent Dirichlet Allocation (LDA) is a widely used

unsupervised machine learning algorithm for finding

the relationship between documents and words in

textual data (Blei et al., 2003). LDA is able to

generalize a number of topics from given documents

with each topic represented by a few words. The

summarization capability makes LDA powerful and

convenient for key feature extraction from large-size

textual datasets. It has been broadly applied to various

research scenarios, including social media content

classification, bibliometric analysis, and scientific

article recommendation (Blei et al., 2003; Iqbal et al.,

2019). Coherence score, which measures how well a

topic model fits the data set, is used to decide the

number of topics. A higher coherence score means

results of such a model represent the documents better

than a lower one. We will also use abstracts of

collected articles to train an LDA topic model. All

analysis for this work is implemented in Python.

4 RESULTS

4.1 Trend & Prolific Analysis

We identified 202 articles with 512 distinctive

authors of 187 institutions from 29 countries/regions

worldwide. By March 2020, these 202 articles have

5590 total citations.

The annual publication and citation count are

presented in Figure 2, where each data point stands

for the number of publications or citations within a

specific year. Beginning in 2008, there was an

increasing trend for both the publication and citation

count despite some fluctuation, indicating growing

research interest towards the field. Since we only

collected work that was available by February, 2020,

it leads to a significant drop in year 2020 on both

lines. In addition, the USA is the most prolific country

with 156 publications, which includes more than 2/3

of the total publications.

In order to view the background distribution,

authors’ affiliated majors were categorized into four

categories: Education, Computer Science, STEM and

Other. The distribution is presented in Figure 3.

Authors with an education background rank first,

closely followed by those with a computer science

background. Meanwhile, 8.8% of authors come from

STEM majors. This indicates that more researchers in

the field are from computer science or STEM majors

rather than education.

4.2 Highly-cited Publications and

Sleeping Beauties

We calculated the adjusted Gini coefficient (Gs) to

measure the imbalance of citation history of identified

highly-cited articles. Table 3 lists articles whose total

citation count is one standard deviation higher the

mean (M = 27.98, SD = 83.60) and their Gs values.

Considering the highest Gs is less than 0.50, it is fair

to say that most highly-cited publications of STEM +

C receive immediate recognition.

Figure 2: Annual publication and citation trend.

Trends and Issues in STEM + C Research: A Bibliometric Perspective

Figure 3: Author affiliated majors’ distribution.

4.3 Keywords Flow Analysis

Figure 4 presents the identified keywords by term

frequency and Figure 5 shows keywords flow over

time. Terms of high frequency yet low information to

our analysis like “education”, “use”, “school”,

“learn” and “integrate” were removed. The term

Table 3: Highly-cited articles and their Gs value.

Articles Citation

The learning effects of computer simulations in

science education

673

Thinking like a wolf, a sheep, or a firefly:

Learning biology through constructing and

testing computational theories—an embodied

modelling approach.

564

Defining computational thinking for

mathematics and science classrooms

406

Computational thinking and tinkering:

Exploration of an early childhood robotics

curriculum

397

Development of system thinking skills in the

context of earth s

stem education

321

Integrating computational thinking with K

–

science education using agent-based

com

utation: A theoretical framewor

283

Computational thinking in K

–

9 education

253*

Visual programming languages integrated

across the curriculum in elementary school: A

two-year case study using “Scratch” in five

schools

225

Computational thinking in compulsory

education: Towards an agenda for research and

ractice

209*

A multidisciplinary approach towards

computational thinking for science majors

166

Designing for deeper learning in a blended

computer science course for middle school

students

142*

Supporting all learners in school-wide

computational thinking: A cross-case

ualitative anal

sis

122

*Gs >.40

“CT” was calculated under the term “computational

thinking”. “Computational thinking” attracts the most

research interest over time. Computing-related terms

like “programming”, “modelling”, “simulation”,

“concept”, and “data” are also listed, indicating

various aspects of CT or computing education were

explored and discussed. Individual disciplines like

science and math, as well as the acronym STEM, all

received increasing attention over the years.

However, research interest in engineering is rather

limited in comparison with others. Meanwhile,

“teachers” is mentioned far less than “students” in the

field, indicating most research focuses on students.

Figure 4: 20 Keywords identified by term frequency.

4.4 LDA Topic Modelling

We first evaluate the performance of our model by

calculating the coherence score with topic numbers

ranging from 1 to 15, setting the model parameter

passes at 50 and random_state at 1. Suggested by c-

oherence score, model has the highest interpretability

when the topic number equals 4. Figure 6 shows the

results of our trained LDA topic model and the topic

distribution. The numbers in Figure 6(a) are the

probability distribution over the topic words. The

topic distribution result is displayed in Figure 6(b).

Some terms are common across topics with

different probabilities, like “student”, “ct”, “comput”,

“scienc” and “school”. This is common as topics are

not mutually exclusive (DiMaggio, 2013). Several

terms are unique to only one or two topics, like

“mathemat”, “program”, “model”, “teacher”, “effect”

and “transfer”. Based on the topic words, we can

summarize Topic 1 as mathematics/STEM research,

Topic 2 as programming learning, Topic 3 as teacher

and integration development, and Topic 4 as learning

outcomes. As shown in Figure 6(b), more than half of

the articles focused on a programming learning

perspective of STEM + C research, around one third

of articles discuss mathematics and STEM learning

CSEDU 2022 - 14th International Conference on Computer Supported Education

and research, while only a few studies address on

teachers’ perspectives and others.

5 CONCLUSION & DISCUSSION

5.1 What Are the Current Trends,

Popular Topics and Their

Dynamics in STEM + C Research?

The number of publications and citations in STEM +

C consistently grow since 2007, and have increased

rapidly for the last 5 years. The USA has contributed

dramatically more publications and received more

citations than any other country/region. Almost half

of the authors are from CS or STEM majors, while

around 40% are from education, indicating STEM +

C has drawn interest from a wide range of majors.

Sleep Beauty analysis suggests work of great

importance has received immediate recognition.

LDA topic modelling was used to identify topics:

mathematics/STEM research, programming learning,

teacher and integration development, and learning

outcomes. Most articles in the field focus on the first

two topics, while the other two are less discussed. In

particular, more than half of the studies focus on the

programming learning, indicating that the most

popular way to conduct STEM + C research is to

integrate programming practice into STEM learning.

Prompting integration of CT through mathematics or

STEM learning has also gained popularity. More than

one third of the collected articles are assigned with

the topic of mathematics/STEM research. The

percentage explicitly suggests the importance of

mathematics in the field.

While the LDA topic modelling sees data on the

document level, a term level analysis, keywords flow,

is conducted to provide a dynamic and

comprehensive view. The term “computational

thinking” (including “CT”) is the single most frequent

keyword and its frequency is about 3 times that of the

second term, “student”. The frequency of “student” is

about 3 times that of “teacher”, indicating that many

studies focus on students’ learning perspectives.

Researchers seem to prefer “science” and “math”

over “engineering” or “technology”. In particular,

both “science” and “math” are more frequently

mentioned than “STEM”. “Engineering” ranks 19th

Figure 5: A general keywords flow of all keywords over time: 1995 – 2020.

Trends and Issues in STEM + C Research: A Bibliometric Perspective

Figure 6: LDA results (a) and the topic distribution (b).

mentioned than “STEM”. “Engineering” ranks 19th

and “technology” is not listed at all. This indicates

that attention is not equally distributed within STEM

disciplines. Meanwhile, several terms are explicitly

computing education related: “programming”,

“modeling”, “simulation” and “data”. These can be

viewed as popular computing education components

in STEM + C research. Among these four, scholars

favour “programming” the most. “Data” seems to

have received some attention only in the past 5 years.

5.2 What is the Role of CT or

Computing Education in

STEM + C Research?

CT has been commonly accepted as an effective

strategy to benefit and advance STEM learning in the

research community (Assaraf & Orion, 2005;

Hambrusch et al., 2009; Jona et al., 2014; Perković et

al., 2010; Swaid, 2015; Weintrop et al., 2016). It is

intuitive to develop CT through programming.

Meanwhile, programming is the most commonly

used way to teach CT (Lye & Koh, 2014). An

alternative way to integrate CT is through

computation concepts, modelling and simulations,

especially for young children (Assaraf & Orion,

2005; Bers et al., 2014; Sáez-López et al., 2016;

Sengupta et al., 2013; Wilensky & Reisman, 2006).

To summarize, programming practice, computation

concepts, modelling and simulation, and data

manipulation are commonly involved in STEM + C

research.

One shared interest in STEM + C research is the

development of a CT framework that can be widely

applied across disciplines in K–12 or higher

education (Hambrusch et al., 2009; Jona et al., 2014;

Perković et al., 2010; Sengupta et al., 2013; Swaid,

2015; Weintrop et al., 2016). Several goals are

extensively addressed by these works, one is to

advance STEM learning with help of CT and prepare

the next generation to be modern citizens (Hambrusch

et al., 2009; Jona et al., 2014; Perković et al., 2010;

Sengupta et al., 2013; Weintrop et al., 2016).

Meanwhile, embedding CT with current K-12 STEM

courses is considered as an alternative solution to K-

12 schools’ inability to offer computer science or

programming classes (Jona et al., 2014; Weintrop et

al., 2016). Works focusing on framework

development can be roughly classified into 2

categories: K–12 or college level education.

College-level practices of STEM + C framework

development often take the form of developing a new

course with joint efforts across disciplines. These

courses are designed for early years of college

education, and mostly involve programming. Several

studies on this provided detailed course descriptions.

Meanwhile, data manipulations, programming

concepts, and simulations are widely adopted by

several frameworks to help students learn scientific

inquiry, STEM gate-keeping courses, and general

courses like Liberal Studies (e.g., Hambrusch et al.,

2009; Perković et al., 2010; Swaid, 2015).

K–12 STEM + C framework development mostly

attempt to embed CT in current STEM courses.

However, programming is not always involved. This

strategy potentially saves schools from the financial

concerns of hiring new teachers and supporting new

courses (Jona et al., 2014; Weintrop et al., 2016).

Meanwhile, all students are required to take STEM

courses in school, the integration will expose a much

wider range of population than a specific course does

(Jona et al., 2014; Weintrop et al., 2016). In

particular, Perković et al. (2010) proposed an agent-

based learning environment for science learning and

modeling. In addition to proposed framework, they

CSEDU 2022 - 14th International Conference on Computer Supported Education

also specified the computational architecture

underlying the learning environment. Their work also

conducted an empirical study using the developed

tools. All these provide valuable reference to future

development. Weintrop et al. (2016) focused on

embedding CT in STEM courses for traditional

classrooms. Through examining literature, practice,

and interviewing teachers, STEM experts and

computer scientists, they define CT through a

taxonomy. They examined developed skills and

generalized four categories: data, modelling and

simulation, computational problem solving, and

systems thinking. The taxonomy provides significant

reference to future research and course development.

Although calls to computing education have

received substantial advocates in the past years, the

lack of qualified teachers, budgets, and standards

barricades its popularization in K–12 education

(Israel et al., 2015; Wang et al., 2016). Addressing CT

provides a feasible solution to many situations. First,

embedding CT within students’ current STEM

workload guarantees students’ exposure while

assuring the teacher’s comfort with learning materials

(Jona et al., 2014). Such practice requires less budget,

expertise and effort than developing and supporting a

new computer science course. Second, learning

programming is not easy, especially for young

children. Learning CT lowers the threshold

significantly. Although the youngest group of

learners are kindergarten children (Bers et al., 2014),

most of the work targets students in 5th grade or

higher. Block-based programming is preferred when

programming is involved. Third, CT addresses

adaptability to STEM courses, which takes form in

programming, data manipulation, modelling and

simulation, or systems thinking (e.g., Assaraf, 2005;

Wilensky & Reisman, 2006).

5.3 What Potential Research Directions

Shall Be Addressed based on

Current Literature?

There are several potential research topics in addition

to what has been identified as popular in the past.

First, from a disciplinary perspective, more efforts

can be made to explore engineering and technology.

Some college-level learning activities are designed to

utilize CT to solve engineering problems (Hambrusch

et al., 2009; Perković et al., 2010; Swaid, 2015).

However, this engineering context is much less

addressed in K–12 education. Second, there is a

potential lack of research on teachers’ professional

development or training. The work of Israel (2015) is

one of the few works that focuses on teacher’s

professional development. This qualitative study

reveals K–12 teachers’ concerns and needs to teach

CT in K–5 classrooms. Third, community college

seems absent from the current research scope. It

remains unclear how the college-level STEM + C

courses can be adopted by community colleges.

Fourth, “data” did not receive much research

attention until the past five years. Related activities

like data literacy, data science, data manipulation

worth more investigation.

5.4 Conclusion

Bibliometric analysis is useful to investigate and

explore existing literature in the field of STEM + C.

Based on 202 identified publications collected from

Web of Science, IEEE Xplore, ACM Digital Library

and Google Scholar, this work presents a

comprehensive overview of the field by showing

publication trends, identifying prolific

countries/regions, institutions and authors,

visualizing collaborations among countries/regions,

institutions and authors, generalizing content-based

topics, recognizing research keywords, popular

research fields and understudied research

perspectives.

There are a few interesting findings. The number

of publications and citations in STEM + C have

consistently grown since 2007, and have increased

rapidly for the last 5 years. STEM + C as an

interdisciplinary field has drawn interest from a wide

range of majors. We anticipate there will be more

publications in the future. Meanwhile, mathematics

has the highest frequency among the four STEM

subjects, making it the most popular STEM subject

for existing STEM + C research practice. When it

comes to computing education, the terms like

“programming”, “modelling”, “simulation” and

“data” have rather high frequency. Among these four

aspects, scholars favour “programming” the most,

while “data” seems to start to receive attention in the

past 5 years only.

5.5 Limitation

Bibliometric analysis, by its nature, focuses on

numbers instead of content. Although we have

conducted both term-level and document-level

content analysis through a data mining method to

address the issue, this work did not fully review all

identified articles. Meanwhile, this work only

searched three databases and one search engine: Web

of Science, IEEE Xplore, ACM Digital Library, and

Google Scholar. Search terms were identified based

Trends and Issues in STEM + C Research: A Bibliometric Perspective

on our understanding of the field as well as search

efficiency. It will be helpful if future work can

identify more efficient search terms or mechanisms

within the field and explore more databases.

Meanwhile, our work did not fully examine other

expressions that are argued similar to CT, like

computational literacy or systems thinking, leading to

unidentified related articles.

ACKNOWLEDGEMENTS

This work is supported by the National Science

Foundation (NSF) of the United States under grant

number 1901704. Any opinions, findings, and

conclusions or recommendations expressed in this

paper, however, are those of the authors and do not

necessarily reflect the views of the NSF.

REFERENCES

Agarwal, A., Durairajanayagam, D., Tatagari, S., Esteves,

S. C., Harlev, A., Henkel, R., Roychoudhury, S., Homa,

S., Puchalt, N. G., Ramasamy, R., Majzoub, A., Dao

Ly, K., Tvrda, E., Assidi, M., Kesari, K., Sharma, R.,

Banihani, S., Ko, E., Abu-Elmagd, M., … Bashiri, A.

(2016). Bibliometrics: Tracking research impact by

selecting the appropriate metrics. In Asian Journal of

Andrology (Vol. 18, Issue 2). https://doi.org/10.4103/

1008-682X.171582

Assaraf, O. B. Z., & Orion, N. (2005). Development of

system thinking skills in the context of earth system

education. Journal of Research in Science Teaching,

42(5). https://doi.org/10.1002/tea.20061

Barcelos, T. S., Munoz, R., Villarroel, R., Merino, E., &

Silveira, I. F. (2018). Mathematics learning through

computational thinking activities: A systematic

literature review. In Journal of Universal Computer

Science (Vol. 24, Issue 7).

Barr, V., & Stephenson, C. (2011). Bringing computational

thinking to K-12: What is involved and what is the role

of the computer science education community? ACM

Inroads, 2(1). https://doi.org/10.1145/1929887.1929905

Bers, M. U., Flannery, L., Kazakoff, E. R., & Sullivan, A.

(2014). Computational thinking and tinkering:

Exploration of an early childhood robotics curriculum.

Computers and Education, 72. https://doi.org/10.1016/

j.compedu.2013.10.020

Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent

Dirichlet allocation. Journal of Machine Learning

Research, 3(4–5). https://doi.org/10.1016/b978-0-12-

411519-4.00006-9

Bundy, A. (2007). Computational Thinking is Pervasive.

Journal of Scientific and Practical Computing, 1(2).

Bybee, R. (2010). Advancing STEM Education: A 2020

Vision. Technology and Engineering Teacher, 70(1).

Campbell, D., Picard-Aitken, M., Côté, G., Caruso, J.,

Valentim, R., Edmonds, S., Williams, G. T., Macaluso,

B., Robitaille, J. P., Bastien, N., Laframboise, M. C.,

Lebeau, L. M., Mirabel, P., Larivière, V., &

Archambault, É. (2010). Bibliometrics as a

performance measurement tool for research evaluation:

The case of research funded by the national cancer

institute of Canada. American Journal of Evaluation,

31(1). https://doi.org/10.1177/1098214009354774

Carmichael, C. C. (2017). A state-by-state policy analysis

of stem education for K–12 public schools. Seton Hall

University.

DiMaggio, P., Nag, M., & Blei, D. (2013). Exploiting

affinities between topic modeling and the sociological

perspective on culture: Application to newspaper

coverage of US government arts funding. Poetics,

41(6), 570-606.

Du, H., Nguyen, L., Yang, Z., Abu-Gellban, H., Zhou, X.,

Xing, W., Cao, G., & Jin, F. (2019). Twitter vs News:

Concern analysis of the 2018 California wildfire event.

Proceedings - International Computer Software and

Applications Conference, 2. https://doi.org/10.1109/

COMPSAC.2019.10208

English, L. D. (2016). STEM education K-12: perspectives

on integration. In International Journal of STEM

Education (Vol. 3, Issue 1). https://doi.org/10.1186/

s40594-016-0036-1

Garneli, V., Giannakos, M. N., & Chorianopoulos, K.

(2015). Computing education in K-12 schools: A

review of the literature. IEEE Global Engineering

Education Conference, EDUCON, 2015-April.

https://doi.org/10.1109/EDUCON.2015.7096023

Glass, R. L. (2006). Call it problem solving, not

computational thinking [8]. In Communications of the

ACM (Vol. 49, Issue 9).

Grover, S., & Pea, R. (2013). Computational Thinking in

K-12: A Review of the State of the Field. In

Educational Researcher (Vol. 42, Issue 1).

https://doi.org/10.3102/0013189X12463051

Grover, S., Pea, R., & Cooper, S. (2015). Designing for

deeper learning in a blended computer science course

for middle school students. Computer Science

Education, 25(2). https://doi.org/10.1080/08993408.20

15.1033142

Hambrusch, S., Hoffmann, C., Korb, J. T., Haugan, M., &

Hosking, A. L. (2009). A multidisciplinary approach

towards computational thinking for science majors.

SIGCSE Bulletin Inroads, 41(1). https://doi.org/

10.1145/1539024.1508931

Hickmott, D., Prieto-Rodriguez, E., & Holmes, K. (2018).

A Scoping Review of Studies on Computational

Thinking in K–12 Mathematics Classrooms. Digital

Experiences in Mathematics Education, 4(1).

https://doi.org/10.1007/s40751-017-0038-8

Hsu, T. C., Chang, S. C., & Hung, Y. T. (2018). How to

learn and how to teach computational thinking:

Suggestions based on a review of the literature.

Computers and Education, 126. https://doi.org/

10.1016/j.compedu.2018.07.004

CSEDU 2022 - 14th International Conference on Computer Supported Education

Hung, J. L. (2012). Trends of e-learning research from 2000

to 2008: Use of text mining and bibliometrics. British

Journal of Educational Technology, 43(1).

https://doi.org/10.1111/j.1467-8535.2010.01144.x

Iqbal, W., Qadir, J., Tyson, G., Mian, A. N., Hassan, S. ul,

& Crowcroft, J. (2019). A bibliometric analysis of

publications in computer networking research.

Scientometrics, 119(2). https://doi.org/10.1007/s111

92-019-03086-z

Israel, M., Pearson, J. N., Tapia, T., Wherfel, Q. M., &

Reese, G. (2015). Supporting all learners in school-

wide computational thinking: A cross-case qualitative

analysis. Computers and Education, 82.

https://doi.org/10.1016/j.compedu.2014.11.022

Jona, K., Wilensky, U., Trouille, L., Horn, M., Orton, K.,

Weintrop, D., & Beheshti, E. (2014). Embedding

Computational Thinking in Science, Technology,

Engineering, and Math (CT-STEM). Future Directions

in Computer Science Education Summit Meeting, 2002.

Jones, S. P., Bell, T., Cutts, Q., Iyer, S., Schulte, C.,

Vahrenhold, J., & Han, B. (2011). Computing at school.

International comparisons. Retrieved May,7, 2013.

Kalelioglu, F., Gulbahar, Y., & Kukul, V. (2016). A

Framework for Computational Thinking Based on a

Systematic Research Review. Baltic Journal of Modern

Computing, 4(3).

Kelley, T. R., & Knowles, J. G. (2016). A conceptual

framework for integrated STEM education. In

International Journal of STEM Education (Vol. 3, Issue

1). https://doi.org/10.1186/s40594-016-0046-z

Krakovsky, M. (2010). CSEdWeek expands its reach.

Communications of the ACM, 53(12).

https://doi.org/10.1145/1859204.1859213

Leonard, J., Buss, A., Gamboa, R., Mitchell, M., Fashola,

O. S., Hubert, T., & Almughyirah, S. (2016). Using

Robotics and Game Design to Enhance Children’s Self-

Efficacy. STEM Attitudes, Compu(Journal of Science

Education and Technology).

Li, J. (2014). Citation curves of “all-elements-sleeping-

beauties”: “flash in the pan” first and then “delayed

recognition.” Scientometrics, 100(2). https://doi.org/

10.1007/s11192-013-1217-z

Li, J., & Ye, F. Y. (2016). Distinguishing sleeping beauties

in science. Scientometrics, 108(2). https://doi.org/

10.1007/s11192-016-1977-3

Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and

learning of computational thinking through

programming: What is next for K-12? In Computers in

Human Behavior (Vol. 41). https://doi.org/10.1016/

j.chb.2014.09.012

Merigó, J. M., Mas-Tur, A., Roig-Tierno, N., & Ribeiro-

Soriano, D. (2015). A bibliometric overview of the

Journal of Business Research between 1973 and 2014.

Journal of Business Research, 68(12). https://doi.org/

10.1016/j.jbusres.2015.04.006

NRC. (2010). Report of a Workshop on The Scope and

Nature of Computational Thinking. In Report of a

Workshop on The Scope and Nature of Computational

Thinking. https://doi.org/10.17226/12840

NSF. (2020). STEM + C call for proposals.

https://www.nsf.gov/funding/pgm_summ.jsp?pims_id

=505006. ([Online; accessed 15-June-2020])

Perković, L., Settle, A., Hwang, S., & Jones, J. (2010). A

framework for computational thinking across the

curriculum. ITiCSE’10 - Proceedings of the 2010 ACM

SIGCSE Annual Conference on Innovation and

Technology in Computer Science Education.

https://doi.org/10.1145/1822090.1822126

Psycharis, S. (2018). Steam in Education: A Literature

Review on the Role of Computational Thinking,

Engineering Epistemology and Computational Science.

Computational Steam Pedagogy (Csp). S. Psycharis

Scientific Culture, 4(2).

Robins, A., Rountree, J., & Rountree, N. (2003). Learning

and teaching programming: A review and discussion.

International Journal of Phytoremediation, 21(1).

https://doi.org/10.1076/csed.13.2.137.14200

Sáez-López, J. M., Román-González, M., & Vázquez-

Cano, E. (2016). Visual programming languages

integrated across the curriculum in elementary school:

A two year case study using “scratch” in five schools.

Computers and Education, 97. https://doi.org/10.1016/

j.compedu.2016.03.003

Sengupta, P., Dickes, A., & Farris, A. (2018). Toward a

phenomenology of computational thinking in STEM

education. In Computational Thinking in the STEM

Disciplines: Foundations and Research Highlights.

https://doi.org/10.1007/978-3-319-93566-9_4

Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., &

Clark, D. (2013). Integrating computational thinking

with K-12 science education using agent-based

computation: A theoretical framework. Education and

Information Technologies, 18(2). https://doi.org/

10.1007/s10639-012-9240-x

Swaid, S. I. (2015). Bringing Computational Thinking to

STEM Education. Procedia Manufacturing, 3.

https://doi.org/10.1016/j.promfg.2015.07.761

Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020).

Assessing computational thinking: A systematic review

of empirical studies. Computers and Education, 148.

https://doi.org/10.1016/j.compedu.2019.103798

The Royal Society. (2012). Shut down or restart? The way

forward for computing in UK schools. Technology,

January.

Tenenberg, J., & McCartney, R. (2014). Computing

education in (k-12) schools from across-national

perspective. ACM New York, NY, USA.

Upadhyaya, B., McGill, M. M., & Decker, A. (2020). A

longitudinal analysis of K–12 computing education

research in the united states: Implications and

recommendations for change. In

Proceedings of the

51st ACM technical symposium on computer science

education (pp. 605–611).

van Raan, A. F. J. (2004). Sleeping Beauties in science.

Scientometrics, 59(3). https://doi.org/10.1023/B:SCI

E.0000018543.82441.f1

Wallin, J. A. (2005). Bibliometric methods: Pitfalls and

possibilities. In Basic and Clinical Pharmacology and

Trends and Issues in STEM + C Research: A Bibliometric Perspective

Toxicology (Vol. 97, Issue 5). https://doi.org/10.1111/

j.1742-7843.2005.pto_139.x

Wang, J., Hong, H., Ravitz, J., & Hejazi Moghadam, S.

(2016, February). Landscape of K-12 computer science

education in the US: Perceptions, access, and barriers.

In Proceedings of the 47th ACM Technical Symposium

on Computing Science Education (pp. 645-650).

Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K.,

Trouille, L., & Wilensky, U. (2016). Defining

Computational Thinking for Mathematics and Science

Classrooms. Journal of Science Education and

Technology, 25(1). https://doi.org/10.1007/s10956-

015-9581-5

Wilensky, U., & Reisman, K. (2006). Thinking like a wolf,

a sheep, or a firefly: Learning biology through

constructing and testing computational theories - An

embodied modeling approach. In Cognition and

Instruction (Vol. 24, Issue 2).

https://doi.org/10.1207/s1532690xci2402_1

Wing, J. M. (2006). Computational thinking.

Communications of the ACM, 49(3),33–35.

Wing, J. M. (2008). Computational thinking and thinking

about computing. Philosophical Transactions of the

Royal Society A: Mathematical, Physical and

Engineering Sciences, 366(1881). https://doi.org/

10.1098/rsta.2008.0118

Zhang, L. C., & Nouri, J. (2019). A systematic review of

learning computational thinking through Scratch in K-

9. Computers and Education, 141. https://doi.org/

10.1016/j.compedu.2019.1

CSEDU 2022 - 14th International Conference on Computer Supported Education