Teaching Practice Using ChatGPT in Higher Education

Edna Dias Canedo

1 a

, Angelica Toffano Seidel Calazans

2 b

, Geovana Ramos Sousa Silva

1 c

Eloisa Toffano Seidel Masson

2 d

and F

abio L

ucio Lopes Mendonc¸a

3 e

University of Bras

ılia (UnB), Department of Computer Science, Bras

ılia, Brazil

University Center of Bras

ılia (UniCEUB), Bras

ılia, DF, 70910-900, Brazil

University of Bras

ılia (UnB), Electrical Engineering Department, Bras

ılia, DF, Brazil

Keywords:

ChatGPT, Teaching in Higher Education, Knowledge Sharing.

Abstract:

ChatGPT is an Artiﬁcial Intelligence (AI) chatbot platform developed by OpenAI. Several studies have high-

lighted the advantages and disadvantages of integrating ChatGPT into teaching methodologies and knowledge

generation within higher education. We conducted a survey involving 86 professors within the Computer Sci-

ence ﬁeld in Brazil. Our ﬁndings indicate that professors are utilizing ChatGPT for content generation and

the creation of teaching materials, including practical exercises, slides, assignments, and tests. Moreover, they

view ChatGPT as a potential facilitator of learning by fostering interaction between students and professors.

In the realm of knowledge production, professors are leveraging ChatGPT for tasks such as aiding in the com-

position of research papers or articles and generating automatic summaries. However, as per the professors’

perceptions, a notable limitation of ChatGPT is its inability to provide bibliographic references for the content

it delivers. Most professors believe that ChatGPT can be used as a support tool in higher education to gen-

erate knowledge. However, it is essential to address the challenges associated with the lack of bibliographic

references in the content provided by ChatGPT.

1 INTRODUCTION

ChatGPT is a chatbot platform with Artiﬁcial

Intelligence (AI) developed by OpenAI (Wu et al.,

2023). ChatGPT was made available to the public

on November 30th, 2022. It relies on a combination

of multilingual natural language and programming

languages to offer comprehensive and adaptive

responses (Zhuo et al., 2023). According to Teubner

et al. (Teubner et al., 2023a): OpenAI’s GPT-3

(Generative Pre-trained Transformer 3) is a premier

Large Language Model (LLM) and can handle a

wide range of natural language processing tasks

without any need for ﬁnetuning. Its largest variant

features 175 billion parameters and has been trained

on 570 GB of a wide range of text data, including

books, press articles, Wikipedia, blogs, and other

https://orcid.org/0000-0002-2159-339X

https://orcid.org/0000-0002-2783-4473

https://orcid.org/0000-0002-0304-0804

https://orcid.org/0000-0002-0409-058X

https://orcid.org/0000-0001-7100-7304

web content (300 billion words in total). As a result,

it can reliably produce texts that are read as if written

by humans.

ChatGPT achieved one million users in just ﬁve

days after its release (Chartr, 2022) and 100 million

users in two months (Paris, 2023). Thus, it has drawn

numerous users’ attention, who interact with the

platform and post comments on social media daily

(Zhuo et al., 2023). In the educational context, mainly

in higher education, studies present the advantages

and disadvantages of students and professors using

ChatGPT. Advantages are: chatGPT has a positive

impact on the teaching-learning process (Rueda

et al., 2023), it offers numerous opportunities for

professors and lecturers to develop ideas (Halaweh,

2023), (Qadir, 2023a), very useful as a support for

educational work (Rahman and Watanobe, 2023),

both students and faculty can beneﬁt from this

tool (Javaid et al., 2023), faculty can save time on

numerous tasks by using these technologies (Javaid

et al., 2023), students can use it as a support tool

(Javaid et al., 2023), this tool can increase student

engagement and satisfaction (Firat, 2023), It can

Canedo, E., Calazans, A., Silva, G., Masson, E. and Mendonça, F.

Teaching Practice Using ChatGPT in Higher Education.

DOI: 10.5220/0012725500003690

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 26th International Conference on Enterprise Information Systems (ICEIS 2024) - Volume 1, pages 541-552

ISBN: 978-989-758-692-7; ISSN: 2184-4992

541

enhance learning experiences and transform the

role of educators (Firat, 2023) , it offers learners

the chance to improve their language skills (Fauzi

et al., 2023), increases collaboration, time efﬁciency

and effectiveness (Fauzi et al., 2023), variations in

translation quality among high-resource languages

(Zhuo et al., 2023), generation of socially safe and

unbiased responses in open-ended dialogues (Zhuo

et al., 2023), and so on.

The disadvantages are the need for professors

to be trained to use the tool properly (Rueda et al.,

2023), (Halaweh, 2023), (Rahman and Watanobe,

2023), (Javaid et al., 2023), lack of common sense in

its use and others Difﬁculties in complex reasoning

(Rahman and Watanobe, 2023), Many students

use ChatGPT to manage and deliver their work

without learning anything (Castillo et al., 2023), it

is important to use these tools with caution, as they

can be misleading (Qadir, 2023a), as inadequate

comprehension of low-resource languages (Zhuo

et al., 2023), the possibility of bypassing safety

features, and generations of dangerous, immoral, or

illegal responses (Zhuo et al., 2023), and the like.

When asked how it could help professors in

higher education, ChatGPT answered the following

it could help by 1) providing them with relevant and

up-to-date information on a wide range of topics;

2) generating ideas for course content, such as

discussion topics, reading materials, or case studies;

3) creating automated assessments, such as quizzes

or exams, that can be graded quickly and efﬁciently;

4) providing language support, such as translations

or explanations of complex concepts; 5) generating

personalized learning recommendations, such as

recommended readings or study materials, based on

individual student needs.

Kung et al. (Kung et al., 2023) analyzed the

performance of ChatGPT when taking the United

States Medical Licensing Exam (USMLE) in Step

1, Step 2CK, and Step 3. According to the authors,

ChatGPT demonstrated a high level of agreement

and perception. These results indicated that ChatGPT

has the potential to help with medical education

and potentially in clinical decision-making. Other

authors have identiﬁed positive and reactive ways

of using ChatGPT in academia (Shoufan, 2023;

Joyner, 2023; Zhai, 2023a). For example, Pickell et

al. (Pickell and Doak, 2023) suggested ﬁve ideas for

helping professors avoid excessively using ChatGPT

in text elaboration, such as requesting comparisons

of theories or subjects with more current models

and integrating their personal experiences in light

of the requested topic. Similarly, Firat (Firat, 2023)

suggested potential applications of ChatGPT in

transforming auto-didactic Experiences and Open

Education, such as offering interactive exercises and

games catered to the learner’s particular learning

needs and delivering personalized suggestions for

learning materials and resources based on a learner’s

individual needs and objectives. According to the

author, this interactive and individualized support

can improve learning and encourage engagement and

motivation.

Zhai (Zhai, 2023b) experimented with ChatGPT

to automatically generate a performance-based

assessment task. The results suggest that ChatGPT

has the potential to solve the most challenging

science learning problems through the development

of automatic assessment and classiﬁcation, learning

guidance, and automatic recommendation of learning

materials. Despite positive ﬁndings, this study

suggests that ChatGPT cannot replace professors,

and professors need professional knowledge to use

ChatGPT for educational purposes. Phillips et al.

(Phillips et al., 2022) explored how ChatGPT3 could

extract and summarize student chat in a collaborative

application in a learning environment. ChatGPT was

able to correctly attribute states such as frustration

and confusion, reliably synthesize missing statements

in the source text, and effectively ignore extraneous

noise in student chat. The paper discussed how this

abstract could help professors understand student

collaboration in collaborative learning environments.

Considering the above, it is vital to research

the current role of ChatGPT in higher education

teaching, how it can be used, and how it is being used

to improve learning and knowledge production in

higher education in the context of computer science.

Given this context, the aim of this research is to i)

identify proposals for using ChatGPT to improve

learning and knowledge production in the higher

education of computer science; ii) identify how

ChatGPT is currently being used by professors in the

computer science area in the context of learning and

knowledge production; and iii) identify advantages

and disadvantages of using ChatGPT, proposing

actions to mitigate.

The ﬁndings of this research demonstrate that

computer science professors utilize ChatGPT as a

supportive tool in the preparation of educational

content to be used with students. For instance,

they employ it in creating exercises, assignments,

and supplementary materials, as well as in the

development and subsequent grading of exams

administered to students. The professors further

afﬁrm that ChatGPT functions as a mediator in the

teaching and learning process, facilitating interaction

between students and professors. This interaction

ICEIS 2024 - 26th International Conference on Enterprise Information Systems

542

holds signiﬁcant importance within the context of

computer science as it motivates students, enabling

them to constantly seek knowledge updates. This

applies to both students and professors who need

to stay up-to-date with new technologies daily.

Moreover, having the support of a computational tool

minimizes workloads and saves time, allowing them

to allocate their efforts toward other activities.

In the remainder of this paper, we describe the

background and related work in Section 2. Afterward,

we introduce our research design in Section 3 and

present our results in Section 4. Eventually, we

discuss the implications for research and practice

along with the study limitations and threats to validity

in Section 5 and draw our conclusion by outlying

future research directions in Section 6.

2 RELATED WORK

The use of ChatGPT in teaching and knowledge

generation has been the subject of many discussions

and studies. While New York City’s Department

of Education bans ChatGPT in public schools

(Rosenblatt, 2023), others encourage its use in the

classroom (Zhai, 2023b; Phillips et al., 2022), as the

author of a scientiﬁc article (Thunstr

om et al., 2023),

for research (Teubner et al., 2023b), among others.

In the context of teaching, George et al. (George

et al., 2023) mentioned that the education industry

has a lot to gain from ChatGPT capabilities as it

can offer students personalized learning experiences

tailored to their speciﬁc needs. According to the

authors, professors can make lesson plans for

each student based on their interests and abilities

and provide immediate feedback as they progress

through material or assignments. GPT chat also

allows professors to grade student work faster by

automatically grading it.

Firat (Firat, 2023) researched how ChatGPT

can transform autodidactic experiences and open

education. According to this author, ChatGPT

chat can encourage student autonomy and improve

learning experiences and can provide individualized

suggestions for reading material and other resources

or interactive assignments and activities that

meet students’ unique requirements and learning

objectives. ChatGPT can offer interactive exercises

and games that meet a student’s speciﬁc learning

needs and offer personalized suggestions for learning

materials and resources based on a student’s

individual needs and goals. This interactive and

individualized support can enhance learning and

encourage engagement and motivation.

Zhai (Zhai, 2023b) presented a pilot study in

the middle school context using ChatGPT. The

author concluded that ChatGPT has the potential

to help educators and school systems address

challenges related to assessment practices in the

implementation of the Science subject. ChatGPT can

improve assessment development efﬁciency, provide

personalized learning, and assist in evaluating

and reporting student performance, optimizing the

educator’s time and effort. However, it cannot

provide emotional support, critical thinking, and the

development of problem-solving skills in science

learning. In short, ChatGPT provides tools to

enhance and complement the work of professors but

is not a substitute for them. Professors must be able

to assess the quality and relevance of the information

provided by ChatGPT and make necessary decisions

about its use in the classroom.

Phillips et al. (Phillips et al., 2022) studied how

a new GPT 3 model can summarize and analyze

student chat conversations in computer-supported

collaborative applications in learning environments.

The authors concluded that ChatGPT could allow

the professor to make inferences about the nature of

student collaboration, such as viewing summaries

of student conversations in real-time to identify

frustrated and confused students. Although ChatGPT

has limitations, according to the authors, it would

not hinder ChatGPT from being a substantial asset to

professors.

In the context of writing texts, Shidig (Shidiq,

2023) identiﬁed that Artiﬁcial Intelligence offers

many educational facilities, such as using various

systems, such as virtual mentors, voice assistants,

innovative content, smart classrooms, automatic

assessment, and personalized learning. ChatGPT’s

understanding of human language greatly facilitates

creative writing, such as poems, short stories, novels,

or other types of writing whose quality is equivalent

to human work. The author discusses ChatGPT and

its impact on students’ lack of creativity in writing

skills.

Pickell et al. (Pickell and Doak, 2023) suggested

ﬁve ideas for dealing with ChatGPT in a learning

environment, also considering the context of writing

texts by students. First, the authors identiﬁed that

generic essay questions are highly susceptible to

cheating through the GPT. Overly generic and

common writing topics were already susceptible to

plagiarism and cheating, and ChatGPT is quite adept

at delivering correct writing but also often cliched.

Therefore, the ﬁrst idea would be to get students to

build better projects that connect, analyze, judge,

design, and present new ideas as an alternative to

Teaching Practice Using ChatGPT in Higher Education

543

minimizing the use of ChatGPT in this context.

Qadir (Qadir, 2023b) analyzed several ChatGPT

responses regarding education and suggested a

series of questions, such as using it responsibly and

as a complement, not the primary source; using

critical thinking over its responses; among others.

In addition, the author states that ChatGPT has

the potential to offer personalized and compelling

learning experiences, providing students with

personalized feedback and explanations, as well as

creating realistic virtual simulations for practical

learning. However, it is also essential to consider that

ChatGPT can perpetuate prejudices or even generate

and spread misinformation. According to the author,

AI in education raises ethical concerns, such as the

potential for unethical or dishonest use by students

and the potential unemployment of humans.

Cotton et al. (Cotton et al., 2023) mentioned

that using GPT-3 in higher education could offer

many beneﬁts, including greater engagement,

collaboration, and accessibility. It can be used

for language translation, summarizing, answering

questions, generating text, and custom assessments.

However, these tools also raise challenges and

concerns, particularly regarding academic honesty

and plagiarism. For example, GPT-3 can facilitate

cheating, and it can be difﬁcult to distinguish between

human and machine-generated writing. The authors

further examine the opportunities and challenges

of using GPT-3 in higher education, focusing on

these tools’ potential risks and rewards and how

universities can address their challenges. According

to the authors, universities should carefully consider

the potential risks and rewards of using these tools

and take steps to ensure they are used ethically and

responsibly. This may involve developing policies

and procedures for their use, providing training and

support for students and faculty, and using various

methods to detect and prevent academic dishonesty.

Researching the identiﬁcation of methods and

tools to detect content generated by ChatGPT,

Pegoralo et al. (Pegoraro et al., 2023) provided

a comprehensive overview of evaluating the latest

techniques’ ability to differentiate between responses

generated by ChatGPT and those produced by

humans. Several tools were evaluated, such

as Stylometric Detection of AI-generated Text,

ZeroGPT, OpenAI Text Classiﬁer, GPTZero, and

Hugging Face. The evaluation results demonstrate

that none of the existing methods can effectively

detect content generated by ChatGPT. According to

the authors, the exceptional ability of GPT to deceive

detectors reinforces the need to ongoing efforts to

improve the accuracy and robustness of text detection

techniques in the face of increasingly sophisticated

technologies generating content by AI.

In the context of knowledge production, Burger

et al. (Burger et al., 2023) presented an overview of

the current state of AI use in research, highlighting

recent trends and developments in the ﬁeld. The

authors present guidelines for using AI in the

scientiﬁc research process. The guidelines have been

developed for case literature review, but the authors

believe these instructions can be adjusted to many

research ﬁelds. The authors highlight the advantages

and limitations of AI today in any research using AI.

Advantages include objectivity and repeatability in

research processes currently subject to human error.

general-purpose models, the understanding of which

is essential to use them correctly in research.

Thunstrom et al. (Thunstr

om et al., 2023)

explored the potential of a system to co-author an

academic work based on the criteria proposed by

the International Committee of Medical Journal

Editors (ICMJE). The authors used the GPT-3 to

write a review article on the topic of their choice:

the effects of sleep deprivation on cognitive function.

The system was asked to comply with the four

main criteria for co-authorship as recommended by

the ICMJE: contribute with design or conception,

critically redact or review, approve the ﬁnal version,

agreement to be held accountable. The results

showed that the system could meet the criteria,

although it has difﬁculties in referencing.

Teubner et al. (Teubner et al., 2023b) analyze

various aspects of how GPT chat and similar

technologies may interfere with knowledge production

and teaching. Regarding the production of knowledge

using AI, they suggest that ChatGPT and other similar

tools can lead to a situation where the ability to read

and interpret different text options becomes more

important than the ability to write them. According

to the authors, knowledge workers interacting

with ChatGPT will prove their worth based on a

combination of skillful solicitation (considering that

questions to be better answered have to be better

crafted), quick quality control, and adaptation of

responses. Faculty and students must be sensitized to

take ultimate responsibility for whatever they deliver.

Academic policies can demand transparency about

where and how AI assistants are used. The current

state of generative AI technology represented by

ChatGPT is impressive but, as demonstrated, can

be ﬂawed and misused. However, it sure is just a

preview of what is to come. Professors in the area of

computer science and other areas need to understand

the implications of this and study how to adapt the

teaching of their disciplines to ensure that the next

ICEIS 2024 - 26th International Conference on Enterprise Information Systems

544

generations of professionals can take advantage

of the beneﬁts offered in an ethical, efﬁcient, and

productive way.

3 STUDY SETTINGS

In this study, we adopted a quantitative approach

using a survey questionnaire as the primary research

method to capture the perceptions of higher education

professors. The following research questions were

used to guide this work:

RQ.1: How ChatGPT is being used in higher

education in the context of computer science?

RQ.2: How does the use of ChatGPT impact

the production of knowledge and teaching

practice (advantages and disadvantages)?

The survey population target was composed of

higher education professors from both public and

private institutions. There was a control question at

the beginning of the survey to ﬁlter the participants

and ensure we got responses only from our target

audience. The participants were recruited through

personal contacts and social networks. In total, we

invited 140 professors to participate in the research.

All the authors of this paper were involved in the

design and validation of the survey questions. Two

authors described the survey questions and the others

validated them. The survey consisted of 18 questions,

12 closed-ended and 6 open-ended questions, as

shown in Table 1. The complete survey is available

in our survey supplementary material available at

https://zenodo.org/records/10257915.

We used the Ofﬁce Forms platform

to create

the survey. The form started with the informed

consent term presenting the conditions to participate

in this survey and the research contacts for eventual

questions regarding the survey. Participants were

not required to present any data that would reveal

their identities to preserve their anonymity, although

there was an optional ﬁeld for them to provide their

emails in case there were any follow-up questions.

The survey was available online for 36 days and

the average time to respond was 11 minutes and 31

seconds.

We carried out a pilot with two professors

from a federal public university and one professor

from a private university, and their suggestions for

improvement were all implemented. The objective

was to validate understanding mainly with open

https://forms.ofﬁce.com

questions. After the pilot, some questions were

changed. Consequently, the pilot responses were

discarded in the data analysis. Table 1 presents all

survey questions.

Surveys collect qualitative and quantitative

information to provide an overview of the current

status relating to a phenomenon (Kitchenham and

Pﬂeeger, 2008). In quantitative analysis, we use

descriptive statistics, for example, to represent

and describe participant characterization data.

We used Grounded Theory to perform open and

axial (Wuetherick, 2010) coding to analyze the

open questions. Grounded Theory is a method of

inductively generating theory from collected data

(Stol et al., 2016). Studies often include unstructured

text, for example, interview transcripts, surveys,

ﬁeld notes, and so on. However, they can include

structured text, diagrams and images, and even

quantitative (Corbin and Strauss, 2014) data.

4 RESULTS AND DISCUSSIONS

4.1 Demographic Information

Eighty-six (86) university professors from different

Brazilian states responded to the survey. Only two

states had no representatives, Minas Gerais and

Piau

ı. Thirty-seven (37) participants work at federal

universities in Brazil, 13 at state universities, 11 at

federal institutes, 7 at private colleges, 9 at private

universities, and 8 at university centers (private).

Most participants have between 10 and 15 years of

experience as professors from higher education and

work with Software Engineering and Programming

Language disciplines. Table 2 provides an overview

of the proﬁle of survey participants.

Regarding how long professors have been using

ChatGPT, 56% of them reported that it was between

1 and 3 months, 31% that they used it less than one

month, 10% that between 4 and 6 months and only 3%

of participants claimed to have used it for more than

six months. It is important to highlight that ChatGPT

was made available to the public on November 30th,

2022, and that this research was carried out between

April to May 2023. Therefore, we can infer that most

respondents had already been using chatGPT for 1 to

3 months or more. Despite the short period of use (1

to 3 months), compared to the availability of chatGPT

(5 months), use over 1 to 3 months was considered a

good representation of the perception of chatGPT.

Teaching Practice Using ChatGPT in Higher Education

545

Table 1: Survey Questions.

Demographic Information

1. Email - Optional response

2. In which state do you currently live?

3. What is the nature of your work and/or organization?

4. How long have you been working as a higher education professor?

5. What is your area of teaching?

6. What is your educational background?

Use of ChatGPT

7. ChatGPT was made available in Nov/2022. For how many months have you been using ChatGPT?

8. In my opinion, ChatGPT will have a signiﬁcant impact on all areas of knowledge.

9. In my opinion, ChatGPT will have a signiﬁcant impact on education, especially in higher education.

10. Have you already used any of these techniques in your teaching with ChatGPT?

11. Please cite the main results of the application of these techniques (positive or negative).

12. What other techniques have you used in your teaching? What were the results (positive and negative)?

13. Have you used any of these techniques when developing research, articles, or texts with ChatGPT?

14. In your perception, what are the main problems with using ChatGPT in higher education? Is there any way to

mitigate them?

15. In your perception, what are the main advantages of using ChatGPT in higher education teaching?

16. In your perception, is there a need for training professors to be able to use ChatGPT in higher education more

effectively? Why?

17. The use of ChatGPT in teaching will be permanent. It will be a new tool to be used in teaching and knowledge

generation.

18. Regarding the previous question, could you inform us why?

Table 2: Demographic Information Sample.

Variable Categories % #

Organization Federal University 43.2 37

State University 15.11 13

Federal Institute 12.8 11

Private College 8.14 7

Private University 10.3 9

University Center 9.3 8

Disciplines Software Engineering 59.30 51

Programming Language 27.9 24

Requirements Engineering 20.9 18

Artiﬁcial Intelligence 18.6 16

Database 9.3 8

Systems Analysis 8.14 7

Software Quality 8.14 7

Data Structure 5.8 5

Computer Networks 2.3 2

Distributed Systems 2.3 2

Other disciplines 16.3 14

Education PhD 38.0 38

Postdoctoral 44.2 24

Master’s degree 18.6 16

Master’s degree student 3.5 3

PhD student 4.6 4

Specialization 1.2 1

4.2 RQ.1

Regarding the changes ChatGPT may induce, 90% of

the professors who participated in the survey strongly

agree or agree that ChatGPT will have much impact

in all areas of knowledge and that ChatGPT will

have much impact in the educational area, mainly

in higher education, as presented in Q08, Q09, and

Q17 of Figure 1. This result conﬁrms the ﬁndings

of George et al. (George et al., 2023) and Kalla et

al. (Kalla and Smith, 2023), who identiﬁed in their

research that ChatGPT would impact several areas of

knowledge. Furthermore, our results regarding the

impact of ChatGPT on education also conﬁrm the

ﬁndings of other related works (Qadir, 2023b; George

et al., 2023; Firat, 2023; Zhai, 2023b; Phillips et al.,

2022).

Figure 1: Participants’ perception about the ChatGPT’s

impact.

As for using ChatGPT as a teaching assistant,

62% of professors have already used ChatGPT in the

creation of content to be worked with students in the

classroom; 42% to generate questions and answers

to speciﬁc domains of knowledge; 35% used it to

answer student questions since once the GPT model

is trained, it can be used to answer student questions

in real-time; 32% of professors used ChatGPT to

ICEIS 2024 - 26th International Conference on Enterprise Information Systems

546

support the preparation of a subject’s teaching plan

or to create a chat room, respectively, as shown

in Figure 2, and previously found by other works

(Firat, 2023; Zhai, 2023b; Phillips et al., 2022).

Furthermore, our ﬁndings also conﬁrm the results

presented by ChatGPT itself, which indicates a series

of suggestions for educational activities that can be

used in higher education, such as creating a chat

room using GPT technology, training the GPT model

with data related to the course content; automatically

generate a performance-based assessment task;

automatic grading of student’s written responses;

designing exercises and interactive games that meet

the speciﬁc learning needs of the student.

Some of the activities most mentioned by the

professors participating in the survey in the use of

ChatGPT to support teaching were not mentioned

in ChatGPT’s response to our query, such as a)

support in the elaboration of content to be worked

with students and b) support in the elaboration of

the teaching plan of a discipline. Another different

response from a professor was:

Figure 2: Ways in which professors have been utilizing

ChatGPT in their teaching activities.

“[...] I have been using GPT templates since version

2 and have had positive results in transcribing my

texts into a language style suitable for students.

Academic and purely formal language can easily

give way when requesting the rewriting of my texts

using other writing styles. Another use that I have

practiced is to request the elaboration of topics

for slides considering a text, generally authorial,

provided by me.”

Regarding the advantages and disadvantages

or limitations of ChatGPT in the perception of

professors, the advantages most cited by them

were: 1) Streamlines the professor’s activities,

supporting the construction of didactic content;

2) Its use provides greater motivation, interaction,

participation, and learning on the part of students;

3) Speeds up the generation of exercises and

questions/answers about the didactic content to be

worked on in the classroom with the students; 4)

Facilitates the elaboration and preparation of the

teaching and discipline plan; and 5) Generates ideas

for interesting activities to be carried out in the

classroom with students.

Some of these advantages had already been

identiﬁed previously by Firat (Firat, 2023; Zhai,

2023b; Cotton et al., 2023). It is essential to highlight

some of the reports of the professors who participated

in the survey on how they used ChatGPT in the

context of teaching in higher education and the

results achieved by them, as seen in the transcripts

below:

“[...] ChatGPT allowed the generation of user

stories in different contexts. I was able to quickly

provide several examples to students in class. It

played the role of the stakeholder while conducting

the business rule deﬁnition activity of the practical

case study.”

“[...] I generated some questions to ask students

during class and collected the responses in

ChatGPT. After I applied the questionnaire to the

students, we compared the students’ answers with

the answers given by ChatGPT and they were very

similar.”

“[...] I was able to identify syllabic for other

subjects similar to my subject and capture speciﬁc

objectives to be added to the teaching plan. In

addition, I was able to elaborate on content of

interest to the students, which is more current and

there was a greater motivation on their part when I

said that I had collected from ChatGPT and that we

could compare the results together and analyze what

they thought of the answers. It was a very interactive

and participatory class.”

Regarding the disadvantages or limitations of the

use of ChatGPT to support professors ’ activities,

the answers most mentioned by the participants

were: 1) Interesting and comprehensive information,

but no bibliographic reference; 2) Need to check

the results obtained and sometimes perform an

adjustment or correction of the ChatGPT responses;

3) It is necessary to elaborate more elaborate and

speciﬁc questions to obtain success in the query; 4)

Need to train the model to use it more effectively;

and 5) Copyright concerns. Some responses from

professors about the disadvantages or limitations of

using ChatGPT were:

Teaching Practice Using ChatGPT in Higher Education

547

“[...] I was able to quickly generate a quiz on a

data structure topic for students to take in class.

However, ChatGPT may have generated the same

questionnaire for several other professors, making

the activities repetitive.”

“[...] The use of the tool allowed an ethical

discussion in the classroom. Much of the content

presented does not provide citations/bibliographic

references, which can lead to disorderly plagiarism.

This is worrying since it is necessary to mention who

carried out certain experiments.”

“[...] ChatGPT tutorials and tips facilitate student

learning, making access to information faster and

more efﬁcient. However, it is necessary to be aware

of the origin of the information sent by ChatGPT,

since it does not share the source of the information

and some of it may not be reliable.”

In none of the previous studies, the lack of

bibliographic references and the need to check the

responses received were mentioned as limitations

of the ChatGPT. This is an important ﬁnding of

our research since it is a limitation that needs to

be considered, mainly due to the problems that

can be provoked by using texts without being

referenced. Although Pickell et al. (Pickell and

Doak, 2023) detected a citation issue, they did

not mention their absence as a disadvantage. The

need for well-designed questions for ChatGPT to

answer more accurately conﬁrms the ﬁndings of

Qadir (Qadir, 2023b) and Teubner et al. (Teubner

et al., 2023b). Regarding the concern with copyright

versus ChatGPT, we identiﬁed this topic as a concern

of other authors from other areas of knowledge

(B.D.Lund and Wang, 2023; Cox and Tzoc, 2023).

Professors also use some techniques to support the

following teaching activities: elaboration of student

evaluations, planning and elaborating activities and

corrections; provision of questions and answers;

creating examples, creating exercises, and solving

more didactic problems. It is important to note that

most of the responses mentioned using ChatGPT

in the most varied contexts within higher education

teaching, demonstrating that professors are using

ChatGPT with great creativity and responsibility.

Some responses were:

“[...] I have been working in remote teaching, and I

have made summarized texts from the transcription

of classes in an automated way. Passing only

the video link, I extract the text and generate the

summary through the Google API and OpenAI.

I also use this summary to validate compliance

with the lesson plan. I used ChatGPT to suggest

contexts for applying Problem-Based Learning. The

quantity and quality of assertive suggestions for

the construction of case studies are phenomenal.

However, it is necessary to have knowledge to

perceive failures of contexts.”

“[...] I’ve already asked a group of students to solve

questions and check the answers with ChatGPT. We

had the same amount of wrong answers that needed

to be evaluated. The lack of knowledge and maturity

of students can lead to a wrong path. I have carried

out these activities in the classroom so that students

can mediate these issues.”

“[...] ChatGPT can support as a mediator in the

preparation of didactic content, but it should not be

the only source to be consulted by the professor, also

because it is necessary to have a theoretical basis

based on the literature.’ ’

Professors also identiﬁed some problems using

ChatGPT and suggested some ways to mitigate them.

The main problems and mitigation identiﬁed were:

1) The lack of bibliographical references – ChatGPT

should cite adequate references for its answers; 2)

Plagiarism by students — always check the generated

information; 3) The constant need to analyze and

evaluate the answers with a critical view — have

a knowledgeable person to analyze the answers;

and 4) The need to formulate questions concisely

and effectively to avoid inconsistent and untrue

information – train the model to mitigate possible

errors.

We transcribed some problems identiﬁed by

professors and possible mitigation for them:

“[...] It is necessary to make students aware that

ChatGPT is a collaborative tool and not a substitute

for the process of generating and transferring

knowledge.”

“[...] The generation (production) of automated

texts can atrophy the human ability to write with

quality. I have tried to mitigate this problem by

having students write the assignments in their own

handwriting, not allowing the digital delivery of any

activity. In addition, I request that each work be

explained orally to the whole class.”

ICEIS 2024 - 26th International Conference on Enterprise Information Systems

548

“[...] Using ChatGPT does not provide correct

answers if the question is not well worded. It is

necessary to analyze and evaluate the answers. One

way to mitigate this would be to use ChatGPT as

a support tool for students to make this analysis in

relation to the answers obtained.”

It is interesting to highlight that Orduna and

Clavijo (Ordu

na-Malea and Cabezas-Clavijo, 2023)

also identiﬁed other problems regarding references to

non-existent papers (ghost bibliographic references).

According to these authors, the use of Large-

Language Models based which generates plausible

but fake bibliographic references.

4.3 RQ.2

In order to understand how the use of ChatGPT could

impact the production of knowledge and the activities

of professors in the classroom, we investigated the

perception of professors regarding the advantages of

using ChatGPT in higher education teaching. Table 3

displays the categories and subcategories after coding

responses. The production of teaching material

category was the most mentioned by the participants,

and most of them use ChatGPT to prepare practical

exercises and slides to be used in the classroom with

students.

We transcribed some of the professors’ answers.

For example, respondent #R27 and #R48 said,

respectively:

“[...] Using ChatGPT can improve student and

professor knowledge in a given area, making the

learning process faster, easier, and more efﬁcient.”

“[...] ChatGPT can be used as a tool support

in the learning process, allowing better interaction

between students and professors, as well as sharing

knowledge between them in a collaborative, fun,

and instantaneous way .”

We investigated whether professors think there

is a need for faculty training to use ChatGPT

more effectively in higher education teaching. Most

professors (60) stated that training is not necessary for

professors to be able to use ChatGPT. According to

participants, ChatGPT is intuitive and user-friendly.

Teubner et al. (Teubner et al., 2023b) have stated

that the increase in productivity with the use of

ChatGPT will depend on the user’s proﬁciency in

using the technology, that is, individuals who already

have difﬁculty with Information Technologies,

certainly not will beneﬁt from the tool. Thus, a new

survey with professors from other areas of knowledge

would be necessary to verify whether the ease and

Table 3: Categories and subcategories of advantages and

disadvantages of using ChatGPT in higher education.

Category Subcategory # Cited

Production of teaching

material

Development of practical

exercises

Preparation of slides 69

Development of user

stories

Development of use

cases

Development of tasks 50

Development of tests 48

Increased efﬁciency

Increased productivity 83

Automation of tasks 79

Support in task execution 71

Advanced knowledge

Support in knowledge

sharing

Quick response time 65

Easy access to

information

Problem-solving 37

Learning mediator

Facilitation of interaction

among students

Acceleration of learning 32

Usability

Ease of use 68

User-friendly interface 8

Disadvantages

Risk of plagiarism by

students

Students cannot fully

trust the information

Students settle in, always

looking for chatGPT and

not reading or consulting

books

You need to know how

to ask questions correctly

to get more appropriate

answers

Mitigation

Train chatGPT to have

more efﬁcient responses

Students and professors

must be trained to be able

to use technology more

efﬁciently

It is necessary to develop

anti-plagiarism tools for

this context

Use chatGPT as a

learning support

technology and not

as a substitute

Use other technologies

in conjunction with

chatGPT

Students must be

encouraged to question

ChatGPT answers

productivity mentioned by professors in the area of

computer science about using ChatGPT are related

or not to the proﬁciency in IT of these professors.

Teaching Practice Using ChatGPT in Higher Education

549

That is, it is necessary to investigate whether it is not

necessary to train users to use ChatGPT efﬁciently

and effectively.

Regarding the continuous adoption of ChatGPT,

85% of professors agreed or fully agreed that

ChatGPT will be a permanent tool in higher

education teaching, coexisting with other existing

methods. Many respondents also noted that ChatGPT

could provide academic and pedagogical support for

professors and students due to its user-friendliness

and quick response time. As examples, participant

#R23 and #R69 stated, respectively:

“[...] Like other research and teaching tools, there

will be some future tool that will replace or bring

other forms of learning, but it is inevitable that its

use will become fundamental in higher education

teaching.”

“[...] ChatGPT helps us a lot in preparing content

to be used in the classroom and provides us with

a lot of updated information in real-time. Thus,

it is impossible for it to fall into disuse, especially

with the evolution of AI, which is present in all our

day-to-day tasks. We cannot close our eyes to the

advancement of educational support tools.”

Regarding preparing research articles or texts

with ChatGPT, the survey results indicated that 74%

of the participating professors used ChatGPT to

generate answers to speciﬁc knowledge domains.

Additionally, 53% of respondents utilized the tool

for research paper or essay writing support by

providing a thesis statement and key points as a

model. Also, 43% of professors reported using

ChatGPT to generate text in a particular style or

tone, allowing for draft versions of research papers or

other written materials. Another 42% of professors

used the tool to automatically summarize articles or

documents, making it easier to stay up-to-date with

current research. Furthermore, 38% of professors

utilized ChatGPT to support their literature review

process, generating article summaries or providing a

list of relevant articles on a speciﬁc topic or keyword.

Finally, 17% of professors used the tool to analyze

large volumes of text data such as social media posts

or news articles, providing insights and identifying

patterns, as demonstrated in Figure 3.

Teubner et al. (Teubner et al., 2023b) mentioned

that one of the positive points of using ChatGPT is

the improvement of researchers’ linguistic skills in a

non-native language. According to the authors, the

use of ChatGPT should facilitate the communication

of these researchers, for example, in academic or legal

writing, helping to prevent linguistic discrimination

Figure 3: Ways in which professors have been utilizing

ChatGPT in their research activities.

with non-native speakers.

5 THREATS TO VALIDITY

This study is subject to some validity threats that

must be considered when interpreting the ﬁndings.

Firstly, although the survey was anonymous and

conﬁdential, some respondents may have provided

inaccurate, incomplete, or biased information for

various reasons, such as social desirability or

fatigue. Moreover, the authors’ subjectivity may have

inﬂuenced the interpretation of the data (Lazar et al.,

2017).

As the survey relied on self-reported data from

educators, the responses may be subject to their lack

of self-knowledge. For instance, some respondents

may have overestimated or underestimated their

ChatGPT use, competence, or attitudes or let their

personal views be inﬂuenced by their professional

roles or responsibilities. Furthermore, while our

study focused on investigating the use of ChatGPT

in the SE ﬁeld, we designed broader research

questions to encompass higher education in general.

This approach aimed to explore similarities and

differences in ChatGPT usage across various

educational contexts, including SE. We acknowledge

that explicitly highlighting the SE ﬁlter in the

questionnaire would have been beneﬁcial to gather

more speciﬁc insights. However, it’s important to

note that our study exclusively involved computing

professors as participants. This deliberate selection

ensured that the data collected remained focused

on the SE ﬁeld, despite the questionnaire’s broader

scope. The expertise and experiences of these

computing professors informed their responses,

providing valuable insights into computing education.

Another limitation of the study is that it focused solely

on the perceptions of Brazilian professors. As the

survey was conducted only in Brazil, the results

may not represent professors’ views, practices,

ICEIS 2024 - 26th International Conference on Enterprise Information Systems

550

and challenges in other countries or cultures. This

limits the external validity and results generalization

considering this sample of participants.

6 CONCLUSIONS

In this study, we investigate proposals for using

ChatGPT to improve learning and knowledge

production in the ﬁeld of computer science; to

identify how ChatGPT is currently being used by

professors in the computer science ﬁeld in the

context of learning and knowledge production;

and to identify the advantages and disadvantages

of using ChatGPT, proposing actions to mitigate

them. ChatGPT offers numerous opportunities for

professors and lecturers to develop ideas that can

be used as support for educational work. It can

enhance learning experiences and transform the

role of educators. We surveyed 86 professors from

public and private universities in Brazil to collect

their perceptions of ChatGPT. Professors are using

ChatGPT to prepare teaching material for classroom

use, mainly in constructing exercises, tasks, and tests.

In addition, they consider that ChatGPT facilitates

the activities that a professor needs to perform

in higher education teaching, such as preparing

teaching plans for the subjects they teach. In the

professors’ perception, ChatGPT also improves

students’ motivation, interaction, participation, and

learning during classes.

Some disadvantages of ChatGPT were identiﬁed,

such as unreliable information, the risk of plagiarism

and accommodation by students, and the need to

know how to ask correctly to receive more accurate

answers. Some actions were identiﬁed to mitigate

or exclude these disadvantages, such as students

must be encouraged to question chatGPT responses,

use of chatGPT to support learning, training to

better use chat effectively, and development of anti-

plagiarism tools, among others. Furthermore, it was

identiﬁed that ChatGPT also provides interesting and

comprehensive information but does not provide any

bibliographic reference of the contents presented to

its users, which is considered a signiﬁcant problem by

professors. In future work, we intend to investigate

the use of ChatGPT in other areas of knowledge and

compare it with the perceptions of professors in the

area of computer science. In addition, we want to

identify which techniques are used by professors

from other areas of knowledge and if they have more

or fewer difﬁculties in using this tool compared to

professors of computer science.

ACKNOWLEDGMENTS

This study was ﬁnanced in part by the Coordenac¸

de Aperfeic¸oamento de Pessoal de N

ıvel Superior –

Brasil (CAPES) – Finance Code 001.

REFERENCES

B.D.Lund and Wang, T. (2023). Chatting about chatgpt:

how may ai and gpt impact academia and libraries?

Library Hi Tech News, ahead-of-print:7.

Burger, B., Kanbach, D. K., , Kraus, S., Breier, M., and

Corvello, V. (2023). On the use of ai-based tools like

chatgpt to support management research. European

Journal of Innovation Management, 26(7):9.

Castillo, A. G. R., Serna, G. J., Silva, Arocutipa, J. P. F.,

Berrios, H. Q., Rodriguez, M. A. M., Reyes, G. Y.,

Lopez, H. R. P., Teves, R. M. V., Herbert, Rivera,

V. H., and Arias-Gonz

ales, J. L. (2023). Effect of

chat gpt on the digitized learning process of university

students. Journal of Namibian Studies : History

Politics Culture, 33.

Chartr (2022). Chatgpt: the ai bot taking the tech world by

storm. Chartr ofﬁce, 1:1.

Corbin, J. and Strauss, A. (2014). Basics of qualitative

research: Techniques and procedures for developing

grounded theory. Canadian Journal of University

Continuing Education, 36(2):1–12.

Cotton, D. R., Cotton, P. A., and Shipway, J. R. (2023).

Chatting and cheating: Ensuring academic integrity

in the era of chatgpt. DREC - Plymouth Marjon

University, 1:11.

Cox, C. and Tzoc, E. (2023). Chatgpt implications for

academic libraries. College & Research Libraries

News, 85(3):4.

Fauzi, F., Tuhuteru, L., Sampe, F., Ausat, A., and Hatta,

H. R. (2023). Analysing the role of chatgpt in

improving student productivity in higher education.

Journal on Education, 5:14886–14891.

Firat, M. (2023). How chat gpt can transform autodidactic

experiences and open education? Department

of Distance Education, Open Education Faculty,

Anadolu University, 1:4.

George, A. S., George, A. H., and Martin, G. (2023).

A review of chatgpt ai’s impact on several business

sectors. Partners Universal International Innovation

Journal (PUIIJ), 1:9.

Halaweh, M. (2023). Chatgpt in education: Strategies

for responsible implementation. Contemporary

Educational Technology, 15.

Javaid, M., Haleem, A., Singh, R. P., Khan, S., and Khan,

I. H. (2023). Unlocking the opportunities through

chatgpt tool towards ameliorating the education

system. BenchCouncil Transactions on Benchmarks,

Standards and Evaluations, 3(2):100115.

Joyner, D. A. (2023). Chatgpt in education: Partner or

pariah? XRDS, 29(3):48–51.

Teaching Practice Using ChatGPT in Higher Education

551

Kalla, D. and Smith, N. (2023). Study and analysis

of chat gpt and its impact on different ﬁelds of

study. International Journal of Innovative Science and

Research Technology, 8(3):7.

Kitchenham, B. A. and Pﬂeeger, S. L. (2008). Personal

opinion surveys. In Shull, F., Singer, J., and Sjøberg,

D. I. K., editors, Guide to Advanced Empirical

Software Engineering, pages 63–92. Springer,

https://doi.org/10.1007/978-1-84800-044-5 3.

Kung, T. H., Cheatham, M., Medenilla, A., , Sillos, C.,

Leon, L. D., Elepa

no, C., Madriaga, M., Aggabao,

R., Diaz-Candido, G., Maningo, J., and Tseng, V.

(2023). Performance of chatgpt on usmle: Potential

for ai-assisted medical education using large language

models. medRxiv and bioRxiv, 1:12.

Lazar, J., Feng, J., and Hochheiser, H. (2017).

Research Methods in Human-Computer Interaction,

2nd Edition. Morgan Kaufmann, https://www.

sciencedirect.com/science/book/9780128053904.

Ordu

na-Malea, E. and Cabezas-Clavijo, A. (2023). Chatgpt

and the potential growing of ghost bibliographic

references. Scientometrics, 128(9):5351–5355.

Paris, M. (2023). Chatgpt hits 100 million users, google

invests in ai bot and catgpt goes viral. Forbes, 1:1.

Pegoraro, A., Kumari, K., Fereidooni, H., and Sadeghi, A.

(2023). To chatgpt, or not to chatgpt: That is the

question! CoRR, abs/2304.01487:6.

Phillips, T., Saleh, A., Glazewski, K. D., and Hmelo-

Silver, C. E. (2022). Exploring the use of gpt-

3 as a tool for evaluating text-based collaborative

discourse. Proceedings 12th International Conference

on Learning Analytics & Knowledge LAK22, 1:2.

Pickell, T. R. and Doak, B. R. (2023). Five ideas for how

professors can deal with gpt-3 ... for now. Faculty

Publications - George Fox School of Theology, 1:7.

Qadir, J. (2023a). Engineering education in the era

of chatgpt: Promise and pitfalls of generative ai

for education. In 2023 IEEE Global Engineering

Education Conference (EDUCON), pages 1–

9, 10.1109/EDUCON54358.2023.10125121.

EDUCON.

Qadir, J. (2023b). Engineering education in the era of

chatgpt: Promise and pitfalls of generative ai for

education. TechRxiv, 1:9.

Rahman, M. M. and Watanobe, Y. (2023). Chatgpt for

education and research: Opportunities, threats, and

strategies. Applied Sciences, 13(9):1–21.

Rosenblatt, K. (2023). Chatgpt banned from new york city

public schools’ devices and networks. NBCNews, 1:1.

Rueda, M. M., Fern

andez-Cerero, J., Fern

andez-Batanero,

J. M., and L

opez-Meneses, E. (2023). Impact of the

implementation of chatgpt in education: A systematic

review. Comput., 12(8):153.

Shidiq, M. (2023). The use of artiﬁcial intelligence-based

chat-gpt and its challenges for the world of education;

from the viewpoint of the development of creative

writing skills. Proceeding of International Conference

on Education, Society and Humanity, 1(3):5.

Shoufan, A. (2023). Exploring students’ perceptions of

chatgpt: Thematic analysis and follow-up survey.

IEEE Access, 11:38805–38818.

Stol, K., Ralph, P., and Fitzgerald, B. (2016). Grounded

theory in software engineering research: a critical

review and guidelines. In Dillon, L. K., Visser, W.,

and Williams, L. A., editors, Proceedings of the 38th

International Conference on Software Engineering,

ICSE 2016, Austin, TX, USA, May 14-22, 2016, pages

120–131, https://doi.org/10.1145/2884781.2884833.

ACM.

Teubner, T., Flath, C. M., Weinhardt, C., van der Aalst, W.,

and Hinz, O. (2023a). Welcome to the era of chatgpt

et al. the prospects of large language models. Bus Inf

Syst Eng, 65(2):95–101.

Teubner, T., Flath, C. M., Weinhardt, C., van der Aalst, W.

M. P., and Hinz, O. (2023b). Welcome to the era of

chatgpt et al. Bus. Inf. Syst. Eng., 65(2):95–101.

Thunstr

om, A. O., Transformer, G. P., and Steingrimsson,

S. (2023). Does gpt-3 qualify as a co-author of a

scientiﬁc paper publishable in peer-review journals

according to the icmje criteria? - a case study.

Discover Artiﬁcial Intelligence, 3(12):20.

Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q., and Tang,

Y. (2023). A brief overview of chatgpt: The history,

status quo and potential future development. IEEE

CAA J. Autom. Sinica, 10(5):1122–1136.

Wuetherick, B. (2010). Basics of qualitative research:

Techniques and procedures for developing grounded

theory. Canadian Journal of University Continuing

Education, 36(2):1–12.

Zhai, X. (2023a). Chatgpt for next generation science

learning. XRDS, 29(3):42–46.

Zhai, X. (2023b). Chatgpt for next generation science

learning. SSRN, 1:8.

Zhuo, T. Y., Huang, Y., Chen, C., and Xing, Z. (2023).

Exploring AI ethics of chatgpt: A diagnostic analysis.

CoRR, abs/2301.12867:1–17.

ICEIS 2024 - 26th International Conference on Enterprise Information Systems

552