Investigating the Quality of AI-Generated Distractors for a Multiple-Choice Vocabulary Test

Wojciech Malec

Research.Publish.Connect.

*Please fill out at least one Field. *Value must be an number!

Title:
ISBN:
Year:
Acronym:
Subject:

Advanced Search Proceedings Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Title:
Author:
Affiliation:
Subject:

Advanced Search Papers Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Name:
Affiliation:
Country:
Conference:
Subject:

Advanced Search Authors Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Name:
Country:
Subject:

Advanced Search Affiliations Search

If you're looking for an exact phrase use quotation marks on text fields.

Proceedings

Proceedings Search *Please fill out at least one Field. *Value must be an number!

Title:
ISBN:
Year:
Acronym:
Subject:

Advanced Search Proceedings Search

If you're looking for an exact phrase use quotation marks on text fields.

Papers

Papers Search *Please fill out at least one Field.

Title:
Author:
Affiliation:
Subject:

Advanced Search Papers Search

If you're looking for an exact phrase use quotation marks on text fields.

Authors

Authors Search *Please fill out at least one Field.

Name:
Affiliation:
Country:
Conference:
Subject:

Advanced Search Authors Search

If you're looking for an exact phrase use quotation marks on text fields.

Advanced Search

Paper

Investigating the Quality of AI-Generated Distractors for a Multiple-Choice Vocabulary Test

Topics: AI Support for Item Creation; Assessment

In Proceedings of the 16th International Conference on Computer Supported Education - Volume 1: AIG, 836-843, 2024 , Angers, France

Author: Wojciech Malec

Affiliation: Institute of Linguistics, John Paul II Catholic University of Lublin, Al. Racławickie, Lublin, Poland

Keyword(s): AI-Generated Items, ChatGPT, Vocabulary Assessment, Multiple-Choice Testing, Distractor Analysis.

Abstract: This paper reports the findings of a study into the effectiveness of distractors generated for a multiple-choice vocabulary test. The distractors were created by OpenAI’s ChatGPT (the free version 3.5) and used for the construction of a vocabulary test administered to 142 students learning English as a foreign language at the advanced level. Quantitative analysis revealed that the test had relatively low reliability, and some of its items had very ineffective distractors. When examined qualitatively, certain items were likewise found to have an ill-matched set of options. Moreover, follow-up queries failed to correct the original errors and produce more appropriate distractors. The results of this study indicate that although the use of artificial intelligence has an unquestionably positive impact on test practicality, ChatGPT-generated multiple-choice items cannot yet be used in operational settings without human moderation.

CC BY-NC-ND 4.0

Guest: Register as new SciTePress user now for free.

SciTePress user: please login.

My Papers

You are not signed in, therefore limits apply to your IP address 3.15.201.103

In the current month:

Recent papers: 100 available of 100 total

2⁺ years older papers: 200 available of 200 total

Paper citation in several formats:

Malec, W. (2024). Investigating the Quality of AI-Generated Distractors for a Multiple-Choice Vocabulary Test. In Proceedings of the 16th International Conference on Computer Supported Education - Volume 1: AIG; ISBN 978-989-758-697-2; ISSN 2184-5026, SciTePress, pages 836-843. DOI: 10.5220/0012762400003693

@conference{aig24,
author={Wojciech Malec},
title={Investigating the Quality of AI-Generated Distractors for a Multiple-Choice Vocabulary Test},
booktitle={Proceedings of the 16th International Conference on Computer Supported Education - Volume 1: AIG},
year={2024},
pages={836-843},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012762400003693},
isbn={978-989-758-697-2},
issn={2184-5026},
}

TY - CONF

JO - Proceedings of the 16th International Conference on Computer Supported Education - Volume 1: AIG
TI - Investigating the Quality of AI-Generated Distractors for a Multiple-Choice Vocabulary Test
SN - 978-989-758-697-2
IS - 2184-5026
AU - Malec, W.
PY - 2024
SP - 836
EP - 843
DO - 10.5220/0012762400003693
PB - SciTePress