Academics' perceptions of ChatGPT-generated written outputs: A practical application of Turing’s Imitation Game

Author(s)
Matthews, Joshua
Volpe, Catherine Rita
Publication Date
2023-12-22
Abstract
<p>Artificial intelligence (AI) technology, such as Chat Generative Pre-trained Transformer (ChatGPT), is evolving quickly and having a significant impact on the higher education sector. Although the impact of ChatGPT on academic integrity processes is a key concern, little is known about whether academics can reliably recognise texts that have been generated by AI. This qualitative study applies Turing's Imitation Game to investigate 16 education academics' perceptions of two pairs of texts written by either ChatGPT or a human. Pairs of texts, written in response to the same task, were used as the stimulus for interviews that probed academics' perceptions of text authorship and the textual features that were important in their decision-making. Results indicated academics were only able to identify AI-generated texts half of the time, highlighting the sophistication of contemporary generative AI technology. Academics perceived the following categories as important for their decision-making: voice, word usage, structure, task achievement and flow. All five categories of decision-making were variously used to rationalise both accurate and inaccurate decisions about text authorship. The implications of these results are discussed with a particular focus on what strategies can be applied to support academics more effectively as they manage the ongoing challenge of AI in higher education.</p>
Citation
Australasian Journal of Educational Technology, 39(5), p. 82-100
ISSN
1449-5554
1449-3098
Link
Publisher
ASCILITE
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International
Title
Academics' perceptions of ChatGPT-generated written outputs: A practical application of Turing’s Imitation Game
Type of document
Journal Article
Entity Type
Publication

Files:

NameSizeformatDescriptionLink
closedpublished/AcademicsMatthewsVolpe2023JournalArticle.pdf 516.416 KB application/pdf Published version View document