Please use this identifier to cite or link to this item:
Title: Academics' perceptions of ChatGPT-generated written outputs: A practical application of Turing’s Imitation Game
Contributor(s): Matthews, Joshua  (author)orcid ; Volpe, Catherine Rita  (author)orcid 
Publication Date: 2023-12-22
Open Access: Yes
Handle Link:

Artificial intelligence (AI) technology, such as Chat Generative Pre-trained Transformer (ChatGPT), is evolving quickly and having a significant impact on the higher education sector. Although the impact of ChatGPT on academic integrity processes is a key concern, little is known about whether academics can reliably recognise texts that have been generated by AI. This qualitative study applies Turing's Imitation Game to investigate 16 education academics' perceptions of two pairs of texts written by either ChatGPT or a human. Pairs of texts, written in response to the same task, were used as the stimulus for interviews that probed academics' perceptions of text authorship and the textual features that were important in their decision-making. Results indicated academics were only able to identify AI-generated texts half of the time, highlighting the sophistication of contemporary generative AI technology. Academics perceived the following categories as important for their decision-making: voice, word usage, structure, task achievement and flow. All five categories of decision-making were variously used to rationalise both accurate and inaccurate decisions about text authorship. The implications of these results are discussed with a particular focus on what strategies can be applied to support academics more effectively as they manage the ongoing challenge of AI in higher education.

Publication Type: Journal Article
Source of Publication: Australasian Journal of Educational Technology, 39(5), p. 82-100
Publisher: ASCILITE
Place of Publication: Australia
ISSN: 1449-5554
Fields of Research (FoR) 2008: 130303 Education Assessment and Evaluation
130213 Vocational Education and Training Curriculum and Pedagogy
130103 Higher Education
130306 Educational Technology and Computing
Fields of Research (FoR) 2020: 390303 Higher education
390114 Vocational education and training curriculum and pedagogy
390201 Education policy
390402 Education assessment and evaluation
Socio-Economic Objective (SEO) 2008: 930203 Teaching and Instruction Technologies
930501 Education and Training Systems Policies and Development
930403 School/Institution Policies and Development
Socio-Economic Objective (SEO) 2020: 280109 Expanding knowledge in education
160102 Higher education
160301 Assessment, development and evaluation of curriculum
160205 Policies and development
Peer Reviewed: Yes
HERDC Category Description: C1 Refereed Article in a Scholarly Journal
Appears in Collections:Journal Article
School of Education

Files in This Item:
2 files
File Description SizeFormat 
open/AcademicsMatthewsVolpe2023JournalArticle.pdfPublished Version504.31 kBAdobe PDF
Download Adobe
Show full item record
Google Media

Google ScholarTM



This item is licensed under a Creative Commons License Creative Commons