Please use this identifier to cite or link to this item:
Title: Exploring relationships between automated and human evaluations of L2 texts
Contributor(s): Matthews, Joshua  (author)orcid ; Wijeyewardene, Ingrid  (author)orcid 
Publication Date: 2018-10-01
Open Access: Yes
DOI: 10125/44661Open Access Link
Handle Link:
Abstract: Despite the current potential to use computers to automatically generate a large range of text-based indices, many issues remain unresolved about how to apply these data in established language teaching and assessment contexts. One way to resolve these issues is to explore the degree to which automatically generated indices, which are reflective of key measures of text quality, align with parallel measures derived from locally relevant, human evaluations of texts. This study describes the automated evaluation of 104 English as a second language texts through use of the computational tool Coh-Metrix, which was used to generate indices reflecting text cohesion, lexical characteristics, and syntactic complexity. The same texts were then independently evaluated by two experienced human assessors through use of an analytic scoring rubric. The interrelationships between the computer and human generated evaluations of the texts are presented in this paper with a particular focus on the automatically generated indices that were most strongly linked to the human generated measures. A synthesis of these findings is then used to discuss the role that such automated evaluation may have in the teaching and assessment of second language writing
Publication Type: Journal Article
Source of Publication: Language Learning & Technology, 22(3), p. 143-158
Publisher: University of Hawai'i, National Foreign Language Resource Center
Place of Publication: United States of America
ISSN: 1094-3501
Fields of Research (FoR) 2008: 130303 Education Assessment and Evaluation
200303 English as a Second Language
130306 Educational Technology and Computing
Fields of Research (FoR) 2020: 390402 Education assessment and evaluation
470306 English as a second language
390405 Educational technology and computing
Socio-Economic Objective (SEO) 2008: 930201 Pedagogy
930301 Assessment and Evaluation of Curriculum
930203 Teaching and Instruction Technologies
Socio-Economic Objective (SEO) 2020: 160302 Pedagogy
160301 Assessment, development and evaluation of curriculum
160304 Teaching and instruction technologies
Peer Reviewed: Yes
HERDC Category Description: C1 Refereed Article in a Scholarly Journal
Publisher/associated links:
Appears in Collections:Journal Article

Files in This Item:
1 files
File SizeFormat 
Show full item record

Page view(s)

checked on Feb 11, 2024


checked on Feb 11, 2024
Google Media

Google ScholarTM



Items in Research UNE are protected by copyright, with all rights reserved, unless otherwise indicated.