Please use this identifier to cite or link to this item:
https://hdl.handle.net/1959.11/5804
Title: | Are they answering what we think we're asking?: The importance of pre-testing survey questions using cognitive interviews | Contributor(s): | McCormack, Cathryn Louise (author) | Publication Date: | 2008 | Handle Link: | https://hdl.handle.net/1959.11/5804 | Abstract: | Evaluation often involves conducting surveys. To create a survey, a typical process consists of: drafting questions using our experience and understanding of basic principles of good question writing to guide us, and obtaining feedback from colleagues. Sometimes a focus group might be included, or conducting a small scale trial. However this process misses the vital pre-testing stage of cognitive interviews. Cognitive interviews are widely used in the development of surveys for health and social sciences, but appear to be underutilised to inform the development of instruments in other fields. In the process of reviewing its unit and teaching evaluation instruments, the University of New England (UNE) investigated both existing and proposed questions with cognitive interviews using a retrospective think aloud protocol. Over 120 interviews were conducted with students. Each student was asked three or four questions from the full set, resulting in about 20 responses for each question. Interviews were recorded, transcribed, and analysed for variation in understanding. The cognitive interviews identified problems with conceptualisation and phrasing in the question wording, as well as problems with variation of interpretation of the items on the response scale. These problems were not at all clear to the panel developing the survey prior to the interviews being conducted. | Publication Type: | Conference Publication | Conference Details: | AES 2008: Australasian Evaluation Society Conference, Perth, Australia, 2008, | Source of Publication: | Proceedings of the Australasian Evaluation Society Conference - Evaluation: Adding Value | Publisher: | Australasian Evaluation Society (AES) | Place of Publication: | Casuarina, Australia | Fields of Research (FoR) 2008: | 160506 Education Policy 160807 Sociological Methodology and Research Methods |
Socio-Economic Objective (SEO) 2008: | 930501 Education and Training Systems Policies and Development | Peer Reviewed: | Yes | HERDC Category Description: | E2 Non-Refereed Scholarly Conference Publication | Publisher/associated links: | http://www.aes.asn.au/conferences/ http://www.aes.asn.au/conferences/2008/papers/p65.pdf |
---|---|
Appears in Collections: | Conference Publication |
Files in This Item:
File | Description | Size | Format |
---|
Page view(s)
894
checked on Mar 7, 2023
Download(s)
2
checked on Mar 7, 2023
Items in Research UNE are protected by copyright, with all rights reserved, unless otherwise indicated.