Capturing Movement: A Tablet App, Geometry Touch, for Recording Onscreen Finger-based Gesture Data

Abstract

This paper presents a novel digital method of capturing finger-based gestures on touchscreen devices for the purposes of exploring tracing gestures in educational research. Given that tracing has been found to support cognition, learning and problem-solving in educational settings, data related to the performance of these gestures is increasingly of interest to researchers. Most educational research methods exploring the use of hand gestures rely on in-person data collection, whether through direct observation, or video recording of participants' behaviour for later analysis. These methods, while effective for observing gross movements, may not provide researchers with detailed insights into how learners interact with learning materials. Using custom tools to record touchscreen engagement on tablet computing devices can address this limitation, while also providing the means to visually represent touch-based interactions with these devices. Geometry Touch is an iPad app developed and tested by the primary author as part of a pilot study. The research study, theoretically grounded in Cognitive Load Theory, demonstrated that Geometry Touch could efficiently collect data on touchscreen interactions, while also providing potential avenues to quantify touchscreen interactions through computational means. The purpose of this paper is to report on the development and testing of this app, while providing an explanation of how it was used as a method of data collection by leveraging touchscreen technology. The paper concludes by discussing how this digital method of capturing movement can provide further insight into how finger-based gestures can influence learning and as such, could increase the reach of gesture-based research.