AI-Assisted vs Traditional Self-Assessment: A Multi-Modal Study of Student Processes, Accuracy, and Cognitive-Emotional Impact
Seval Kemal
Supervisors: Ernesto Panadero & Darina Scully
Abstract
Artificial intelligence is rapidly entering higher education assessment, yet its impact on students' self-regulated learning (SRL) processes remains underexplored. This PhD project investigates how AI-assisted self-assessment, Microsoft Copilot alongside a structured rubric, shapes metacognition, cognitive engagement, emotional responses, and accuracy in evaluating writing, compared to rubric-only methods. A randomized controlled trial (RCT) with DCU students contrasts control and intervention conditions on a reflective essay task. Multimodal data include think-aloud protocols (SEFEMO-coded), eye-tracking (Tobii Pro Fusion), electrodermal activity (Shimmer GSR), questionnaires, expert ratings, and self-generated feedback. Data collection ongoing with 33 participants secured.
Background
SRL drives academic success through cognitive, metacognitive, and emotional processes (Panadero, 2017). Self-assessment, central to SRL, often misaligns with expert judgments, especially among novices (Andrade, 2019). While AI tools like Copilot support feedback, causal SRL effects lack multimodal study. SEFEMO application to AI self-assessment contexts is novel.
Objectives
RQ: How does AI-assisted self-assessment influence SRL processes, cognitive-emotional dynamics, and self-evaluation accuracy vs. traditional methods? Targets: SEFEMO actions; eye-tracking attention; EDA arousal; accuracy/feedback; AI trust.
Methods
Mixed-methods RCT: Students complete 750-word essay, randomized to control (rubric-only, Copilot disabled) or intervention (rubric + Copilot). Data collection: SEFEMO-coded think-alouds (Panadero et al., 2025); Tobii fixation/gaze; Shimmer EDA + cognitive load measures (Boucsein, 2012); blind expert ratings; feedback coding; baseline/AI perception questionnaires. Planned analyses: t-tests/Mann-Whitney, correlations, triangulation (de Mooij et al., 2025).
Expected Results
AI expected to increase Compare/Redo frequencies and cyclical patterns (vs. linear Read/Recall in control); greater rubric attention; higher arousal (engagement vs. strain); accuracy gains for low-achievers but shallower feedback; mixed AI trust. Clarifies AI as scaffold vs. replacement.
Significance
Theoretical: First SEFEMO application to AI; multimodal SRL extension. Practical: Evidence for Copilot in formative assessment. Methodological/policy: RCT template for AI-EdTech; informs ethics, autonomy, critical AI literacies.
Feedback Questions
-
With this multimodal approach (SEFEMO + eye-tracking + EDA), which analyses should I prioritize first?
-
How can we protect student autonomy in AI self-assessment practices?
-
Even offering 15€ vouchers, I am struggling with participant recruitment, especially students uninterested in SRL. What practical strategies work best?
References
Andrade, H. L. (2019). A critical review of research on student self-assessment. Frontiers in Education, 4, 87. https://doi.org/10.3389/feduc.2019.00087
Boucsein, W. (2012). Electrodermal activity (2nd ed.). Springer. https://doi.org/10.1007/978-1-4614-1126-0
de Mooij, S., Lams, J., Lim, L., Järvelä, S., Bannert, M., & Azevedo, R. (2025). A systematic review of self-regulated learning through multimodal data and AI. Educational Psychology Review, 37, Article 54. https://doi.org/10.1007/s10648-025-10028-0
Panadero, E. (2017). A review of self-regulated learning. Frontiers in Psychology, 8, Article 422. https://doi.org/10.3389/fpsyg.2017.00422
Panadero, E., Pinedo, L., & Fernández Ruiz, J. (2025). Unleashing think-aloud data. Learning and Instruction, 95, Article 102031. https://doi.org/10.1016/j.learninstruc.2024.102031
