Conference Agenda

This paper provides a descriptive history of AI chatbots and an overview of current debates surrounding their use in higher education informed by a structured narrative literature review of 50 academic articles. It aligns with the conference’s call for retrospectives, retrocomputing homages and EdTech histories while also contributing to strands related to critical AI literacies, practices and conceptual frames of learning and technology. The paper contributes an informative and impartial briefing for colleagues who hold a wide spectrum of views related to AI in education.

The paper describes the functionality and applications of early AI chatbots in the 1960s, ‘70s and ‘80s such as ELIZA and SCHOLAR before examining debates relating to the visual design elements of chatbots in the 1990s followed by the impact of Generative AI on chatbot design in the 2020s. The paper gives an overview of current definitions and taxonomies relating to AI and chatbots in the GenAI era to provide a foundation for colleagues to understand contemporary discourses. The paper then describes some of the benefits and challenges of AI chatbots through the concept of the pharmakon (an ancient Greek word for something that is both a remedy and a poison) (Wegerif and Casebourne, 2025)¹ to conceptualise the potential impact of AI on education.

The paper also looks at the challenges of developing robust frameworks for evaluating educational AI chatbots due to a lack of empirical studies and issues that arise from using pre-existing approaches such as the Technology Acceptance Model in this domain. The paper concludes by examining Retrieval-Augmented Generation (RAG) chatbots and some of the emerging approaches for evaluating the pedagogical impact of this technology. The accompanying presentation will give an overview of these aspects using clear diagrams, screenshots and historical examples to provide attendees with an engaging and approachable briefing to this field of research.

1. Wegerif, R., & Casebourne, I. (2025). A dialogic theoretical foundation for integrating generative AI into pedagogical design. British Journal of Educational Technology.

The rapid mainstream acceptance of generative artificial intelligence (GenAI) creates an opportunity to strengthen student learning in teacher education. By embedding responsible GenAI use within core learning processes, such as receiving and acting on feedback for assignments and tasks, programmes can develop student teachers’ AI literacy while promoting quality learning and feedback literacy (e.g., Prompiengchai et al. 2025). In this presentation, we will present a project, funded by the Strategic Alignment of Teaching and Learning Enhancement Funding in Higher Education, aimed at developing and documenting a GenAI formative feedback agent (RebelBot) to support student teachers’ planning tasks within the professional B.Ed. Physical Education, Sports Studies, and Arts at University College Cork (UCC).

The project, currently underway, adopts a Design-Based Research-inspired (Reeves et al. 2006), multi-phase cycle of co-design, piloting, and refinement of RebelBot. Phase 1 comprised two co-creation workshops with five paid Student Partners (SPs), centring on (i) previous experience with GenAI and desired features for RebelBot and (ii) feedback structure, preferences, and guardrails.

Phase 2 comprises an iterative pilot of RebelBot, created within Microsoft Copilot Studio under UCC licensing and EU Data Boundary controls, using the OpenAI GPT-5 Reasoning model (preview), and following the overall specifications agreed in Phase 1. Four SPs completed three submission cycles of planning artefacts, each followed by a survey collecting insights on usability (User Experience Questionnaire; Schrepp et al.) and structured suggestions to inform iterative refinements. This piloting phase also includes a preliminary reliability and validity study reusing consenting students’ historic planning assignments. This seeks to evaluate (i) test–retest reliability, examining stability of agent outputs when the same artefact is processed on separate occasions under controlled prompting, and (ii) convergent validity, testing alignment between agent outputs and existing criterion-level marking by a human marker.

Phase 3 will provide cohort-wide access for all Year 2 students (n = 52) to support equitable access and will also evaluate pre–post feedback literacy (Feedback Literacy Behaviour Scale; Dawson et al.) and AI ethical learning (AI Literacy Questionnaire; Ng et al.). It will also include a reflexive focus group with lecturers (n = 3) on the perceived value and impact of the agent’s implementation.

Phase 4 will entail the creation and dissemination of materials and tutorials to support the development of similar agents across disciplines and contexts.

This presentation will summarise the design decisions, phased evaluation, and ethical and guardrail structures underpinning RebelBot, offering a transferable methodology for responsible GenAI feedback with students in accredited third-level programmes.

As generative AI tools rapidly enter educational and professional environments, organizations face a critical challenge: how to harness AI to enhance learning and knowledge access while maintaining accuracy, governance and alignment with institutional standards. Recent work has highlighted both the potential and risks of large language models in medical education, particularly regarding reliability, hallucinations, and alignment with evidence-based teaching (Kung et al., 2023; Bobula, 2024). This paper presents the design, deployment and evaluation of a course-specific AI Teaching Assistant (AI TA) as a practical model for responsible, controlled AI adoption with clear relevance for pharmacology teaching to undergraduate medical students.

The project involved the development of an AI TA using OpenAI’s ChatGPT platform, deliberately constrained to operate within a curated and verified knowledge base rather than unrestricted Internet data. The AI TA was trained exclusively using pharmacology course materials and supplementary resources that met explicit inclusion criteria for factual accuracy, licensing compliance, relevance to learning outcomes and pedagogical clarity. This governance-first approach addresses common AI risks such as hallucinations, content drift, and misalignment with instructor or organizational expectations.

The AI TA was piloted with 35 second-year medical students following completion of the pharmacokinetics component of a content-intensive pharmacology module. Student engagement and perceived effectiveness were evaluated using a mixed-methods approach, with 23 participants (65.7%) completing a post-pilot survey that combined quantitative Likert-scale items (100% completion) with an open-ended qualitative feedback question (91.3% response rate). Findings indicate that 66% of respondents reported a positive overall experience, highlighting improved accessibility to complex material, clearer explanations and increased confidence during revision and exam preparation. Students valued the AI TA’s consistency with instructor-provided content and its availability for on-demand clarification, reinforcing trust in the system. Feedback also identified limitations, including occasional inaccuracies in calculation-based responses and performance latency, underscoring the importance of ongoing oversight and refinement.

This paper argues that narrowly scoped, curriculum-aligned AI systems offer a scalable and low-risk blueprint for AI adoption in educational settings.Importantly, the presentation will also outline the practical steps involved in designing, configuring, and governing such a system, including knowledge base curation, prompt structuring, and evaluation strategies, to support replication in other educational contexts. The findings provide actionable insights for legal technology leaders, educators and innovation teams seeking to deploy AI-enabled learning tools while preserving trust, quality and accountability.

References:

1. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, et al. (2023) Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health 2(2): e0000198. https://doi.org/10.1371/journal.pdig.0000198

2. Bobula, M. (2024). Generative artificial intelligence (AI) in higher education: a comprehensive review of challenges, opportunities, and implications. Journal of Learning Development in Higher Education, (30). https://doi.org/10.47408/jldhe.vi30.1137

The rapid emergence of generative AI has prompted widespread concern across higher education about academic integrity, assessment design, and the future role of educators. Institutional responses have often focused on detection technologies, policy restrictions, or tool-specific guidance. While necessary, these responses risk overlooking deeper questions about the values and purposes that should shape the integration of AI into education.

This practitioner paper reflects on the development of the Manifesto for Generative AI in Higher Education, created through the GenAI:N3 network across Irish technological universities. Rather than presenting the manifesto as a policy or prescriptive framework, we position it as a collaborative artefact designed to stimulate dialogue about teaching, learning, ethics, and institutional responsibility in an age of AI.

The manifesto comprises thirty short statements organised across three themes: Rethinking Teaching and Learning, Responsibility, Ethics, and Power, and Imagination, Humanity, and the Future. These statements emerged through conversations with educators, workshops, conference presentations, and iterative reflection across the sector. They are intentionally concise and open-ended, designed to provoke discussion rather than prescribe solutions.

This paper explores the process of creating the manifesto and the role such artefacts can play in fostering critical AI literacy and community dialogue within higher education. We reflect on how manifesto-based approaches can create space for multiple voices, including educators and students, to engage critically with the opportunities and tensions introduced by generative AI.

We argue that participatory artefacts such as manifestos can support the higher education community in moving beyond reactive policy responses towards more reflective, values-driven approaches to educational technology.