As generative AI tools rapidly enter educational and professional environments, organizations face a critical challenge: how to harness AI to enhance learning and knowledge access while maintaining accuracy, governance and alignment with institutional standards. Recent work has highlighted both the potential and risks of large language models in medical education, particularly regarding reliability, hallucinations, and alignment with evidence-based teaching (Kung et al., 2023; Bobula, 2024). This paper presents the design, deployment and evaluation of a course-specific AI Teaching Assistant (AI TA) as a practical model for responsible, controlled AI adoption with clear relevance for pharmacology teaching to undergraduate medical students.

The project involved the development of an AI TA using OpenAI’s ChatGPT platform, deliberately constrained to operate within a curated and verified knowledge base rather than unrestricted Internet data. The AI TA was trained exclusively using pharmacology course materials and supplementary resources that met explicit inclusion criteria for factual accuracy, licensing compliance, relevance to learning outcomes and pedagogical clarity. This governance-first approach addresses common AI risks such as hallucinations, content drift, and misalignment with instructor or organizational expectations.

The AI TA was piloted with 35 second-year medical students following completion of the pharmacokinetics component of a content-intensive pharmacology module. Student engagement and perceived effectiveness were evaluated using a mixed-methods approach, with 23 participants (65.7%) completing a post-pilot survey that combined quantitative Likert-scale items (100% completion) with an open-ended qualitative feedback question (91.3% response rate). Findings indicate that 66% of respondents reported a positive overall experience, highlighting improved accessibility to complex material, clearer explanations and increased confidence during revision and exam preparation. Students valued the AI TA’s consistency with instructor-provided content and its availability for on-demand clarification, reinforcing trust in the system. Feedback also identified limitations, including occasional inaccuracies in calculation-based responses and performance latency, underscoring the importance of ongoing oversight and refinement.

This paper argues that narrowly scoped, curriculum-aligned AI systems offer a scalable and low-risk blueprint for AI adoption in educational settings.Importantly, the presentation will also outline the practical steps involved in designing, configuring, and governing such a system, including knowledge base curation, prompt structuring, and evaluation strategies, to support replication in other educational contexts. The findings provide actionable insights for legal technology leaders, educators and innovation teams seeking to deploy AI-enabled learning tools while preserving trust, quality and accountability.

References:

1. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, et al. (2023) Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health 2(2): e0000198. https://doi.org/10.1371/journal.pdig.0000198

2. Bobula, M. (2024). Generative artificial intelligence (AI) in higher education: a comprehensive review of challenges, opportunities, and implications. Journal of Learning Development in Higher Education, (30). https://doi.org/10.47408/jldhe.vi30.1137