Project work or essay work is a vital way in which our students learn. Gen AI can now successfully complete most of this work with little assistance and potentially almost no learning by learners. Removing these assessments in favour of exams removes a vital component of learning. Interviewing students about their work does enable their understanding to be evaluated; however, it is extremely time consuming for the lecturers rendering it unviable in most courses.

We have piloted a novel approach to this problem:

1. A student submits their project work / essay work.

2. We develop a standard with a set of exemplars of what good questions look like to test the understanding of their individual work.

3. From each individual piece of work, we generate (with a local LLM) a set of basic and advanced versions of these questions (with answers).

4. In a locked down environment (other websites cannot be accessed), each student answers their personalised questions.

5. The lecturers can use a selection of the answers to help evaluate the understanding of the student and be used as a component of their project/essay grade (at its most extreme, where very limited understanding is demonstrated - indicating that little of the work is the student’s own work - a project which would otherwise have done well will fail).

We also explored using this for formative assessment - our approach is intended to support learners to use AI tools in ways that enhance learning and understanding rather than undermine it. We piloted this across two diverse student cohorts (computing and psychology) this semester. In this talk we show the details of the process and present the lessons we learned. We also surveyed learners and educators as the first steps towards codesign of the next iteration of this work.