Conference Agenda

Within the current cultural heritage and wider research landscape, discussions around AI are predominating. Particularly, there are poignant and critical discussions around ethical concerns of engaging these tools in conjunction with the complex landscape of humanities data (e.g. cultural heritage collections). These concerns affect communities of all scales: individuals and groups represented within datasets, those curating and preserving these data, those engaging in research with the data, the public more widely and/or policy makers, to name a few examples. AI and its responsible use should be able to be accounted for and be accountable to the breadth and depth of human experience – something that is currently woefully short of being achieved. Quality assessment and safeguarding methods, such as auditing, usually rest in the hands of individuals developing AI or working in associated disciplinary areas. This limitation severely limits the diversity of voices, perspectives and audiences represented within these audits and therefore the feedback represented in AI developments.

This panel opens discussion to concepts of participatory AI auditing – a movement that advocates for inclusive approaches to audit, that allow anybody, with or without an AI background, to appraise and contribute to the governance of AI. Intended as a democratised approach to assessing the appropriateness of AI applications in given contexts, participatory auditing is an evolving research area that sits across multiple disciplines. The RAI UK funded PHAWM (Participatory Harm Auditing Workbenches and Methodologies) project is one research group whose aim is to co-design and develop participatory AI auditing tools and methodologies. This panel will illuminate some of the project’s central findings and developments, with particular focus on a cultural heritage – a use cases of PHAWM. The panel aims to stimulate critical discussion around the broader applicability of participatory auditing to facilitate the use of AI for engaged and responsible scholarship within the humanities.

The Papers:

The first paper, entitled “Co-designing understandings of participatory audits”, will explore the stakeholder research underpinning the workbench and methodological outputs. This will offer meaningful insights into how auditing is perceived by cultural heritage practitioners, as well as the specific requirements to align auditing methods to established practice and workflows. It will also provide context as to the importance of co-design methodologies within this sphere, analysing the current landscape of research in this area and the challenges it faces.

The second paper is entitled “Embedding participatory frameworks into auditing methodologies”. It will discuss how the co-designed findings were translated into a methodological framework that can support the critical assessment of AI with or without an associated audit workbench. The PHAWM Methodology adopts accessibility as a central consideration, offering multiple entry points to support useful navigation of and engagement with participatory auditing for cultural heritage practitioners and others without training in AI development. Beyond this, the paper will explore use cases for the methodology, examining how it performs ‘in action’ across multiple contexts.

The third paper, entitled “The PHAWM workbench: a participatory AI auditing tool”, will present the PHAWM workbench prototype offering a walkthrough of the functionality, as well as discussing, in more detail, a range of features included and the research involved in their creation. In particular, the paper will comment on the design of the interface, indicating how diverse auditors, such as those present within cultural heritage institutions are supported to explore and assess AI applications in a way that emboldens them to give robust and critical responses respective of their own area of expertise.

The Speakers:

This panel represents the cumulative work of the PHAWM team as a whole (cf. phawm.org) with particular contributions made by the Cultural Heritage Use-Case. The speakers will be Dr Zoe Bartliff, Dr Emily O’Hara, Dr Ayah Soufan and other direct authors include Dr Yunhyong Kim, Dr Kate Simpson, Dr Yashar Moshfeghi and Dr Ian Ruthven.

Participation:

A key element of the panel discussion is to secure participant feedback for integration into the emergent research, with particular attention to the relevance, appropriateness and areas for development of the workbenches and methodologies. The project is centred upon a co-design philosophy meaning that events such as this are essential for enriching the research underpinning the workbench and methodology developments, as well as for finetuning the shape of these tools. As such, all attendees will be asked to contribute fully to workshop style discussions at periodic points within the session. Participation in the panel indicates consent to this data being gathered and used in the ongoing research, however, all data will be aggregated and no personal data will be gathered.