XAI Evaluation Cards: A Practical Method for Designing Human-Centred XAI Evaluations
XAI Evaluation Cards provide a card-sorting method to systematically design human-centered evaluations of explainable AI systems across disciplines.
Published Oct 1, 2026arXiv ↗
Only vote on papers you've read. Sign in with GitHub to vote.
The paper offers a valuable 82-study taxonomy and practical printable cards that help teams prioritize XAI evaluation dimensions, though critics argue the framework catalogs confusion rather than resolving it and the card-sorting method remains an unproven…
Abstract
Evaluating explainable AI (XAI) systems from a human-centred approach requires researchers to select from numerous evaluation dimensions and measures, often in an ad hoc and fragmented manner. This paper introduces a method to help HCI, computer science, designers and social science researchers systematically evaluate XAI systems. The approach is based on an updated XAI-specific evaluation framework derived from an analysis of 82 studies. Using this framework, we developed a card-sorting method with 36 cards to help researchers prioritise relevant evaluation aspects. The process was tested with two research groups (n = 13) across five projects. The XAI Evaluation Cards are available as a printable appendix, along with an online repository of methods from previous XAI studies. Although not exhaustive, our findings indicate that the card-sorting approach can organise and streamline the design of the evaluation process, encouraging a more comprehensive and multidisciplinary assessment of XAI systems in research and development.