Human and large language model judgments of cognitive impairment from language: An explainable artificial intelligence approach

Abstract

INTRODUCTION: We examine how non-expert humans (young adults unfamiliar with dementia) and large language models (LLMs) perceive dementia in transcribed texts-recognizing signs that may indicate cognitive decline. Human perception is important, as it is often the driver for seeking medical evaluation. LLM perception is equally interesting given their potential as screening tools. METHODS: Humans and LLMs intuitively judged whether transcribed picture descriptions came from dementia patients or healthy controls. We represented texts using high-level, expert-guided features and used logistic regression to model perceptions and analyze coefficients. RESULTS: Human judgments are inconsistent, relying on a narrow and sometimes misleading set of cues. LLMs use a richer, more clinically aligned feature set. Both groups show a tendency toward false negatives. DISCUSSION: This work highlights the need to educate humans and LLMs to recognize a broader range of dementia-related linguistic signals. It also underscores the value of interpretability in dementia research. Highlights: Explainable artificial intelligence (AI) uncovers linguistic cues that humans and large language models (LLMs) associate with dementia.LLMs allow scalable extraction of expert-defined features on picture descriptions.LLMs use broader cues than humans to detect dementia and better align with diagnoses.Humans and LLMs exhibit false negatives; LLMs view fluency as cognitive health.Understanding non-expert perceptions can guide education and improve early awareness.