Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration
Modular Pluralism plugs specialized community LMs into base LLMs to enable Overton, steerable, and distributional pluralistic alignment across diverse communities.
Published 202415 citationsPaper ↗
Only vote on papers you've read. Sign in with GitHub to vote.
Modular Pluralism delivers an elegant, black-box-compatible framework for patching LLMs with specialized community modules to serve diverse values.
Abstract
While existing alignment paradigms have been integral in developing large language models (LLMs), LLMs often learn an averaged human preference and struggle to model diverse preferences across cultures, demographics, and communities.We propose MODULAR PLU-RALISM, a modular framework based on multi-LLM collaboration for pluralistic alignment: it "plugs into" a base LLM a pool of smaller but specialized community LMs, where models collaborate in distinct modes to flexibility support three modes of pluralism: Overton, steerable, and distributional (Sorensen et al., 2024b).MODULAR PLURALISM is uniquely compatible with black-box LLMs and offers the modular control of adding new community LMs for previously underrepresented communities.We evaluate MODULAR PLURAL-ISM with six tasks and four datasets featuring questions/instructions with value-laden and perspective-informed responses.Extensive experiments demonstrate that MODULAR PLU-RALISM advances the three pluralism objectives across six black-box and open-source LLMs.Further analysis reveals that LLMs are generally faithful to the inputs from smaller community LLMs, allowing seamless patching by adding a new community LM to better cover previously underrepresented communities.1