Awesome AI Agents › Multi-Agent Frameworks

composable-models/llm_multiagent_debate

⭐ 557 Python added to this list on 2025-04-19 repository created 2023-05-23

The project "Improving Factuality and Reasoning in Language Models through Multiagent Debate" is a research implementation based on a paper presented at ICML 2024. It focuses on enhancing the factual accuracy and reasoning capabilities of language models by employing a multiagent debate framework. This approach involves multiple agents debating to improve the quality of answers generated by language models, aiming to reduce errors and increase reliability in various tasks. The repository provides preliminary code and resources to run experiments on several tasks including arithmetic, Grade School Math (GSM), biographies, and the Massive Multitask Language Understanding (MMLU) benchmark. Each task has dedicated subfolders with scripts to generate and evaluate answers using the multiagent debate method. For example, users can generate and evaluate math problem solutions by running specific Python scripts in the respective directories. The project also offers access to debate logs for further analysis and references an external implementation of multiagent debate on open-source language models. The datasets used for evaluation, such as GSM and MMLU, are linked for easy access. The repository encourages citation of the original paper and provides a bibtex entry for academic use. Overall, this project is valuable for researchers and practitioners interested in improving language model performance through collaborative agent interactions. It demonstrates a novel technique to address challenges in factuality and reasoning, providing tools and datasets to replicate and extend the research. The modular structure allows users to experiment with different tasks and evaluate the effectiveness of multiagent debate in enhancing language model outputs.

https://github.com/composable-models/llm_multiagent_debate

arithmeticbiographiescollaborative-agentsdatasetsdebate-logsevaluationfactualitygrade-school-mathgsmicml-2024language-modelsmmlumultiagent-debatereasoningresearch-implementation