Awesome AI Agents › Multi-Agent Frameworks

microsoft/JARVIS

⭐ 25262 Python added to this list on 2025-04-19 repository created 2023-03-30

JARVIS is a collaborative AI system developed by Microsoft that integrates large language models (LLMs) with a wide range of expert machine learning models from the HuggingFace Hub to solve complex AI tasks. The system uses language as an interface to connect LLMs with various AI models, enabling sophisticated task automation and multi-model collaboration. The core architecture consists of an LLM acting as a controller and numerous expert models serving as executors. The workflow is divided into four main stages: Task Planning, where the LLM (e.g., ChatGPT) analyzes user requests and breaks them down into solvable tasks; Model Selection, where the LLM selects appropriate expert models based on their capabilities; Task Execution, where the selected models are invoked to perform their tasks; and Response Generation, where the LLM integrates the outputs from all models to generate a coherent response. JARVIS supports various deployment modes including server, web, Gradio demo, and CLI, providing flexibility for different user needs. It can run locally with substantial hardware requirements or leverage cloud-based HuggingFace Inference Endpoints for lighter setups. The system is designed to be extensible and continuously evolving, with recent additions like EasyTool for simplified tool usage and TaskBench for benchmarking task automation capabilities of LLMs. JARVIS aims to push the boundaries of artificial general intelligence (AGI) research by enabling seamless collaboration between LLMs and specialized AI models. The project includes comprehensive documentation, setup instructions, and example use cases demonstrating its ability to handle complex multi-step tasks such as image processing, text analysis, and video generation. It also provides APIs for accessing intermediate task planning and execution results, enhancing transparency and control. Overall, JARVIS represents a significant step towards building intelligent systems that can autonomously plan, execute, and integrate diverse AI capabilities to address real-world problems.

https://github.com/microsoft/JARVIS

agiai-modelsai-researchapisartificial-general-intelligencechatgptcli-modedeep-learningeasytoolgradio-demohuggingface-hubimage-processingjarvislarge-language-modelsllmmodel-selectionmulti-model-collaborationplatformpytorchresponse-generationserver-modetask-automationtask-executiontask-planningtaskbenchtext-analysisvideo-generationweb-interface