Awesome AI Agents › AI Workflow Orchestrators

anarchy-ai/LLM-VM

⭐ 490 Python added to this list on 2025-03-09 repository created 2023-05-19

The Anarchy LLM-VM is an open-source project aimed at creating a highly optimized and opinionated backend server for running large language models (LLMs) with advanced features. It serves as a virtual machine or interpreter for human language, coordinating between data, models, prompts, and tools to enhance the capabilities and speed of LLMs. The project supports open-source LLMs and allows running them locally, which helps reduce costs, increase flexibility, and speed up AGI development. Key features include tool usage, persistent stateful memory, live data augmentation, data and task fine-tuning, output templating, a web playground, API endpoints, student-teacher distillation, data synthesis, load-balancing, and orchestration. The LLM-VM is designed to optimize batch calls and is model and architecture agnostic, ensuring efficient use of resources. The project is currently in beta and development is paused, but it offers a rich set of functionalities such as implicit agents for tool integration, inference optimization through batching, sparse inference, quantization, and distillation, and task auto-optimization to train smaller efficient models from larger ones. It provides a Python library and HTTP endpoints for easy integration and usage. The roadmap includes features like live data augmentation, a web playground, load-balancing, output templating, and persistent stateful memory. Installation is straightforward with support for Python 3.10 and above, and the project can be installed via pip or by cloning the repository. It supports multiple operating systems including MacOS, Linux, and Windows, with detailed setup instructions provided. The project emphasizes community involvement and transparency, aligning with its mission to democratize AGI development using open-source models.

https://github.com/anarchy-ai/LLM-VM

agiagi-developmentanarchyapi-endpointsartificial-intelligencebatchingcommunitycost-reductiondata-augmentationdata-synthesisdeep-learningdistillationdistillation-modelfine-tuningflexibilityhttp-endpointsinference-optimizationinterpreterlarge-language-modelsllmllm-agentllm-inferencellm-localllm-trainingllm-vmllmsload-balancinglocal-model-runningmachine-learningopen-sourceopen-source-modelsorchestrationoutput-templatingpersistent-memorypython-libraryquantizationsparse-inferencestudent-teacher-distillationtask-auto-optimizationtool-usagetransparencyvirtual-machineweb-playground