Awesome AI AgentsGenerative Software Frameworks

SqueezeAILab/LLMCompiler

⭐ 1881 Python added to this list on 2025-04-19 repository created 2023-12-06

LLMCompiler is a cutting-edge framework designed to optimize the orchestration of parallel function calling in large language models (LLMs), including both open-source and closed-source models. The framework addresses the limitations of current multi-function calling methods, which often require sequential execution of tasks, leading to high latency, increased costs, and sometimes inaccurate results. By leveraging the reasoning capabilities of LLMs, LLMCompiler automatically identifies which tasks can be executed in parallel and which are interdependent, thereby decomposing complex problems into multiple parallelizable tasks. This results in significant improvements in latency, cost efficiency, and accuracy across various benchmarks such as HotpotQA, Movie Recommendation, and ParallelQA. Users specify the tools and optional in-context examples, and LLMCompiler computes an optimized orchestration plan for function calls. The framework supports integration with popular models like LLaMA, OpenAI’s GPT models, and custom models served via the vLLM framework. It also supports endpoints like Azure and Friendli, enhancing its versatility. Installation involves setting up a conda environment and installing dependencies, with straightforward commands to run benchmarks and custom models. The project is actively maintained with plans to incorporate features like Tree-of-Thoughts evaluation. LLMCompiler is part of ongoing research presented at ICML 2024, and its source code and documentation provide comprehensive guidance for users to implement and extend the framework for their specific needs. The project is notable for its ability to improve the efficiency and effectiveness of multi-function calling in LLMs, making it a valuable tool for developers and researchers working with advanced language models.

https://github.com/SqueezeAILab/LLMCompiler

accuracy-improvementazure-endpointbenchmarkscost-savingefficient-inferencefriendli-endpointfunction-callinghotpotqaicml-2024large-language-modelslatency-reductionllamallama2llmllm-agentllm-agentsllm-frameworkllmcompilerllmsmovie-recommendationmulti-function-callingnatural-language-processingnlpopenai-gptparallel-function-callparallel-function-callingparallelqatask-orchestrationtransformertree-of-thoughtsvllm

Also in Generative Software Frameworks

nexu-io/open-design

Open Design is a local-first, open-source desktop application that serves as an alternative to Claude Design, integrating with various coding agents to generate web, desktop, and mobile prototypes,...

deepset-ai/haystack

Haystack is an AI orchestration framework for building customizable, production-ready NLP applications using large language models, vector search, and advanced retrieval techniques.

op7418/guizang-ppt-skill

AI agent skill for generating polished HTML slide decks with various design styles, image prompts, social covers, and a WebGL/low-power presentation runtime.

HKUDS/Agentic-AIGC

ViMax is an agentic AI system for end-to-end video generation, transforming raw ideas, novels, or scripts into complete video content by orchestrating scriptwriting, storyboarding, character creati...

The-Pocket/PocketFlow

Pocket Flow is a minimalist 100-line LLM framework that uses a graph-based abstraction to enable expressive multi-agent systems, workflows, and retrieval-augmented generation, designed for high productivity and minimal bloat.

Netflix/metaflow

Metaflow is a human-centric framework by Netflix that streamlines building, managing, and deploying real-life AI and machine learning systems from prototyping to production at scale.

presenton/presenton

Presenton is an open-source AI-powered presentation generator and API that runs locally, offering customizable layouts, multiple AI model integrations, versatile image generation, and privacy-focused presentation creation and export.

thesysdev/openui

OpenUI is a full-stack Generative UI framework that offers a compact, streaming-first language for model-generated UI, a React runtime, and token-efficient chat interfaces.