mudler/LocalAI
LocalAI is an open-source AI engine that enables running various AI models (LLMs, vision, voice, image, video) on any hardware, including CPU-only systems, with a composable and extensible architec...
Awesome AI Agents › LLM Framework Tools
lemony-ai/cascadeflow
cascadeflow is an in-process intelligence layer for AI agents, published as the cascadeflow Python package and the @cascadeflow/core npm package under an MIT license. Instead of sitting at the HTTP boundary like an external proxy, it runs inside the agent execution loop and makes per-step decisions about which model handles a given query or tool call. The library uses speculative execution: a smaller or domain-specific model answers first, the response is scored, and the request escalates to a flagship model only when the quality check fails. The README bases this on the observation that 40 to 70 percent of queries do not require slow, expensive flagship models, and that smaller domain-specific models often outperform large general-purpose ones on specialised tasks. Reported savings are 69 percent on MT-Bench, 93 percent on GSM8K, 52 percent on MMLU and 80 percent on TruthfulQA while retaining 96 percent of GPT-5 quality. Beyond cost, cascadeflow exposes controls for latency, quality, budget, compliance and energy, including per-tool-call budget gating and runtime enforcement actions such as stop, deny_tool and switch_model. It records per-step decision traces for auditing and accumulates insight from every model call, tool result and quality score across runs, and business KPI weights and targets can be injected during agent loops. The stated overhead is under five milliseconds because the decisions happen in process rather than over a network round trip. Integrations ship for LangChain, the OpenAI Agents SDK, CrewAI, PydanticAI, Google ADK, n8n, the Vercel AI SDK, OpenClaw and Hermes Agent, the last of which supports per-skill model cascading, task-complexity cascading, topic-aware subagent cascading and an observe-only rollout mode without taking over provider credentials, base URLs, fallback chains or API modes. It targets teams running production agents that need cost control and auditable routing decisions.
https://github.com/lemony-ai/cascadeflow
Also in LLM Framework Tools
langfuse/langfuse
Langfuse is an open source LLM engineering platform that enables collaborative development, monitoring, evaluation, and debugging of AI applications with comprehensive observability, prompt management, and integration features.
567-labs/instructor
Instructor is a popular Python library that simplifies working with structured outputs from large language models by providing validation, retries, and streaming support across multiple LLM providers.
TEN-framework/ten-framework
TEN Framework is a real-time, distributed, cloud-edge collaborative multimodal AI agent framework supporting multiple programming languages for building high-performance AI applications.
google/adk-samples
A collection of sample agents built with the Agent Development Kit (ADK) to accelerate development of conversational and multi-agent workflows, designed for customization and integration with Google Cloud services.
truefoundry/cognita
Cognita is an open-source, modular, and production-ready RAG framework by TrueFoundry that simplifies building, customizing, and deploying scalable Retrieval Augmented Generation applications with a no-code UI and extensive integration features.
shroominic/codeinterpreter-api
An open-source API that implements the ChatGPT Code Interpreter for sandboxed Python code execution, enabling data analysis, visualization, and interactive AI-driven computational tasks with local execution and scalable production deployment.
agentscope-ai/ReMe
Local-first memory layer for AI agents that converts conversations and resources into readable markdown memory nodes and retrieves them through a hybrid of wikilinks, BM25 and embeddings.