Awesome AI Agents › Stateful Serverless Frameworks
ToolBench is an open-source platform designed to facilitate the training, serving, and evaluation of large language models (LLMs) with advanced tool-use capabilities. The project focuses on constructing a large-scale, high-quality instruction tuning dataset that enables LLMs to master thousands of diverse real-world APIs, primarily sourced from RapidAPI, a platform hosting over 16,000 REST APIs. The dataset is automatically generated using the latest ChatGPT (gpt-3.5-turbo-16k) with enhanced function call capabilities, and it includes detailed annotations such as reasoning processes, tool execution steps, and results. This comprehensive dataset supports both single-tool and multi-tool scenarios, with multi-tool cases further divided into intra-category and intra-collection types. A novel depth-first search based decision tree (DFSDT) method is employed to improve annotation efficiency and handle complex instructions that traditional methods struggle with. ToolBench provides not only the dataset but also training and evaluation scripts, along with a fine-tuned model called ToolLLaMA, which demonstrates performance comparable to ChatGPT in tool use. The project includes a RapidAPI backend service to simplify API access for users, and it offers a web demo showcasing ToolLLaMA's capabilities. The platform is continuously updated, with recent improvements including a more stable evaluation tool (ToolEval), enhanced model versions, and expanded dataset annotations. The project is intended for research and educational purposes and is distributed under the Apache License 2.0. It aims to empower open-source LLMs with general tool-use abilities by providing a scalable, high-quality resource for instruction tuning. The repository also links to related projects like StableToolBench, which offers a stable local toolbench server based on API response simulation. Overall, ToolBench is a comprehensive ecosystem for advancing the integration of LLMs with real-world APIs, enabling sophisticated tool learning and execution through large-scale, richly annotated datasets and robust evaluation frameworks.
https://github.com/OpenBMB/ToolBench
LiteLLM is a Python SDK and proxy server that provides a unified OpenAI-compatible interface to call over 100 large language model APIs from multiple providers, featuring consistent output, retry logic, budget controls, and extensive observability integrations.
Flowise is an open-source drag-and-drop platform that enables users to easily build and deploy customized Large Language Model (LLM) application workflows with a user-friendly interface and flexible deployment options.
Parlant is a Conversation Modeling engine that enables precise, consistent, and reliable control over GenAI-driven conversational agents by enforcing structured behavioral guidelines and adapting dynamically to user interactions.
RivetKit is a stateful serverless framework that enables building scalable, real-time, and collaborative applications deployable across multiple platforms including Rivet, Cloudflare Workers, Bun, and Node.js.
LlamaDeploy is an async-first framework that enables seamless deployment, scaling, and productionization of agentic multi-service workflows built with llama_index, facilitating easy transition from development to cloud-based production environments.
Ragbits is a modular framework providing building blocks for rapid development, deployment, and monitoring of scalable and reliable Generative AI applications with flexible LLM integration and advanced document processing capabilities.
Mirascope is a flexible and user-friendly library that provides a unified interface to work with multiple large language model providers, simplifying AI-driven text generation and information extraction tasks.
Strands Agents Tools is a versatile toolkit that equips AI agents with powerful capabilities for file operations, system interaction, API communication, multimedia processing, memory management, and advanced reasoning to build intelligent applications efficiently.