FlowiseAI/Flowise
Flowise is an open-source drag-and-drop platform that enables users to easily build and deploy customized Large Language Model (LLM) application workflows with a user-friendly interface and flexible deployment options.
Awesome AI Agents › Stateful Serverless Frameworks
BerriAI/litellm
LiteLLM is a Python SDK and proxy server designed to unify access to over 100 large language model (LLM) APIs through a consistent OpenAI-compatible interface. It supports a wide range of LLM providers including Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq, and more. The project offers a gateway that translates user inputs into the appropriate API calls for completion, embedding, and image generation endpoints across these providers, ensuring consistent output formatting. This means that text responses are always accessible in a standardized location within the response structure, simplifying integration and usage. LiteLLM includes advanced features such as retry and fallback logic to handle multiple deployments and providers seamlessly, allowing for robust and reliable API calls. It also supports setting budgets and rate limits per project, API key, and model, making it suitable for enterprise use cases where cost control and usage monitoring are critical. The proxy server can be deployed easily using Docker images, with stable releases undergoing rigorous load testing. The SDK supports synchronous, asynchronous, and streaming API calls, enabling developers to choose the best approach for their applications. It also integrates with various observability and logging tools like Lunary, MLflow, Langfuse, DynamoDB, S3 Buckets, Helicone, Promptlayer, Traceloop, Athina, and Slack, providing comprehensive monitoring and analytics capabilities. LiteLLM is actively maintained and encourages community contributions for adding support for new providers. It is backed by Y Combinator and offers extensive documentation, including migration guides, usage examples, and deployment instructions. The project aims to simplify and standardize interaction with diverse LLM APIs, making it easier for developers to build AI-powered applications without worrying about provider-specific differences.
https://github.com/BerriAI/litellm
Also in Stateful Serverless Frameworks
emcie-co/parlant
Parlant is a Conversation Modeling engine that enables precise, consistent, and reliable control over GenAI-driven conversational agents by enforcing structured behavioral guidelines and adapting dynamically to user interactions.
OpenBMB/ToolBench
ToolBench is an open platform for training, serving, and evaluating large language models with advanced tool-use capabilities using a large-scale, richly annotated dataset of real-world APIs.
rivet-gg/rivetkit
RivetKit is a stateful serverless framework that enables building scalable, real-time, and collaborative applications deployable across multiple platforms including Rivet, Cloudflare Workers, Bun, and Node.js.
run-llama/llama_deploy
LlamaDeploy is an async-first framework that enables seamless deployment, scaling, and productionization of agentic multi-service workflows built with llama_index, facilitating easy transition from development to cloud-based production environments.
deepsense-ai/ragbits
Ragbits is a modular framework providing building blocks for rapid development, deployment, and monitoring of scalable and reliable Generative AI applications with flexible LLM integration and advanced document processing capabilities.
Mirascope/mirascope
Mirascope is a flexible and user-friendly library that provides a unified interface to work with multiple large language model providers, simplifying AI-driven text generation and information extraction tasks.
strands-agents/tools
Strands Agents Tools is a versatile toolkit that equips AI agents with powerful capabilities for file operations, system interaction, API communication, multimedia processing, memory management, and advanced reasoning to build intelligent applications efficiently.