Awesome AI Agents › Stateful Serverless Frameworks
Ragbits is a comprehensive framework designed to facilitate the rapid development of Generative AI (GenAI) applications. It provides a modular and scalable set of building blocks that enable developers to build reliable GenAI apps with ease. One of the key features of Ragbits is its flexibility in working with large language models (LLMs), allowing users to swap between over 100 LLMs via LiteLLM or run local models, ensuring adaptability to various use cases. The framework emphasizes type safety by using Python generics to enforce strict type safety in model interactions, enhancing code reliability. Ragbits supports integration with multiple vector stores such as Qdrant and PgVector, allowing users to bring their own vector storage solutions. It also includes developer tools for managing vector stores, query pipelines, and testing prompts directly from the terminal, streamlining the development workflow. The modular installation approach lets users install only the components they need, reducing dependencies and improving performance. For retrieval-augmented generation (RAG) processing, Ragbits can ingest over 20 document formats including PDFs, HTML, spreadsheets, and presentations. It leverages tools like Docling and Unstructured for data processing and supports complex data extraction including tables and images with built-in vision-language model (VLM) support. The framework can connect to various data sources such as S3, GCS, and Azure, and uses Ray-based parallel processing to scale ingestion for large datasets. Deployment and monitoring are made robust with real-time observability through OpenTelemetry and CLI insights. Ragbits includes built-in testing with promptfoo to validate prompts before deployment and features auto-optimization to continuously improve model performance. Additionally, it offers a chat UI for deploying chatbot interfaces with API support, persistence, and user feedback mechanisms. Ragbits is distributed as a set of packages covering core functionalities, agents, document search, evaluation, chat infrastructure, and CLI tools. It is suitable for developers looking to build, deploy, and maintain advanced GenAI applications efficiently with a focus on scalability, flexibility, and developer experience.
https://github.com/deepsense-ai/ragbits
LiteLLM is a Python SDK and proxy server that provides a unified OpenAI-compatible interface to call over 100 large language model APIs from multiple providers, featuring consistent output, retry logic, budget controls, and extensive observability integrations.
Flowise is an open-source drag-and-drop platform that enables users to easily build and deploy customized Large Language Model (LLM) application workflows with a user-friendly interface and flexible deployment options.
Parlant is a Conversation Modeling engine that enables precise, consistent, and reliable control over GenAI-driven conversational agents by enforcing structured behavioral guidelines and adapting dynamically to user interactions.
ToolBench is an open platform for training, serving, and evaluating large language models with advanced tool-use capabilities using a large-scale, richly annotated dataset of real-world APIs.
RivetKit is a stateful serverless framework that enables building scalable, real-time, and collaborative applications deployable across multiple platforms including Rivet, Cloudflare Workers, Bun, and Node.js.
LlamaDeploy is an async-first framework that enables seamless deployment, scaling, and productionization of agentic multi-service workflows built with llama_index, facilitating easy transition from development to cloud-based production environments.
Mirascope is a flexible and user-friendly library that provides a unified interface to work with multiple large language model providers, simplifying AI-driven text generation and information extraction tasks.
Strands Agents Tools is a versatile toolkit that equips AI agents with powerful capabilities for file operations, system interaction, API communication, multimedia processing, memory management, and advanced reasoning to build intelligent applications efficiently.