mudler/LocalAI
LocalAI is an open-source AI engine that enables running various AI models (LLMs, vision, voice, image, video) on any hardware, including CPU-only systems, with a composable and extensible architec...
Awesome AI Agents › LLM Framework Tools
awslabs/generative-ai-cdk-constructs
AWS Generative AI CDK Constructs is an open-source library that extends the AWS Cloud Development Kit (AWS CDK) to provide well-architected, multi-service patterns for building generative AI solutions on AWS. The library offers high-level abstractions called constructs, which are reusable architectural patterns designed to simplify the deployment and management of generative AI infrastructure. These constructs encapsulate best practices and default configurations, enabling developers to quickly define and deploy AI solutions in a predictable and repeatable manner using infrastructure as code. The project supports multiple programming languages including TypeScript, Python, C#, Go, and Java, making it accessible to a wide range of developers. It includes constructs for deploying foundation models using Amazon SageMaker, including models from Amazon SageMaker JumpStart, Hugging Face, and custom models stored in S3. Additionally, it provides constructs for monitoring Amazon Bedrock services with Amazon CloudWatch dashboards. The library is designed to be modular and extensible, organized into logical modules using object-oriented principles. It is actively developed and marked as experimental, meaning it may undergo non-backward compatible changes. The project encourages contributions and provides design and development guidelines for adding new constructs. AWS Generative AI CDK Constructs aims to accelerate the development of generative AI applications by abstracting complex AWS service integrations into simple, reusable components. This helps developers focus on building AI capabilities rather than managing infrastructure details. The project also maintains compatibility with specific AWS CDK versions, ensuring smooth integration and updates. Overall, this library is a valuable tool for developers looking to leverage AWS services for generative AI, providing a structured and efficient way to deploy AI models and related infrastructure using AWS best practices.
https://github.com/awslabs/generative-ai-cdk-constructs
Also in LLM Framework Tools
langfuse/langfuse
Langfuse is an open source LLM engineering platform that enables collaborative development, monitoring, evaluation, and debugging of AI applications with comprehensive observability, prompt management, and integration features.
567-labs/instructor
Instructor is a popular Python library that simplifies working with structured outputs from large language models by providing validation, retries, and streaming support across multiple LLM providers.
TEN-framework/ten-framework
TEN Framework is a real-time, distributed, cloud-edge collaborative multimodal AI agent framework supporting multiple programming languages for building high-performance AI applications.
google/adk-samples
A collection of sample agents built with the Agent Development Kit (ADK) to accelerate development of conversational and multi-agent workflows, designed for customization and integration with Google Cloud services.
truefoundry/cognita
Cognita is an open-source, modular, and production-ready RAG framework by TrueFoundry that simplifies building, customizing, and deploying scalable Retrieval Augmented Generation applications with a no-code UI and extensive integration features.
lemony-ai/cascadeflow
In-process Python and TypeScript library that cascades model calls inside the agent loop, routing each query or tool call between small and flagship models by quality, cost, latency and budget.
shroominic/codeinterpreter-api
An open-source API that implements the ChatGPT Code Interpreter for sandboxed Python code execution, enabling data analysis, visualization, and interactive AI-driven computational tasks with local execution and scalable production deployment.