ollama/ollama
Ollama is a versatile platform that enables easy deployment, customization, and management of large language models locally across multiple operating systems with support for various model formats and a REST API.
Awesome AI Agents › AI Agent Deployment
palico-ai/palico-ai
Palico AI is an integrated framework designed to facilitate the iterative development, performance improvement, and production deployment of Large Language Model (LLM) applications. It addresses the complexities involved in building LLM applications by providing a comprehensive tech stack that supports continuous experimentation with different models, prompts, retrieval-augmented generation (RAG) datasets, call chaining, and custom code. The framework offers complete flexibility to developers, allowing them to build any LLM application by implementing a simple Chat function. It supports advanced features such as streaming messages, memory management for conversation states, tool executions for client and server-side agents, feature flags for runtime swapping of models and logic, and monitoring through detailed logs and traces. Palico AI integrates seamlessly with popular tools and libraries in the LLM ecosystem, including Langchain, LlamaIndex, Portkey, OpenAI, Anthropic, Cohere, Azure, AWS Bedrock, GCP Vertex AI, Pinecone, PG Vector, and Chroma. This extensive integration capability ensures that developers can leverage their preferred services and databases while using Palico as the core framework. The framework also includes a Playground UI for local preview and testing of changes, enabling rapid development cycles. It supports continuous integration and pull-request previews, making it suitable for production environments. Additionally, Palico provides tools for creating experiments and evaluations to systematically improve application performance, along with client SDKs and REST APIs for frontend integration. A control panel called Palico Studio is available for managing applications efficiently. Overall, Palico AI is a powerful and flexible solution for developers looking to build, optimize, and deploy LLM applications with ease, offering a rich set of features to handle the iterative nature of LLM development and deployment.
https://github.com/palico-ai/palico-ai
Also in AI Agent Deployment
alibaba/nacos
Nacos is a dynamic service discovery, configuration, and service management platform designed to simplify building and managing cloud-native applications and microservices.
e2b-dev/E2B
E2B is an open-source secure cloud runtime infrastructure that enables safe execution of AI-generated code in isolated sandboxes for AI applications and agents.
GoogleCloudPlatform/agent-starter-pack
A collection of production-ready Generative AI Agent templates for Google Cloud that accelerates development by addressing deployment, evaluation, customization, and observability challenges.
zebbern/claude-code-guide
Claude Code Guide is a comprehensive and authoritative resource providing detailed instructions, tips, and references for using the Claude Code CLI tool to interact with the Claude AI model, including advanced features and MCP integration.
boxlite-ai/boxlite
Rust engine giving AI agents hardware-isolated persistent micro-VMs that run any OCI image, embeddable as a daemonless library in Python, Node, Go, Rust or C, or run as a CLI, REST server or cloud control plane.
jim-schwoebel/awesome_ai_agents
A comprehensive and community-driven repository offering over 1,500 curated resources, tools, and projects related to AI agents for various applications and development needs.
Forethought-Technologies/AutoChain
AutoChain is a lightweight and extensible framework that simplifies building and automatically evaluating custom generative agents powered by large language models using custom tools and simulated multi-turn conversations.