run-llama/llama_index
LlamaIndex is a leading data framework that enables building LLM-powered applications by providing tools for data ingestion, structuring, and advanced querying to augment large language models with private and external data.
Awesome AI Agents › Data Integration and Specialized Solutions
vectordotdev/vector
Vector is an advanced, high-performance, and end-to-end observability data pipeline written in Rust, serving as both an agent and an aggregator. It provides comprehensive control over observability data by enabling the collection, transformation, and routing of logs, metrics, and coming soon, traces. This platform aims to significantly reduce observability costs, facilitate seamless vendor transitions, enhance data quality, and consolidate monitoring agents, thereby improving overall performance and reliability within an observability stack. Maintained by Datadog's Community Open Source Engineering team, Vector emphasizes reliability, end-to-end functionality, and unified data handling. It supports various use cases including cost reduction, vendor independence, data enrichment, and improved performance. With over 100,000 daily downloads and a user base processing over 500TB daily, Vector is utilized by numerous startups and enterprises. The project also provides extensive documentation and robust performance comparisons against alternatives like Filebeat, FluentBit, FluentD, Logstash, and Splunk, demonstrating superior performance in many scenarios. Its correctness tests highlight its reliability in areas such as disk buffer persistence and file rotation.
https://github.com/vectordotdev/vector
Also in Data Integration and Specialized Solutions
pingcap/tidb
TiDB is an open-source, cloud-native, distributed SQL database offering ACID guarantees, horizontal scalability, high availability, HTAP capabilities, and MySQL compatibility.
getzep/graphiti
Graphiti is a framework for building and querying real-time, temporally-aware knowledge graphs designed to support AI agents in dynamic environments with efficient incremental updates and hybrid retrieval methods.
dolthub/dolt
Dolt is a SQL database with Git-like features, enabling full version control over data, including branching, merging, and diffing of data.
cube-js/cube
Cube Core is an open-source semantic layer that enables AI, business intelligence, and embedded analytics by providing a flexible, API-driven platform supporting multiple SQL data sources and real-time analytics.
llmware-ai/llmware
llmware is a unified framework for building enterprise Retrieval-Augmented Generation (RAG) pipelines using small, specialized language models integrated with secure knowledge sources for efficient AI applications.
cocoindex-io/cocoindex
CoCoIndex is an ultra-performant data transformation framework for AI, specializing in incremental processing for data indexing and real-time semantic search with Python and Rust.
electric-sql/electric
Electric is a Postgres sync engine that provides real-time data synchronization, partial replication, and data delivery for modern applications and AI agents.