run-llama/llama_index
LlamaIndex is a leading data framework that enables building LLM-powered applications by providing tools for data ingestion, structuring, and advanced querying to augment large language models with private and external data.
Awesome AI Agents › Data Integration and Specialized Solutions
liweiphys/layra
LAYRA is an advanced visual-first Retrieval-Augmented Generation (RAG) system designed to understand and process documents in a way that preserves their original layout and visual structure. Unlike traditional RAG systems that rely heavily on OCR and tokenization, which often result in loss of layout fidelity and semantic continuity, LAYRA uses pure visual embeddings to treat documents as visually structured artifacts. This approach allows it to maintain the integrity of tables, paragraphs, graphical elements such as charts and figures, and overall document hierarchy, enabling a more human-like understanding of documents. The system is built with a modern technology stack, featuring a responsive and developer-friendly web UI developed using Next.js and TailwindCSS. The backend is powered by FastAPI and is designed with an asynchronous-first architecture that integrates various scalable components including Redis, MySQL, MongoDB, MinIO, and Milvus for efficient data storage and retrieval. LAYRA currently supports PDF document uploads and parsing, converting pages into rich semantic vectors using the Colpali project and the colqwen2.5 visual embedding model. LAYRA leverages the Qwen2.5-VL large language model series for multimodal understanding and plans to extend support to other advanced models like GPT-4o, Claude, and Gemini. The system's architecture supports fast vector search for scalable dense retrieval, enabling users to query documents and receive answers that preserve the visual layout context. This makes LAYRA suitable for enterprise deployment and research exploration, providing a plug-and-play platform for unstructured document understanding enhanced by multimodal AI. The project is actively developed, with future plans to support additional document formats such as Word, PPT, Excel, images, and Markdown, as well as integrating intelligent agents for multi-hop reasoning and advanced document analysis. LAYRA aims to bridge the gap between unstructured document data and AI-driven insights by offering a visual-first, layout-preserving RAG system that goes beyond traditional OCR-based methods.
https://github.com/liweiphys/layra
Also in Data Integration and Specialized Solutions
pingcap/tidb
TiDB is an open-source, cloud-native, distributed SQL database offering ACID guarantees, horizontal scalability, high availability, HTAP capabilities, and MySQL compatibility.
getzep/graphiti
Graphiti is a framework for building and querying real-time, temporally-aware knowledge graphs designed to support AI agents in dynamic environments with efficient incremental updates and hybrid retrieval methods.
dolthub/dolt
Dolt is a SQL database with Git-like features, enabling full version control over data, including branching, merging, and diffing of data.
vectordotdev/vector
Vector is a high-performance, end-to-end observability data pipeline designed to collect, transform, and route logs, metrics, and traces.
cube-js/cube
Cube Core is an open-source semantic layer that enables AI, business intelligence, and embedded analytics by providing a flexible, API-driven platform supporting multiple SQL data sources and real-time analytics.
llmware-ai/llmware
llmware is a unified framework for building enterprise Retrieval-Augmented Generation (RAG) pipelines using small, specialized language models integrated with secure knowledge sources for efficient AI applications.
cocoindex-io/cocoindex
CoCoIndex is an ultra-performant data transformation framework for AI, specializing in incremental processing for data indexing and real-time semantic search with Python and Rust.