infiniflow/ragflow
RAGFlow is an open-source Retrieval-Augmented Generation engine that leverages deep document understanding and Large Language Models to provide accurate, citation-backed question-answering from complex and diverse data sources.
Awesome AI Agents › NL AI Frameworks
bosun-ai/swiftide
Swiftide is a Rust-based library designed for building applications that leverage large language models (LLMs) with a focus on fast, streaming data indexing, querying, and agentic functionalities. It enables efficient data ingestion, transformation, and indexing to support Retrieval Augmented Generation (RAG), a technique that enhances prompt responses by integrating relevant external data. The library offers modular and extendable APIs with minimal abstractions, allowing developers to create custom pipelines and agents tailored to their specific needs. Swiftide supports asynchronous and parallel processing, making it suitable for high-performance applications. It integrates with popular LLMs and storage providers, providing ready-to-use pipeline transformations for rapid development from concept to production with minimal code. Key features include a streaming indexing pipeline, an experimental query pipeline, and an agent framework that facilitates building intelligent agents capable of complex tasks. The project is part of the bosun.ai initiative, aiming to advance autonomous code improvement through AI. Swiftide is under active development, with frequent updates and improvements documented on their blog and GitHub repository. The library includes various components such as loaders, transformers, semantic chunkers, and embedders, and supports extending functionality by implementing custom transformers. Example use cases demonstrate indexing local code projects, enriching data with metadata, embedding, storing in vector databases like Qdrant, and querying with subquestion generation and answer retrieval. The agent framework allows building agents that can perform tasks like code search by combining LLMs with specialized tools. Swiftide emphasizes performance, flexibility, and ease of use, making it a powerful tool for developers building advanced LLM-powered applications in Rust.
https://github.com/bosun-ai/swiftide
Also in NL AI Frameworks
ScrapeGraphAI/Scrapegraph-ai
ScrapeGraphAI is an AI-powered Python library that creates intelligent web scraping pipelines using large language models to extract structured data from websites and local documents efficiently.
pydantic/pydantic
Pydantic is a Python library for fast and extensible data validation using Python type hints, enabling developers to define and validate data models efficiently.
apache/doris
Apache Doris is a high-performance, real-time analytical database with a storage-compute integrated architecture, designed for sub-second query response and high concurrency in diverse data analysis scenarios.
business-science/ai-data-science-team
AI Data Science Team is a Python library featuring AI-powered agents that automate and accelerate common data science tasks, including data cleaning, feature engineering, machine learning, and exploratory data analysis, to improve productivity and efficiency.
ucbepic/docetl
DocETL is a system for creating and executing complex document processing and data transformation pipelines powered by large language models, featuring an interactive UI playground and a Python package for production use.
hitsz-ids/synthetic-data-generator
Synthetic Data Generator (SDG) is a comprehensive framework for generating high-quality, privacy-compliant synthetic structured tabular data using advanced statistical and LLM-based models, optimized for big data applications.
Mintplex-Labs/vector-admin
VectorAdmin is a universal and user-friendly tool suite for managing multiple vector databases, providing full control over vector data with multi-user support, embedding management, and cloud deployment capabilities.