infiniflow/ragflow
RAGFlow is an open-source Retrieval-Augmented Generation engine that leverages deep document understanding and Large Language Models to provide accurate, citation-backed question-answering from complex and diverse data sources.
Awesome AI Agents › NL AI Frameworks
ucbepic/docetl
DocETL is a comprehensive system designed to facilitate complex document processing and data transformation pipelines using large language models (LLMs). It provides a dual approach for users: an interactive UI playground called DocWrangler for iterative pipeline development and prompt engineering, and a Python package for running production pipelines programmatically. The system is particularly suited for tasks that require sophisticated document processing workflows, leveraging the power of LLMs like ChatGPT or Claude to assist in pipeline creation and execution. The interactive playground, DocWrangler, allows users to experiment with different prompts and pipeline configurations in real-time, making it easier to refine and optimize data processing tasks. Users can build pipelines step-by-step and export the final configurations for production use. This playground can be accessed online or run locally via Docker or manual setup, providing flexibility for development environments. For production use, the Python package supports running pipelines from the command line or within Python code, requiring Python 3.10+ and an OpenAI API key or equivalent for LLM operations. The system also supports integration with AWS Bedrock, enabling the use of AWS-hosted models for processing. DocETL emphasizes ease of use and extensibility, offering detailed documentation, tutorials, and community projects that extend its capabilities, such as conversation generation, text-to-speech, and topic extraction from YouTube transcripts. The project also provides educational resources on UI/UX design and techniques to improve output quality. Overall, DocETL is a powerful tool for developers and data scientists looking to harness LLMs for advanced document processing and ETL workflows, combining interactive development tools with robust production capabilities.
https://github.com/ucbepic/docetl
Also in NL AI Frameworks
ScrapeGraphAI/Scrapegraph-ai
ScrapeGraphAI is an AI-powered Python library that creates intelligent web scraping pipelines using large language models to extract structured data from websites and local documents efficiently.
pydantic/pydantic
Pydantic is a Python library for fast and extensible data validation using Python type hints, enabling developers to define and validate data models efficiently.
apache/doris
Apache Doris is a high-performance, real-time analytical database with a storage-compute integrated architecture, designed for sub-second query response and high concurrency in diverse data analysis scenarios.
business-science/ai-data-science-team
AI Data Science Team is a Python library featuring AI-powered agents that automate and accelerate common data science tasks, including data cleaning, feature engineering, machine learning, and exploratory data analysis, to improve productivity and efficiency.
hitsz-ids/synthetic-data-generator
Synthetic Data Generator (SDG) is a comprehensive framework for generating high-quality, privacy-compliant synthetic structured tabular data using advanced statistical and LLM-based models, optimized for big data applications.
Mintplex-Labs/vector-admin
VectorAdmin is a universal and user-friendly tool suite for managing multiple vector databases, providing full control over vector data with multi-user support, embedding management, and cloud deployment capabilities.
scouter-project/scouter
Scouter is an open source Application Performance Management (APM) tool that monitors and manages the performance of software applications and system resources across various platforms and services.