ScrapeGraphAI/Scrapegraph-ai
ScrapeGraphAI is an AI-powered Python library that creates intelligent web scraping pipelines using large language models to extract structured data from websites and local documents efficiently.
Awesome AI Agents › NL AI Frameworks
infiniflow/ragflow
RAGFlow is an open-source Retrieval-Augmented Generation (RAG) engine designed for deep document understanding and knowledge extraction from complex and unstructured data formats. It provides a streamlined workflow that integrates Large Language Models (LLMs) to deliver accurate and truthful question-answering capabilities, supported by well-founded citations from diverse data sources. The engine is capable of handling a wide variety of document types including Word, slides, Excel, text files, images, scanned copies, structured data, and web pages, making it highly versatile for business applications of any scale. Key features of RAGFlow include deep document understanding that enables knowledge extraction from complicated formats, intelligent and explainable template-based chunking for data processing, and grounded citations that reduce hallucinations by providing traceable references to support answers. The system architecture supports multi-modal models that can interpret images within documents like PDFs or DOCX files, and it integrates internet search capabilities for enhanced reasoning and research functionalities. RAGFlow offers an automated and effortless RAG workflow with configurable LLMs and embedding models, multiple recall mechanisms paired with fused re-ranking, and intuitive APIs for seamless integration into business environments. It supports Docker deployment with options for CPU or GPU acceleration, and it requires moderate system resources to run effectively. The project is actively maintained with frequent updates that improve document layout analysis, keyword extraction, related question generation, and knowledge graph optimization. The project is well-documented and provides a demo for users to explore its capabilities. It is licensed under Apache 2.0 and encourages community contributions. Overall, RAGFlow is a powerful tool for organizations looking to leverage advanced AI-driven document understanding and retrieval-augmented generation to enhance their data processing and question-answering systems.
https://github.com/infiniflow/ragflow
Also in NL AI Frameworks
pydantic/pydantic
Pydantic is a Python library for fast and extensible data validation using Python type hints, enabling developers to define and validate data models efficiently.
apache/doris
Apache Doris is a high-performance, real-time analytical database with a storage-compute integrated architecture, designed for sub-second query response and high concurrency in diverse data analysis scenarios.
business-science/ai-data-science-team
AI Data Science Team is a Python library featuring AI-powered agents that automate and accelerate common data science tasks, including data cleaning, feature engineering, machine learning, and exploratory data analysis, to improve productivity and efficiency.
ucbepic/docetl
DocETL is a system for creating and executing complex document processing and data transformation pipelines powered by large language models, featuring an interactive UI playground and a Python package for production use.
hitsz-ids/synthetic-data-generator
Synthetic Data Generator (SDG) is a comprehensive framework for generating high-quality, privacy-compliant synthetic structured tabular data using advanced statistical and LLM-based models, optimized for big data applications.
Mintplex-Labs/vector-admin
VectorAdmin is a universal and user-friendly tool suite for managing multiple vector databases, providing full control over vector data with multi-user support, embedding management, and cloud deployment capabilities.
scouter-project/scouter
Scouter is an open source Application Performance Management (APM) tool that monitors and manages the performance of software applications and system resources across various platforms and services.