infiniflow/ragflow
RAGFlow is an open-source Retrieval-Augmented Generation engine that leverages deep document understanding and Large Language Models to provide accurate, citation-backed question-answering from complex and diverse data sources.
Awesome AI Agents › NL AI Frameworks
hitsz-ids/synthetic-data-generator
The Synthetic Data Generator (SDG) is a specialized framework designed to generate high-quality structured tabular data that mimics the essential characteristics of original datasets without containing any sensitive information. This makes the synthetic data exempt from privacy regulations such as GDPR and ADPPA, allowing safe use in various domains including data sharing, model training, debugging, system development, and testing. SDG supports a wide range of statistical data synthesis algorithms and integrates advanced models such as LLM-based synthetic data generation, which can generate synthetic data without requiring training data by leveraging metadata. It also offers off-table feature inference, where new column data can be inferred based on existing table data and the knowledge embedded in large language models. The framework includes a Data Processor module that handles data preprocessing and postprocessing, such as converting datetime columns to appropriate formats, managing null values, and supporting plugin systems for customization. SDG is optimized for big data, significantly reducing memory consumption and enabling training on large-scale datasets with thousands of categorical entries. It features models like GaussianCopula integrated into its system, improving synthetic data quality by detecting and specifying data column relationships automatically. SDG is actively maintained with continuous improvements, including performance enhancements, new model integrations, and expanded metadata support for single and multiple tables. The project provides extensive documentation, colab examples for practical use cases, and a roadmap for future development. It is designed to be a robust, scalable, and privacy-conscious solution for generating synthetic tabular data, making it valuable for researchers, developers, and organizations needing high-quality synthetic datasets for various applications.
https://github.com/hitsz-ids/synthetic-data-generator
Also in NL AI Frameworks
ScrapeGraphAI/Scrapegraph-ai
ScrapeGraphAI is an AI-powered Python library that creates intelligent web scraping pipelines using large language models to extract structured data from websites and local documents efficiently.
pydantic/pydantic
Pydantic is a Python library for fast and extensible data validation using Python type hints, enabling developers to define and validate data models efficiently.
apache/doris
Apache Doris is a high-performance, real-time analytical database with a storage-compute integrated architecture, designed for sub-second query response and high concurrency in diverse data analysis scenarios.
business-science/ai-data-science-team
AI Data Science Team is a Python library featuring AI-powered agents that automate and accelerate common data science tasks, including data cleaning, feature engineering, machine learning, and exploratory data analysis, to improve productivity and efficiency.
ucbepic/docetl
DocETL is a system for creating and executing complex document processing and data transformation pipelines powered by large language models, featuring an interactive UI playground and a Python package for production use.
Mintplex-Labs/vector-admin
VectorAdmin is a universal and user-friendly tool suite for managing multiple vector databases, providing full control over vector data with multi-user support, embedding management, and cloud deployment capabilities.
scouter-project/scouter
Scouter is an open source Application Performance Management (APM) tool that monitors and manages the performance of software applications and system resources across various platforms and services.