infiniflow/ragflow
RAGFlow is an open-source Retrieval-Augmented Generation engine that leverages deep document understanding and Large Language Models to provide accurate, citation-backed question-answering from complex and diverse data sources.
Awesome AI Agents › NL AI Frameworks
Pravko-Solutions/FlashLearn
FlashLearn is a Python library designed to seamlessly integrate large language models (LLMs) into typical data workflows and ETL pipelines using a fit/predict pattern. It provides a simple interface and orchestration capabilities that support up to 1000 calls per minute, enabling users to perform data transformations, classifications, summarizations, rewriting, and custom multi-step tasks with LLMs as if they were standard machine learning components. The core concept revolves around defining each step and task in a pipeline using compact JSON definitions, which makes the pipelines easy to understand, maintain, and extend. FlashLearn supports multiple LLM providers including LiteLLM, Ollama, OpenAI, DeepSeek, and any other OpenAI-compatible clients, offering flexibility in choosing the backend model. Users can "learn" new skills by providing instructions and sample data, similar to training a model, and save these skills as JSON files for reuse across systems. The input data is expected as a list of dictionaries, making it compatible with data from APIs, spreadsheets, or user forms. Once a skill is defined or learned, it can be loaded and applied to data in just a few lines of code, producing structured JSON outputs that map directly to the input records. These outputs can then be used in downstream processes such as database storage, filtering, or further analysis. FlashLearn emphasizes JSON input and output for all operations, facilitating easy chaining and integration with other tools. It includes examples across various domains such as customer service, finance, marketing, personal assistance, product intelligence, sales, and software development, demonstrating its versatility. The library also supports parallel task execution to improve performance. Overall, FlashLearn simplifies the incorporation of LLMs into existing data pipelines, making advanced language model capabilities accessible and manageable within standard data processing workflows.
https://github.com/Pravko-Solutions/FlashLearn
Also in NL AI Frameworks
ScrapeGraphAI/Scrapegraph-ai
ScrapeGraphAI is an AI-powered Python library that creates intelligent web scraping pipelines using large language models to extract structured data from websites and local documents efficiently.
pydantic/pydantic
Pydantic is a Python library for fast and extensible data validation using Python type hints, enabling developers to define and validate data models efficiently.
apache/doris
Apache Doris is a high-performance, real-time analytical database with a storage-compute integrated architecture, designed for sub-second query response and high concurrency in diverse data analysis scenarios.
business-science/ai-data-science-team
AI Data Science Team is a Python library featuring AI-powered agents that automate and accelerate common data science tasks, including data cleaning, feature engineering, machine learning, and exploratory data analysis, to improve productivity and efficiency.
ucbepic/docetl
DocETL is a system for creating and executing complex document processing and data transformation pipelines powered by large language models, featuring an interactive UI playground and a Python package for production use.
hitsz-ids/synthetic-data-generator
Synthetic Data Generator (SDG) is a comprehensive framework for generating high-quality, privacy-compliant synthetic structured tabular data using advanced statistical and LLM-based models, optimized for big data applications.
Mintplex-Labs/vector-admin
VectorAdmin is a universal and user-friendly tool suite for managing multiple vector databases, providing full control over vector data with multi-user support, embedding management, and cloud deployment capabilities.