Awesome AI Agents › NL AI Frameworks

apache/doris

⭐ 15901 Java added to this list on 2025-06-22 repository created 2017-08-10

Apache Doris is a high-performance, real-time analytical database designed with a Massively Parallel Processing (MPP) architecture. It is known for its ease of use and ability to deliver sub-second query response times even when handling massive datasets. The system supports both high-concurrency point queries and high-throughput complex analytical queries, making it suitable for a wide range of data analysis scenarios. Typical use cases include report analysis, ad-hoc querying, unified data warehousing, and accelerating queries on data lakes. Users can leverage Apache Doris to build applications such as user behavior analysis, A/B testing platforms, log retrieval and analysis, user profiling, and order analysis. The architecture of Apache Doris is streamlined with a storage-compute integrated design, consisting mainly of two types of processes: Frontend (FE) and Backend (BE). The FE handles user requests, query parsing and planning, metadata management, and node management, while the BE is responsible for data storage and query execution. Data is partitioned into shards and replicated across multiple BE nodes to ensure reliability and availability. The FE nodes can be deployed in multiple roles (Master, Follower, Observer) to support disaster recovery and increase query concurrency. Both FE and BE components are horizontally scalable, allowing the system to support clusters with hundreds of machines and petabytes of data. Apache Doris uses the MySQL protocol and supports standard SQL, making it compatible with various client tools and BI platforms. It is widely used for real-time data analysis, including real-time reporting, decision-making, ad hoc analysis, user profiling, and behavior analysis. It also supports lakehouse analytics by accelerating queries on data lakes and enabling federated analytics across multiple data sources. Additionally, it facilitates SQL-based observability for log and event analysis in distributed systems. The system ensures high availability through data replication and quorum-based synchronization, supporting both same-city and cross-region disaster recovery configurations. This robust architecture and feature set make Apache Doris a powerful and reliable choice for modern analytical database needs.

https://github.com/apache/doris

a/b-testingad-hoc-queryagentaianalytical-databaseapache-dorisbackendbigquerycomplex-analysisdata-lake-accelerationdata-replicationdatabasedbtdelta-lakedisaster-recoveryeltfederated-analyticsfrontendhigh-concurrencyhigh-performancehorizontal-scalabilityhudiiceberglakehouselakehouse-analyticslog-analysislog-and-event-analysismetadata-managementmpp-architecturemysql-protocololaporder-analysispaimonquery-enginequery-executionreal-timereal-time-analyticsredshiftreport-analysissnowflakesparksqlsql-based-observabilitysql-compatibilitystorage-compute-integrated-architecturesub-second-queryunified-data-warehouseuser-behavior-analysisuser-profiling

Also in NL AI Frameworks

infiniflow/ragflow

RAGFlow is an open-source Retrieval-Augmented Generation engine that leverages deep document understanding and Large Language Models to provide accurate, citation-backed question-answering from complex and diverse data sources.

ScrapeGraphAI/Scrapegraph-ai

ScrapeGraphAI is an AI-powered Python library that creates intelligent web scraping pipelines using large language models to extract structured data from websites and local documents efficiently.

pydantic/pydantic

Pydantic is a Python library for fast and extensible data validation using Python type hints, enabling developers to define and validate data models efficiently.

business-science/ai-data-science-team

AI Data Science Team is a Python library featuring AI-powered agents that automate and accelerate common data science tasks, including data cleaning, feature engineering, machine learning, and exploratory data analysis, to improve productivity and efficiency.

ucbepic/docetl

DocETL is a system for creating and executing complex document processing and data transformation pipelines powered by large language models, featuring an interactive UI playground and a Python package for production use.

hitsz-ids/synthetic-data-generator

Synthetic Data Generator (SDG) is a comprehensive framework for generating high-quality, privacy-compliant synthetic structured tabular data using advanced statistical and LLM-based models, optimized for big data applications.

Mintplex-Labs/vector-admin

VectorAdmin is a universal and user-friendly tool suite for managing multiple vector databases, providing full control over vector data with multi-user support, embedding management, and cloud deployment capabilities.

scouter-project/scouter

Scouter is an open source Application Performance Management (APM) tool that monitors and manages the performance of software applications and system resources across various platforms and services.