typedef-ai/fenic

⭐ 670 Python repository created 2025-06-16

fenic is a semantic DataFrame query engine for processing unstructured and structured data with language models as first-class operators. Users write familiar PySpark or SQL style operations, select, filter, join, group by and aggregate, and mix in semantic operators including extract, classify, summarize, embed and semantic join. Models are configured once on a session object, pipelines are built lazily, and the engine compiles and executes the plan on a runtime designed for inference workloads with automatic batching, rate limiting, retries, response caching and token and cost accounting. Extraction is schema-bound: a Pydantic model defines the desired shape and fenic returns validated typed columns that can be queried like any other data, replacing regex parsing and one-off prompts. The design argument the project makes is that inference belongs inside the query model rather than being orchestrated by hand around it, and that the pipeline itself should be the durable artifact. Because the work is expressed as typed operators, a pipeline is inspectable through row-level lineage, an explain method and per-query metrics; it is rerunnable through lazy plans and caching; and it can be promoted into a named table, a view, or an MCP tool that an agent calls directly. That last point is the agent angle: a discovery an agent made while exploring messy data becomes a callable, reusable pipeline instead of dying in a chat transcript, and humans and agents author and read the same artifact. Typical inputs named in the documentation are documents, transcripts, logs, evaluation traces, support tickets, tables and APIs. Several model providers are supported through environment keys, including OpenAI, Anthropic, Google, Cohere and OpenRouter. The package installs from PyPI, and a bundled skill plus a lint command help coding agents write correct fenic code.

https://github.com/typedef-ai/fenic

agentsaidataframe-librarydataframesetlllmmcporchestrationpolarspythonsemantic-operatorsunstructured-data

Also in NL AI Frameworks

infiniflow/ragflow

RAGFlow is an open-source Retrieval-Augmented Generation engine that leverages deep document understanding and Large Language Models to provide accurate, citation-backed question-answering from complex and diverse data sources.

ScrapeGraphAI/Scrapegraph-ai

ScrapeGraphAI is an AI-powered Python library that creates intelligent web scraping pipelines using large language models to extract structured data from websites and local documents efficiently.

pydantic/pydantic

Pydantic is a Python library for fast and extensible data validation using Python type hints, enabling developers to define and validate data models efficiently.

apache/doris

Apache Doris is a high-performance, real-time analytical database with a storage-compute integrated architecture, designed for sub-second query response and high concurrency in diverse data analysis scenarios.

business-science/ai-data-science-team

AI Data Science Team is a Python library featuring AI-powered agents that automate and accelerate common data science tasks, including data cleaning, feature engineering, machine learning, and exploratory data analysis, to improve productivity and efficiency.