Awesome AI AgentsWorkflow Automation Assistants

xxyQwQ/ComfyBench

⭐ 205 Python added to this list on 2025-05-02 repository created 2024-09-01

ComfyBench is a comprehensive benchmarking framework designed to evaluate the capabilities of large language model (LLM)-based agents in autonomously designing collaborative AI systems within the ComfyUI environment. The project implements the concepts presented in the paper titled "ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systems." The core idea is to assess how well agents can interpret task instructions, learn from extensive documentation, and generate effective workflows that represent collaborative AI systems to accomplish specified tasks. The benchmark measures agent performance using two key metrics: pass rate and resolve rate. The pass rate indicates whether the generated workflow can be executed correctly, while the resolve rate reflects whether the workflow successfully meets the task requirements. This dual-metric evaluation provides a robust understanding of an agent's ability to autonomously design and implement AI workflows. ComfyAgent, a component of the project, is an agent that constructs collaborative AI systems by generating workflows in ComfyUI. These workflows are translated into equivalent code to enhance comprehension by LLMs, enabling the agent to learn from existing workflows and autonomously create new ones. The generated workflows serve as interpretable representations of collaborative AI systems tailored to complete given tasks. The repository includes detailed instructions for setup, including environment configuration, dependency installation, and preparation of ComfyUI with necessary models and extensions. It also provides scripts for executing the ComfyAgent pipeline and evaluating agent performance on the ComfyBench dataset, which contains extensive documentation, curriculum workflows, and task instructions. The project is actively maintained and has been accepted for presentation at CVPR 2025, highlighting its significance in the AI research community. It offers a valuable resource for researchers and developers interested in advancing autonomous AI system design through LLM-based agents and workflow generation in ComfyUI.

https://github.com/xxyQwQ/ComfyBench

ai-workflowsautonomous-ai-designbenchmarkingcollaborative-ai-systemscomfyagentcomfybenchcomfyuicvpr-2025evaluationllm-based-agentspass-rateresolve-ratetask-instructionsworkflow-generation

Also in Workflow Automation Assistants

Significant-Gravitas/AutoGPT

AutoGPT is a powerful AI platform that enables users to create, deploy, and manage continuous AI agents to automate complex workflows with options for self-hosting and cloud deployment.

enescingoz/awesome-n8n-templates

A comprehensive open-source collection of over 280 free n8n automation templates for various services, including Gmail, Telegram, Slack, Notion, OpenAI, and more.

n8n-io/self-hosted-ai-starter-kit

The Self-hosted AI Starter Kit by n8n is an open-source Docker Compose template that quickly sets up a secure, local AI and low-code development environment integrating n8n, Ollama, Qdrant, and PostgreSQL for building self-hosted AI workflows.

genspark-ai/genoffice

Open-source AI-native alternative to Microsoft Office, editing Word, Excel, PowerPoint and PDF files locally with built-in document-editing agents.

czlonkowski/n8n-skills

A set of seven Claude Code skills designed to help AI assistants build flawless, production-ready n8n workflows using the n8n-mcp MCP server.

dtyq/magic

Magic is the first open-source all-in-one AI productivity platform offering a comprehensive product matrix to help enterprises build and deploy intelligent AI applications for enhanced productivity.

lmnr-ai/lmnr

Laminar is an open-source all-in-one platform for engineering, tracing, evaluating, and managing AI products with scalable infrastructure and rich telemetry features.

eracle/OpenOutreach

A self-hosted email-first sales agent that turns a product description into an ideal-customer profile, discovers and scores B2B leads from a licensed data source, then runs personalised outreach from your own mailbox.