RayVentura/ShortGPT
ShortGPT is an AI-powered framework that automates video content creation and editing for platforms like YouTube Shorts and TikTok, supporting multi-language voiceovers and automated footage sourcing.
Awesome AI Agents › Video Processing Agents
GetStream/Vision-Agents
Open Vision Agents by Stream is a cutting-edge project designed to build real-time, multi-modal AI agents that can watch, listen, and understand video content with ultra-low latency. The platform leverages Stream's edge network to ensure rapid response times, with video and audio latency kept under 30 milliseconds, enabling seamless interactive video AI experiences. It supports integration with various AI models and video providers, including popular object detection models like YOLO and Roboflow, as well as advanced language models such as Gemini, OpenAI, and Claude. The project offers native SDKs for multiple platforms including React, Android, iOS, Flutter, React Native, and Unity, making it versatile for developers targeting different environments. Key features include true real-time video AI via WebRTC, a flexible interval/processor pipeline for frame processing, turn detection and diarization for natural conversations, voice activity detection for efficient resource use, and speech-to-text and text-to-speech capabilities for smooth conversational voice user experiences. Additionally, the platform supports tool and function calling to execute arbitrary code or APIs during conversations, and built-in memory through Stream Chat to maintain context across interactions. The project also provides a variety of out-of-the-box integrations with services like AWS Bedrock, AWS Polly, Deepgram, ElevenLabs, and more, enhancing its capabilities in speech recognition, text-to-speech, and video transformation. Practical applications demonstrated include sports coaching AI, such as golf coaching using real-time object detection combined with language models, and an invisible assistant for real-time coaching in scenarios like sales or job interviews. The project is open and flexible, allowing use with any video edge network, and is accessible via easy installation commands and free API keys from Stream. Overall, Open Vision Agents by Stream empowers developers to create intelligent, responsive, and interactive video AI agents tailored to diverse use cases, from gaming and sports to professional coaching and real-time assistance.
https://github.com/GetStream/Vision-Agents
Also in Video Processing Agents
showlab/Awesome-Video-Diffusion
A curated list of recent diffusion models and resources for video generation, editing, restoration, and various AI-driven video applications.
dramaclaw/dramaclaw
Source-available AIGC video production pipeline that turns a manuscript into a finished short drama, handling character extraction, episode planning, scripts, storyboards, voice-over and final editing.
nexu-io/html-video
Apache-2.0 toolkit that lets local coding agents turn HTML, CSS and data into real MP4 video through pluggable render engines, 21 animated templates and an optional AI soundtrack.
HKUDS/VideoAgent
Multi-agent framework for video understanding, editing and generation that parses user intent, builds a graph-based tool workflow and produces edited or newly created videos from natural language.
video-db/Director
Director is an AI video agents framework that enables complex video tasks like search, editing, and generation through natural language commands and real-time streaming, built on VideoDB's video-as-data infrastructure.
video-db/StreamRAG
StreamRAG is a GPT-powered video search and streaming agent that enables real-time video search, summarization, and publishing of searchable video collections on the ChatGPT store.
microsoft/DeepVideoDiscovery
Microsoft research agent for long-form video question answering that treats segmented clips as an environment and iteratively selects multi-granular search tools, reaching state of the art on LVBench.