Awesome AI Agents › Post-Exploitation Agents

AI-secure/AdvAgent

⭐ 26 Jupyter Notebook added to this list on 2025-06-16 repository created 2024-10-16

AdvAgent is a project focused on controllable blackbox red-teaming for web agents, designed to test and improve the security and robustness of large language model-based web agents. The project provides a comprehensive pipeline to conduct adversarial attacks on various large language models such as GPT, Gemini, and Claude, with a primary example targeting GPT. It leverages datasets like Mind2Web and SeeAct to perform experiments and evaluate the effectiveness of its approach against state-of-the-art web agent frameworks. The repository includes detailed setup instructions, including environment creation, dependency installation, and API key configuration for OpenAI and Hugging Face. Users can generate training and test datasets by filtering and augmenting source data, followed by constructing datasets suitable for supervised fine-tuning (SFT) and direct preference optimization (DPO). The project supports fine-tuning models using Together AI's API, providing scripts and commands for uploading datasets, training models with LoRA, and downloading the fine-tuned models. Additionally, it offers a DPO training pipeline with scripts for training, model selection, and merging. Evaluation scripts are included to assess the performance of the trained models. The project is well-documented with a clear directory structure for data and models, facilitating reproducibility and further research. Overall, AdvAgent aims to advance the field of AI security by enabling controlled adversarial testing of web agents, helping to identify vulnerabilities and improve their resilience against malicious inputs.

https://github.com/AI-secure/AdvAgent

adversarial-attacksai-securityblackbox-red-teamingclaudedata-generationdirect-preference-optimizationevaluationgeminigptlarge-language-modelsmind2web-datasetmodel-trainingrobustnessseeact-frameworksupervised-fine-tuningtogether-aiweb-agents