Awesome AI Agents › Reasoning Frameworks

guidance-ai/guidance

⭐ 21750 Jupyter Notebook added to this list on 2025-04-19 repository created 2022-11-10

Guidance is a programming paradigm and library designed to efficiently control and steer large language models (LLMs) to produce high-quality, structured outputs tailored to specific use cases. It offers a powerful alternative to conventional prompting or fine-tuning by reducing latency and cost while enhancing control over the generation process. The core idea behind Guidance is to allow users to write programs that interleave control logic (such as conditionals, loops, and tool calls) with text generation seamlessly, enabling more precise and flexible interactions with LLMs. The library supports a wide range of backends including Transformers, llama.cpp, OpenAI, AzureAI, and VertexAI, allowing users to write a single Guidance program that can run on multiple platforms. This cross-compatibility makes it versatile for different deployment environments. Guidance provides features like constrained generation using selects (sets of options), regular expressions, and context-free grammars, which help in guiding the model to produce outputs that meet specific structural or content requirements. One of the standout features is the ability to automatically interleave tool use with generation, where the model can pause generation to call external tools (e.g., calculators) and then resume generation based on the tool's output. This integration supports complex workflows and enhances the model's capabilities beyond text generation alone. Guidance also emphasizes ease of use with pure Python syntax enhanced by additional language model functionalities, rich templates using f-strings, and reusable components. It handles tokenization issues transparently, allowing users to work directly with text rather than tokens. The library supports streaming outputs and integrates well with Jupyter notebooks for interactive development. Overall, Guidance is a comprehensive framework for developers and researchers looking to harness the power of large language models with fine-grained control, efficiency, and flexibility, making it suitable for a wide range of applications from chatbots to automated content generation and beyond.

https://github.com/guidance-ai/guidance

azureaichat-interfaceconstrained-generationcontext-free-grammarsf-stringsguidancejupyter-notebookslarge-language-modelsllama.cppllmopenaiprogramming-paradigmpythonregular-expressionsreusable-componentsselectsstreamingtext-generationtoken-healingtool-integrationtransformersvertexai