cogito

Cogito is a powerful Go library for building intelligent, co-operative agentic software and LLM-powered workflows, focusing on improving results for small, open source language models that scales to any LLM.

🧪 Tested on Small Models ! Our test suite runs on 0.6b Qwen (not fine-tuned), proving effectiveness even with minimal resources.

📝 Working on Official Paper
I am currently working on the official academic/white paper for Cogito. The paper will provide detailed theoretical foundations, experimental results, and comprehensive analysis of the framework's capabilities.

🏗️ Architecture

Cogito is the result of building LocalAI, LocalAGI and LocalOperator (yet to be released).

Cogito uses an internal pipeline to first make the LLM reason about a specific task, forcing the model to reason and later extracts with BNF grammars exact data structures from the LLM. This is applied to every primitive exposed by the framework.

It provides a comprehensive framework for creating conversational AI systems with advanced reasoning, tool execution, goal-oriented planning, iterative content refinement capabilities, and seamless integration with external tools also via the Model Context Protocol (MCP).

🔧 Composable Primitives
Cogito primitives can be combined to form more complex pipelines, enabling sophisticated AI workflows.

🚀 Quick Start

Installation

go get github.com/mudler/cogito

Basic Usage

package main

import (
    "context"
    "fmt"
    "github.com/mudler/cogito"
    "github.com/mudler/cogito/clients"

)

func main() {
    // Create an LLM client
    llm := clients.NewOpenAILLM("your-model", "api-key", "https://api.openai.com")
    
    // Create a conversation fragment
    fragment := cogito.NewEmptyFragment().
        AddMessage("user", "Tell me about artificial intelligence")
    
    // Get a response
    newFragment, err := llm.Ask(context.Background(), fragment)
    if err != nil {
        panic(err)
    }
    
    fmt.Println(newFragment.LastMessage().Content)
}

Using Tools

Creating Custom Tools

To create a custom tool, implement the Tool[T] interface:

type MyToolArgs struct {
    Param string `json:"param" description:"A parameter"`
}

type MyTool struct{}

// Implement the Tool interface
func (t *MyTool) Run(args MyToolArgs) (string, error) {
    // Your tool logic here
    return fmt.Sprintf("Processed: %s", args.Param), nil
}

// Create a ToolDefinition using NewToolDefinition helper
myTool := cogito.NewToolDefinition(
    &MyTool{},
    MyToolArgs{},
    "my_tool",
    "A custom tool",
)

Tools in Cogito are added by calling NewToolDefinition on your tool, which automatically generates openai.Tool via the Tool() method. Tools are then passed by to cogito.WithTools:

// Define tool argument types
type WeatherArgs struct {
    City string `json:"city" description:"The city to get weather for"`
}

type SearchArgs struct {
    Query string `json:"query" description:"The search query"`
}

// Create tool definitions - these automatically generate openai.Tool
weatherTool := cogito.NewToolDefinition(
    &WeatherTool{},
    WeatherArgs{},
    "get_weather",
    "Get the current weather for a city",
)

searchTool := cogito.NewToolDefinition(
    &SearchTool{},
    SearchArgs{},
    "search",
    "Search for information",
)

// Create a fragment with user input
fragment := cogito.NewFragment(openai.ChatCompletionMessage{
    Role:    "user",
    Content: "What's the weather in San Francisco?",
})

// Execute with tools - you can pass multiple tools with different types
result, err := cogito.ExecuteTools(llm, fragment, 
    cogito.WithTools(weatherTool, searchTool))
if err != nil {
    panic(err)
}

// result.Status.ToolsCalled will contain all the tools being called
// The LLM can select one or multiple tools per iteration
// When parallel execution is enabled, multiple tools can run concurrently

Tool Call Callbacks and Adjustments

Cogito allows you to intercept and adjust tool calls before they are executed. This enables interactive workflows where users can review, approve, modify, or directly edit tool calls.

ToolCallDecision Struct:

The callback returns a ToolCallDecision struct that provides fine-grained control:

type ToolCallDecision struct {
    Approved   bool         // true to proceed, false to interrupt
    Adjustment string       // Feedback for LLM to interpret (empty = no adjustment)
    Modified   *ToolChoice  // Direct modification (takes precedence over Adjustment)
    Skip       bool         // Skip this tool call but continue execution
}

Basic Tool Call Callback:

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithToolCallBack(func(tool *cogito.ToolChoice, state *cogito.SessionState) cogito.ToolCallDecision {
        fmt.Printf("Agent wants to run: %s with args: %v\n", tool.Name, tool.Arguments)
        return cogito.ToolCallDecision{Approved: true} // Proceed without adjustment
    }))

Interactive Tool Call Approval with Adjustments:

import (
    "bufio"
    "encoding/json"
    "os"
    "strings"
)

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithMaxAdjustmentAttempts(3), // Limit adjustment attempts
    cogito.WithToolCallBack(func(tool *cogito.ToolChoice, state *cogito.SessionState) cogito.ToolCallDecision {
        args, _ := json.Marshal(tool.Arguments)
        fmt.Printf("The agent wants to run the tool %s with arguments: %s\n", tool.Name, string(args))
        fmt.Println("Do you want to run the tool? (y/n/adjust)")
        
        reader := bufio.NewReader(os.Stdin)
        text, _ := reader.ReadString('\n')
        text = strings.TrimSpace(text)
        
        switch text {
        case "y":
            return cogito.ToolCallDecision{Approved: true}
        case "n":
            return cogito.ToolCallDecision{Approved: false}
        default:
            // Provide adjustment feedback - the LLM will re-evaluate the tool call
            return cogito.ToolCallDecision{
                Approved:   true,
                Adjustment: text,
            }
        }
    }))

Direct Tool Modification:

You can directly modify tool arguments without relying on LLM interpretation:

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithToolCallBack(func(tool *cogito.ToolChoice, state *cogito.SessionState) cogito.ToolCallDecision {
        // Validate and fix arguments directly
        if tool.Name == "search" {
            query, ok := tool.Arguments["query"].(string)
            if !ok || len(query) < 3 {
                // Directly modify instead of asking LLM
                fixed := *tool
                fixed.Arguments = map[string]any{
                    "query": "default search query",
                }
                return cogito.ToolCallDecision{
                    Approved: true,
                    Modified: &fixed,
                }
            }
        }
        return cogito.ToolCallDecision{Approved: true}
    }))

Skipping Tool Calls:

You can skip a tool call while continuing execution (useful for conditional tool execution):

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithToolCallBack(func(tool *cogito.ToolChoice, state *cogito.SessionState) cogito.ToolCallDecision {
        // Skip certain tools based on conditions
        if tool.Name == "search" && someCondition {
            return cogito.ToolCallDecision{
                Approved: true,
                Skip:     true, // Skip this tool but continue execution
            }
        }
        return cogito.ToolCallDecision{Approved: true}
    }))

When a tool is skipped:

• The tool call is added to the conversation fragment
• A message indicating the tool was skipped is added
• Execution continues to the next iteration
• This is different from Approved: false which interrupts execution entirely

Session State and Resuming Execution:

The SessionState contains the current tool choice and fragment, allowing you to save and resume execution:

var savedState *cogito.SessionState

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithToolCallBack(func(tool *cogito.ToolChoice, state *cogito.SessionState) cogito.ToolCallDecision {
        // Save the state for later resumption
        savedState = state
        
        // Interrupt execution
        return cogito.ToolCallDecision{Approved: false}
    }))

// Later, resume execution from the saved state
if savedState != nil {
    resumedFragment, err := savedState.Resume(llm, cogito.WithTools(searchTool))
    if err != nil {
        panic(err)
    }
    // Continue with resumedFragment
}

Starting with a Specific Tool Choice:

You can pre-select one or more tools to start execution with:

// Single tool
initialTool := &cogito.ToolChoice{
    Name: "search",
    Arguments: map[string]any{
        "query": "artificial intelligence",
    },
}

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithStartWithAction(initialTool))

// Multiple tools
initialTools := []*cogito.ToolChoice{
    {
        Name: "search",
        Arguments: map[string]any{"query": "AI"},
    },
    {
        Name: "get_weather",
        Arguments: map[string]any{"city": "San Francisco"},
    },
}

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool, weatherTool),
    cogito.WithStartWithAction(initialTools...))

Forcing Reasoning:

Enable forced reasoning to ensure the LLM provides detailed reasoning for tool selections:

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithForceReasoning())

Injecting Messages During Tool Execution

Cogito allows you to inject new conversation messages during the main tool execution loop. This enables dynamic user interaction where messages can be added while the agent is executing tools, and you can track whether injected messages were successfully added to the conversation.

Basic Message Injection:

Create a channel to send messages and pass it to ExecuteTools:

import (
    "github.com/sashabaranov/go-openai"
    "github.com/mudler/cogito"
)

// Create a channel for message injection
messageInjectionChan := make(chan openai.ChatCompletionMessage, 10)

// Start tool execution in a goroutine
go func() {
    result, err := cogito.ExecuteTools(llm, fragment,
        cogito.WithTools(searchTool, weatherTool),
        cogito.WithMessageInjectionChan(messageInjectionChan))
    if err != nil {
        panic(err)
    }
    fmt.Println(result.LastMessage().Content)
}()

// Inject messages while execution is running
messageInjectionChan <- openai.ChatCompletionMessage{
    Role:    "user",
    Content: "Also check for recent updates on this topic",
}

// Continue injecting messages as needed
messageInjectionChan <- openai.ChatCompletionMessage{
    Role:    "user",
    Content: "Focus on the most recent data",
}

// Close the channel to stop accepting messages
close(messageInjectionChan)

Receiving Feedback About Injected Messages:

Use a result channel to get feedback about whether messages were successfully injected:

messageInjectionChan := make(chan openai.ChatCompletionMessage, 10)
resultChan := make(chan cogito.MessageInjectionResult, 10)

go func() {
    result, err := cogito.ExecuteTools(llm, fragment,
        cogito.WithTools(searchTool),
        cogito.WithMessageInjectionChan(messageInjectionChan),
        cogito.WithMessageInjectionResultChan(resultChan))  // Receive feedback
    // ...
}()

// Inject a message and get feedback
messageInjectionChan <- openai.ChatCompletionMessage{
    Role:    "user",
    Content: "Prioritize recent sources",
}

// Wait for feedback
result := <-resultChan
if result.Err != nil {
    fmt.Printf("Injection failed: %v\n", result.Err)
} else {
    fmt.Printf("Successfully injected 1 message at position %d\n", result.Position)
}

MessageInjectionResult Structure:

The feedback channel returns MessageInjectionResult with:

• Count: Number of messages successfully injected
• Position: Position in the fragment where messages were added
• Err: Error if injection failed (validation errors, etc.)

Complete Interactive Example:

package main

import (
    "bufio"
    "os"
    "strings"
    "github.com/sashabaranov/go-openai"
    "github.com/mudler/cogito"
)

func main() {
    messageInjectionChan := make(chan openai.ChatCompletionMessage, 10)
    resultChan := make(chan cogito.MessageInjectionResult, 10)

    // Start execution in background
    go func() {
        _, err := cogito.ExecuteTools(llm, fragment,
            cogito.WithTools(searchTool),
            cogito.WithMessageInjectionChan(messageInjectionChan),
            cogito.WithMessageInjectionResultChan(resultChan))
        if err != nil {
            panic(err)
        }
        close(messageInjectionChan) // Signal no more messages will be sent
    }()

    // Interactive message injection from stdin
    scanner := bufio.NewScanner(os.Stdin)
    for {
        fmt.Print("Enter message to inject (or 'quit'): ")
        if !scanner.Scan() {
            break
        }
        
        text := strings.TrimSpace(scanner.Text())
        if text == "quit" {
            close(messageInjectionChan)
            break
        }
        
        // Send message
        messageInjectionChan <- openai.ChatCompletionMessage{
            Role:    "user",
            Content: text,
        }
        
        // Receive feedback (non-blocking)
        select {
        case result := <-resultChan:
            if result.Err != nil {
                fmt.Printf("Error: %v\n", result.Err)
            } else {
                fmt.Printf("Message injected at position %d\n", result.Position)
            }
        default:
        }
    }
}

How It Works:

1. Message Injection Timing: Messages are injected at the end of each iteration after tool results are added, but before the next tool selection. This ensures injected messages are considered in the next planning step.

2. Context Awareness: Injected messages are tracked in Fragment.Status.InjectedMessages with the iteration number, allowing you to see when each message was added.

3. Channel Semantics:

   • Nil channels are safe and simply ignored (idiomatic Go)
   • Closed channels signal that no more messages will be sent
   • Non-blocking sends on result channels prevent deadlocks if no one is listening

4. Concurrent Safety: The tool loop uses a select statement to monitor both the context cancellation and message injection channel, ensuring responsive message handling.

Use Cases:

• Interactive Workflows: Allow users to provide real-time guidance while the agent is executing tools
• Adaptive Execution: Inject corrective instructions based on intermediate results
• Multi-Stage Tasks: Guide the agent through complex multi-step processes
• User Feedback Integration: Incorporate user input during long-running tool executions

Notes:

• The message injection channel is optional and completely independent of tool call callbacks
• Both WithMessageInjectionChan and WithMessageInjectionResultChan accept nil (default) - simply don't use them if you don't need this feature
• Channel capacity should account for the maximum concurrent messages you expect to inject
• The result feedback channel uses non-blocking sends, so it won't block the tool execution loop
• Successfully injected messages are part of the returned fragment and appear in Fragment.Status.InjectedMessages
• When the message injection channel is closed, the loop continues to accept context cancellation and normal tool execution

Multiple Tool Selection and Parallel Execution

Cogito supports selecting and executing multiple tools in a single iteration. When enabled, the LLM can select multiple tools that can be executed concurrently, improving efficiency for independent operations.

Multiple Tool Selection (Sequential):

By default, Cogito can select multiple tools, but they execute sequentially:

// The LLM can select multiple tools, executed one after another
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool, weatherTool, factCheckTool))
// Tools are selected and executed sequentially

Parallel Tool Execution:

Enable parallel execution to run multiple tools concurrently:

// Enable parallel tool execution
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool, weatherTool, factCheckTool),
    cogito.EnableParallelToolExecution)

// When parallel execution is enabled:
// - The LLM uses a different intention tool that allows selecting multiple tools
// - Selected tools are executed concurrently using goroutines
// - Results are collected and added to the conversation in order

When to Use Parallel Execution:

• Use parallel execution when:

  • Tools are independent and don't depend on each other's results
  • You want to improve performance for multiple independent operations
  • Tools can safely run concurrently (e.g., fetching weather for multiple cities, searching multiple queries)

• Use sequential execution (default) when:

  • Tools depend on each other's results
  • You need to maintain strict execution order
  • You want simpler debugging and error handling

Sink State with Multiple Tools:

When multiple tools are selected and one of them is a sink state tool, Cogito will:

1. Execute all non-sink tools first (in parallel or sequential based on the option)
2. Stop execution after non-sink tools complete
3. The sink state indicates that no further tools are needed

// Example: LLM selects [searchTool, weatherTool, replyTool]
// - searchTool and weatherTool execute first
// - replyTool (sink state) is detected
// - Execution stops after searchTool and weatherTool complete
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool, weatherTool),
    cogito.EnableParallelToolExecution)

Error Handling:

When a tool call callback interrupts execution, Cogito returns cogito.ErrToolCallCallbackInterrupted:

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithToolCallBack(func(tool *cogito.ToolChoice, state *cogito.SessionState) cogito.ToolCallDecision {
        return cogito.ToolCallDecision{Approved: false} // Interrupt
    }))

if err != nil {
    if errors.Is(err, cogito.ErrToolCallCallbackInterrupted) {
        fmt.Println("Execution was interrupted by tool call callback")
    }
}

Notes:

• The callback receives both the ToolChoice and SessionState for full context
• Approved: false interrupts execution entirely
• Approved: true, Skip: true skips the tool call but continues execution (useful for conditional execution)
• Adjustment (non-empty) triggers an adjustment loop where the LLM re-evaluates the tool call
• Modified (non-nil) directly uses the modified tool choice without re-querying the LLM
• When a tool is skipped, it's added to the conversation with a "skipped" message, preserving history
• The adjustment loop has a maximum attempt limit (default: 5, configurable via WithMaxAdjustmentAttempts)
• SessionState can be serialized to JSON for persistence
• The adjustment prompt has been improved to provide better guidance to the LLM

Configuring Sink State

When the LLM determines that no tool is needed to respond to the user, Cogito uses a "sink state" tool to handle the response. By default, Cogito uses a built-in reply tool, but you can customize or disable this behavior.

Disable Sink State:

// Disable sink state entirely - the LLM will return an error if no tool is selected
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(weatherTool, searchTool),
    cogito.DisableSinkState)

Custom Sink State Tool:

// Define a custom sink state tool
type CustomReplyArgs struct {
    Reasoning string `json:"reasoning" description:"The reasoning for the reply"`
}

type CustomReplyTool struct{}

func (t *CustomReplyTool) Run(args CustomReplyArgs) (string, error) {
    // Custom logic to process the reasoning and generate a response
    return fmt.Sprintf("Based on: %s", args.Reasoning), nil
}

// Create a custom sink state tool
customSinkTool := cogito.NewToolDefinition(
    &CustomReplyTool{},
    CustomReplyArgs{},
    "custom_reply",
    "Custom tool for handling responses when no other tool is needed",
)

// Use the custom sink state tool
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(weatherTool, searchTool),
    cogito.WithSinkState(customSinkTool))

Notes:

• The sink state tool is enabled by default with a built-in reply tool
• When enabled, the sink state tool appears as an option in the tool selection enum
• The sink state tool receives a reasoning parameter containing the LLM's reasoning about why no tool is needed
• Custom sink state tools must accept a reasoning parameter in their arguments

Sub-Agent Spawning

Cogito supports spawning sub-agents via tools, allowing the LLM to delegate tasks to independent child agents. Sub-agents can run in the foreground (blocking — waits for result) or background (non-blocking — returns an ID immediately so the parent can continue working).

Enabling Sub-Agents:

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool, weatherTool),
    cogito.EnableAgentSpawning, // Adds spawn_agent, check_agent, get_agent_result tools
    cogito.WithIterations(10),
)

When enabled, three built-in tools are injected:

• spawn_agent — Spawns a sub-agent with a task. Set background: true for non-blocking execution.
• check_agent — Checks the status of a background agent by ID.
• get_agent_result — Retrieves the result of a background agent. Set wait: true to block until done.

Foreground Agents (Blocking):

The LLM calls spawn_agent with background: false. The sub-agent runs synchronously inside the tool call, and its result is returned as the tool output:

// The LLM might call: spawn_agent(task="Research quantum computing", background=false)
// → Sub-agent runs ExecuteTools with the same tools (minus agent tools)
// → Sub-agent completes and returns result
// → Parent receives result as tool output and continues
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.EnableAgentSpawning,
    cogito.WithIterations(5),
)

Background Agents (Non-Blocking):

The LLM calls spawn_agent with background: true. The sub-agent launches in a goroutine, and the parent immediately gets back an agent ID. ExecuteTools automatically stays alive while background agents are running — when the LLM has no more tools to call, the loop blocks until a background agent finishes. The completion notification is injected into the conversation, and the LLM can react to it:

// The LLM might call: spawn_agent(task="Analyze data", background=true)
// → Returns "Agent spawned in background with ID: abc-123"
// → Parent continues working on other tasks
// → When sub-agent finishes, parent sees:
//   "Background agent abc-123 has completed. Task: Analyze data. Result: ..."
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool, analysisTool),
    cogito.EnableAgentSpawning,
    cogito.WithIterations(10),
)

Sharing Agent State Across Calls:

Use WithAgentManager to share the agent registry across multiple ExecuteTools calls, allowing you to track background agents across conversation turns:

manager := cogito.NewAgentManager()

// First turn: spawns a background agent
result1, _ := cogito.ExecuteTools(llm, fragment1,
    cogito.WithTools(searchTool),
    cogito.EnableAgentSpawning,
    cogito.WithAgentManager(manager),
    cogito.WithIterations(5),
)

// Second turn: can check on or retrieve results from previously spawned agents
result2, _ := cogito.ExecuteTools(llm, fragment2,
    cogito.WithTools(searchTool),
    cogito.EnableAgentSpawning,
    cogito.WithAgentManager(manager),
    cogito.WithIterations(5),
)

// Programmatic access to all agents
for _, agent := range manager.List() {
    fmt.Printf("Agent %s: %s\n", agent.ID, agent.Status)
}

Using a Separate LLM for Sub-Agents:

mainLLM := clients.NewOpenAILLM("gpt-4", apiKey, baseURL)
subAgentLLM := clients.NewOpenAILLM("gpt-3.5-turbo", apiKey, baseURL)

result, err := cogito.ExecuteTools(mainLLM, fragment,
    cogito.WithTools(searchTool),
    cogito.EnableAgentSpawning,
    cogito.WithAgentLLM(subAgentLLM), // Sub-agents use a different model
    cogito.WithIterations(5),
)

Completion Callbacks:

Use WithAgentCompletionCallback to get programmatic notification when any background sub-agent finishes — useful for UI updates or external monitoring:

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.EnableAgentSpawning,
    cogito.WithAgentCompletionCallback(func(agent *cogito.AgentState) {
        fmt.Printf("Agent %s finished with status: %s\n", agent.ID, agent.Status)
        if agent.Status == cogito.AgentStatusCompleted {
            fmt.Println("Result:", agent.Result)
        }
    }),
    cogito.WithIterations(10),
)

Tool Filtering:

By default, sub-agents receive all parent tools except the agent management tools themselves (preventing unbounded recursion). The LLM can also specify a subset of tools:

// The LLM can call: spawn_agent(task="Search only", tools=["search"])
// → Sub-agent only has access to the "search" tool

To allow sub-agents to also spawn their own sub-agents, the LLM can explicitly include agent tools:

// spawn_agent(task="Complex task", tools=["search", "spawn_agent", "check_agent", "get_agent_result"])

Streaming Sub-Agent Events:

When streaming is enabled, sub-agent events are tagged with a StreamEventSubAgent type and include the agent's ID:

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.EnableAgentSpawning,
    cogito.WithTools(searchTool),
    cogito.WithStreamCallback(func(ev cogito.StreamEvent) {
        if ev.Type == cogito.StreamEventSubAgent {
            fmt.Printf("[Agent %s] %s\n", ev.AgentID, ev.Content)
        } else {
            fmt.Print(ev.Content)
        }
    }),
    cogito.WithIterations(5),
)

When to Use Sub-Agents:

• Use foreground agents when the parent needs the result before continuing (e.g., research a topic, then summarize)
• Use background agents when tasks are independent and the parent can continue working (e.g., start multiple research tasks in parallel)
• Use WithAgentManager when you need to track agents across multiple conversation turns
• Use WithAgentLLM when sub-agents should use a cheaper/faster model

Field Annotations for Tool Arguments

Cogito supports several struct field annotations to control how tool arguments are defined in the generated JSON schema:

Available Annotations:

• json:"field_name" - Required. Defines the JSON field name for the parameter.
• description:"text" - Provides a description for the field that helps the LLM understand what the parameter is for.
• enum:"value1,value2,value3" - Restricts the field to a specific set of allowed values (comma-separated).
• required:"false" - Makes the field optional. By default, all fields are required unless marked with required:"false".

Examples:

// Basic required field with description
type BasicArgs struct {
    Query string `json:"query" description:"The search query"`
}

// Optional field
type OptionalArgs struct {
    Query string `json:"query" required:"false" description:"Optional search query"`
    Limit int    `json:"limit" required:"false" description:"Maximum number of results"`
}

// Field with enum values
type EnumArgs struct {
    Action string `json:"action" enum:"create,read,update,delete" description:"The action to perform"`
}

// Field with enum and description
type WeatherArgs struct {
    City        string `json:"city" description:"The city name"`
    Unit        string `json:"unit" enum:"celsius,fahrenheit" description:"Temperature unit"`
    Format      string `json:"format" enum:"short,detailed" required:"false" description:"Output format"`
}

// Complete example with multiple field types
type AdvancedSearchArgs struct {
    // Required field with description
    Query string `json:"query" description:"The search query"`
    
    // Optional field with enum
    SortBy string `json:"sort_by" enum:"relevance,date,popularity" required:"false" description:"Sort order"`
    
    // Optional numeric field
    Limit int `json:"limit" required:"false" description:"Maximum number of results"`
    
    // Optional boolean field
    IncludeImages bool `json:"include_images" required:"false" description:"Include images in results"`
}

// Create tool with advanced arguments
searchTool := cogito.NewToolDefinition(
    &AdvancedSearchTool{},
    AdvancedSearchArgs{},
    "advanced_search",
    "Advanced search with sorting and filtering options",
)

Notes:

• Fields without required:"false" are automatically marked as required in the JSON schema
• Enum values are case-sensitive and should match exactly what you expect in Run()
• The json tag is required for all fields that should be included in the tool schema
• Descriptions help the LLM understand the purpose of each parameter, leading to better tool calls

Alternatively, you can implement ToolDefinitionInterface directly if you prefer more control:

type CustomTool struct{}

func (t *CustomTool) Tool() openai.Tool {
    return openai.Tool{
        Type: openai.ToolTypeFunction,
        Function: &openai.FunctionDefinition{
            Name:        "custom_tool",
            Description: "A custom tool",
            Parameters: jsonschema.Definition{
                // Define your schema
            },
        },
    }
}

func (t *CustomTool) Execute(args map[string]any) (string, error) {
    // Your execution logic
    return "result", nil
}

Guidelines for Intelligent Tool Selection

Guidelines provide a powerful way to define conditional rules for tool usage. The LLM intelligently selects which guidelines are relevant based on the conversation context, enabling dynamic and context-aware tool selection.

// Create tool definitions
searchTool := cogito.NewToolDefinition(
    &SearchTool{},
    SearchArgs{},
    "search",
    "Search for information",
)

weatherTool := cogito.NewToolDefinition(
    &WeatherTool{},
    WeatherArgs{},
    "get_weather",
    "Get weather information",
)

// Define guidelines with conditions and associated tools
guidelines := cogito.Guidelines{
    cogito.Guideline{
        Condition: "User asks about information or facts",
        Action:    "Use the search tool to find information",
        Tools: cogito.Tools{
            searchTool,
        },
    },
    cogito.Guideline{
        Condition: "User asks for the weather in a city",
        Action:    "Use the weather tool to find the weather",
        Tools: cogito.Tools{
            weatherTool,
        },
    },
}

// Get relevant guidelines for the current conversation
fragment := cogito.NewEmptyFragment().
    AddMessage("user", "When was Isaac Asimov born?")

// Execute tools with guidelines
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithGuidelines(guidelines),
    cogito.EnableStrictGuidelines) // Only use tools from relevant guidelines
if err != nil {
    panic(err)
}

Automatic Tool Guidance with EnableGuidedTools

The EnableGuidedTools option enables intelligent filtering of tools through guidance, even when explicit guidelines aren't provided for all tools. This feature automatically creates "virtual guidelines" from tool descriptions, allowing the LLM to intelligently filter and select tools based on their descriptions.

When No Guidelines Exist:

When EnableGuidedTools is enabled and no guidelines are provided, Cogito automatically creates virtual guidelines for all tools using their descriptions as the condition. This allows the LLM to filter tools based on relevance to the conversation context.

// Create tools without any guidelines
searchTool := cogito.NewToolDefinition(
    &SearchTool{},
    SearchArgs{},
    "search",
    "A search engine to find information about a topic",
)

weatherTool := cogito.NewToolDefinition(
    &WeatherTool{},
    WeatherArgs{},
    "get_weather",
    "Get weather information for a specific city",
)

// EnableGuidedTools creates virtual guidelines from tool descriptions
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool, weatherTool),
    cogito.EnableGuidedTools) // Creates virtual guidelines from descriptions

When Guidelines Exist:

When guidelines are provided but some tools aren't included in any guideline, EnableGuidedTools creates virtual guidelines only for those "unguided" tools. This allows you to have explicit guidelines for some tools while automatically handling others.

// Define guidelines for search tool only
guidelines := cogito.Guidelines{
    cogito.Guideline{
        Condition: "User asks about information or facts",
        Action:    "Use the search tool to find information",
        Tools: cogito.Tools{
            searchTool,
        },
    },
}

// weatherTool is not in any guideline - EnableGuidedTools will create
// a virtual guideline for it using its description
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithGuidelines(guidelines...),
    cogito.WithTools(weatherTool),
    cogito.EnableGuidedTools) // Creates virtual guidelines for unguided tools

How It Works:

1. No Guidelines Scenario: When no guidelines exist and EnableGuidedTools is enabled:

   • Virtual guidelines are created for ALL tools
   • Each tool's description becomes the condition/guidance in the template
   • The LLM filters tools based on how well their descriptions match the conversation context

2. With Guidelines Scenario: When guidelines exist and EnableGuidedTools is enabled:

   • Only tools NOT in any guideline get virtual guidelines
   • Virtual guidelines use the format: "The task requires: [tool description]"
   • Real guidelines and virtual guidelines are merged and filtered together

Benefits:

• Automatic Tool Filtering: No need to create guidelines for every tool
• Better Tool Selection: LLM can intelligently filter tools based on descriptions
• Flexible Configuration: Mix explicit guidelines with automatic guidance
• Reduced Configuration: Especially useful when you have many tools

Notes:

• Tool descriptions should be meaningful and descriptive for best results
• Virtual guidelines follow the same filtering process as real guidelines
• When no guidelines exist, tool descriptions serve as both condition and guidance in the template
• The feature adds LLM call overhead for filtering virtual guidelines

Goal-Oriented Planning

// Extract a goal from conversation
goal, err := cogito.ExtractGoal(llm, fragment)
if err != nil {
    panic(err)
}

// Create tool definition
searchTool := cogito.NewToolDefinition(
    &SearchTool{},
    SearchArgs{},
    "search",
    "Search for information",
)

// Create a plan to achieve the goal
plan, err := cogito.ExtractPlan(llm, fragment, goal, 
    cogito.WithTools(searchTool))
if err != nil {
    panic(err)
}

// Execute the plan
result, err := cogito.ExecutePlan(llm, fragment, plan, goal,
    cogito.WithTools(searchTool))
if err != nil {
    panic(err)
}

Planning with TODOs

Planning with TODOs addresses context accumulation by starting each iteration with fresh context while persisting TODOs and feedback between iterations. This pattern uses separate worker and judge models: the worker executes tasks, and one or more judge LLMs review the work to determine if goal execution is completed or needs rework.

How it works:

1. Work Phase: Worker model executes tasks with fresh context that includes:
   • Overall goal
   • TODO list with progress (markdown checkboxes)
   • Previous feedback from review phase
2. Review Phase: One or more judge LLMs review the work and decide if goal execution is completed or incomplete (needs rework). When multiple reviewers are provided, a majority vote determines the final decision.
3. Persistence: TODOs and feedback persist between iterations; conversation history is cleared
4. Iteration: Loop continues until goal execution is completed or max iterations

Automatic TODO Generation (Recommended):

TODOs are automatically generated from plan subtasks when WithReviewerLLM() (judge LLM) is provided:

workerLLM := cogito.NewOpenAILLM("worker-model", "key", "url")
judgeLLM := cogito.NewOpenAILLM("judge-model", "key", "url")

goal, _ := cogito.ExtractGoal(workerLLM, fragment)
plan, _ := cogito.ExtractPlan(workerLLM, fragment, goal)

// Execute plan with Planning with TODOs enabled
// TODOs are automatically generated from plan subtasks
result, err := cogito.ExecutePlan(
    workerLLM,  // worker LLM
    fragment,
    plan,
    goal,
    cogito.WithTools(searchTool, writeTool),
    cogito.WithIterations(5),  // TODO iterations
    cogito.WithReviewerLLM(judgeLLM),  // Provide judge LLM for review (enables Planning with TODOs)
    cogito.WithTODOPersistence("./todos.json"),  // Optional: file persistence
)

Manual TODO List:

You can also provide a manual TODO list:

import "github.com/mudler/cogito/structures"

// Initialize TODO list manually
todoList := &structures.TODOList{
    TODOs: []structures.TODO{
        {ID: "1", Description: "Research topic", Completed: false},
        {ID: "2", Description: "Write draft", Completed: false},
    },
}

// Execute plan with manually provided TODOs
result, err := cogito.ExecutePlan(
    workerLLM,
    fragment,
    plan,
    goal,
    cogito.WithTools(searchTool, writeTool),
    cogito.WithIterations(5),
    cogito.WithTODOs(todoList),   // Manually provide TODO list
    cogito.WithReviewerLLM(judgeLLM),  // Provide judge LLM for review
    cogito.WithTODOPersistence("./todos.json"),
)

When to use Planning with TODOs:

• When dealing with complex, multi-step tasks that may require multiple iterations
• When you want to prevent context accumulation from failed attempts
• When you need separate models for work and review phases (worker and judge)
• When you want to track progress explicitly with checkboxes

Multiple Reviewer LLMs (Majority Voting):

You can provide multiple reviewer LLMs for more robust decision-making. When multiple reviewers are provided, Cogito uses majority voting to determine if the goal has been achieved:

workerLLM := cogito.NewOpenAILLM("worker-model", "key", "url")
judgeLLM1 := cogito.NewOpenAILLM("judge-model-1", "key", "url")
judgeLLM2 := cogito.NewOpenAILLM("judge-model-2", "key", "url")
judgeLLM3 := cogito.NewOpenAILLM("judge-model-3", "key", "url")

goal, _ := cogito.ExtractGoal(workerLLM, fragment)
plan, _ := cogito.ExtractPlan(workerLLM, fragment, goal)

// Execute plan with multiple reviewers
// The goal is considered achieved if more than half of reviewers agree
result, err := cogito.ExecutePlan(
    workerLLM,
    fragment,
    plan,
    goal,
    cogito.WithTools(searchTool, writeTool),
    cogito.WithIterations(5),
    cogito.WithReviewerLLM(judgeLLM1, judgeLLM2, judgeLLM3),  // Multiple reviewers
    cogito.WithTODOPersistence("./todos.json"),
)

How Majority Voting Works:

• Each reviewer LLM independently evaluates whether the goal has been achieved
• If more than half of the reviewers determine the goal is achieved, the overall decision is positive
• The review result from the majority is used as the final feedback
• This approach provides more reliable and consistent goal achievement detection, especially when using smaller or less reliable models

File-based Persistence:

TODOs can be persisted to a file and loaded between sessions:

result, err := cogito.ExecutePlan(
    workerLLM,
    fragment,
    plan,
    goal,
    cogito.WithReviewerLLM(judgeLLM),  // Single reviewer (or multiple)
    cogito.WithTODOPersistence("./todos.json"),  // Save/load TODOs from file
)

The TODO file is automatically saved after each iteration and loaded at the start of execution.

Content Refinement

// Create tool definition
searchTool := cogito.NewToolDefinition(
    &SearchTool{},
    SearchArgs{},
    "search",
    "Search for information",
)

// Refine content through iterative improvement
refined, err := cogito.ContentReview(llm, fragment,
    cogito.WithIterations(3),
    cogito.WithTools(searchTool))
if err != nil {
    panic(err)
}

Iterative Content Improvement

An example on how to iteratively improve content by using two separate models:

llm := cogito.NewOpenAILLM("your-model", "api-key", "https://api.openai.com")
reviewerLLM := cogito.NewOpenAILLM("your-reviewer-model", "api-key", "https://api.openai.com")

// Create content to review
initial := cogito.NewEmptyFragment().
    AddMessage("user", "Write about climate change")

response, _ := llm.Ask(ctx, initial)

// Create tool definitions
searchTool := cogito.NewToolDefinition(
    &SearchTool{},
    SearchArgs{},
    "search",
    "Search for information",
)

factCheckTool := cogito.NewToolDefinition(
    &FactCheckTool{},
    FactCheckArgs{},
    "fact_check",
    "Verify facts",
)

// Iteratively improve with tool support
improvedResponse, _ := cogito.ContentReview(reviewerLLM, response,
    cogito.WithIterations(3),
    cogito.WithTools(searchTool, factCheckTool),
    cogito.EnableToolReasoner)

Model Context Protocol (MCP) Integration

Cogito supports the Model Context Protocol (MCP) for seamless integration with external tools and services. MCP allows you to connect to remote tool providers and use their capabilities directly within your Cogito workflows.

import (
    "github.com/modelcontextprotocol/go-sdk/mcp"
)

// Create MCP client sessions
command := exec.Command("docker", "run", "-i", "--rm", "ghcr.io/mudler/mcps/weather:master")
transport := &mcp.CommandTransport{ Command: command }

client := mcp.NewClient(&mcp.Implementation{Name: "test", Version: "v1.0.0"}, nil)
mcpSession, _ := client.Connect(context.Background(), transport, nil)

// Use MCP tools in your workflows
result, _ := cogito.ExecuteTools(llm, fragment,
    cogito.WithMCPs(mcpSession))

MCP with Guidelines

// Define guidelines that include MCP tools
guidelines := cogito.Guidelines{
    cogito.Guideline{
        Condition: "User asks about information or facts",
        Action:    "Use the MCP search tool to find information",
    },
}

// Execute with MCP tools and guidelines
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithMCPs(searchSession),
    cogito.WithGuidelines(guidelines),
    cogito.EnableStrictGuidelines)

Automatic Conversation Compaction

Cogito can automatically compact conversations to prevent context overflow when token usage exceeds a threshold. This is useful for long-running conversations with LLMs that have context limits.

How it works:

1. After each LLM call, Cogito checks if the token count exceeds the threshold
2. If exceeded, it generates a summary of the conversation history using an LLM
3. The original messages are replaced with a condensed summary, preserving context

Basic Usage:

// Enable automatic compaction with a token threshold of 4000
// This will trigger compaction when the conversation exceeds 4000 tokens
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithCompactionThreshold(4000))

Customizing Compaction:

// Set custom compaction options
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithTools(searchTool),
    cogito.WithCompactionThreshold(4000),      // Trigger at 4000 tokens
    cogito.WithCompactionKeepMessages(5),      // Keep last 5 messages (default: 10)
)

Notes:

• Compaction requires token usage data from the LLM (supported by OpenAI, LocalAI with token usage enabled)
• If LastUsage is not available, Cogito falls back to estimating tokens from message count
• The summary prompt uses the conversation compaction prompt type
• Compaction preserves Status fields like LastUsage, ToolsCalled, etc.

Auto-Improving Agent (Self-Editing System Prompt)

Cogito supports an "autoimproving" feature where the agent can self-edit an additional system prompt across executions. After each ExecuteTools run, a review step analyzes the conversation and optionally updates the system prompt to improve future performance.

How it works:

1. Before the main loop: If the state contains a non-empty SystemPrompt, it is injected as a system message at the start of the conversation.
2. Main loop: Runs normally (unchanged).
3. After the main loop: A review step runs that analyzes the conversation and may update the system prompt via an internal edit_system_prompt tool.

Basic Usage:

// Create a state object — the caller owns and persists this across calls
state := &cogito.AutoImproveState{}

// First execution — state starts empty, review may create an initial system prompt
result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithAutoImproveState(state),
    cogito.WithTools(searchTool),
)
// state.SystemPrompt was updated in-place by the review step
saveState(state) // Persist however you like (JSON, database, etc.)

// Next execution — load the state and pass it again
loadState(state)
result, err = cogito.ExecuteTools(llm, fragment,
    cogito.WithAutoImproveState(state),
    cogito.WithTools(searchTool),
)
// state.SystemPrompt may have been further refined

Using a Separate Reviewer LLM:

You can use a different (potentially larger or more capable) LLM for the review step:

workerLLM := clients.NewOpenAILLM("worker-model", "key", "url")
reviewerLLM := clients.NewOpenAILLM("reviewer-model", "key", "url")

state := &cogito.AutoImproveState{}

result, err := cogito.ExecuteTools(workerLLM, fragment,
    cogito.WithAutoImproveState(state),
    cogito.WithAutoImproveReviewerLLM(reviewerLLM),
    cogito.WithTools(searchTool),
)

AutoImproveState Structure:

type AutoImproveState struct {
    SystemPrompt string `json:"system_prompt"` // The self-editing system prompt
    ReviewCount  int    `json:"review_count"`  // Number of reviews performed
}

Notes:

• The state is mutated in-place through the pointer — no need to read it back from the fragment
• The review step is non-fatal: if it fails, the main execution result is returned unchanged
• The review step uses an allowlist of safe options (no callbacks, guidelines, or autoimprove) to prevent recursion
• ReviewCount is incremented on each successful review, regardless of whether the prompt was changed
• The reviewer sees the complete conversation including the final response
• Serialize AutoImproveState to JSON for easy persistence between sessions

Custom Prompts

customPrompt := cogito.NewPrompt(`Your custom prompt template with {{.Context}}`)

result, err := cogito.ExecuteTools(llm, fragment,
    cogito.WithPrompt(cogito.ToolReasonerType, customPrompt))

🎮 Examples

Interactive Chat Bot

# Run the example chat application
make example-chat

This starts an interactive chat session with tool support including web search capabilities.

Custom Tool Implementation

See examples/internal/search/search.go for a complete example of implementing a DuckDuckGo search tool.

🧪 Testing

The library includes comprehensive test coverage using Ginkgo and Gomega. Tests use containerized LocalAI for integration testing.

Running Tests

# Run all tests
make test

# Run with specific log level
LOG_LEVEL=debug make test

# Run with custom arguments
GINKGO_ARGS="--focus=Fragment" make test

📄 License

Ettore Di Giacinto 2025-now. Cogito is released under the Apache 2.0 License.

📚 Citation

If you use Cogito in your research or academic work, please cite our paper:

@article{cogito2025,
  title={Cogito: A Framework for Building Intelligent Agentic Software with LLM-Powered Workflows},
  author={Ettore Di Giacinto <mudler@localai.io>},
  journal={https://github.com/mudler/cogito},
  year={2025},
  note={}
}