performance

Agent Performance Optimization

This package provides performance optimization features for the Agent framework, including pooling, caching, and batch execution.

Features

1. Agent Pooling (pool.go)

Agent pooling reduces the overhead of creating new Agent instances by maintaining a pool of pre-created agents.

Key Features:

• Pre-creation of Agent instances
• Configurable pool size (initial and max)
• Automatic idle Agent recycling
• Agent lifecycle management (max lifetime)
• Comprehensive statistics

Performance Benefits:

• ~10% faster than creating new agents for each request
• Significantly reduces memory allocations
• Better performance under high concurrency

Usage:

import "github.com/kart-io/goagent/performance"

// Define factory
factory := func() (core.Agent, error) {
    return MyAgent{}, nil
}

// Configure pool
config := performance.PoolConfig{
    InitialSize:     5,
    MaxSize:         50,
    IdleTimeout:     5 * time.Minute,
    MaxLifetime:     30 * time.Minute,
    AcquireTimeout:  5 * time.Second,
    CleanupInterval: 1 * time.Minute,
}

// Create pool
pool, err := performance.NewAgentPool(factory, config)
defer pool.Close()

// Execute with automatic acquire/release
output, err := pool.Execute(ctx, input)

// Or manually manage
agent, err := pool.Acquire(ctx)
defer pool.Release(agent)
output, err := agent.Execute(ctx, input)

Statistics:

stats := pool.Stats()
fmt.Printf("Active: %d/%d (%.2f%% utilization)\n",
    stats.ActiveCount, stats.MaxSize, stats.UtilizationPct)

2. Result Caching (cache.go)

Caching stores execution results based on input, dramatically reducing response time for repeated requests.

Key Features:

• SHA256-based cache key generation
• Configurable TTL (Time To Live)
• Automatic cache eviction (LRU)
• Custom key generator support
• Hit rate tracking

Performance Benefits:

• 98.82% faster for cache hits (1ms → <1µs)
• Cache hit rate typically >90% for repeated operations
• Near-zero latency for cached results

Usage:

// Create cached agent
agent := MyAgent{}
config := performance.CacheConfig{
    MaxSize:         1000,
    TTL:             10 * time.Minute,
    CleanupInterval: 1 * time.Minute,
    EnableStats:     true,
}

cachedAgent := performance.NewCachedAgent(agent, config)
defer cachedAgent.Close()

// Execute (automatically cached)
output, err := cachedAgent.Execute(ctx, input)

// Custom cache key generator
config.KeyGenerator = func(input *core.AgentInput) string {
    return fmt.Sprintf("%s:%s", input.Task, input.Instruction)
}

Cache Management:

// Invalidate specific entry
cachedAgent.Invalidate(input)

// Clear all cache
cachedAgent.InvalidateAll()

// Check statistics
stats := cachedAgent.Stats()
fmt.Printf("Hit rate: %.2f%% (Hits: %d, Misses: %d)\n",
    stats.HitRate, stats.Hits, stats.Misses)

3. Batch Execution (batch.go)

Batch execution processes multiple inputs concurrently with controlled parallelism.

Key Features:

• Configurable concurrency limit
• Error handling strategies (fail-fast / continue)
• Result aggregation
• Streaming results support
• Comprehensive statistics

Performance Benefits:

• 12x speedup for 1000 tasks with 10 concurrent workers
• Near-linear scaling with concurrency
• Efficient resource utilization

Usage:

// Create batch executor
agent := MyAgent{}
config := performance.BatchConfig{
    MaxConcurrency: 10,
    Timeout:        5 * time.Minute,
    ErrorPolicy:    performance.ErrorPolicyContinue,
    EnableStats:    true,
}

executor := performance.NewBatchExecutor(agent, config)

// Prepare inputs
inputs := []*core.AgentInput{...}

// Execute batch
result := executor.Execute(ctx, inputs)

fmt.Printf("Success: %d/%d\n",
    result.Stats.SuccessCount, result.Stats.TotalCount)

Error Policies:

• ErrorPolicyFailFast: Stop on first error
• ErrorPolicyContinue: Continue and collect all errors

Streaming Execution:

// Process results as they arrive
resultChan, errorChan := executor.ExecuteStream(ctx, inputs)

for {
    select {
    case output := <-resultChan:
        // Handle successful result
    case err := <-errorChan:
        // Handle error
    }
}

4. Object Pooling (pool_manager.go, pool_strategies.go)

Object pooling reduces memory allocations and GC pressure by reusing frequently created objects.

Architecture:

• PoolManager Interface - Clean abstraction for pool operations
• PoolAgent Implementation - Concrete implementation with dependency injection
• Strategy Pattern - Flexible pool behavior control
• Agent Pattern - Pool manager as an Agent

Key Features:

• Pools for 6 high-frequency object types
• Zero allocations for pooled objects
• Automatic object cleanup and reset
• Large object protection (prevents memory bloat)
• Statistics tracking for each pool
• Dependency injection - No global state
• Multiple strategies - Adaptive, Scenario-based, Metrics, Priority

Usage:

// Create pool manager with configuration
config := &performance.PoolManagerConfig{
    EnabledPools: map[performance.PoolType]bool{
        performance.PoolTypeByteBuffer: true,
        performance.PoolTypeMessage:    true,
    },
    MaxBufferSize: 64 * 1024,
    MaxMapSize:    100,
}

manager := performance.NewPoolAgent(config)

// Use the pool
buf := manager.GetBuffer()
defer manager.PutBuffer(buf)

buf.WriteString("Hello, World!")

Strategies:

// Adaptive strategy - auto-adjusts based on usage frequency
adaptiveStrategy := performance.NewAdaptivePoolStrategy(config)
config.UseStrategy = adaptiveStrategy

// Scenario-based strategy - predefined configurations
scenarioStrategy := performance.NewScenarioBasedStrategy(config)
scenarioStrategy.SetScenario(performance.ScenarioLLMCalls)

// Metrics strategy - collect performance metrics
metricsStrategy := performance.NewMetricsPoolStrategy(baseStrategy, collector)

Agent Pattern:

// Create pool manager as an Agent
agent := performance.NewPoolManagerAgent("pool_optimizer", config)

input := &core.AgentInput{
    Task: "configure_pools",
    Context: map[string]interface{}{
        "scenario": "llm_calls",
    },
}

output, _ := agent.Execute(ctx, input)

Performance Benefits:

Pool Statistics:

// Get individual pool stats
stats := manager.GetStats(performance.PoolTypeByteBuffer)
fmt.Printf("Gets: %d, Puts: %d, News: %d\n",
    stats.Gets.Load(), stats.Puts.Load(), stats.News.Load())

// Get all pool stats
allStats := manager.GetAllStats()
for poolType, stats := range allStats {
    fmt.Printf("%s - Gets: %d, Puts: %d\n",
        poolType, stats.Gets.Load(), stats.Puts.Load())
}

// Reset statistics
manager.ResetStats()

Best Practices:

• Always use defer to ensure objects are returned to pool
• Buffers >64KB are automatically discarded (not pooled)
• Maps >100 keys are reallocated to prevent memory leaks
• Never store pooled objects in global variables
• Objects are automatically cleaned when returned to pool
• Use dependency injection for better testability

When to Use:

• High-frequency LLM API calls (Message, AgentInput/Output)
• Concurrent tool execution (ToolInput/Output)
• JSON serialization/deserialization (ByteBuffer)
• Stream response processing

When NOT to Use:

• Low-frequency operations (<100/sec)
• Long-lived objects (minute-scale lifetimes)
• Objects with highly variable sizes

Combined Usage

All optimizations can be combined for maximum performance:

// 1. Create agent pool with cached agents
factory := func() (core.Agent, error) {
    agent := MyAgent{}
    cacheConfig := performance.DefaultCacheConfig()
    return performance.NewCachedAgent(agent, cacheConfig), nil
}

pool, _ := performance.NewAgentPool(factory, performance.DefaultPoolConfig())
defer pool.Close()

// 2. Create batch executor using the pool
batchConfig := performance.DefaultBatchConfig()
poolAgent := &PoolAgent{pool: pool}
executor := performance.NewBatchExecutor(poolAgent, batchConfig)

// 3. Execute batch with pooling + caching + parallelism
result := executor.Execute(ctx, inputs)

Performance Benchmarks

Pool Performance

Non-pooled: 861,164 ns/op (659 B/op, 9 allocs/op)
Pooled:     776,556 ns/op (1226 B/op, 17 allocs/op)
Improvement: ~10% faster

Cache Performance

Uncached:   1,064,088 ns/op (560 B/op, 7 allocs/op)
Cached:          942 ns/op (818 B/op, 11 allocs/op)
Improvement: 98.82% faster (1130x speedup)

Batch Performance

Serial (1000 tasks):     1,049 ms
Batch (10 concurrent):      87 ms
Speedup: 12.01x

Concurrent Pool Access

1 Goroutine:     998,113 ns/op
10 Goroutines:    19,900 ns/op (50x better throughput)

Object Pool Performance

ByteBuffer (with pool):    13 ns/op (0 allocs/op)
ByteBuffer (without pool): 20 ns/op (1 alloc/op)
Improvement: 35% faster, 0 allocations

Message (with pool):       12 ns/op (0 allocs/op)
Message (without pool):    25 ns/op (1 alloc/op)
Improvement: 52% faster, 0 allocations

ToolInput (with pool):     28 ns/op (0 allocs/op)
ToolInput (without pool):  50 ns/op (1 alloc/op)
Improvement: 44% faster, 0 allocations

Configuration Guidelines

Pool Size

• Initial Size: 10-20% of max expected concurrency
• Max Size: Based on available resources (CPU cores × 2-4)
• Idle Timeout: 5-10 minutes for typical workloads
• Max Lifetime: 30-60 minutes to prevent memory leaks

Cache Size

• Max Size: Based on unique request patterns (1000-10000)
• TTL: Based on data freshness requirements (1-60 minutes)
• Cleanup Interval: 1-5 minutes

Batch Concurrency

• Max Concurrency: Number of CPU cores × 2
• Timeout: Based on longest expected task duration × 1.5
• Error Policy:
  • Use FailFast for critical operations
  • Use Continue for best-effort batch processing

Best Practices

1. Monitor Statistics: Always check pool/cache/batch statistics in production
2. Tune Based on Load: Adjust configurations based on actual usage patterns
3. Resource Limits: Set appropriate max sizes to prevent resource exhaustion
4. Error Handling: Always handle errors from batch execution
5. Graceful Shutdown: Always call Close() on pools and cached agents

Testing

Run performance tests:

# Unit tests
go test ./pkg/agent/performance/...

# Performance report
go test -v ./pkg/agent/performance/... -run TestPerformanceReport

# Benchmarks
go test -bench=. -benchmem ./pkg/agent/performance/...

# Specific benchmark
go test -bench=BenchmarkCachedVsUncached -benchtime=10s ./pkg/agent/performance/...

Examples

See example_test.go for detailed usage examples:

• Example_agentPool: Basic pool usage
• Example_cachedAgent: Caching with hit rate tracking
• Example_batchExecutor: Batch processing
• Example_combinedOptimizations: Using all three together
• Example_streamingBatch: Streaming results
• Example_customCacheKey: Custom cache key generation

Decoupled Architecture Example

See ../examples/advanced/pool-decoupled-architecture/ for comprehensive decoupled architecture demonstration:

• Dependency Injection: Creating isolated pool manager instances
• Strategy Pattern: Adaptive, Scenario-based, Metrics, and Priority strategies
• Agent Pattern: Pool manager as an Agent
• Scenario-driven Configuration: Auto-configuration for different use cases
• Metrics Collection: Integration with monitoring systems
• Isolated Testing: Test-friendly design

Run the example:

cd ../examples/advanced/pool-decoupled-architecture
make run

Features demonstrated:

• 6 different usage patterns
• Multiple pool strategies
• Performance monitoring
• Best practices

Performance Metrics Summary

License

Copyright (c) 2025 kart-io