aleutian

Overview ¶

Package main provides CachePathResolver for optimal cache location resolution.

CachePathResolver determines the best location for model caches (HuggingFace, Ollama) with support for external drive detection, container accessibility verification, and platform-specific path handling.

Security Context ¶

This is a MODERATE-RISK component. It handles filesystem paths and may trigger Podman machine restarts. Improper path resolution could lead to data being written to unexpected locations.

Resolution Priority ¶

Caches are resolved using this priority:

1. Environment variable override (ALEUTIAN_MODELS_CACHE)
2. User-designated preferred drive (config.PreferredDrive)
3. External drive with existing cache (aleutian_data/models_cache)
4. External drive with most free space
5. Local stack directory fallback

Platform Behavior ¶

• macOS: External paths under /Volumes require Podman VM mounts
• Linux: Direct filesystem access, no VM mounts needed

Design Principles ¶

• Interface-first design for testability
• Dependencies injected (ProcessManager, UserPrompter)
• Thread-safe for concurrent use
• Single responsibility per method

Package main contains the Aleutian CLI chat runner interfaces and implementations.

This file defines the ChatRunner interface for abstracting chat loop execution, enabling different implementations for RAG and direct chat modes. It follows the interface-first pattern from claude.md Section 3.2.

Architecture:

cmd_chat.go → ChatRunner Interface → RAGChatRunner / DirectChatRunner
                                     ↓
                                     ChatService (from chat_service.go)
                                     InputReader (stdin abstraction)
                                     ChatUI (from pkg/ux)

See docs/designs/pending/streaming_chat_integration.md for full design.

Package main contains the DirectChatRunner implementation.

This file implements the ChatRunner interface for direct LLM chat mode (no RAG). It coordinates between the DirectChatService, ChatUI, and InputReader to provide an interactive chat experience without knowledge base retrieval.

See docs/designs/pending/streaming_chat_integration.md for full design.

Package main contains the RAGChatRunner implementation.

This file implements the ChatRunner interface for RAG-enabled chat mode. It coordinates between the ChatService (HTTP communication), ChatUI (display), and InputReader (user input) to provide an interactive chat experience.

See docs/designs/pending/streaming_chat_integration.md for full design.

Package main contains the Aleutian CLI chat service implementations.

This file defines the ChatService interface and its implementations for communicating with the Aleutian orchestrator's chat endpoints. It follows the patterns established in the orchestrator's datatypes package:

• All requests and responses have Id and CreatedAt fields for tracing
• Interfaces enable dependency injection and testability
• Configuration structs group related settings

Architecture:

CLI Loop → ChatService Interface → HTTPClient Interface → http.Client
          ↓                       ↓
          ragChatService         mockHTTPClient (tests)
          directChatService

See docs/code_quality_lessons/002_http_clients_microservices.md for patterns.

Package main provides LogSanitizer for removing PII from logs before LLM processing.

LogSanitizer is a critical security component that ensures no sensitive data (emails, API keys, credit cards, etc.) is passed to the LLM summarizer. All log content MUST pass through sanitization before any AI processing.

Design Rationale ¶

Direct log content cannot be safely included in LLM prompts because:

• Logs may contain PII (emails, usernames, IP addresses)
• Logs may contain secrets (API keys, tokens, passwords)
• LLM providers (even local) should follow data minimization

By applying regex-based sanitization, we ensure defense-in-depth:

• Redact known patterns before processing
• Allow custom patterns for domain-specific data
• Log what was redacted for audit purposes

Package main provides MetricsStore for ephemeral metric storage.

MetricsStore enables metric trend analysis without requiring external infrastructure like Prometheus. It stores metrics in a rolling window with optional JSONL persistence for historical queries.

Design Rationale ¶

Health intelligence needs historical data to detect trends, but we don't want to require Prometheus/Grafana for the local use case. By providing a lightweight in-memory store with optional JSONL persistence, we can:

• Track metrics without external dependencies
• Compute baselines for anomaly detection
• Persist data across restarts when needed
• Avoid SQL injection attack vectors by using JSONL

Security ¶

JSONL files are human-readable and can be encrypted at rest using the system's encryption facilities. Unlike SQLite, there are no query injection vulnerabilities to consider.

Package main contains model_ensurer.go which coordinates pre-flight model verification and automatic downloading for the Aleutian stack.

Problem Statement ¶

When users run `aleutian stack start`, required Ollama models must be available:

1. Embedding model (always required for RAG)
2. LLM model (required if backend=ollama)
3. Tool router model (optional, for trace agent tool routing)
4. Parameter extractor model (optional, for trace agent parameter extraction)

Previously, users encountered cryptic errors like "model not found" deep in the stack startup. This component provides clear, early verification with automatic remediation.

Solution ¶

ModelEnsurer coordinates between SystemChecker and OllamaModelManager:

┌─────────────────────────────────────────────────────────────────┐
│                         ModelEnsurer                            │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  EnsureModels()                                                 │
│      │                                                          │
│      ├── Build list of required models from config              │
│      │                                                          │
│      ├── For each model:                                        │
│      │   ├── OllamaModelManager.HasModel()                      │
│      │   └── OllamaModelManager.IsCustomModel()                 │
│      │                                                          │
│      ├── IF models need pulling:                                │
│      │   ├── SystemChecker.CheckNetworkConnectivity()           │
│      │   ├── SystemChecker.CheckDiskSpace()                     │
│      │   └── OllamaModelManager.PullModel() for each            │
│      │                                                          │
│      └── Return ModelEnsureResult                               │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Model Versioning ¶

Models support Ollama's name:tag format:

• "nomic-embed-text-v2-moe" → uses :latest implicitly
• "nomic-embed-text-v2-moe:latest" → explicit latest
• "gpt-oss:7b-q4" → specific quantization
• "llama3:70b" → specific size variant

Graceful Degradation ¶

If network is unavailable but required models exist locally, the ensurer returns CanProceed=true with OfflineMode=true and appropriate warnings.

Usage ¶

ensurer := NewDefaultModelEnsurer(ModelEnsurerConfig{
    EmbeddingModel: "nomic-embed-text-v2-moe",
    LLMModel:       "gpt-oss",
    DiskLimitGB:    50,
    BackendType:    "ollama",
})

ensurer.SetProgressCallback(func(status string, completed, total int64) {
    fmt.Printf("\r%s: %d/%d", status, completed, total)
})

result, err := ensurer.EnsureModels(ctx)
if err != nil {
    log.Fatal(err)
}

if !result.CanProceed {
    log.Fatal("Required models not available")
}

Configuration ¶

The ensurer respects these environment variables (which override config):

• EMBEDDING_MODEL: Override embedding model name
• OLLAMA_MODEL: Override LLM model name

Related Files ¶

• system_checker.go: System pre-flight checks
• ollama_client.go: Ollama API client
• cmd_stack.go: Integration point
• docs/designs/pending/ollama_model_management.md: Architecture

Package main provides No-Op (Null Object) implementations for all major interfaces.

These implementations satisfy interface contracts without performing any actual work. Use them as safe defaults when optional dependencies are not provided, preventing nil pointer panics while maintaining type safety.

Design Rationale ¶

In Go, a nil pointer to a struct can satisfy an interface check, but calling methods on it causes a panic. By providing No-Op implementations, we can:

• Prevent nil pointer panics in production
• Simplify testing by not requiring mock setup for unused dependencies
• Make optional dependencies explicit in the type system

Usage ¶

manager, err := NewSomeManager(deps)
// If deps.MetricsStore is nil, use NoOpMetricsStore internally

Thread Safety ¶

All No-Op implementations are safe for concurrent use (they do nothing).

Package main contains ollama_client.go which provides model management operations for the Ollama local model server.

Problem Statement ¶

When users run `aleutian stack start`, we need to ensure required models are available before starting containers. This requires:

1. Listing models currently available in Ollama
2. Detecting which models need to be downloaded
3. Pulling models with progress feedback
4. Distinguishing between registry models and user-created custom models

Previously, users had to manually run `ollama pull <model>` before using Aleutian, which created friction and confusion, especially for first-time users.

Solution ¶

OllamaModelManager provides a clean interface for model management:

┌─────────────────────────────────────────────────────────────────┐
│                    aleutian stack start                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  1. SystemChecker.IsOllamaInstalled()  ← Verify Ollama ready    │
│                                                                 │
│  2. OllamaClient.ListModels()          ← Get available models   │
│     └─ Cached for performance                                   │
│                                                                 │
│  3. OllamaClient.HasModel(embedModel)  ← Check if we need pull  │
│     OllamaClient.HasModel(llmModel)                             │
│                                                                 │
│  4. IF models missing:                                          │
│     ├─ OllamaClient.GetModelSize()     ← For disk space check   │
│     └─ OllamaClient.PullModel()        ← Download with progress │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Custom Model Detection ¶

Aleutian supports user-created GGUF models that are imported into Ollama. These models cannot be "pulled" from the registry - they only exist locally.

Detection strategy: Models created via `ollama create` or imported from GGUF have a `template` field in their metadata (from Modelfile). Registry-only models like embedding models don't have templates.

// Detect custom model
isCustom, _ := client.IsCustomModel(ctx, "my-fine-tuned-llm")
if isCustom {
    fmt.Println("This is a locally-created model")
}

Progress Callback ¶

Model pulls report progress via callback, using Ollama's native progress:

err := client.PullModel(ctx, "nomic-embed-text-v2-moe", func(status string, completed, total int64) {
    if total > 0 {
        percent := float64(completed) / float64(total) * 100
        fmt.Printf("\r  %s: %.1f%% (%s/%s)", status, percent,
            formatBytes(completed), formatBytes(total))
    } else {
        fmt.Printf("\r  %s...", status)
    }
})

Model Caching ¶

ListModels results are cached for 30 seconds to avoid redundant API calls. Use RefreshModelCache to force an update if needed.

Usage ¶

client := NewOllamaClient("http://localhost:11434")

// List all models
models, err := client.ListModels(ctx)

// Check if model exists
exists, _ := client.HasModel(ctx, "nomic-embed-text-v2-moe")

// Pull missing model with progress
if !exists {
    err = client.PullModel(ctx, "nomic-embed-text-v2-moe", progressCallback)
}

Configuration ¶

The client respects these environment variables:

• OLLAMA_HOST: Override default Ollama URL (http://localhost:11434)

Related Files ¶

• system_checker.go: Pre-flight system checks
• cmd_stack.go: Integration point (ensureOllamaModels function)
• docs/designs/pending/ollama_model_management.md: Full architecture

Package main provides ProfileResolver for hardware-based configuration optimization.

The ProfileResolver determines optimal environment configuration based on detected hardware capabilities. It supports automatic detection, explicit profile selection, and user-defined custom profiles.

Profile Tiers ¶

Built-in profiles match hardware to optimal model configurations:

• Low (< 16GB): gemma3:4b, 2048 tokens - Basic local inference
• Standard (16-32GB): qwen3:14b, 4096 tokens - Balanced performance
• Performance (32-128GB): gpt-oss:20b, 8192 tokens - Enhanced context with thinking
• Ultra (128GB+): gpt-oss:120b, 32768 tokens - Enterprise grade with thinking

Hardware Detection ¶

On macOS: Uses unified memory (sysctl hw.memsize) On Linux: Prefers NVIDIA VRAM (nvidia-smi), falls back to system RAM On Windows: Safe default (8GB assumed)

Custom Profiles ¶

Users can define custom profiles in aleutian.yaml:

profiles:
  - name: my-custom
    ollama_model: mixtral:8x7b
    max_tokens: 16384
    reranker_model: cross-encoder/ms-marco-MiniLM-L-6-v2
    min_ram_mb: 48000

Package main provides SecretsManager for secure secret management.

SecretsManager provides a centralized, secure abstraction for retrieving and managing secrets (API keys, tokens, credentials). It supports multiple backends with automatic fallback.

Security Context ¶

This is a CRITICAL-RISK component because it handles authentication credentials for external services (Anthropic, OpenAI) and internal components. Improper handling could lead to credential exposure, unauthorized access, or compliance violations.

Security Features ¶

• Zero Value Logging: Secret values are NEVER logged (even at debug level)
• Audit Trail: All access is recorded (secret name only, not value)
• Fail-Secure: Missing secrets result in clear errors with setup help
• Format Validation: Known secrets validated for expected patterns

Backend Priority ¶

Backends are tried in order until a secret is found:

1. HashiCorp Vault (if configured) - Phase 6B, not yet implemented
2. 1Password CLI (if enabled and available)
3. macOS Keychain (if enabled, darwin only)
4. Linux libsecret (if enabled, linux only)
5. Environment variables (if enabled)

Design Principles ¶

• Interface-first design for testability
• Dependencies injected (config, metrics)
• Thread-safe for concurrent use
• Single responsibility per method

Package main provides StackManager for orchestrating Aleutian stack lifecycle.

StackManager is the primary orchestrator that coordinates all stack operations: infrastructure provisioning, secrets management, model verification, profile resolution, container orchestration, and health checking.

Architecture ¶

StackManager sits at the top of the dependency hierarchy:

┌─────────────────────────────────────────────────────────────────┐
│                        StackManager                             │
│  (Orchestrates startup, shutdown, status, logs)                 │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  Start() sequence:                                              │
│    1. InfrastructureManager.EnsureReady()   // Podman machine   │
│    2. ModelEnsurer.EnsureModels()           // Ollama models    │
│    3. SecretsManager.EnsureExists()         // API keys         │
│    4. CachePathResolver.Resolve()           // Model cache      │
│    5. ProfileResolver.Resolve()             // Environment vars │
│    6. ComposeExecutor.Up()                  // Containers       │
│    7. HealthChecker.WaitForServices()       // Health check     │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Design Principles ¶

• Dependency Injection: All operations go through injected interfaces
• Single Responsibility: Each dependency handles one concern
• Testability: Full mock support for all dependencies
• Error Context: Errors are wrapped with diagnostic information
• Graceful Degradation: ModelEnsurer is optional (nil-safe)

Phase Integration ¶

This component integrates all previous refactoring phases:

• Phase 0: Config (config package)
• Phase 1: ProcessManager (used by dependencies)
• Phase 2: UserPrompter (used by InfrastructureManager)
• Phase 3: DiagnosticsCollector
• Phase 4: ProfileResolver
• Phase 5: InfrastructureManager
• Phase 6: SecretsManager
• Phase 7: CachePathResolver
• Phase 8: ComposeExecutor
• Phase 9: HealthChecker
• ModelEnsurer (already implemented)

Thread Safety ¶

StackManager is safe for concurrent use. However, only one Start/Stop/Destroy operation should be in progress at a time. Concurrent operations are serialized via mutex.

Usage ¶

// Create all dependencies
proc := NewDefaultProcessManager()
infra := NewDefaultInfrastructureManager(proc, prompter, metrics)
secrets := NewDefaultSecretsManager(proc, config.Global.Secrets.UseEnv)
cache := NewDefaultCachePathResolver(proc, prompter)
compose, _ := NewDefaultComposeExecutor(composeCfg, proc)
health := NewDefaultHealthChecker(proc, healthCfg)
models := NewDefaultModelEnsurer(modelCfg)
profile := NewDefaultProfileResolver(proc, customProfiles)
diagnostics := NewDefaultDiagnosticsCollector(proc, formatter, storage)

// Create and use StackManager
mgr := NewDefaultStackManager(
    infra, secrets, cache, compose, health,
    models, profile, diagnostics, config.Global,
)

err := mgr.Start(ctx, StartOptions{Profile: "performance"})
if err != nil {
    log.Fatal(err)
}

Package main contains the Aleutian CLI streaming chat service implementations.

This file defines the StreamingChatService interface and its implementations for communicating with the Aleutian orchestrator's streaming chat endpoints. It follows the layered streaming architecture:

HTTP Response Body → SSEParser → SSEStreamReader → StreamRenderer → StreamResult

Architecture ¶

CLI Loop → StreamingChatService Interface → HTTPClient Interface → http.Client
                    ↓                                ↓
          ragStreamingChatService           SSEParser → SSEStreamReader
          directStreamingChatService                         ↓
                                                      StreamRenderer

File Organization ¶

This file follows optimal Go code style:

1. Interfaces (contracts first)
2. Configuration structs
3. Implementation structs
4. Constructor functions
5. Methods on structs

See docs/code_quality_lessons/005_layered_streaming_architecture.md for design.