Documentation
¶
Overview ¶
Package initializer provides local initialization for Aleutian Trace.
This package implements the core logic for the `aleutian init` command, which creates a `.aleutian/` directory containing:
- index.db: Symbol index (SQLite)
- graph.db: Call graph (SQLite)
- manifest.json: File hashes for incremental updates
- config.yaml: Project settings
Thread Safety ¶
The Initializer is safe for concurrent use. File-level locking prevents multiple init operations on the same project.
Atomic Writes ¶
Index writes use the temp-directory-swap pattern to ensure the .aleutian/ directory is never in a partial state.
Buffered Channel Architecture ¶
File parsing uses buffered channels for work distribution:
┌─────────┐ ┌───────────────────┐ ┌───────────────────┐
│ Scanner │────▶│ fileChan (buffer) │────▶│ Worker Pool (N) │
└─────────┘ └───────────────────┘ └───────────────────┘
│
▼
┌─────────┐ ┌───────────────────┐ ┌───────────────────┐
│ Writer │◀────│ resultChan (buf) │◀────│ Parse Results │
└─────────┘ └───────────────────┘ └───────────────────┘
Index ¶
- Constants
- Variables
- func OptimalWorkerCount() int
- type BatchError
- type Config
- type Edge
- type FileEntry
- type FileLock
- type IndexBuilder
- type Initializer
- type ManifestFile
- type MemoryIndex
- func (m *MemoryIndex) BuildIndexes()
- func (m *MemoryIndex) EdgeCount() int
- func (m *MemoryIndex) GetByFile(filePath string) []*Symbol
- func (m *MemoryIndex) GetByID(id string) *Symbol
- func (m *MemoryIndex) GetByName(name string) []*Symbol
- func (m *MemoryIndex) GetCallees(symbolID string, limit int) []Edge
- func (m *MemoryIndex) GetCallers(symbolID string, limit int) []Edge
- func (m *MemoryIndex) SymbolCount() int
- type ParseFileError
- type ParseJob
- type ParseResult
- type Progress
- type ProgressCallback
- type ProjectConfig
- type Result
- type Storage
- func (s *Storage) AleutianPath() string
- func (s *Storage) Exists() bool
- func (s *Storage) LoadIndex(ctx context.Context) (*MemoryIndex, error)
- func (s *Storage) LoadManifest(validateChecksums bool) (*ManifestFile, error)
- func (s *Storage) WriteIndex(ctx context.Context, symbols []Symbol, edges []Edge, manifest *ManifestFile, ...) (*WriteResult, error)
- type StorageError
- type StorageReader
- type StorageWriter
- type Symbol
- type WriteResult
Constants ¶
const ( ExitSuccess = 0 // Operation completed successfully ExitFailure = 1 // Initialization failed ExitBadArgs = 2 // Invalid arguments )
Exit codes for the init command.
const ( DefaultMaxWorkers = 8 DefaultMaxFileSize = 10 * 1024 * 1024 // 10MB DefaultFileTimeout = 30 * time.Second DefaultChannelBuffer = 100 DefaultProgressBatch = 100 DefaultMemoryLimitMB = 500 AleutianDir = ".aleutian" LockFileName = ".lock" ManifestFileName = "manifest.json" IndexFileName = "index.json" ConfigFileName = "config.yaml" )
Default configuration values.
const APIVersion = "1.0"
APIVersion is the JSON output API version.
const FormatVersion = "1.0"
FormatVersion is the current index format version. Increment when making breaking changes to storage format.
const StaleLockDuration = 1 * time.Hour
StaleLockDuration is the time after which a lock is considered stale.
Variables ¶
var ( // Configuration errors ErrEmptyProjectRoot = errors.New("project root must not be empty") ErrInvalidMaxWorkers = errors.New("max workers must be greater than 0") ErrInvalidMaxFileSize = errors.New("max file size must be greater than 0") ErrPathNotExist = errors.New("path does not exist") ErrPathNotDirectory = errors.New("path is not a directory") ErrPathTraversal = errors.New("path traversal detected") // Lock errors ErrLockAcquireFailed = errors.New("failed to acquire lock") ErrLockHeld = errors.New("another init operation is in progress") // Storage errors ErrIndexCorrupted = errors.New("index file is corrupted") ErrChecksumMismatch = errors.New("checksum validation failed") ErrVersionMismatch = errors.New("index format version mismatch") ErrAtomicSwapFailed = errors.New("atomic directory swap failed") ErrDatabaseOpenFailed = errors.New("failed to open database") // Parsing errors ErrNoSupportedFiles = errors.New("no supported files found") ErrNoLanguages = errors.New("no supported languages detected") ErrParseTimeout = errors.New("parse operation timed out") // Context errors ErrContextCancelled = errors.New("operation was cancelled") )
Sentinel errors for initialization.
Functions ¶
func OptimalWorkerCount ¶
func OptimalWorkerCount() int
OptimalWorkerCount returns the optimal number of workers for the system.
Types ¶
type BatchError ¶
type BatchError struct {
Errors []error
}
BatchError holds multiple errors from batch operations.
func (*BatchError) Error ¶
func (e *BatchError) Error() string
Error implements the error interface.
func (*BatchError) HasErrors ¶
func (e *BatchError) HasErrors() bool
HasErrors returns true if there are any errors.
func (*BatchError) ToError ¶
func (e *BatchError) ToError() error
ToError returns nil if no errors, or the BatchError if there are errors.
type Config ¶
type Config struct {
ProjectRoot string
Languages []string
ExcludePatterns []string
MaxWorkers int
MaxFileSize int64
FileTimeout time.Duration
Force bool
DryRun bool
Quiet bool
Verbose bool
}
Config holds initialization configuration.
Fields ¶
- ProjectRoot: Absolute path to project root. Must not be empty.
- Languages: Languages to parse. Empty means auto-detect.
- ExcludePatterns: Glob patterns to exclude.
- MaxWorkers: Maximum parallel workers. Must be > 0.
- MaxFileSize: Maximum file size in bytes. Files larger are skipped.
- FileTimeout: Per-file parse timeout. Zero means no timeout.
- Force: If true, rebuild index even if exists.
- DryRun: If true, show what would be indexed without writing.
- Quiet: If true, suppress progress output.
- Verbose: If true, show detailed per-file output.
func DefaultConfig ¶
DefaultConfig returns a Config with sensible defaults.
type Edge ¶
type Edge struct {
FromID string `json:"from_id"`
ToID string `json:"to_id"`
Kind string `json:"kind"` // calls, imports, implements, etc.
FilePath string `json:"file_path"`
Line int `json:"line"`
}
Edge represents a relationship between symbols.
type FileEntry ¶
type FileEntry struct {
Path string `json:"path"`
Hash string `json:"hash"`
Mtime int64 `json:"mtime"`
Size int64 `json:"size"`
}
FileEntry represents a single file in the manifest.
type FileLock ¶
type FileLock struct {
// contains filtered or unexported fields
}
FileLock provides advisory file locking to prevent concurrent init operations.
Thread Safety ¶
FileLock is NOT safe for concurrent use. Each goroutine should have its own instance.
Platform Support ¶
Uses flock(2) on Unix systems. On Windows, uses LockFileEx.
func NewFileLock ¶
NewFileLock creates a new FileLock for the given project root.
Description ¶
Creates a lock file at {projectRoot}/.aleutian/.lock. The .aleutian/ directory is created if it doesn't exist.
Inputs ¶
- projectRoot: Absolute path to project root. Must not be empty.
Outputs ¶
- *FileLock: The lock instance (not yet acquired).
- error: ErrEmptyProjectRoot if projectRoot is empty.
func (*FileLock) Acquire ¶
Acquire attempts to acquire an exclusive lock.
Description ¶
Creates the lock file and attempts to acquire an exclusive advisory lock. If the lock is already held by another process, returns ErrLockHeld.
Inputs ¶
- None
Outputs ¶
- error: ErrLockHeld if lock is held, or other error on failure.
Limitations ¶
- Advisory lock only - other processes can ignore it.
- Must call Release() to free the lock.
func (*FileLock) ForceRelease ¶
ForceRelease removes a stale lock file.
Description ¶
Only call this after confirming the lock is stale via IsStale(). This is a recovery mechanism for crashed processes.
Inputs ¶
- None
Outputs ¶
- error: Any error from removing the file.
Limitations ¶
- Race condition: Another process could grab the lock between IsStale() and ForceRelease().
func (*FileLock) HolderPID ¶
HolderPID returns the PID of the process holding the lock.
Description ¶
Reads the lock file to get the PID of the holder. Returns 0 if the lock file doesn't exist or is empty.
Inputs ¶
- None
Outputs ¶
- int: PID of lock holder, or 0 if unknown.
func (*FileLock) IsHeld ¶
IsHeld checks if the lock is currently held by another process.
Description ¶
Attempts to acquire the lock non-blocking. If successful, releases it immediately. This is a quick check without actually holding the lock.
Inputs ¶
- None
Outputs ¶
- bool: True if lock is held by another process.
- error: Any error (other than lock-held condition).
func (*FileLock) IsStale ¶
IsStale checks if the lock file appears to be stale.
Description ¶
A lock is considered stale if:
- The lock file is older than StaleLockDuration
- The holding process no longer exists
Inputs ¶
- None
Outputs ¶
- bool: True if the lock appears stale.
type IndexBuilder ¶
type IndexBuilder interface {
// Init initializes the index for a project.
Init(ctx context.Context, cfg Config, progress ProgressCallback) (*Result, error)
}
IndexBuilder defines the interface for building the code index.
Thread Safety ¶
Implementations must be safe for concurrent use.
type Initializer ¶
type Initializer struct {
// contains filtered or unexported fields
}
Initializer builds the code index for a project.
Description ¶
Initializer orchestrates the parsing and indexing of source files. It uses buffered channels for work distribution to achieve parallel file processing.
Architecture ¶
┌─────────┐ ┌───────────────────┐ ┌───────────────────┐
│ Scanner │────▶│ fileChan (buffer) │────▶│ Worker Pool (N) │
└─────────┘ └───────────────────┘ └───────────────────┘
│
▼
┌─────────┐ ┌───────────────────┐ ┌───────────────────┐
│ Collector│◀────│ resultChan (buf) │◀────│ Parse Results │
└─────────┘ └───────────────────┘ └───────────────────┘
Thread Safety ¶
Initializer is safe for concurrent use.
func NewInitializer ¶
func NewInitializer(storage StorageWriter) *Initializer
NewInitializer creates a new Initializer.
Description ¶
Creates an Initializer with the given storage backend.
Inputs ¶
- storage: Storage backend for persisting the index. Must not be nil.
Outputs ¶
- *Initializer: The initializer instance.
Thread Safety ¶
The returned Initializer is safe for concurrent use.
func (*Initializer) Init ¶
func (i *Initializer) Init(ctx context.Context, cfg Config, progress ProgressCallback) (*Result, error)
Init initializes the index for a project.
Description ¶
Performs full initialization:
- Acquires exclusive lock to prevent concurrent inits
- Detects languages in project (if not specified)
- Scans for source files matching language extensions
- Parses files in parallel using buffered channels
- Collects symbols and builds call graph
- Writes index atomically to .aleutian/
Inputs ¶
- ctx: Context for cancellation. Must not be nil.
- cfg: Configuration for initialization. Must be valid (cfg.Validate() == nil).
- progress: Callback for progress updates. May be nil.
Outputs ¶
- *Result: Initialization results. Never nil on success.
- error: Non-nil on failure. See errors.go for possible errors.
Limitations ¶
- Only one init can run per project at a time (file lock)
- Files larger than cfg.MaxFileSize are skipped
- Unparseable files are skipped with warnings (not errors)
Assumptions ¶
- cfg.ProjectRoot is an absolute path to an existing directory
- Caller has write permission on the project root
type ManifestFile ¶
type ManifestFile struct {
FormatVersion string `json:"format_version"`
ProjectRoot string `json:"project_root"`
Files map[string]FileEntry `json:"files"`
IndexChecksum string `json:"index_checksum"`
GraphChecksum string `json:"graph_checksum"`
CreatedAtMilli int64 `json:"created_at_milli"`
UpdatedAtMilli int64 `json:"updated_at_milli"`
}
ManifestFile holds the file manifest with checksums.
type MemoryIndex ¶
type MemoryIndex struct {
Symbols []Symbol `json:"symbols"`
Edges []Edge `json:"edges"`
// contains filtered or unexported fields
}
MemoryIndex holds the in-memory index for fast queries.
Description ¶
MemoryIndex provides O(1) lookups by symbol ID and efficient filtering by name, kind, and file path. Loaded from JSON files and kept in memory for the duration of operations.
Thread Safety ¶
MemoryIndex is safe for concurrent read access. Write operations (Add*) must be synchronized externally or done during initialization.
func NewMemoryIndex ¶
func NewMemoryIndex() *MemoryIndex
NewMemoryIndex creates an empty MemoryIndex.
Description ¶
Creates an initialized MemoryIndex ready for adding symbols and edges.
Outputs ¶
- *MemoryIndex: Empty index with initialized maps.
Thread Safety ¶
The returned MemoryIndex is safe for concurrent use after initialization.
func (*MemoryIndex) BuildIndexes ¶
func (m *MemoryIndex) BuildIndexes()
BuildIndexes builds the in-memory lookup indexes from Symbols and Edges.
Description ¶
Call this after loading symbols/edges from JSON to build the fast lookup maps. Must be called before query methods are used.
Thread Safety ¶
NOT safe for concurrent use. Call during initialization only.
func (*MemoryIndex) EdgeCount ¶
func (m *MemoryIndex) EdgeCount() int
EdgeCount returns the number of edges in the index.
func (*MemoryIndex) GetByFile ¶
func (m *MemoryIndex) GetByFile(filePath string) []*Symbol
GetByFile retrieves symbols in a file.
Inputs ¶
- filePath: File path to look up.
Outputs ¶
- []*Symbol: Symbols in the file, or empty slice if none.
Thread Safety ¶
Safe for concurrent use.
func (*MemoryIndex) GetByID ¶
func (m *MemoryIndex) GetByID(id string) *Symbol
GetByID retrieves a symbol by its ID.
Inputs ¶
- id: Symbol ID to look up.
Outputs ¶
- *Symbol: The symbol, or nil if not found.
Thread Safety ¶
Safe for concurrent use.
func (*MemoryIndex) GetByName ¶
func (m *MemoryIndex) GetByName(name string) []*Symbol
GetByName retrieves symbols by name.
Inputs ¶
- name: Symbol name to look up.
Outputs ¶
- []*Symbol: Matching symbols, or empty slice if none found.
Thread Safety ¶
Safe for concurrent use.
func (*MemoryIndex) GetCallees ¶
func (m *MemoryIndex) GetCallees(symbolID string, limit int) []Edge
GetCallees retrieves edges originating FROM the given symbol.
Inputs ¶
- symbolID: Symbol ID to find callees for.
- limit: Maximum edges to return (0 = unlimited).
Outputs ¶
- []Edge: Edges where FromID matches symbolID.
Thread Safety ¶
Safe for concurrent use.
func (*MemoryIndex) GetCallers ¶
func (m *MemoryIndex) GetCallers(symbolID string, limit int) []Edge
GetCallers retrieves edges pointing TO the given symbol.
Inputs ¶
- symbolID: Symbol ID to find callers for.
- limit: Maximum edges to return (0 = unlimited).
Outputs ¶
- []Edge: Edges where ToID matches symbolID.
Thread Safety ¶
Safe for concurrent use.
func (*MemoryIndex) SymbolCount ¶
func (m *MemoryIndex) SymbolCount() int
SymbolCount returns the number of symbols in the index.
type ParseFileError ¶
ParseFileError represents an error parsing a specific file.
func (*ParseFileError) Error ¶
func (e *ParseFileError) Error() string
Error implements the error interface.
func (*ParseFileError) Unwrap ¶
func (e *ParseFileError) Unwrap() error
Unwrap returns the underlying error.
type ParseResult ¶
ParseResult holds the result of parsing a single file.
type Progress ¶
type Progress struct {
Phase string
FilesTotal int
FilesScanned int
FilesCurrent string
Percent int
}
Progress represents initialization progress.
type ProgressCallback ¶
type ProgressCallback func(Progress)
ProgressCallback is called during initialization with progress updates.
type ProjectConfig ¶
type ProjectConfig struct {
FormatVersion string `yaml:"format_version" json:"format_version"`
Languages []string `yaml:"languages" json:"languages"`
ExcludePatterns []string `yaml:"exclude_patterns" json:"exclude_patterns"`
CreatedAt string `yaml:"created_at" json:"created_at"`
UpdatedAt string `yaml:"updated_at" json:"updated_at"`
}
ProjectConfig holds project-specific settings stored in config.yaml.
type Result ¶
type Result struct {
APIVersion string `json:"api_version"`
ProjectRoot string `json:"project_root"`
Languages []string `json:"languages"`
FilesIndexed int `json:"files_indexed"`
SymbolsFound int `json:"symbols_found"`
EdgesBuilt int `json:"edges_built"`
DurationMs int64 `json:"duration_ms"`
IndexPath string `json:"index_path"`
Warnings []string `json:"warnings,omitempty"`
Incremental bool `json:"incremental"`
FilesChanged int `json:"files_changed,omitempty"`
}
Result holds the initialization result.
Fields ¶
- ProjectRoot: The initialized project path.
- Languages: Languages detected/used.
- FilesIndexed: Number of files successfully indexed.
- SymbolsFound: Total symbols extracted.
- EdgesBuilt: Call graph edges created.
- DurationMs: Total initialization time in milliseconds.
- IndexPath: Path to .aleutian/ directory.
- Warnings: Non-fatal issues encountered.
- Incremental: True if this was an incremental update.
- FilesChanged: Number of files changed (for incremental).
type Storage ¶
type Storage struct {
// contains filtered or unexported fields
}
Storage provides persistent storage for the index using JSON files.
Description ¶
Storage implements both StorageWriter and StorageReader interfaces, providing JSON-based persistence for symbols and call graph edges. Data is loaded into memory for fast queries.
File Structure ¶
.aleutian/ ├── manifest.json # File hashes and metadata ├── index.json # Symbols and edges └── config.yaml # Project settings
Thread Safety ¶
Storage is safe for concurrent use. Write operations are serialized through atomic directory swaps.
func NewStorage ¶
NewStorage creates a new Storage for the given project root.
Description ¶
Creates a Storage instance for managing the .aleutian/ directory. Does not create any files or directories until WriteIndex is called.
Inputs ¶
- projectRoot: Absolute path to project root. Must not be empty.
Outputs ¶
- *Storage: The storage instance.
Thread Safety ¶
The returned Storage is safe for concurrent use.
Assumptions ¶
- projectRoot is an absolute path
- Caller has validated projectRoot exists and is a directory
func (*Storage) AleutianPath ¶
AleutianPath returns the path to the .aleutian directory.
Description ¶
Returns the full path to the .aleutian directory within the project.
Outputs ¶
- string: Absolute path to .aleutian directory.
Thread Safety ¶
This method is safe for concurrent use.
func (*Storage) Exists ¶
Exists checks if the .aleutian directory exists with valid files.
Description ¶
Checks for the existence of the .aleutian directory and the manifest file. Does not validate checksums or file contents.
Outputs ¶
- bool: True if .aleutian directory exists with manifest file.
Thread Safety ¶
This method is safe for concurrent use.
func (*Storage) LoadIndex ¶
func (s *Storage) LoadIndex(ctx context.Context) (*MemoryIndex, error)
LoadIndex loads the full index into memory.
Description ¶
Reads and parses the index.json file, then builds in-memory lookup indexes for fast queries.
Inputs ¶
- ctx: Context for cancellation. Must not be nil.
Outputs ¶
- *MemoryIndex: The loaded index with built lookup maps.
- error: Non-nil on failure.
Thread Safety ¶
This method is safe for concurrent use.
func (*Storage) LoadManifest ¶
func (s *Storage) LoadManifest(validateChecksums bool) (*ManifestFile, error)
LoadManifest loads the manifest file from disk.
Description ¶
Reads and parses the manifest.json file. Optionally validates checksums of the index.json file against stored values.
Inputs ¶
- validateChecksums: If true, verify index checksum matches stored value.
Outputs ¶
- *ManifestFile: The loaded manifest. Never nil on success.
- error: Non-nil on failure. Returns ErrIndexCorrupted if file is invalid, ErrVersionMismatch if format version differs, ErrChecksumMismatch if checksums don't match.
Thread Safety ¶
This method is safe for concurrent use.
func (*Storage) WriteIndex ¶
func (s *Storage) WriteIndex( ctx context.Context, symbols []Symbol, edges []Edge, manifest *ManifestFile, config *ProjectConfig, ) (*WriteResult, error)
WriteIndex atomically writes the index to disk as JSON files.
Description ¶
Uses the temp-directory-swap pattern for atomic writes:
- Write to temp directory (.aleutian.tmp.{timestamp})
- Compute SHA256 checksum of index.json
- Backup existing .aleutian (if any) to .aleutian.backup.{timestamp}
- Atomic rename temp to .aleutian
- Remove backup on success, restore on failure
Inputs ¶
- ctx: Context for cancellation. Must not be nil.
- symbols: Symbols to write to index.json. May be empty.
- edges: Edges to write to index.json. May be empty.
- manifest: Manifest with file hashes. Must not be nil.
- config: Project configuration. Must not be nil.
Outputs ¶
- *WriteResult: Path and checksum of written index. Never nil on success.
- error: Non-nil on failure. Returns ErrAtomicSwapFailed if rename fails, context.Canceled or context.DeadlineExceeded if cancelled.
Thread Safety ¶
This method is safe for concurrent use. Uses atomic operations.
Limitations ¶
- Requires write permission on project root directory
- Temp directory must be on same filesystem for atomic rename
type StorageError ¶
StorageError wraps storage-related errors with context.
func (*StorageError) Error ¶
func (e *StorageError) Error() string
Error implements the error interface.
func (*StorageError) Unwrap ¶
func (e *StorageError) Unwrap() error
Unwrap returns the underlying error.
type StorageReader ¶
type StorageReader interface {
// Exists checks if the .aleutian directory exists with valid files.
Exists() bool
// LoadManifest loads the manifest file from disk.
LoadManifest(validateChecksums bool) (*ManifestFile, error)
// LoadIndex loads the full index into memory.
LoadIndex(ctx context.Context) (*MemoryIndex, error)
}
StorageReader defines the interface for reading index data.
Thread Safety ¶
Implementations must be safe for concurrent use.
type StorageWriter ¶
type StorageWriter interface {
// WriteIndex atomically writes the index to disk.
WriteIndex(ctx context.Context, symbols []Symbol, edges []Edge, manifest *ManifestFile, config *ProjectConfig) (*WriteResult, error)
}
StorageWriter defines the interface for writing index data.
Thread Safety ¶
Implementations must be safe for concurrent use.
type Symbol ¶
type Symbol struct {
ID string `json:"id"`
Name string `json:"name"`
Kind string `json:"kind"`
FilePath string `json:"file_path"`
StartLine int `json:"start_line"`
EndLine int `json:"end_line"`
StartCol int `json:"start_col"`
EndCol int `json:"end_col"`
Signature string `json:"signature,omitempty"`
Parent string `json:"parent,omitempty"`
Metadata map[string]string `json:"metadata,omitempty"`
}
Symbol represents a code symbol (function, type, etc.).
type WriteResult ¶
WriteResult holds the result of writing the index.
Source Files