engine

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 30, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package engine is the in-repo façade over the squasher: a small, typed entry point (Config, SquashFiles, SquashDirectory, AnalyzeDirectory) that the fixture and fuzz suites drive without going through cobra. The CLI in internal/cli shares its safety-level parsing. It is not an import surface for other modules; capysquash is consumed as a binary.

Package engine - Engine wrapper with advanced features

Package engine - Statistics and progress tracking types

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AnalysisResult

type AnalysisResult struct {
	// TotalFiles is the number of migration files analyzed
	TotalFiles int

	// TotalStatements is the total number of SQL statements
	TotalStatements int

	// TotalObjects is the number of database objects
	TotalObjects int

	// Redundancies lists redundant operations that can be consolidated
	Redundancies []Redundancy

	// ObjectsByType maps object types to counts
	ObjectsByType map[string]int

	// DataOperations contains detailed data operation counts
	DataOperations DataOpCounts

	// Warnings contains validation warnings
	Warnings []string
}

AnalysisResult contains the results of migration analysis.

func AnalyzeDirectory

func AnalyzeDirectory(directory string, config *Config) (*AnalysisResult, error)

AnalyzeDirectory analyzes migration files without making modifications.

If config is nil, DefaultConfig() is used.

Analysis is a read-only, static pass over the migration set. The provided Config is validated up front (SafetyLevel, OutputFormat, and RuleOverrides names are rejected when invalid) and Verbose controls logging. The remaining Config options are ignored by design because analysis neither consolidates nor writes anything: SafetyLevel/RuleOverrides select consolidation rules (analysis applies none), and the output, streaming, backup, rollback, and cycle-detection options only affect squash runs.

Example:

analysis, err := engine.AnalyzeDirectory("./migrations", nil)
if err != nil {
    return err
}
fmt.Printf("Found %d redundancies\n", len(analysis.Redundancies))

type Config

type Config struct {
	// Context controls cancellation and request lifetime for all engine work.
	// Nil uses context.Background.
	Context context.Context

	// SafetyLevel determines consolidation aggressiveness (default: Standard)
	SafetyLevel SafetyLevel

	// RuleOverrides force-enables (true) or force-disables (false) specific
	// named consolidation rules relative to the SafetyLevel baseline. Rule
	// names must match the catalog served by consolidation.GetRegistry (e.g.
	// "create_alter_consolidation", "function_deduplication"). Unknown rule
	// names are rejected with an error when the engine is constructed, never
	// silently ignored. A nil/empty map applies the baseline unchanged.
	//
	// This is the per-request integration point for org-scoped rule overrides
	// (the global rule registry is an immutable catalog and is never mutated).
	RuleOverrides map[string]bool

	// ProdDBDSN is the production database connection string required by the
	// Paranoid safety level (database validation) and used by backup
	// generation. When empty, the PROD_DB_DSN environment variable is used.
	ProdDBDSN string

	// ValidationDSN is the empty, disposable database the Paranoid safety
	// level applies the squashed baseline to before comparing its catalog
	// with production's. When empty, CAPYSQUASH_VALIDATION_DSN is used.
	ValidationDSN string

	// Version is the caller's tool version stamped into provenance metadata
	// (SquashResult.ProvenanceInfo.Version / .squashmap.json). Callers should
	// set it from their release metadata; empty falls back to "dev".
	Version string

	// OutputFormat determines file organization (default: FormatSingle)
	OutputFormat OutputFormat

	// SeparateDataOps when true, returns separate DDL and data operations files
	SeparateDataOps bool

	// EnableStreaming enables memory-efficient processing for large datasets (default: false)
	EnableStreaming bool

	// MemoryLimitMB sets memory limit for streaming mode (default: 256)
	MemoryLimitMB int

	// BatchSize controls how many migrations are processed in each batch (streaming mode)
	// Default: 50. Increase for better throughput, decrease to reduce memory usage.
	BatchSize int

	// WorkerCount sets the number of parallel workers (streaming mode)
	// Default: 4. Should match available CPU cores for optimal performance.
	WorkerCount int

	// ProgressCallback is called during processing to report progress
	// The callback receives: (processed, total, phase)
	//
	// Example:
	//   config.ProgressCallback = func(processed, total int64, phase string) {
	//       percent := float64(processed) / float64(total) * 100
	//       fmt.Printf("[%s] %.1f%% complete\n", phase, percent)
	//   }
	ProgressCallback ProgressCallback

	// EnableBackup enables backup generation before squashing (default: false)
	// Creates timestamped backups of original migrations for safety
	EnableBackup bool

	// BackupPath specifies where to store backups (default: "./backups")
	// Only used if EnableBackup is true
	BackupPath string

	// BackupRetentionDays specifies how long to keep backups (default: 30)
	// Older backups are automatically cleaned up
	BackupRetentionDays int

	// EnableRollback enables rollback script generation (default: false)
	// Creates scripts to undo the squashing operation
	EnableRollback bool

	// RollbackPath specifies where to store rollback scripts (default: "./rollbacks")
	// Only used if EnableRollback is true
	RollbackPath string

	// EnableCycleDetection enables DDL cycle detection (default: false)
	// Detects circular dependencies in migrations
	EnableCycleDetection bool

	// ShowCycleDetails shows detailed cycle information (default: false)
	// Only used if EnableCycleDetection is true
	ShowCycleDetails bool

	// CycleDetectionDepth sets maximum depth for cycle detection (default: 10)
	// Higher values detect deeper cycles but use more memory
	CycleDetectionDepth int

	// Verbose enables detailed logging (default: false)
	Verbose bool

	// DryRun performs a trial run that writes nothing to disk (default: false).
	// The engine returns SQL strings either way; DryRun additionally disables
	// the file-writing side features (pg_dump backups and rollback plans).
	DryRun bool
}

Config contains configuration for the squashing engine.

func DefaultConfig

func DefaultConfig() *Config

DefaultConfig returns a configuration with sensible defaults.

type DataOpCounts

type DataOpCounts struct {
	Total   int `json:"total"`
	Inserts int `json:"inserts"`
	Updates int `json:"updates"`
	Deletes int `json:"deletes"`
}

DataOpCounts contains detailed counts of data operations

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine wraps the internal squashing engine and provides access to advanced features like statistics, progress tracking, and resource management.

Use NewEngine to create an engine instance, then call Squash methods. Always call Close() when done to free resources.

Example:

config := engine.DefaultConfig()
config.ProgressCallback = func(processed, total int64, phase string) {
    fmt.Printf("Progress: %d/%d - %s\n", processed, total, phase)
}

eng, err := engine.NewEngine(config)
if err != nil {
    log.Fatal(err)
}
defer eng.Close()

result, err := eng.SquashDirectory("./migrations")
if err != nil {
    log.Fatal(err)
}

stats := eng.GetStats()
fmt.Printf("Processed %d objects in %s\n", stats.ObjectsTracked, stats.ProcessingTime)

func NewEngine

func NewEngine(config *Config) (*Engine, error)

NewEngine creates a new engine instance with the provided configuration.

The engine must be closed when done using Close() to free resources.

Example:

eng, err := engine.NewEngine(config)
if err != nil {
    return err
}
defer eng.Close()

func (*Engine) Close

func (e *Engine) Close() error

Close releases resources used by the engine.

Always call this when done with the engine, preferably using defer. A failure to release resources (e.g. closing the production database connection) is returned, never swallowed.

Example:

eng, err := engine.NewEngine(config)
if err != nil {
    return err
}
defer eng.Close()

func (*Engine) GetAuthCompatibilitySQL

func (e *Engine) GetAuthCompatibilitySQL() string

GetAuthCompatibilitySQL returns authentication compatibility SQL for Docker validation.

This SQL is used to set up auth_users compatibility when using Supabase-style authentication with vanilla PostgreSQL.

Example:

authSQL := eng.GetAuthCompatibilitySQL()
if authSQL != "" {
    fmt.Println("Auth compatibility required")
}

func (*Engine) GetExtensions

func (e *Engine) GetExtensions() []string

GetExtensions returns the list of PostgreSQL extensions detected in migrations.

Example:

exts := eng.GetExtensions()
if len(exts) > 0 {
    fmt.Printf("Required extensions: %v\n", exts)
}

func (*Engine) GetMemoryStats

func (e *Engine) GetMemoryStats() MemoryStats

GetMemoryStats returns memory usage statistics (streaming mode only).

Example:

if config.EnableStreaming {
    memStats := eng.GetMemoryStats()
    fmt.Printf("Memory: %dMB / %dMB\n",
        memStats.CurrentUsageMB, memStats.LimitMB)
}

func (*Engine) GetResult

func (e *Engine) GetResult() *SquashResult

Returns nil if no squashing operation has been performed yet.

func (*Engine) GetSafetyLevel

func (e *Engine) GetSafetyLevel() SafetyLevel

GetSafetyLevel returns the configured safety level.

Example:

level := eng.GetSafetyLevel()
fmt.Printf("Using safety level: %s\n", level)

func (*Engine) GetStats

func (e *Engine) GetStats() SquashStats

GetStats returns current squashing statistics.

This can be called during or after squashing operations.

Example:

stats := eng.GetStats()
fmt.Printf("Phase: %s\n", stats.Phase)
fmt.Printf("Progress: %d/%d\n", stats.MigrationsProcessed, stats.TotalMigrations)

func (*Engine) GetWarnings

func (e *Engine) GetWarnings() []string

GetWarnings returns accumulated warnings from the squashing operation.

Example:

warnings := eng.GetWarnings()
for _, w := range warnings {
    log.Printf("Warning: %s\n", w)
}

func (*Engine) SquashDirectory

func (e *Engine) SquashDirectory(directory string) (*SquashResult, error)

SquashDirectory consolidates all migration files in a directory.

Example:

eng, err := engine.NewEngine(config)
defer eng.Close()

result, err := eng.SquashDirectory("./migrations")

func (*Engine) SquashFiles

func (e *Engine) SquashFiles(migrations map[int]string) (*SquashResult, error)

SquashFiles consolidates specific migration files.

The migrations map can contain either file paths OR SQL content. If a value looks like a file path (.sql extension), it will be read. Otherwise, it's treated as SQL content directly.

Example with file paths:

migrations := map[int]string{
    1: "001_create_users.sql",
    2: "002_create_posts.sql",
}
result, err := eng.SquashFiles(migrations)

Example with SQL content:

migrations := map[int]string{
    1: "CREATE TABLE users (id INT);",
    2: "CREATE TABLE posts (id INT);",
}
result, err := eng.SquashFiles(migrations)

type MemoryStats

type MemoryStats struct {
	// CurrentUsageMB is the current memory usage in megabytes
	CurrentUsageMB int64 `json:"current_usage_mb"`

	// PeakUsageMB is the peak memory usage in megabytes
	PeakUsageMB int64 `json:"peak_usage_mb"`

	// LimitMB is the configured memory limit in megabytes
	LimitMB int64 `json:"limit_mb"`

	// UsagePercent is the percentage of limit currently used
	UsagePercent float64 `json:"usage_percent"`
}

MemoryStats provides memory usage statistics for streaming operations

type OutputFormat

type OutputFormat string

OutputFormat determines how squashed SQL is formatted.

const (
	// FormatSingle outputs all migrations in a single file
	FormatSingle OutputFormat = "single"

	// FormatSplit outputs migrations in multiple files by category
	FormatSplit OutputFormat = "split"
)

type ProgressCallback

type ProgressCallback func(processed, total int64, phase string)

ProgressCallback is called during squashing to report progress.

Parameters:

  • processed: Number of items processed so far
  • total: Total number of items to process
  • phase: Current processing phase (see ProgressPhase constants)

Example:

config.ProgressCallback = func(processed, total int64, phase string) {
    if total > 0 {
        percent := float64(processed) / float64(total) * 100
        fmt.Printf("[%s] Progress: %.1f%% (%d/%d)\n", phase, percent, processed, total)
    }
}

type ProgressPhase

type ProgressPhase string

ProgressPhase represents a processing phase with progress information

const (
	PhaseInitializing  ProgressPhase = "Initializing"
	PhaseParsing       ProgressPhase = "Parsing"
	PhaseTracking      ProgressPhase = "Tracking"
	PhaseAnalyzing     ProgressPhase = "Analyzing"
	PhaseConsolidating ProgressPhase = "Consolidating"
	PhaseGenerating    ProgressPhase = "Generating SQL"
	PhaseValidating    ProgressPhase = "Validating"
	PhaseComplete      ProgressPhase = "Complete"
)

type ProvenanceInfo

type ProvenanceInfo struct {
	// Version of the squasher used
	Version string `json:"version"`

	// SafetyLevel applied during squashing
	SafetyLevel string `json:"safety_level"`

	// InputFiles lists source migration files
	InputFiles []string `json:"input_files,omitempty"`

	// OutputFiles lists generated files
	OutputFiles []string `json:"output_files,omitempty"`

	// ContentHash for integrity verification
	ContentHash string `json:"content_hash,omitempty"`
}

ProvenanceInfo contains metadata about the squashing operation

type Redundancy

type Redundancy struct {
	// Type is the redundancy type (e.g., "duplicate_create", "overridden_alter")
	Type string

	// ObjectName is the affected database object
	ObjectName string

	// Description explains the redundancy
	Description string

	// Severity indicates importance (low, medium, high)
	Severity string
}

Redundancy represents a redundant database operation.

type SafetyLevel

type SafetyLevel string

SafetyLevel determines how aggressively migrations are consolidated.

const (
	// Conservative applies minimal consolidation, preserving most operations
	Conservative SafetyLevel = "conservative"

	// Standard balances consolidation and safety (recommended)
	Standard SafetyLevel = "standard"

	// Aggressive maximizes consolidation, suitable for development
	Aggressive SafetyLevel = "aggressive"

	// Paranoid preserves everything, minimal changes
	Paranoid SafetyLevel = "paranoid"
)

func ParseSafetyLevel

func ParseSafetyLevel(s string) (SafetyLevel, error)

ParseSafetyLevel parses a safety level string case-insensitively (surrounding whitespace is ignored) and rejects unknown values. Consumers (API, Studio, CLI wrappers) should use this instead of reimplementing safety-level parsing with divergent semantics.

func ValidSafetyLevels

func ValidSafetyLevels() []SafetyLevel

ValidSafetyLevels returns every valid safety level, delegating to the internal validation as the single source of truth.

type SquashResult

type SquashResult struct {
	// BaselineSQL contains the consolidated DDL SQL.
	BaselineSQL string `json:"baseline_sql"`

	// DataOperationsSQL contains data operations SQL (INSERT, UPDATE, DELETE)
	DataOperationsSQL string `json:"data_operations_sql,omitempty"`

	// Warnings contains any warnings generated during squashing
	Warnings []string `json:"warnings,omitempty"`

	// FilesProcessed is the number of migration files processed
	FilesProcessed int `json:"files_processed"`

	// ObjectsConsolidated is the number of database objects consolidated
	ObjectsConsolidated int `json:"objects_consolidated"`

	// ProcessingTime is the duration of the squashing operation
	ProcessingTime string `json:"processing_time"`

	// Extensions contains detected/required PostgreSQL extensions
	Extensions []string `json:"extensions,omitempty"`

	// AuthCompatibilitySQL contains SQL to mock authentication services for validation
	AuthCompatibilitySQL string `json:"auth_compatibility_sql,omitempty"`

	// ProvenanceInfo contains metadata about the squashing operation
	ProvenanceInfo *ProvenanceInfo `json:"provenance_info,omitempty"`
}

SquashResult contains the results of a squashing operation.

func SquashDirectory

func SquashDirectory(directory string, config *Config) (*SquashResult, error)

SquashDirectory consolidates all migration files in a directory.

If config is nil, DefaultConfig() is used.

Example:

result, err := engine.SquashDirectory("./migrations", nil)
if err != nil {
    return err
}
fmt.Println(result.BaselineSQL)

func SquashFiles

func SquashFiles(migrations map[int]string, config *Config) (*SquashResult, error)

SquashFiles consolidates specific migration files.

The migrations map can contain either: - File paths: values ending in .sql will be read from disk - SQL content: multi-line SQL strings will be used directly

Example with file paths:

migrations := map[int]string{
    1: "001_create_users.sql",
    2: "002_create_posts.sql",
}
result, err := engine.SquashFiles(migrations, nil)

Example with SQL content:

migrations := map[int]string{
    1: "CREATE TABLE users (id INT);\nCREATE TABLE posts (id INT);",
}
result, err := engine.SquashFiles(migrations, nil)

type SquashStats

type SquashStats struct {
	// Phase is the current processing phase
	// Examples: "Initializing", "Parsing", "Analyzing", "Consolidating", "Generating SQL"
	Phase string `json:"current_phase"`

	// MigrationsProcessed is the number of migrations processed so far
	MigrationsProcessed int64 `json:"migrations_processed"`

	// TotalMigrations is the total number of migrations to process
	TotalMigrations int64 `json:"total_migrations"`

	// ObjectsTracked is the total number of database objects being tracked
	ObjectsTracked int64 `json:"objects_tracked"`

	// ConsolidationsApplied is the number of consolidations successfully applied
	ConsolidationsApplied int64 `json:"consolidations_applied"`

	// ProcessingTime is the total time spent processing
	ProcessingTime time.Duration `json:"processing_time"`

	// PeakMemoryUsage is the peak memory usage in bytes (streaming mode only)
	PeakMemoryUsage int64 `json:"peak_memory_usage"`

	// ThroughputMPS is the throughput in migrations per second
	ThroughputMPS float64 `json:"throughput_migrations_per_sec"`
}

SquashStats tracks detailed squashing statistics for monitoring and observability.

This type provides real-time metrics during squashing operations, useful for progress tracking, performance monitoring, and debugging.

Example:

eng, err := engine.NewEngine(config)
defer eng.Close()

// Get stats during or after operation
stats := eng.GetStats()
fmt.Printf("Phase: %s\n", stats.Phase)
fmt.Printf("Progress: %d/%d migrations\n", stats.MigrationsProcessed, stats.TotalMigrations)
fmt.Printf("Throughput: %.2f migrations/sec\n", stats.ThroughputMPS)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL