engine

package
v0.0.36 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 5, 2026 License: GPL-3.0 Imports: 23 Imported by: 0

Documentation

Overview

Package engine provides the core concurrency and execution logic for Corral.

Index

Examples

Constants

View Source
const LegacyStateFileName = ".corral-state.json"

LegacyStateFileName is the pre-v0.0.20 sidecar location, in the working tree. Still read so existing clones keep their smart-sync state across the upgrade, and removed on the next successful write.

View Source
const StateFileName = "corral-state.json"

StateFileName is the basename of the per-clone sidecar file Corral writes *inside* the repository's Git directory. It records the most recent push timestamp the engine has observed for the upstream so subsequent runs can skip a no-op `git pull`.

It lives in the Git directory rather than the working tree deliberately. Before v0.0.20 the sidecar sat at <repo>/.corral-state.json, which left an untracked file in every managed clone: `git status` reported every repo as dirty, and because both `corralctl prune` and the MCP delete tool refuse to touch a repo with local changes, neither could ever run. Anything under the Git directory is invisible to `git status` by construction, so no ignore-file management is required.

Variables

This section is empty.

Functions

func FetchRepos added in v0.0.30

func FetchRepos(ctx context.Context, owner string, opts github.FetchOptions, forgeName, forgeURL string) ([]github.Repo, error)

FetchRepos lists an owner's repositories from the named forge.

Exported because prune needs it. prune used to call the GitHub client directly, so `--forge codeberg` was accepted and then ignored: it compared Codeberg clones against a GitHub listing. Everything that decides what is missing upstream has to come through one path, or the flag lies.

func ParseLayoutTemplate added in v0.0.20

func ParseLayoutTemplate(layout string) (*template.Template, error)

ParseLayoutTemplate validates a --layout template without rendering it, so the CLI can reject a malformed template during flag validation instead of after a full paginated GitHub fetch has already happened.

func Run

func Run(ctx context.Context, opts RunOptions)

Run executes the core Corral workflow and terminates the process when the run fails. It is the entry point for the CLI; embedders should call RunE, which returns the error instead of exiting.

Example

ExampleRun demonstrates a complete run: clone and organise every public Go and Rust repository for an owner over SSH using shallow single-branch clones, syncing existing checkouts, reporting orphaned local repositories, and emitting a machine-readable JSON summary.

package main

import (
	"context"

	"github.com/sebastienrousseau/corral/internal/engine"
	"github.com/sebastienrousseau/corral/internal/git"
	"github.com/sebastienrousseau/corral/internal/github"
)

func main() {
	ctx := context.Background()
	engine.Run(ctx, engine.RunOptions{
		Owner:       "sebastienrousseau",
		BaseDir:     "/home/me/Code",
		Concurrency: 8,
		DoSync:      true,
		Orphans:     true,
		Protocol:    "ssh",
		Output:      engine.OutputJSON,
		Fetch: github.FetchOptions{
			Limit:            500,
			Visibility:       "public",
			IncludeLanguages: []string{"go", "rust"},
			AuthMode:         github.AuthModeAuto,
		},
		Clone: git.CloneOptions{
			SingleBranch: true,
			Depth:        1,
		},
		Sync: engine.SyncOptions{
			Force: false, // set to true to bypass cache and force pull
		},
	})
}
Example (DryRun)

ExampleRun_dryRun demonstrates previewing actions without making any changes, streaming one NDJSON record per repository.

package main

import (
	"context"

	"github.com/sebastienrousseau/corral/internal/engine"
	"github.com/sebastienrousseau/corral/internal/github"
)

func main() {
	ctx := context.Background()
	engine.Run(ctx, engine.RunOptions{
		Owner:       "sebastienrousseau",
		BaseDir:     "/home/me/Code",
		Concurrency: 1,
		DryRun:      true,
		Protocol:    "https",
		Output:      engine.OutputNDJSON,
		Fetch:       github.FetchOptions{Limit: 1000},
	})
}

func RunE added in v0.0.26

func RunE(ctx context.Context, opts RunOptions) error

RunE executes the core Corral workflow, orchestrating GitHub API fetches, legacy layout migrations, concurrent Git operations, and orphaned repository detection. It returns nil on a clean run, and otherwise an error — an *ExitError when the failure maps to a specific exit status.

Types

type ExitError added in v0.0.26

type ExitError struct {
	// Code is the process exit status the CLI should terminate with.
	Code int
	// Silent is true when the failure has already been reported to the user
	// and printing the error again would duplicate it.
	Silent bool
	// Err is the underlying cause, or nil when Silent.
	Err error
}

ExitError carries the process exit status a failed run should produce.

The engine is a library: it reports failure by returning an error rather than by calling os.Exit, so an embedder can decide what a failure means. Run is the thin wrapper that turns this back into a process exit for the CLI. Silent is set when the condition was already reported to the user — per-repository failures are printed as results, and cancellation is announced by emitCancellation — so the caller must not print it a second time.

func (*ExitError) Error added in v0.0.26

func (e *ExitError) Error() string

Error implements the error interface.

func (*ExitError) Unwrap added in v0.0.26

func (e *ExitError) Unwrap() error

Unwrap returns the underlying cause so errors.Is and errors.As reach it.

type Job

type Job struct {
	// Repo is the GitHub repository to be processed.
	Repo github.Repo
	// Target is the destination directory for the repository under the new layout.
	Target string
	// Legacy is the directory where the repository may exist under the old layout.
	Legacy string
	// Existing is an identity-matched clone at a previous layout path.
	Existing string
}

Job encapsulates a repository to be processed along with its target directories.

type OutputFormat added in v0.0.3

type OutputFormat string

OutputFormat controls how operation results are emitted.

const (
	// OutputText emits human-readable, line-oriented progress output.
	OutputText OutputFormat = "text"
	// OutputJSON emits a single aggregated JSON document at the end of the run.
	OutputJSON OutputFormat = "json"
	// OutputNDJSON emits one JSON object per result as a stream of newline-delimited records.
	OutputNDJSON OutputFormat = "ndjson"
)

type RepoResult added in v0.0.3

type RepoResult struct {
	// RepoName is the name of the processed repository.
	RepoName string `json:"repo"`
	// Action is the outcome verb, such as CLONE, SYNC, SKIP, ERROR, or DRY-RUN.
	Action string `json:"action"`
	// Message is a human-readable description of the outcome.
	Message string `json:"message"`
	// Target is the destination directory for the repository.
	Target string `json:"target"`
	// Visibility is the repository visibility (e.g. Public or Private).
	Visibility string `json:"visibility"`
	// Language is the normalized primary language directory name.
	Language string `json:"language"`
	// DryRun indicates whether the run was performed in dry-run mode.
	DryRun bool `json:"dry_run"`
	// Protocol is the clone transport used (https or ssh).
	Protocol string `json:"protocol"`
	// ClonedURL is the URL used for cloning, if a clone was attempted.
	ClonedURL string `json:"clone_url,omitempty"`
	// SyncAttempt indicates whether a sync (pull) was attempted.
	SyncAttempt bool `json:"sync_attempt"`
	// Moved indicates that an identity-matched clone was relocated.
	Moved bool `json:"moved,omitempty"`
}

RepoResult represents the final status of processing a repository.

type RunOptions added in v0.0.3

type RunOptions struct {
	// Owner is the GitHub user or organization whose repositories are processed.
	Owner string
	// BaseDir is the root directory under which repositories are laid out.
	BaseDir string
	// Concurrency is the number of worker goroutines processing repositories; must be >= 1.
	Concurrency int
	// DryRun, when true, reports intended actions without performing clone or pull operations.
	DryRun bool
	// Orphans, when true, enables detection of local repositories no longer present upstream.
	Orphans bool
	// Protocol selects the clone transport and must be either "https" or "ssh".
	Protocol string
	// DoSync, when true, pulls updates into existing repositories.
	DoSync bool
	// Output selects the result emission format (text, json, or ndjson).
	Output OutputFormat
	// Interactive, when true, displays an interactive selector before processing.
	Interactive bool

	// Fetch holds the options passed to the repository listing call.
	Fetch github.FetchOptions
	// Forge names the hosting service to list from: github, gitlab, gitea,
	// forgejo or codeberg. Empty means github, which is what corral did
	// before other forges were supported — a default that changes under
	// people is worse than a narrow one.
	Forge string
	// ForgeURL is a self-hosted instance's base address. Required for
	// gitea and forgejo, which have no single public instance; optional
	// for the others, which do.
	ForgeURL string
	// Clone holds the options passed to each Git clone operation.
	Clone git.CloneOptions
	// Sync controls when an already-cloned repository is actually pulled.
	Sync SyncOptions
	// Layout specifies the templated path structure for repositories. Custom
	// layouts may use Collection and Bucket in addition to the legacy fields.
	Layout string
	// FinderTags enables managed macOS Finder metadata on repository folders.
	FinderTags bool
	// Version is the build version of Corral.
	Version string
}

RunOptions contains all execution controls for a run.

type Summary added in v0.0.3

type Summary struct {
	// Total is the number of repositories scheduled for processing.
	Total int `json:"total"`
	// Cloned is the number of repositories successfully cloned.
	Cloned int `json:"cloned"`
	// Synced is the number of repositories successfully synced.
	Synced int `json:"synced"`
	// Moved is the number of repositories relocated to their desired layout.
	Moved int `json:"moved"`
	// Skipped is the number of repositories skipped.
	Skipped int `json:"skipped"`
	// Failed is the number of repositories that failed to process.
	Failed int `json:"failed"`
	// Canceled is true when the run was interrupted by ctx cancellation
	// (typically SIGINT/SIGTERM). The result set in that case is partial:
	// a scripted consumer reading json/ndjson output should treat it as
	// "not all repositories were processed" rather than a clean run.
	Canceled bool `json:"canceled,omitempty"`
}

Summary tracks aggregate run outcomes.

type SyncOptions added in v0.0.7

type SyncOptions struct {
	// Force, when true, runs `git pull` even when the cached state shows
	// the upstream pushed_at is unchanged.
	Force bool
	// IgnoreSubmoduleFailures, when true with Clone.RecurseSubmodules, allows
	// the parent repository to update even when a submodule sync fails (e.g.
	// the submodule repo was deleted upstream or its access revoked). The
	// failure is logged as a WARN but not propagated.
	IgnoreSubmoduleFailures bool
}

SyncOptions configures the engine's per-repo sync decision. Kept separate from git.CloneOptions because forcing a sync is a corral-level policy choice, not a clone-time git flag.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL