reposync

package
v0.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 19, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

@index Repository/branch admission, ref normalization, and namespace policy for repository sync.

@index Clone URL resolution for webhook-triggered repository sync.

@index Deduplicating work queue for webhook-driven repository sync.

@index Repository checkout-to-graph synchronization application workflow.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrSyncQueueFull         = errors.New("sync queue full")
	ErrSyncQueueShuttingDown = errors.New("sync queue shutting down")
)

Functions

func AllowRuleOwners 

func AllowRuleOwners(rules []RepoRule) []string

AllowRuleOwners returns the positive repository owner prefixes present in allow rules. @intent let server startup warn when repo-name namespace extraction is used with multi-owner webhook allowlists. @domainRule deny rules are ignored because they only narrow the positive admission surface.

func AllowRulesSpanMultipleOwners 

func AllowRulesSpanMultipleOwners(rules []RepoRule) (bool, []string)

AllowRulesSpanMultipleOwners reports whether webhook rules admit repositories under more than one owner prefix. @intent identify configurations that can collide under the current repo-name namespace strategy.

func ExtractNamespace 

func ExtractNamespace(repoFullName string) string

ExtractNamespace derives the existing repo-name namespace strategy. @intent preserve repository-backed namespace compatibility while removing owner segments.

func IsNonRetryable 

func IsNonRetryable(err error) bool

IsNonRetryable reports whether a sync error should bypass queue retries. @intent let retry logic stop early when a failure is known to be permanent for the current payload. @param err is the sync failure returned by the queue handler. @return returns true when err unwraps to nonRetryableError.

func NonRetryable 

func NonRetryable(err error) error

@intent wrap permanent sync failures so queue retry logic can short-circuit them.

func NormalizeBranchRef 

func NormalizeBranchRef(ref string) (string, bool)

NormalizeBranchRef extracts a branch name only from refs/heads references. @intent reject tags and other Git refs before repository sync admission.

func ResolveCloneURL 

func ResolveCloneURL(repoFullName, payloadCloneURL string, baseURLs []string, allowPayload bool) (string, error)

ResolveCloneURL selects a safe clone target from configured base URLs or the payload. @intent keep repository sync deterministic and prevent untrusted webhook payloads from choosing arbitrary clone endpoints unless explicitly allowed.

func ValidateRepoNameNamespaceRules 

func ValidateRepoNameNamespaceRules(rules []RepoRule) error

ValidateRepoNameNamespaceRules rejects admission surfaces that can map distinct repositories to one repo-name namespace. @intent fail webhook startup before equal repo names from different owners can share checkout and graph state. @domainRule repo-name namespaces are safe only when all positive allow rules are constrained to one non-wildcard owner.

Types

type BuildScope 

type BuildScope struct {
	IncludePaths    []string
	ExcludePatterns []string
}

BuildScope captures the repository-local source selection policy for one webhook sync. @intent carry include and exclude configuration together so every webhook update uses one coherent build scope.

type BuildScopeLoader 

type BuildScopeLoader interface {
	Load(repoDir string) (BuildScope, error)
}

BuildScopeLoader loads repository-local graph source selection configuration. @intent separate repository config file parsing from repository sync orchestration.

type CacheInvalidator 

type CacheInvalidator interface{ Invalidate() }

CacheInvalidator drops query results only after a successful graph update. @intent keep derived query cache invalidation after successful repository graph commit.

type CacheInvalidatorFunc 

type CacheInvalidatorFunc func()

CacheInvalidatorFunc adapts a runtime closure to the application port. @intent adapt runtime cache invalidation closures to the app-owned port.

func (CacheInvalidatorFunc) Invalidate 

func (f CacheInvalidatorFunc) Invalidate()

Invalidate adapts a closure to the cache invalidation port. @intent invoke the configured cache invalidation only when one exists.

type Checkout 

type Checkout interface {
	Sync(ctx context.Context, request CheckoutRequest) (repoDir string, err error)
}

Checkout synchronizes a remote branch into its isolated repository directory. @intent isolate checkout locking and Git implementation from sync ordering policy.

type CheckoutRequest 

type CheckoutRequest struct{ RepoFullName, CloneURL, Namespace, Branch string }

CheckoutRequest identifies one admitted remote branch and its local namespace. @intent carry only trusted admission output into the checkout adapter.

type GraphRequest 

type GraphRequest struct {
	RepoDir, Namespace                string
	IncludePaths                      []string
	ExcludePatterns                   []string
	MaxFileBytes, MaxTotalParsedBytes int64
	FailOnUnreadable                  bool
}

GraphRequest carries the exact webhook update contract to the ingest adapter. @intent preserve namespace, source scope, replace limits, and readability policy across the app boundary.

type GraphUpdater 

type GraphUpdater interface {
	Update(ctx context.Context, request GraphRequest) (UpdateStats, error)
}

GraphUpdater replaces one repository namespace from its synchronized checkout. @intent adapt repository sync to the ingest application without importing workflow types.

type Observability 

type Observability interface {
	Start(ctx context.Context, operation, repo, branch string) (context.Context, func())
	LogArgs(ctx context.Context) []any
}

Observability wraps queue process/attempt lifecycles without coupling app policy to tracing infrastructure. @intent allow tracing adapters to enrich contexts and logs while queue behavior remains infrastructure-free.

type QueueConfig 

type QueueConfig struct {
	RetryConfig
	ShutdownTimeout time.Duration
	MaxTrackedRepos int
	Observability   Observability
}

@intent configure queue retry policy and memory bounds when constructing a SyncQueue.

type RepoFilter 

type RepoFilter struct {
	// contains filtered or unexported fields
}

RepoFilter holds the ordered rule list used to decide repo and branch admission. @intent provide a single matcher whose result depends on rule declaration order, where later matching rules override earlier ones.

func NewRepoFilter 

func NewRepoFilter(patterns []string) *RepoFilter

NewRepoFilter expands simple string patterns into webhook repo rules. @intent keep CLI-style allowlist configuration compatible with the richer rule matcher. @param patterns is the ordered allow/deny rule list in compact string form. @ensures returns a filter whose rule order matches the caller input order.

func NewRepoFilterFromRules 

func NewRepoFilterFromRules(rules []RepoRule) *RepoFilter

NewRepoFilterFromRules compiles repository and branch rules into a matcher. @intent centralize Atlantis-style repo filtering so webhook dispatch can make one consistent allow decision. @param rules is the ordered repo rule list whose later matches override earlier ones. @ensures returns a filter that preserves the source rule order for evaluation.

func (*RepoFilter) IsAllowed 

func (f *RepoFilter) IsAllowed(repoFullName string) bool

IsAllowed reports whether a repository matches the configured allow and deny rules. @intent let callers gate repository-level sync before looking at branch-specific restrictions. @domainRule rules are evaluated in declaration order and the last matching rule wins; a later allow rule overrides an earlier deny match and vice versa. @domainRule a repository with no matching rule is denied (default-deny).

func (*RepoFilter) IsAllowedBranch 

func (f *RepoFilter) IsAllowedBranch(repoFullName, branch string) bool

IsAllowedBranch reports whether a repository and branch pair should be processed. @intent combine repo allowlist and branch policy so webhook handlers can skip unsupported pushes cheaply. @domainRule with no rules configured the result is always false (default-deny). @domainRule rules are evaluated in declaration order; each later matching rule replaces the prior decision, so a negate rule clears any previous allow and a subsequent allow rule re-enables the repo. @domainRule when a non-negate rule matches but specifies no branches, the built-in defaults ("main", "master") are used.

func (*RepoFilter) IsAllowedRef 

func (f *RepoFilter) IsAllowedRef(repoFullName, ref string) bool

IsAllowedRef resolves a git ref to a branch and applies branch allowlist rules. @intent reject non-branch webhook refs before they can enter the sync pipeline. @return returns false for non-branch refs and otherwise defers to branch policy evaluation.

type RepoRule 

type RepoRule struct {
	Pattern  string
	Branches []string
}

RepoRule is the user-facing config form pairing a repo pattern with optional branch globs. @intent expose a stable shape for CLI/YAML config without leaking the internal compiled rule layout.

func ParseRepoRule 

func ParseRepoRule(s string) RepoRule

ParseRepoRule decodes a repo rule string with optional comma-separated branch patterns. @intent preserve compact CLI config while still supporting per-repository branch restrictions. @param s follows pattern or pattern:branch1,branch2 syntax. @return returns a RepoRule with branch restrictions only when the colon form is present.

type RepoStats 

type RepoStats struct {
	Repo            string    `json:"repo"`
	Branch          string    `json:"branch"`
	Queued          bool      `json:"queued"`
	Processing      bool      `json:"processing"`
	EnqueuedAt      time.Time `json:"enqueued_at,omitempty"`
	ProcessingAt    time.Time `json:"processing_at,omitempty"`
	LastSuccessTime time.Time `json:"last_success_time,omitempty"`
	LastErrorTime   time.Time `json:"last_error_time,omitempty"`
	LastError       string    `json:"last_error,omitempty"`
}

@intent expose per-repository queue state so operators can inspect backlog and failure hotspots.

type RetryConfig 

type RetryConfig struct {
	// MaxAttempts is the total number of attempts (1 = no retry). Default: 3.
	MaxAttempts int
	// BaseDelay is the initial backoff duration. Default: 1s.
	BaseDelay time.Duration
	// MaxDelay caps the backoff duration. Default: 30s.
	MaxDelay time.Duration
}

RetryConfig configures exponential backoff retry for sync handlers. @intent tune retry backoff so transient webhook sync failures can recover without overwhelming the remote.

type Service 

type Service struct {
	Checkout            Checkout
	BuildScope          BuildScopeLoader
	Graph               GraphUpdater
	Cache               CacheInvalidator
	Observability       Observability
	AttemptTimeout      time.Duration
	MaxFileBytes        int64
	MaxTotalParsedBytes int64
	FailOnUnreadable    bool
	Logger              *slog.Logger
}

@intent coordinate admitted repository checkout, config loading, graph replacement, and cache invalidation.

func (*Service) Sync 

func (s *Service) Sync(ctx context.Context, repoFullName, cloneURL, branch string) error

Sync checks out one admitted repository, updates its graph namespace, and invalidates query cache on success. @intent make repository synchronization ordering reusable outside HTTP server composition.

type SyncHandlerFunc 

type SyncHandlerFunc func(ctx context.Context, repoFullName, cloneURL, branch string) error

SyncHandlerFunc processes one deduplicated repository sync payload. @intent provide the queue-to-service invocation contract without transport ownership.

type SyncQueue 

type SyncQueue struct {
	// contains filtered or unexported fields
}

@intent coordinate deduplicated per-repository sync execution across a worker pool.

func NewSyncQueue 

func NewSyncQueue(workers int, handler SyncHandlerFunc) *SyncQueue

NewSyncQueue creates a queue with background workers and default lifecycle settings. @intent provide the smallest constructor for production webhook dispatch. @param workers is the number of background workers to start. @param handler processes one deduplicated repository sync payload. @sideEffect starts background worker goroutines immediately.

func NewSyncQueueWithConfig 

func NewSyncQueueWithConfig(ctx context.Context, workers int, handler SyncHandlerFunc, cfg QueueConfig) *SyncQueue

NewSyncQueueWithConfig creates a deduplicating repo work queue and starts its workers. @intent coalesce bursty webhook pushes per repository while still allowing different repos to sync concurrently. @param cfg controls retry behavior and how many repositories may be tracked at once. @sideEffect starts worker goroutines and allocates queue state maps. @ensures non-positive MaxAttempts becomes 1, non-positive ShutdownTimeout becomes 30s, and non-positive MaxTrackedRepos becomes 1024.

func NewSyncQueueWithContext 

func NewSyncQueueWithContext(ctx context.Context, workers int, handler SyncHandlerFunc) *SyncQueue

NewSyncQueueWithContext binds queue workers to a parent lifecycle context. @intent allow server shutdown to cancel retries and worker waits cleanly. @param ctx controls worker lifetime and retry cancellation. @sideEffect starts background worker goroutines immediately.

func NewSyncQueueWithOptions 

func NewSyncQueueWithOptions(ctx context.Context, workers int, handler SyncHandlerFunc, retry RetryConfig) *SyncQueue

NewSyncQueueWithOptions applies retry tuning while keeping default queue limits. @intent expose backoff customization without forcing every caller to build a full queue config. @param retry overrides exponential backoff behavior for sync retries. @ensures queue memory bounds still use default MaxTrackedRepos when not specified.

func (*SyncQueue) Add 

func (q *SyncQueue) Add(ctx context.Context, repoFullName, cloneURL, branch string) error

Add records the latest payload for a repository and enqueues it if needed. @intent collapse repeated push events into one queued sync while preserving the newest branch and clone data. @mutates SyncQueue.queue, SyncQueue.dirty, SyncQueue.payloads. @param repoFullName identifies the deduplication key for queueing. @return returns ErrSyncQueueFull when a new repository would exceed the bounded tracked-repo set and ErrSyncQueueShuttingDown after graceful shutdown begins. @domainRule repeated events for the same repository replace payload data instead of enqueueing duplicate work.

func (*SyncQueue) Shutdown 

func (q *SyncQueue) Shutdown()

Shutdown stops accepting new work and waits for workers to finish or time out. @intent give the server a bounded, graceful shutdown path for in-flight webhook sync. @sideEffect flips queue shutdown state, wakes waiting workers, and may record a timeout failure.

func (*SyncQueue) Stats 

func (q *SyncQueue) Stats() SyncQueueStats

Stats snapshots queue health and recent repository activity for operators. @intent expose enough queue state to diagnose backlog, failures, and hot repositories. @return returns a point-in-time snapshot without mutating queue execution state.

type SyncQueueStats 

type SyncQueueStats struct {
	Queued              int           `json:"queued"`
	Dirty               int           `json:"dirty"`
	Processing          int           `json:"processing"`
	TrackedRepos        int           `json:"tracked_repos"`
	MaxTrackedRepos     int           `json:"max_tracked_repos"`
	QueueFullTotal      int64         `json:"queue_full_total"`
	FailureTotal        int64         `json:"failure_total"`
	LastError           string        `json:"last_error,omitempty"`
	LastErrorTime       time.Time     `json:"last_error_time,omitempty"`
	OldestQueuedAge     time.Duration `json:"oldest_queued_age"`
	OldestProcessingAge time.Duration `json:"oldest_processing_age"`
	LastSuccessTime     time.Time     `json:"last_success_time,omitempty"`
	Shutdown            bool          `json:"shutdown"`
	RecentRepos         []RepoStats   `json:"recent_repos,omitempty"`
}

@intent summarize queue-wide health and recent repository activity for observability.

type UpdateStats 

type UpdateStats struct{ Added, Modified, Skipped, Deleted int }

UpdateStats is the sync result needed for repository operation logging. @intent report only update counts needed by repository sync observability.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.