Documentation
¶
Overview ¶
Phase-5 eventing/sync persistence contract (docs/reference/webhooks.md webhooks, docs/reference/http-api.md git-credentials + upstream sync): webhook registrations, git credentials, and repo upstream links live in the meta database next to the repo rows they reference.
INTEGRATION CONTRACT: both backends (pg_eventing.go / sqlite_eventing.go) implement EventingStore, and Store embeds it once the implementations land. Consumers never use these methods directly — the events and sync packages consume them through thin adapters (events.MetaWebhookStore, sync.MetaStores) that translate types and sentinel errors.
Package meta declares the metadata-store contract for organizations, org public keys, and repositories (docs/explanation/domain-model.md domain model, docs/reference/http-api.md HTTP API).
FROZEN CONTRACT: parallel phase-1 slices implement Store against Postgres (schema: orgs / org_public_keys / repos with UNIQUE(org_id, url) spanning soft-deleted rows) and build HTTP handlers on top of these exact signatures. Do not change them without coordinating across slices.
purge.go implements the repository-purge metadata seams (issue #52). The purge orchestrator (jobs.RepoPurger) permanently reclaims soft-deleted repositories; the soft-deleted repos row is the durable marker that drives crash resumption, so PurgeRepoMeta — which stamps purged_at — is the purge's COMPLETION commit point and runs strictly last.
The repos row itself is KEPT forever as a name tombstone: JWT repo claims are URL-scoped and may outlive any retention window (git tokens last up to a year), so releasing the burned (org, url) name would let a stale token authorize against a successor repository created at that URL (Codex security round on #105). Purging therefore reclaims storage and eventing rows but never un-burns the name, matching soft-delete's name semantics; restore after purge is impossible by construction.
Like the ref-store purge seams (ref/purge.go), these live on the concrete backends rather than meta.Store, whose test mirrors must not grow with purge-only surface.
purge_sqlite.go mirrors purge.go's repository-purge metadata seams for the SQLite backend (issue #52) — see purge.go for the rationale.
S3 implementation of Backend for the s3wal zero-infrastructure mode (issue #133, docs/plans/s3wal-plan.md §5-Q4): ALL metadata tables live in ONE conditional JSON object, `_wal/meta.json`, advanced by CAS through the same s3wal.ConditionalStore substrate the ref manifests use (If-None-Match:* create, If-Match advance, seq + fresh writer nonce per write for ambiguous-PUT recovery — plan §2.3, simplified for a single low-write-rate object).
Semantics mirror sqlite.go/sqlite_eventing.go 1:1 wherever a SQL twin exists: same sentinels, soft delete with the 404/409 distinction, (org_id, url) uniqueness spanning soft-deleted rows, the same opaque cursors, and Go-side unixnano timestamps (WithNowFunc injects the clock, so claim-lease expiry is a pure function of it — the DB-clock stand-in, exactly like meta.SQLite). Cross-writer arbitration never reads clocks: it is the object CAS itself, so the §2.4 manifest-clock machinery is deliberately absent here.
Concurrency: one object ⇒ one process-local mutex serializes every operation (the meta twin of ref.S3WAL.lockBase); cross-process writers serialize on the object CAS. Reads validate a process-local (etag, body) cache via conditional GET (304) and decode fresh per use, so no decoded state is ever aliased across operations.
S3WAL metadata format conversion (ADR-51): the only code that reads and writes a `_wal/meta.json` version other than s3walMetaVersion. The caller fences every other writer first (`openstorage s3wal-meta convert` holds the instance lease while serve is stopped). Correctness against a writer the fence missed still holds: every write is an If-Match CAS on the version this conversion read, so a stale writer loses with 412 and then fails to decode the converted document.
Phase-5 eventing/sync persistence on the s3wal meta document. Semantics mirror sqlite_eventing.go 1:1 (contract in eventing.go) with two documented deviations:
No legacy claim adoption: an empty-holder or LegacyClaimHolder claim can only be written by a pre-lease SQL replica, and no pre-s3walmeta binary exists (the same rationale as the ref store's absent legacy reservation spellings) — such states are asserted impossible instead of adopted.
Sync-claim write-through (plan §2.8): with a SyncClaimSink wired (WithSyncClaimSink), TryBeginSync commits the meta claim FIRST and then installs the holder token in the base repo group's ref manifest so a fenced CASRef checks the holder in the same object it CASes. Meta-first ordering means only the meta claim's winner ever touches the fence. If the manifest write fails, the meta claim is rolled back best-effort (restoring a consumed sync_pending marker) and the claim is refused — a failed rollback merely leaves the claim to expire, and the missing manifest token keeps every fenced write failing CLOSED either way. After the fence lands, the claim is revalidated against the meta store: a steal that raced the fence install (our manifest write landing after the thief's) is detected, the fence is cleared owner-guarded, and the claim is refused — so a stolen holder can never write under a late-reinstalled fence. Stealing an expired claim additionally clears the victim's manifest token BEFORE the steal CAS commits, so no thief crash can leave a superseded (expired-but-alive) victim still fenced in — the SQL twins exclude the victim atomically because their fence reads the stolen row itself. FinishSync clears the manifest token (owner-guarded, so a thief's token is never touched) BEFORE finalizing the meta row; a failure there aborts the finish (retryable; expiry recovers the lease). DeleteRepoUpstream revokes the token through a resumable three-phase detach (durable Detaching mark → owner-guarded clear → row delete; ordering proof on the method) so a detached upstream's in-flight worker can never publish after the detach committed and no crash leaves the cleanup holder only in process memory. The push lane has no write-through: the outbound claim protects upstream refs, which no local CASRef fences.
s3walmeta_purge.go mirrors purge.go's repository-purge metadata seams for the s3wal backend (issue #52) — see purge.go for the rationale. The repos row is kept forever as a name tombstone (burned-name invariant); purge deletes the eventing rows and stamps purged_at in ONE document CAS, so the guard read and the deletes are same-atom by construction (the atomicity the SQL twins buy with transactions).
Index ¶
- Constants
- Variables
- func CheckFormat(ctx context.Context, b Backend) error
- func NewEventingID(prefix string) string
- func PGRequestSync(ctx context.Context, tx pgx.Tx, repoID string) error
- type Backend
- type CorruptDocumentError
- type CreateOutcome
- type CreateState
- type EventingStore
- type FormatVersionError
- type GitCredential
- type LaneOutcome
- type ListParams
- type ListResult
- type LocalClaim
- type LocalScheduleStore
- type LocalWork
- type Option
- type Org
- type PG
- func (s *PG) AddSigningKey(ctx context.Context, org, keyText string) (SigningKey, error)
- func (s *PG) AdoptUpstreamDefaultBranch(ctx context.Context, repoID, branch string) (bool, error)
- func (s *PG) Close()
- func (s *PG) CompleteRepoCreate(ctx context.Context, org, repoID string) (CreateOutcome, error)
- func (s *PG) CreateGitCredential(ctx context.Context, cred GitCredential) (GitCredential, error)
- func (s *PG) CreateLinkedRepo(ctx context.Context, org, url, defaultBranch string, up RepoUpstream, ...) (Repo, error)
- func (s *PG) CreateRepo(ctx context.Context, org, url, defaultBranch string) (Repo, error)
- func (s *PG) CreateRepoWithState(ctx context.Context, org, url, defaultBranch string, state CreateState) (Repo, error)
- func (s *PG) CreateWebhook(ctx context.Context, org, url, secret string) (Webhook, error)
- func (s *PG) DeleteGitCredential(ctx context.Context, id string) error
- func (s *PG) DeleteRepoUpstream(ctx context.Context, repoID string) error
- func (s *PG) DeleteWebhook(ctx context.Context, org, id string) error
- func (s *PG) FinishPush(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
- func (s *PG) FinishSync(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
- func (s *PG) GetGitCredential(ctx context.Context, id string) (GitCredential, error)
- func (s *PG) GetRepoByID(ctx context.Context, org, repoID string) (Repo, error)
- func (s *PG) GetRepoByURL(ctx context.Context, org, url string) (Repo, error)
- func (s *PG) GetRepoUpstream(ctx context.Context, repoID string) (RepoUpstream, error)
- func (s *PG) GetWebhook(ctx context.Context, id string) (Webhook, error)
- func (s *PG) GitCredentialForRepo(ctx context.Context, repoID string) (GitCredential, error)
- func (s *PG) GitCredentialMeta(ctx context.Context, id string) (GitCredential, error)
- func (s *PG) InitSchema(ctx context.Context) error
- func (s *PG) ListRepos(ctx context.Context, org string, params ListParams) (ListResult, error)
- func (s *PG) ListWebhooks(ctx context.Context, org string) ([]Webhook, error)
- func (s *PG) OrgNames(ctx context.Context) ([]string, error)
- func (s *PG) PublicKeysForOrg(ctx context.Context, name string) ([]string, error)
- func (s *PG) PurgeRepoMeta(ctx context.Context, repoID string) error
- func (s *PG) PurgeableRepos(ctx context.Context, deletedBefore time.Time, after PurgeCandidate, limit int) ([]PurgeCandidate, error)
- func (s *PG) RemoveSigningKey(ctx context.Context, org string, id int64) error
- func (s *PG) RepoPurgeState(ctx context.Context, repoID string) (state PurgeState, found bool, err error)
- func (s *PG) RequestPush(ctx context.Context, repoID string, force bool) error
- func (s *PG) RequestSync(ctx context.Context, repoID string) error
- func (s *PG) RestorePush(ctx context.Context, repoID, holder string, force bool) (RepoUpstream, error)
- func (s *PG) RestoreSync(ctx context.Context, repoID, holder string) (RepoUpstream, error)
- func (s *PG) SetRepoUpstream(ctx context.Context, up RepoUpstream) (RepoUpstream, error)
- func (s *PG) SigningKeysForOrg(ctx context.Context, org string) ([]SigningKey, error)
- func (s *PG) SoftDeleteRepo(ctx context.Context, org, url string) (Repo, error)
- func (s *PG) TryBeginPush(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, bool, string, error)
- func (s *PG) TryBeginSync(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, string, error)
- func (s *PG) UpdateGitCredential(ctx context.Context, id, username, password string) (GitCredential, error)
- func (s *PG) UpdateRepoDefaultBranch(ctx context.Context, org, url, oldBranch, newBranch string) (bool, error)
- func (s *PG) UpsertOrgWithKeys(ctx context.Context, name string, pemPublicKeys []string) (Org, error)
- type PurgeCandidate
- type PurgeState
- type PushOutcome
- type QuarantinedError
- type Repo
- type RepoUpstream
- type RunRestoreStore
- type S3WAL
- func (s *S3WAL) AddSigningKey(ctx context.Context, org, keyText string) (SigningKey, error)
- func (s *S3WAL) AdoptUpstreamDefaultBranch(ctx context.Context, repoID, branch string) (bool, error)
- func (s *S3WAL) Close()
- func (s *S3WAL) CompleteRepoCreate(ctx context.Context, org, repoID string) (CreateOutcome, error)
- func (s *S3WAL) CreateGitCredential(ctx context.Context, cred GitCredential) (GitCredential, error)
- func (s *S3WAL) CreateLinkedRepo(ctx context.Context, org, url, defaultBranch string, up RepoUpstream, ...) (Repo, error)
- func (s *S3WAL) CreateRepo(ctx context.Context, org, url, defaultBranch string) (Repo, error)
- func (s *S3WAL) CreateRepoWithState(ctx context.Context, org, url, defaultBranch string, state CreateState) (Repo, error)
- func (s *S3WAL) CreateWebhook(ctx context.Context, org, url, secret string) (Webhook, error)
- func (s *S3WAL) DeleteGitCredential(ctx context.Context, id string) error
- func (s *S3WAL) DeleteRepoUpstream(ctx context.Context, repoID string) error
- func (s *S3WAL) DeleteWebhook(ctx context.Context, org, id string) error
- func (s *S3WAL) FinishPush(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
- func (s *S3WAL) FinishSync(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
- func (s *S3WAL) GetGitCredential(ctx context.Context, id string) (GitCredential, error)
- func (s *S3WAL) GetRepoByID(ctx context.Context, org, repoID string) (Repo, error)
- func (s *S3WAL) GetRepoByURL(ctx context.Context, org, url string) (Repo, error)
- func (s *S3WAL) GetRepoUpstream(ctx context.Context, repoID string) (RepoUpstream, error)
- func (s *S3WAL) GetWebhook(ctx context.Context, id string) (Webhook, error)
- func (s *S3WAL) GitCredentialForRepo(ctx context.Context, repoID string) (GitCredential, error)
- func (s *S3WAL) GitCredentialMeta(ctx context.Context, id string) (GitCredential, error)
- func (s *S3WAL) InitSchema(ctx context.Context) error
- func (s *S3WAL) ListRepos(ctx context.Context, org string, params ListParams) (ListResult, error)
- func (s *S3WAL) ListWebhooks(ctx context.Context, org string) ([]Webhook, error)
- func (s *S3WAL) OrgNames(ctx context.Context) ([]string, error)
- func (s *S3WAL) PublicKeysForOrg(ctx context.Context, name string) ([]string, error)
- func (s *S3WAL) PurgeRepoMeta(ctx context.Context, repoID string) error
- func (s *S3WAL) PurgeableRepos(ctx context.Context, deletedBefore time.Time, after PurgeCandidate, limit int) ([]PurgeCandidate, error)
- func (s *S3WAL) RemoveSigningKey(ctx context.Context, org string, id int64) error
- func (s *S3WAL) RepoPurgeState(ctx context.Context, repoID string) (PurgeState, bool, error)
- func (s *S3WAL) RequestPush(ctx context.Context, repoID string, force bool) error
- func (s *S3WAL) RequestSync(ctx context.Context, repoID string) error
- func (s *S3WAL) SetRepoUpstream(ctx context.Context, up RepoUpstream) (RepoUpstream, error)
- func (s *S3WAL) SigningKeysForOrg(ctx context.Context, org string) ([]SigningKey, error)
- func (s *S3WAL) SoftDeleteRepo(ctx context.Context, org, url string) (Repo, error)
- func (s *S3WAL) TryBeginPush(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, bool, string, error)
- func (s *S3WAL) TryBeginSync(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, string, error)
- func (s *S3WAL) UpdateGitCredential(ctx context.Context, id, username, password string) (GitCredential, error)
- func (s *S3WAL) UpdateRepoDefaultBranch(ctx context.Context, org, url, oldBranch, newBranch string) (bool, error)
- func (s *S3WAL) UpsertOrgWithKeys(ctx context.Context, name string, pemPublicKeys []string) (Org, error)
- type S3WALConvertOptions
- type S3WALMetaConversion
- type S3WALMetaCounts
- type S3WALPendingCreate
- type S3WALPendingCreatesError
- type SQLite
- func (s *SQLite) AddSigningKey(ctx context.Context, org, keyText string) (SigningKey, error)
- func (s *SQLite) AdoptUpstreamDefaultBranch(ctx context.Context, repoID, branch string) (bool, error)
- func (s *SQLite) ClaimDuePush(ctx context.Context, repoID, holder string, lease time.Duration) (LocalClaim, error)
- func (s *SQLite) ClaimDueSync(ctx context.Context, repoID, holder string, lease time.Duration) (LocalClaim, error)
- func (s *SQLite) Close()
- func (s *SQLite) CompleteRepoCreate(ctx context.Context, org, repoID string) (CreateOutcome, error)
- func (s *SQLite) CompleteSyncRepoCreate(ctx context.Context, org, repoID string) (CreateOutcome, error)
- func (s *SQLite) CreateGitCredential(ctx context.Context, cred GitCredential) (GitCredential, error)
- func (s *SQLite) CreateLinkedRepo(ctx context.Context, org, url, defaultBranch string, up RepoUpstream, ...) (Repo, error)
- func (s *SQLite) CreateRepo(ctx context.Context, org, url, defaultBranch string) (Repo, error)
- func (s *SQLite) CreateRepoWithState(ctx context.Context, org, url, defaultBranch string, state CreateState) (Repo, error)
- func (s *SQLite) CreateWebhook(ctx context.Context, org, url, secret string) (Webhook, error)
- func (s *SQLite) DeleteGitCredential(ctx context.Context, id string) error
- func (s *SQLite) DeleteRepoUpstream(ctx context.Context, repoID string) error
- func (s *SQLite) DeleteWebhook(ctx context.Context, org, id string) error
- func (s *SQLite) FinishPush(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
- func (s *SQLite) FinishPushOutcome(ctx context.Context, repoID, holder string, o PushOutcome) (RepoUpstream, error)
- func (s *SQLite) FinishSync(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
- func (s *SQLite) FinishSyncOutcome(ctx context.Context, repoID, holder string, o SyncOutcome) (RepoUpstream, error)
- func (s *SQLite) GetGitCredential(ctx context.Context, id string) (GitCredential, error)
- func (s *SQLite) GetRepoByID(ctx context.Context, org, repoID string) (Repo, error)
- func (s *SQLite) GetRepoByURL(ctx context.Context, org, url string) (Repo, error)
- func (s *SQLite) GetRepoUpstream(ctx context.Context, repoID string) (RepoUpstream, error)
- func (s *SQLite) GetWebhook(ctx context.Context, id string) (Webhook, error)
- func (s *SQLite) GitCredentialForRepo(ctx context.Context, repoID string) (GitCredential, error)
- func (s *SQLite) GitCredentialMeta(ctx context.Context, id string) (GitCredential, error)
- func (s *SQLite) InitSchema(ctx context.Context) error
- func (s *SQLite) ListLocalWork(ctx context.Context, org, after string, limit int) ([]LocalWork, error)
- func (s *SQLite) ListRepos(ctx context.Context, org string, params ListParams) (ListResult, error)
- func (s *SQLite) ListWebhooks(ctx context.Context, org string) ([]Webhook, error)
- func (s *SQLite) OrgNames(ctx context.Context) ([]string, error)
- func (s *SQLite) PublicKeysForOrg(ctx context.Context, name string) ([]string, error)
- func (s *SQLite) PurgeRepoMeta(ctx context.Context, repoID string) error
- func (s *SQLite) PurgeableRepos(ctx context.Context, deletedBefore time.Time, after PurgeCandidate, limit int) ([]PurgeCandidate, error)
- func (s *SQLite) RemoveSigningKey(ctx context.Context, org string, id int64) error
- func (s *SQLite) RepoPurgeState(ctx context.Context, repoID string) (state PurgeState, found bool, err error)
- func (s *SQLite) RequestPush(ctx context.Context, repoID string, force bool) error
- func (s *SQLite) RequestPushNow(ctx context.Context, repoID string, force bool) error
- func (s *SQLite) RequestSync(ctx context.Context, repoID string) error
- func (s *SQLite) SetRepoUpstream(ctx context.Context, up RepoUpstream) (RepoUpstream, error)
- func (s *SQLite) SigningKeysForOrg(ctx context.Context, org string) ([]SigningKey, error)
- func (s *SQLite) SoftDeleteRepo(ctx context.Context, org, url string) (Repo, error)
- func (s *SQLite) TryBeginPush(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, bool, string, error)
- func (s *SQLite) TryBeginSync(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, string, error)
- func (s *SQLite) UpdateGitCredential(ctx context.Context, id, username, password string) (GitCredential, error)
- func (s *SQLite) UpdateRepoDefaultBranch(ctx context.Context, org, url, oldBranch, newBranch string) (bool, error)
- func (s *SQLite) UpsertOrgWithKeys(ctx context.Context, name string, pemPublicKeys []string) (Org, error)
- type SigningKey
- type StaleDocumentError
- type Store
- type SyncClaimSink
- type SyncOutcome
- type Webhook
Constants ¶
const EnvSyncStealLegacy = "OPENSTORAGE_SYNC_STEAL_LEGACY"
EnvSyncStealLegacy enables stealing adopted legacy claims whose grace has lapsed. Any value strconv.ParseBool accepts as true enables it; malformed values stay disabled (the safe direction).
const LegacyClaimHolder = "!legacy-grace"
LegacyClaimHolder marks a sync claim taken by a pre-lease replica and adopted into the lease regime by a new one (TryBeginSync mixed-fleet staging). The '!' prefix cannot occur in real holder tokens, whose process tag is hostname-pid (RFC 1123 hostnames have no '!').
Adopted legacy claims are NEVER stolen by default: a pre-lease binary has neither the ref-update fence nor a holder-guarded FinishSync, so no finite grace can make stealing from a possibly-alive one safe. Stealing them requires EnvSyncStealLegacy — the operator's assertion that no pre-lease replica remains (same fleet-prerequisite pattern as delete-mode sweeping), after which legacy rows can only be pre-upgrade wedges. Claims from lease-aware replicas heal automatically regardless.
const OmittedDefaultBranch = "main"
OmittedDefaultBranch is the default branch a sync create stores when the request names none. A GitHub sync later adopts the upstream HEAD in its place (RepoUpstream.DefaultBranchOmitted, issue #599).
const ViewCheckLogEnv = "OPENSTORAGE_TEST_VIEWCHECK_LOG"
ViewCheckLogEnv names a file that builds with the openstorage_viewcheck tag append each failed view check to before they panic. Serve can lose the panic (the API turns a handler panic into a 500), so the e2e harness in internal/githttp sets it and fails the run on any line (issue #503).
Variables ¶
var ( // ErrNotFound: unknown org, or repo absent/soft-deleted → 404. ErrNotFound = errors.New("not found") // ErrAlreadyExists: CreateRepo unique violation on (org, url), including // against a soft-deleted row → 409. ErrAlreadyExists = errors.New("repository already exists") // ErrAlreadyDeleted: SoftDeleteRepo on an already soft-deleted repo → 409. ErrAlreadyDeleted = errors.New("repository already deleted") // ErrClaimLost: FinishSync by a holder whose expired sync claim was // stolen (issue #86) — the late finisher must abandon silently. ErrClaimLost = errors.New("sync claim lost") // ErrInvalidCursor: ListRepos received an undecodable cursor → 400. ErrInvalidCursor = errors.New("invalid cursor") )
Sentinel errors implementations must return (possibly wrapped; callers match with errors.Is) so HTTP handlers can map storage outcomes onto the docs/reference/http-api.md status contract.
var ErrRepoNotDeleted = errors.New("repository is not soft-deleted")
ErrRepoNotDeleted guards PurgeRepoMeta: the repos row still exists and is live, so purging its metadata would orphan a serving repository. Callers verified soft-deletion before starting — hitting this means the check was skipped or the repo was re-created mid-purge.
var ErrReservedSecret = fmt.Errorf("secret values must not start with the reserved prefix %q", secrets.ReservedPrefix)
ErrReservedSecret: CreateWebhook/CreateGitCredential/UpdateGitCredential rejected a secret value starting with the reserved envelope prefix ("enc:") → 400. Enforced regardless of whether encryption is enabled — see the internal/secrets package comment for the collision argument.
var ErrValueNotUTF8 = errors.New("s3walmeta: string value is not valid UTF-8")
ErrValueNotUTF8 rejects caller-supplied strings carrying invalid UTF-8 before any document mutation (the ref.ErrRefNameNotUTF8 rationale): encoding/json silently rewrites invalid bytes to U+FFFD, so two distinct raw values (or map keys — org names key the orgs table) would collapse onto one stored spelling, silently corrupting lookups and uniqueness. The SQL twins diverge here exactly like the ref stores: PostgreSQL TEXT rejects invalid UTF-8 at the wire, SQLite stores raw bytes verbatim. Reads need no guard — no document can contain such a value, so raw-byte lookups are clean misses.
Functions ¶
func CheckFormat ¶ added in v0.19.0
CheckFormat reads b's stored metadata document once and returns its error: a *FormatVersionError when the format is wrong, a *CorruptDocumentError when the document is damaged, a store error when the read failed, nil otherwise (including a missing document, the empty state). Backends without a format version (PostgreSQL, SQLite) return nil without I/O. meta.Open never reads the document, so this is the cheap way to learn the format before a write that must not proceed on the wrong one.
func NewEventingID ¶
NewEventingID returns "<prefix>_<16 hex>" — the shared ID shape for webhooks (wh_) and git credentials (ggc_, openapi example ggc_1234567890abcdef).
func PGRequestSync ¶ added in v0.18.0
PGRequestSync is RequestSync inside the caller's transaction. It locks the upstream row, so a concurrent detach either waits for the commit or makes this fail with ErrNotFound (issue #388: a sync create must not complete for a detached repo).
Types ¶
type Backend ¶
type Backend interface {
Store
// InitSchema creates the metadata tables (idempotent, safe under
// concurrent startups).
InitSchema(ctx context.Context) error
Close()
}
Backend extends Store with the lifecycle the wire seam needs: serve runs InitSchema once at startup and Close on shutdown. Consumers keep coding against Store; hooks never run DDL.
func Open ¶
Open dispatches on the URL scheme to the matching Backend:
postgres:// or postgresql:// → OpenPG sqlite:///abs/path/db.sqlite → pure-Go SQLite store s3wal://bucket?endpoint=… → single-object CAS store over conditional S3
Open never runs InitSchema: hooks reconstruct stores from env URLs and must not run DDL — the serve/wire seam owns schema init (for s3wal that means the conditional-write probe). Options (WithSecretsCodec) apply to any backend.
type CorruptDocumentError ¶ added in v0.19.0
type CorruptDocumentError struct {
Err error // what is wrong with the document
}
CorruptDocumentError reports a stored metadata document that does not decode or fails a structural guard. Like FormatVersionError it is not transient: the store answered, every metadata read fails on the same bytes, and only a restore helps. The hook refuses on it (issue #466).
func (*CorruptDocumentError) Error ¶ added in v0.19.0
func (e *CorruptDocumentError) Error() string
func (*CorruptDocumentError) Unwrap ¶ added in v0.19.0
func (e *CorruptDocumentError) Unwrap() error
type CreateOutcome ¶
type CreateOutcome int
CreateOutcome is the result of CompleteRepoCreate.
const ( CreateCompleted CreateOutcome = iota + 1 // this call finished the create CreateAlreadyComplete // an earlier call finished it CreateDeleted // the repo was deleted first; the create must not report success )
func PGCompleteRepoCreate ¶
func PGCompleteRepoCreate(ctx context.Context, tx pgx.Tx, org, repoID string, schedule func(context.Context) error) (CreateOutcome, error)
PGCompleteRepoCreate is CompleteRepoCreate inside the caller's transaction on the metadata database. It locks the repo row; only when the row is live and pending does it run schedule(ctx) (nil = nothing) and mark the row complete, so both commit together or not at all (issue #388: River inserts a sync create's first-sync job in tx). A complete or deleted row runs nothing.
type CreateState ¶
type CreateState int
CreateState records whether a repo's create finished (issue #388). A repeat POST /repos may answer 200 only for CreateStateComplete.
const ( // CreateStateLegacy: the create is not tracked. The row predates create // tracking (on S3-WAL, a metadata v1 row), or an untracked path created // it (CreateRepo). A crashed fork can look like this, so the row is // never treated as complete. CreateStateLegacy CreateState = 0 // CreateStatePending: inserted, and the create has not finished. CreateStatePending CreateState = 1 // CreateStateComplete: the create finished. CreateStateComplete CreateState = 2 )
type EventingStore ¶
type EventingStore interface {
// CreateWebhook inserts a webhook with a server-generated ID.
// ErrNotFound when the org is unknown.
CreateWebhook(ctx context.Context, org, url, secret string) (Webhook, error)
// ListWebhooks returns the org's webhooks in registration order.
// ErrNotFound when the org is unknown. ListWebhooks is an enumeration
// surface: a row whose stored secret cannot be decrypted (issue #54)
// is returned with Secret == "" rather than failing the listing, so
// event fan-out and the list API (which never expose secrets) keep
// working and the bad row's ID stays discoverable for deletion.
// Consumers that need the secret must use GetWebhook, which fails
// closed per webhook.
ListWebhooks(ctx context.Context, org string) ([]Webhook, error)
// GetWebhook returns the webhook with the given ID regardless of org
// (delivery jobs carry only the ID); ErrNotFound when absent.
GetWebhook(ctx context.Context, id string) (Webhook, error)
// DeleteWebhook removes the org's webhook; ErrNotFound when the org is
// unknown or the ID does not belong to it.
DeleteWebhook(ctx context.Context, org, id string) error
// CreateGitCredential stores cred (ID/CreatedAt assigned by the store;
// caller-set ID/CreatedAt are ignored). ErrAlreadyExists when the repo
// already has a credential (docs/reference/http-api.md: 409).
CreateGitCredential(ctx context.Context, cred GitCredential) (GitCredential, error)
// GetGitCredential returns the credential by ID; ErrNotFound when absent.
GetGitCredential(ctx context.Context, id string) (GitCredential, error)
// GitCredentialMeta returns the credential row with Password always ""
// — the password column is never read, so this works even when the
// stored envelope can no longer be decrypted (issue #54). Ownership
// prechecks for update/delete use it so operators can always replace or
// remove a credential whose key was retired prematurely or whose row is
// corrupt. ErrNotFound when absent.
GitCredentialMeta(ctx context.Context, id string) (GitCredential, error)
// UpdateGitCredential replaces username/password; ErrNotFound when absent.
UpdateGitCredential(ctx context.Context, id, username, password string) (GitCredential, error)
// DeleteGitCredential removes the credential; ErrNotFound when absent.
DeleteGitCredential(ctx context.Context, id string) error
// GitCredentialForRepo returns the repo's credential; ErrNotFound when
// none is stored (public upstream).
GitCredentialForRepo(ctx context.Context, repoID string) (GitCredential, error)
// SetRepoUpstream inserts the upstream row (CreatedAt assigned by the
// store; RunCount/SuccessCount/Syncing start zeroed). ErrAlreadyExists
// when the repo already has one (base_repo is set exactly once).
SetRepoUpstream(ctx context.Context, up RepoUpstream) (RepoUpstream, error)
// CreateLinkedRepo is CreateRepo plus SetRepoUpstream (and
// CreateGitCredential when cred is non-nil) in one atomic write, so no
// reader sees the repo before its upstream link (issue #387: a
// concurrent create of the same name compares upstreams). The store
// assigns every ID; caller-set RepoIDs are ignored. ErrAlreadyExists
// when (org, url) is taken, and then nothing is written. The repo is
// CreateStatePending until CompleteRepoCreate.
CreateLinkedRepo(ctx context.Context, org, url, defaultBranch string, up RepoUpstream, cred *GitCredential) (Repo, error)
// AdoptUpstreamDefaultBranch sets the live repo's default branch to
// branch and clears DefaultBranchOmitted in one write, only while the
// flag is set (issue #599). adopted is false, and nothing changes, when
// the flag is clear or the repo is detached. When the branch is no
// longer OmittedDefaultBranch (a PATCH by an older binary, which does
// not clear the flag, during a rolling upgrade), it only clears the
// flag: the explicit choice wins. ErrNotFound when the repo is deleted.
AdoptUpstreamDefaultBranch(ctx context.Context, repoID, branch string) (adopted bool, err error)
// GetRepoUpstream returns the row; ErrNotFound when detached/never linked.
GetRepoUpstream(ctx context.Context, repoID string) (RepoUpstream, error)
// DeleteRepoUpstream detaches the repo; ErrNotFound when already detached.
DeleteRepoUpstream(ctx context.Context, repoID string) error
// RequestSync marks a pending sync request (sync_pending = true).
// Callers that promise a sync (POST /repos/pull-upstream 202) set it
// BEFORE enqueueing: if the enqueue coalesces with a run that already
// claimed (whose upstream fetch may predate this request), the flag
// survives to that run's FinishSync and the finisher schedules a
// follow-up run. ErrNotFound when detached.
RequestSync(ctx context.Context, repoID string) error
// TryBeginSync atomically claims a sync run (syncing false -> true)
// under a lease and returns the row as of the claim. holder is the
// caller's per-run claim token; the claim records it with an expiry
// of now+lease. A row whose current claim has EXPIRED is stolen
// atomically (read-check-write inside one transaction/row lock — no
// read-then-write window): stolenFrom then carries the previous
// holder for observability, and the previous holder's late
// FinishSync fails with ErrClaimLost.
//
// Mixed-fleet staging (rolling upgrades from pre-lease versions): a
// claim with an EMPTY holder was taken by an old replica that writes
// no lease — it may be a live run, not a crashed one. Such a claim
// is never stolen on sight; it is ADOPTED instead (holder set to
// LegacyClaimHolder, expiry now+lease, claimed=false) and becomes
// stealable only when that grace passes without the old replica's
// FinishSync clearing it. The grace equals the caller's lease, which
// strictly dominates any live pre-lease run window.
//
// The claim consumes sync_pending: every request marked before the
// claim is observed by the claimed run's upstream fetch, which
// happens after it. claimed=false (row returned) when an unexpired
// run is in flight or a legacy claim was just adopted; ErrNotFound
// when detached.
TryBeginSync(ctx context.Context, repoID, holder string, lease time.Duration) (up RepoUpstream, claimed bool, stolenFrom string, err error)
// FinishSync completes the run claimed by holder: increments RunCount
// (and SuccessCount when success), clears Syncing and the claim
// lease, returns the updated row. SyncPending is NOT touched: a set
// flag in the returned row tells the finisher to schedule a follow-up
// run (the follow-up's claim consumes it). ErrNotFound when the repo
// was detached mid-run or no run is in flight; ErrClaimLost when the
// row is claimed by a DIFFERENT holder — the caller's expired claim
// was stolen, and the late finisher must abandon without touching the
// thief's claim or the counters.
FinishSync(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
// RequestPush raises PushPending to at least 1 (2 when force) —
// levels only coalesce upward (issue #90). Same mark-before-enqueue
// protocol as RequestSync. ErrNotFound when detached.
RequestPush(ctx context.Context, repoID string, force bool) error
// TryBeginPush is TryBeginSync on the push_* lane: it claims the
// outbound run under a lease and consumes PushPending whole,
// reporting whether the run must force. There is no legacy adoption —
// an expired push lease is stolen outright, and the steal INHERITS
// the dead claim's persisted force mandate (PushClaimForce), so a
// crash between claim and finish cannot silently downgrade an
// accepted forced request.
TryBeginPush(ctx context.Context, repoID, holder string, lease time.Duration) (up RepoUpstream, claimed bool, force bool, stolenFrom string, err error)
// FinishPush is FinishSync on the push_* lane.
FinishPush(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
}
EventingStore is the phase-5 persistence surface. Org-scoped methods key by org name (JWT `iss`) and return ErrNotFound for unknown orgs, matching the Store conventions above.
type FormatVersionError ¶ added in v0.19.0
type FormatVersionError struct {
Got int // the stored document's version
Want []int // the versions this read accepts
}
FormatVersionError reports a readable metadata document whose format version this read does not accept (ADR-51). Unlike a store error it is not transient: the store answered, and only a format conversion (or the matching binary) helps. Callers that fail open on store trouble, such as the hook's eventing (issue #445), refuse on this error instead.
func (*FormatVersionError) Error ¶ added in v0.19.0
func (e *FormatVersionError) Error() string
type GitCredential ¶
type GitCredential struct {
ID string // server-generated "ggc_<16 hex>"
RepoID string
Username string // optional; token-only providers omit it
Password string
CreatedAt time.Time // UTC
}
GitCredential is one stored HTTPS credential for a repo's git upstream (docs/reference/http-api.md "Git credentials"). Password is plaintext at this interface and gets the same optional at-rest envelope encryption as Webhook.Secret; it must never be logged.
type LaneOutcome ¶
type LaneOutcome int
LaneOutcome is how a run ends.
const ( // OutcomeDone ends the run: success, permanent failure, or retry // budget spent. Resets attempt and due time. OutcomeDone LaneOutcome = iota + 1 // OutcomeRetry (push only) ends a transiently failed run: re-marks the // run's level, advances the attempt, and sets due = now + Delay, unless // RequestPushNow recorded an explicit request during the run: that // request stays due now. OutcomeRetry // OutcomeRestore ends a run that never started its transfer (failed // publication wait) or that the process's shutdown cancelled (issue // #342): re-marks, keeps the attempt, due = now + Delay (an explicit // request made during the run stays due now), and counts no run. OutcomeRestore )
type ListParams ¶
type ListParams struct {
Q string // case-insensitive substring filter on URL, pre-trimmed; "" = no filter
Cursor string // opaque cursor from a previous ListResult; "" = first page
Limit int // page size, already clamped/validated by the caller
}
ListParams are the GET /repos query parameters after handler-side validation: handlers reject bad `limit` with 400 and pass the resolved value here (default 20, max 100); the opaque cursor is validated by the store (ErrInvalidCursor).
type ListResult ¶
type ListResult struct {
Repos []Repo // never nil; empty slice when no matches
NextCursor string // opaque; "" when exhausted
HasMore bool
}
ListResult is one page of repos ordered by (created_at DESC, repo_id DESC).
type LocalClaim ¶
type LocalClaim struct {
Up RepoUpstream // the row after the claim, or the current row when refused
Claimed bool
Busy bool // refused: a live claim (or legacy grace) holds the lane
Force bool // push only: the run must force-push
Attempt int // push only: the run's retry position (0 = first attempt)
StolenFrom string // holder of the expired claim this claim replaced
}
LocalClaim is the result of a conditional claim. Exactly one of Claimed, Busy, or neither (no durable work due) holds.
type LocalScheduleStore ¶
type LocalScheduleStore interface {
// CompleteSyncRepoCreate marks a pending sync create complete and
// records its first-sync request in one write (issue #388).
CompleteSyncRepoCreate(ctx context.Context, org, repoID string) (CreateOutcome, error)
// RequestPushNow is RequestPush for an explicit caller request (API
// push-upstream): it also makes the pending work due now, keeping the
// retry position. One write, so no request/expedite gap exists.
RequestPushNow(ctx context.Context, repoID string, force bool) error
// ClaimDuePush claims the push lane only when it has durable work: a
// pending level whose due time has passed and no live claim, or an
// expired claim (stolen with its force mandate). Otherwise it writes
// nothing and reports Busy (a live claim holds the lane) or neither.
ClaimDuePush(ctx context.Context, repoID, holder string, lease time.Duration) (LocalClaim, error)
// ClaimDueSync is ClaimDuePush for the pull lane (sync_pending,
// sync_due); legacy-claim adoption is unchanged from TryBeginSync.
ClaimDueSync(ctx context.Context, repoID, holder string, lease time.Duration) (LocalClaim, error)
// FinishPushOutcome ends holder's push run and schedules what follows
// in the same write. ErrClaimLost if another holder owns the lane;
// ErrNotFound if detached or no run is in flight.
FinishPushOutcome(ctx context.Context, repoID, holder string, o PushOutcome) (RepoUpstream, error)
// FinishSyncOutcome is FinishPushOutcome for the pull lane.
FinishSyncOutcome(ctx context.Context, repoID, holder string, o SyncOutcome) (RepoUpstream, error)
// ListLocalWork returns at most limit lanes of org's live repos whose
// claim guard holds now, ordered by RepoID strictly after after. An
// empty page ends a traversal; an unknown org has no work.
ListLocalWork(ctx context.Context, org, after string, limit int) ([]LocalWork, error)
}
LocalScheduleStore is the local serve process's durable schedule. Get it with LocalSchedule: *SQLite implements it, and S3WAL provides it through an adapter (ADR-51).
func LocalSchedule ¶ added in v0.19.0
func LocalSchedule(ms any) (LocalScheduleStore, bool)
LocalSchedule returns ms's durable local schedule (ADR-50). SQLite is one; S3WAL provides it through s3walLocalSchedule (ADR-51). ok is false for PostgreSQL, which schedules through River.
type Option ¶
type Option func(*storeOptions)
Option configures Open/OpenPG/OpenSQLite.
func WithNowFunc ¶
WithNowFunc pins the store's Go-side clock (nil = time.Now, the default). SQLite only: every SQLite timestamp — including the sync-claim lease expiry comparison in TryBeginSync — is written and evaluated Go-side, so injecting a logical clock makes lease expiry a pure function of that clock (the DST harness, internal/simtest, depends on this for deterministic traces). PG ignores it deliberately: its lease comparisons evaluate on the DATABASE clock so node clock skew never participates (issue #86).
func WithSecretsCodec ¶
WithSecretsCodec enables envelope encryption of the secret columns (nil codec = plaintext, the default).
func WithSyncClaimSink ¶
func WithSyncClaimSink(sink SyncClaimSink) Option
WithSyncClaimSink wires the §2.8 sync-claim write-through sink (ref.S3WAL in zero-infra mode). Only the s3wal meta backend consumes it; the SQL backends ignore it because their CASRef fence reads the shared repo_upstreams row directly. Nil restores today's behavior.
type Org ¶
Org is one registered organization (single-org self-host mode seeds exactly one at startup).
type PG ¶
type PG struct {
// contains filtered or unexported fields
}
PG implements Backend on Postgres.
func (*PG) AddSigningKey ¶
AddSigningKey registers a commit-signing key for the org and returns the stored row. Format validation (SSH line / armored PGP block) is the caller's responsibility per the Store contract; the text is stored verbatim.
func (*PG) AdoptUpstreamDefaultBranch ¶ added in v0.19.0
AdoptUpstreamDefaultBranch (meta.EventingStore contract). The repos row is locked first, as UpdateRepoDefaultBranch does, so a concurrent PATCH either clears the flag before this reads it or waits for this commit.
func (*PG) CompleteRepoCreate ¶
CompleteRepoCreate moves a pending repo to complete (meta.Store contract).
func (*PG) CreateGitCredential ¶
func (s *PG) CreateGitCredential(ctx context.Context, cred GitCredential) (GitCredential, error)
CreateGitCredential stores cred (ID/CreatedAt assigned by the store). ErrAlreadyExists when the repo already has a credential; ErrReservedSecret when the password starts with the reserved envelope prefix.
func (*PG) CreateLinkedRepo ¶
func (s *PG) CreateLinkedRepo(ctx context.Context, org, url, defaultBranch string, up RepoUpstream, cred *GitCredential) (Repo, error)
CreateLinkedRepo runs CreateRepo, CreateGitCredential and SetRepoUpstream in one transaction (meta.EventingStore contract). A concurrent insert of the same (org, url) blocks on the unique index until this commits, so its conflict path reads the upstream too.
func (*PG) CreateRepo ¶
CreateRepo inserts a live repo with a server-generated RepoID. ErrAlreadyExists when (org, url) is taken — even by a soft-deleted row.
func (*PG) CreateRepoWithState ¶
func (s *PG) CreateRepoWithState(ctx context.Context, org, url, defaultBranch string, state CreateState) (Repo, error)
CreateRepoWithState is CreateRepo with an explicit create state.
func (*PG) CreateWebhook ¶
CreateWebhook inserts a webhook with a server-generated ID. ErrNotFound when the org is unknown; ErrReservedSecret when the secret starts with the reserved envelope prefix.
func (*PG) DeleteGitCredential ¶
DeleteGitCredential removes the credential; ErrNotFound when absent.
func (*PG) DeleteRepoUpstream ¶
DeleteRepoUpstream detaches the repo; ErrNotFound when already detached.
func (*PG) DeleteWebhook ¶
DeleteWebhook removes the org's webhook; ErrNotFound when the org is unknown or the ID does not belong to it.
func (*PG) FinishPush ¶
func (s *PG) FinishPush(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
FinishPush is FinishSync on the push_* lane: holder-guarded completion, per-direction counters, ErrClaimLost for stolen claims.
func (*PG) FinishSync ¶
func (s *PG) FinishSync(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
FinishSync completes the run claimed by holder: increments RunCount (and SuccessCount when success), clears Syncing and the claim lease, returns the updated row. ErrNotFound when the repo was detached mid-run — or when no run is in flight: the `AND syncing` guard keeps a stale finisher (detached mid-run, then re-attached) from polluting the fresh row's counters. ErrClaimLost when the row is claimed by a DIFFERENT holder: the caller's expired claim was stolen (issue #86), and the late finisher must not clear the thief's claim or touch the counters.
func (*PG) GetGitCredential ¶
GetGitCredential returns the credential by ID; ErrNotFound when absent.
func (*PG) GetRepoByID ¶
GetRepoByID returns the live repo with the given RepoID; ErrNotFound when absent or soft-deleted.
func (*PG) GetRepoByURL ¶
GetRepoByURL returns the live repo with the given url; ErrNotFound when absent or soft-deleted.
func (*PG) GetRepoUpstream ¶
GetRepoUpstream returns the row; ErrNotFound when detached/never linked.
func (*PG) GetWebhook ¶
GetWebhook returns the webhook with the given ID regardless of org (delivery jobs carry only the ID); ErrNotFound when absent.
func (*PG) GitCredentialForRepo ¶
GitCredentialForRepo returns the repo's credential; ErrNotFound when none is stored (public upstream).
func (*PG) GitCredentialMeta ¶
GitCredentialMeta returns the row with Password "" (the password column is never read — see the interface comment: recovery for undecryptable rows). ErrNotFound when absent.
func (*PG) InitSchema ¶
InitSchema creates the metadata tables (idempotent). The advisory lock serializes concurrent node startups: two racing CREATE TABLE IF NOT EXISTS can fail with a duplicate pg_type key error (same pattern as internal/ref).
func (*PG) ListRepos ¶
func (s *PG) ListRepos(ctx context.Context, org string, params ListParams) (ListResult, error)
ListRepos pages live repos for the org, keyset-ordered by (created_at DESC, repo_id DESC).
func (*PG) ListWebhooks ¶
ListWebhooks returns the org's webhooks in registration order. ErrNotFound when the org is unknown.
func (*PG) PublicKeysForOrg ¶
PublicKeysForOrg returns the org's registered PEM public key blocks in registration order.
func (*PG) PurgeRepoMeta ¶
PurgeRepoMeta finalizes a purge in one transaction: deletes the repo's git_credentials and repo_upstreams rows and stamps the repos row's purged_at tombstone, guarded on the row still being soft-deleted. The row itself is retained forever (see the package comment — the name stays burned). A missing or already-stamped row is an idempotent no-op; a live row aborts with ErrRepoNotDeleted before anything is touched.
func (*PG) PurgeableRepos ¶
func (s *PG) PurgeableRepos(ctx context.Context, deletedBefore time.Time, after PurgeCandidate, limit int) ([]PurgeCandidate, error)
PurgeableRepos returns up to limit repos soft-deleted at or before deletedBefore and not yet purged, ordered by (deleted_at, repo_id) ascending and strictly after the `after` cursor — the background purge pass's rotating work queue (PR #105 round 9): a batch of persistently failing repos stays in the eligible set, so the caller carries the last attempted candidate as the cursor and wraps on exhaustion; without the cursor, oldest-first alone would let such a batch monopolize every pass and starve the rest of the fleet.
func (*PG) RemoveSigningKey ¶
RemoveSigningKey deletes the signing key with the given ID. ErrNotFound when the org is unknown or the ID does not belong to it.
func (*PG) RepoPurgeState ¶
func (s *PG) RepoPurgeState(ctx context.Context, repoID string) (state PurgeState, found bool, err error)
RepoPurgeState returns the repo row for purge eligibility checks: found=false when no row exists; Repo.DeletedAt==nil means the repo is live and must not be purged; PurgedAt non-nil means a purge already completed. Keyed by repo_id alone — purging is a fleet-wide operator action, not an org-scoped API call.
func (*PG) RequestPush ¶
RequestPush raises the outbound pending level (issue #90); levels only coalesce upward so force is never downgraded. ErrNotFound when detached.
func (*PG) RequestSync ¶
RequestSync marks a pending sync request; ErrNotFound when detached. See the EventingStore contract (eventing.go) for the pending-flag protocol (issue #79).
func (*PG) RestorePush ¶
func (s *PG) RestorePush(ctx context.Context, repoID, holder string, force bool) (RepoUpstream, error)
RestorePush is RestoreSync on the push_* lane: the claim is released and the run's consumed level (force when force) re-marked; levels only coalesce upward. The attempt lives in the River job, which snoozes.
func (*PG) RestoreSync ¶
RestoreSync ends the run claimed by holder without counting it (issue #343): serve's shutdown cancelled it. The claim is released and the request re-marked, so the next run redoes it. Errors as FinishSync.
func (*PG) SetRepoUpstream ¶
func (s *PG) SetRepoUpstream(ctx context.Context, up RepoUpstream) (RepoUpstream, error)
SetRepoUpstream inserts the upstream row (CreatedAt assigned by the store; counters/syncing/sync_pending/claim lease zeroed). ErrAlreadyExists when the repo already has one.
func (*PG) SigningKeysForOrg ¶
SigningKeysForOrg returns the org's registered signing keys in registration order; empty slice when none are registered.
func (*PG) SoftDeleteRepo ¶
SoftDeleteRepo sets deleted_at and returns the affected repo. ErrNotFound when no row exists; ErrAlreadyDeleted when already soft-deleted.
func (*PG) TryBeginPush ¶
func (s *PG) TryBeginPush(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, bool, string, error)
TryBeginPush is TryBeginSync on the push_* lane (issue #90): row-locked read-check-write on the database clock, expired leases stolen atomically. No legacy adoption exists here — push claims only ever come from #90-aware binaries, so an expired lease is always provably dead. A steal inherits the dead claim's persisted force mandate (push_claim_force), so a crash between claim and finish cannot downgrade an accepted forced request.
func (*PG) TryBeginSync ¶
func (s *PG) TryBeginSync(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, string, error)
TryBeginSync atomically claims a sync run (syncing false -> true) under a lease and consumes sync_pending (the claimed run's fetch postdates every request marked before the claim). An EXPIRED claim is stolen in the same guarded UPDATE — the row lock taken by the initial SELECT FOR UPDATE makes the read-check-write atomic against concurrent claimants (issue #86: the only recovery for a holder that died mid-run). Both sides of the expiry comparison evaluate on the database clock, so node clock skew does not participate. claimed=false with the current row when an unexpired run is in flight; ErrNotFound when detached.
func (*PG) UpdateGitCredential ¶
func (s *PG) UpdateGitCredential(ctx context.Context, id, username, password string) (GitCredential, error)
UpdateGitCredential replaces username/password; ErrNotFound when absent; ErrReservedSecret when the password starts with the reserved envelope prefix.
func (*PG) UpdateRepoDefaultBranch ¶
func (s *PG) UpdateRepoDefaultBranch(ctx context.Context, org, url, oldBranch, newBranch string) (bool, error)
UpdateRepoDefaultBranch CASes the live repo's default branch (Store contract). One UPDATE carries the whole CAS; zero rows affected then splits inside the same transaction into "no live repo" (ErrNotFound) and "default branch changed concurrently" (won=false).
func (*PG) UpsertOrgWithKeys ¶
func (s *PG) UpsertOrgWithKeys(ctx context.Context, name string, pemPublicKeys []string) (Org, error)
UpsertOrgWithKeys idempotently creates the org and replaces its registered PEM public keys (startup bootstrap in single-org mode).
The org row is locked (SELECT ... FOR UPDATE) before the DELETE/INSERT of org_public_keys so concurrent upserts (e.g. two replicas booting) serialize: without the lock, a blocked DELETE re-evaluates against a statement snapshot that misses the winner's freshly inserted rows, committing a union of both key sets instead of last-writer-wins replacement.
type PurgeCandidate ¶
PurgeCandidate is one purge-queue entry: the repo plus the keyset position its deletion time defines. The zero value doubles as the start-of-queue cursor for PurgeableRepos.
type PurgeState ¶
PurgeState is one repos row as the purge sees it: the row plus the purge-completion tombstone timestamp (nil until a purge completed).
type PushOutcome ¶
type PushOutcome struct {
Kind LaneOutcome
Success bool // Done: counts a successful run
MirrorBack bool // Done+Success: also sets sync_pending in the same write
Force bool // Retry/Restore: the run's consumed level was force
Delay time.Duration // Retry/Restore: due = now + Delay
}
PushOutcome describes how a push run ends.
type QuarantinedError ¶ added in v0.19.0
type QuarantinedError struct {
Err error
}
QuarantinedError answers every metadata operation and key lookup of a process that quarantined its view of meta.json (issue #516). It wraps the refusal that caused it.
func (*QuarantinedError) Error ¶ added in v0.19.0
func (e *QuarantinedError) Error() string
func (*QuarantinedError) Unwrap ¶ added in v0.19.0
func (e *QuarantinedError) Unwrap() error
type Repo ¶
type Repo struct {
RepoID string // server-generated "repo_<12 hex>"
OrgID int64
URL string // JWT `repo` claim value at create time
DefaultBranch string // validated ref component, default "main"
CreatedAt time.Time // UTC; rendered as RFC 3339 in responses
DeletedAt *time.Time // nil while live (soft delete per docs/explanation/domain-model.md)
CreateState CreateState
}
Repo is one repository row.
func ResolveRepoKey ¶
ResolveRepoKey resolves key — a git URL path element (docs/explanation/git-protocol-and-namespaces.md, after the caller strips any `+ephemeral`/`+import` namespace suffix) or a JWT repo claim (docs/reference/authentication.md, url or repo_id) — to its live repo row. RepoID takes precedence (docs/explanation/git-protocol-and-namespaces.md addresses repos by repo_id; ADR-21), then url, so name-style URLs work too. Unknown or soft-deleted keys yield a wrapped ErrNotFound.
type RepoUpstream ¶
type RepoUpstream struct {
RepoID string
Provider string
Owner string
Name string
UpstreamHost string // resolved at create time: provider default or explicit upstream_host
RunCount int // completed sync runs (success + failure)
SuccessCount int // completed successful runs
Syncing bool // one run in flight (TryBeginSync..FinishSync)
// SyncPending marks a sync request (RequestSync) not yet covered by a
// claimed run: TryBeginSync consumes it, so a flag still set when
// FinishSync returns means requests arrived AFTER the finished run
// claimed (its fetch may predate them) and a follow-up run is due
// (issue #79 liveness).
SyncPending bool
// SyncClaimHolder/SyncClaimExpiresAt lease the Syncing claim (issue
// #86): every claim records a per-run holder token and an expiry, and
// a conflicting claimant may STEAL a claim whose lease has provably
// expired — the only recovery for a holder process that died between
// TryBeginSync and FinishSync. FinishSync clears both (holder "" and
// expiry at the zero instant, epoch/0 in storage).
SyncClaimHolder string
SyncClaimExpiresAt time.Time
// ForwardOnPush opts the repo into forwarding won local pushes to the
// upstream (issue #90, base_repo.forward_pushes; default off).
ForwardOnPush bool
// PushPending is the outbound lane's pending marker (issue #90): a
// three-level value (0 none / 1 push / 2 force) that coalesces upward
// via RequestPush and is consumed by TryBeginPush. The level lives on
// the durable row — not the queue job — because job deduplication is
// keyed by repo and a crash-rescued job must still remember force.
PushPending int
// PushSyncing/PushClaimHolder/PushClaimExpiresAt lease the outbound
// claim exactly like the inbound triple. The lanes are separate on
// purpose: the inbound claim protects local refs (the loopback
// force-push), the outbound claim protects upstream refs. No legacy
// adoption exists here — no pre-#90 binary ever holds a push claim,
// so an expired push lease is always stealable.
PushSyncing bool
PushClaimHolder string
PushClaimExpiresAt time.Time
// PushClaimForce persists the force mandate the in-flight claim
// consumed from the marker, until FinishPush clears it: a claim whose
// holder crashed still owes the accepted forced request, so stealing
// an expired claim inherits its force (OR-ed with the thief's own
// consumed level) instead of silently downgrading the rescue.
PushClaimForce bool
PushRunCount int // completed outbound runs (success + failure)
PushSuccessCount int // completed successful outbound runs
// DefaultBranchOmitted marks a GitHub sync create that named no default
// branch (issue #599): the repo starts on "main" until a successful sync
// adopts the upstream HEAD (AdoptUpstreamDefaultBranch), which clears
// it. An explicit default-branch change clears it too. S3-WAL stores no
// such field (metadata v2 is frozen) and always reads false.
DefaultBranchOmitted bool
CreatedAt time.Time
}
RepoUpstream is one repo's upstream mirror link plus its sync-run state (docs/explanation/domain-model.md invariant 11). is_first_sync := SuccessCount == 0; detach = delete the row.
type RunRestoreStore ¶
type RunRestoreStore interface {
RestoreSync(ctx context.Context, repoID, holder string) (RepoUpstream, error)
RestorePush(ctx context.Context, repoID, holder string, force bool) (RepoUpstream, error)
}
RunRestoreStore ends runs that serve's shutdown cancelled without counting them (issue #343). Only *PG implements it: SQLite and S3WAL restore through LocalSchedule (ADR-50, ADR-51).
type S3WAL ¶
type S3WAL struct {
// contains filtered or unexported fields
}
S3WAL implements Backend (plus the purge seams) over one conditional meta object. Construct via OpenS3WAL (production, real store + probe) or NewS3WAL (tests/DST, injectable substrate).
func NewS3WAL ¶
func NewS3WAL(cs s3wal.ConditionalStore, opts ...Option) *S3WAL
NewS3WAL wraps an injectable conditional store, probe-less: the constructor unit tests and the DST harness build meta backends from (mirroring ref.NewS3WAL). Production serving goes through OpenS3WAL.
func OpenS3WAL ¶
OpenS3WAL opens s3wal://bucket?endpoint=…®ion=… — the exact URL grammar of the ref store (s3wal.Dial). Open performs no network I/O; the conditional-write probe runs at InitSchema (ADR-12: Open never runs DDL, and hooks reconstructing stores from env URLs must not probe).
func (*S3WAL) AddSigningKey ¶
AddSigningKey registers a commit-signing key for the org and returns the stored row. Recovery class PRED: the per-run (ID, key text, stamp) triple identifies OUR insert on the fresh snapshot (IDs are minted from the monotonic counter and never reused).
func (*S3WAL) AdoptUpstreamDefaultBranch ¶ added in v0.19.0
func (s *S3WAL) AdoptUpstreamDefaultBranch(ctx context.Context, repoID, branch string) (bool, error)
AdoptUpstreamDefaultBranch never adopts (meta.EventingStore contract): metadata v2 is frozen, so S3-WAL stores no DefaultBranchOmitted flag and its sync repos keep "main" (issue #599, a recorded deviation).
func (*S3WAL) Close ¶
func (s *S3WAL) Close()
Close stops the org-key refresher — joining its goroutine, so no background read outlives Close — and releases the underlying store. Idempotent.
func (*S3WAL) CompleteRepoCreate ¶
CompleteRepoCreate marks a pending row complete (Store contract, issue #388). Recovery class PRED: an ambiguous PUT whose write is still the latest answers CreateCompleted (writer nonce); one that landed and was superseded reruns and answers CreateAlreadyComplete (or CreateDeleted), which callers treat like a lost race against a concurrent completion.
func (*S3WAL) CreateGitCredential ¶
func (s *S3WAL) CreateGitCredential(ctx context.Context, cred GitCredential) (GitCredential, error)
CreateGitCredential stores cred (ID/CreatedAt assigned by the store). ErrAlreadyExists when the repo already has a credential; ErrNotFound for deleted or unknown repos (the live-repo guard, issue #52 — the single-object CAS is the atomicity the SQL twins buy with statement serialization); ErrReservedSecret for reserved-prefix passwords. Recovery class PRED: the per-run random ID proves OUR insert.
func (*S3WAL) CreateLinkedRepo ¶
func (s *S3WAL) CreateLinkedRepo(ctx context.Context, org, url, defaultBranch string, up RepoUpstream, cred *GitCredential) (Repo, error)
CreateLinkedRepo is CreateRepo, CreateGitCredential and SetRepoUpstream in one document mutation (meta.EventingStore contract): the single-object CAS publishes the three rows together. Recovery class PRED, like CreateRepo: the per-run random RepoID on the fresh snapshot proves OUR mutation landed, and with it every row it carried.
func (*S3WAL) CreateRepo ¶
CreateRepo inserts a live repo with a server-generated RepoID. ErrAlreadyExists when (org, url) is taken — even by a soft-deleted or purged row (the burned-name invariant). Recovery class PRED: the per-run RepoID is random and never reused, so its presence on the fresh snapshot proves OUR insert landed.
func (*S3WAL) CreateRepoWithState ¶
func (s *S3WAL) CreateRepoWithState(ctx context.Context, org, url, defaultBranch string, state CreateState) (Repo, error)
CreateRepoWithState is CreateRepo with an explicit create state, stored in the v2 create_state field (ADR-51).
func (*S3WAL) CreateWebhook ¶
CreateWebhook inserts a webhook with a server-generated ID. ErrNotFound when the org is unknown; ErrReservedSecret when the secret starts with the reserved envelope prefix. Recovery class PRED: the per-run random ID proves OUR insert.
func (*S3WAL) DeleteGitCredential ¶
DeleteGitCredential removes the credential; ErrNotFound when absent. Recovery class PRED: credential IDs are never reused.
func (*S3WAL) DeleteRepoUpstream ¶
DeleteRepoUpstream detaches the repo; ErrNotFound when never linked or already detached. Recovery class PRED per phase: rows are identified by their creation stamp, so a row re-linked after a landed delete is never re-deleted by a recovery re-run.
With a SyncClaimSink wired, detach must also revoke the §2.8 manifest fence — the SQL twins get that for free (their fence reads the very repo_upstreams row the delete removes), but here the token lives in the ref manifest, and a token outliving the detach would let the detached upstream's in-flight sync worker publish AFTER the detach committed. The revocation must be RESUMABLE: no intermediate state may hold the holder needed for cleanup only in a dead process's memory (a post-delete clear failure would otherwise hide a live token behind an ErrNotFound the API reads as idempotent success). Hence three phases, each durable before the next:
- Mark the row Detaching (one CAS), capturing the row's claim holder atomically with the mark (a resuming retry re-captures it fresh from the still-marked row). The mark is never unset and, once durable, TryBeginSync refuses the row — in its claim fn AND in its post-install revalidation — so no NEW fence can be installed and handed out from here on; by meta-first ordering, every claim committed before the mark already has its holder on the row.
- Clear that holder's manifest token (owner-guarded). A failure aborts the detach with the marked row — and thus the holder — still durable in the doc: a retry from ANY process re-enters at the no-op mark and re-runs this clear; ErrNotFound is never returned with the fence possibly live. A token installed AFTER this clear belongs to a claim still awaiting revalidation (installs precede revalidation), which then sees the mark or the missing row and self-revokes; a claimant crashing between install and revalidation leaves a never-handed-out token, inert by construction (a fence token only refuses OTHER holders). Post-mark the row's holder only ever transitions to empty (FinishSync and the claim rollback clear it; nothing re-installs), so the captured holder cannot go stale.
- Delete the marked row (one CAS, creation-stamp predicate; a row gone or replaced means a concurrent detach completed — the same terminal state). Landing requires phases 1-2 durable, so a later call's ErrNotFound proves every handed-out fence was revoked.
Steal interaction (window CLOSED by TryBeginSync's steal pre-clear): a steal only commits after the superseded victim's manifest token is durably cleared, so a pre-mark steal cannot strand the victim's token under a row naming the thief. The only tokens that can outlive phase 2's clear of the marked row's holder are therefore late installs by claims still awaiting revalidation, which self-revoke on the mark or the missing row — or linger inert when the claimant crashes first, never having been handed out. Remaining assumption: fence tokens are only ever installed by TryBeginSync's write-through and only honored for its returned, revalidated holders.
Audit of the remaining row ops against the mark: RequestSync and RequestPush raise pending flags that die with the row; TryBeginPush hands out the outbound claim, which has no fence write-through (file comment) — an in-flight push worker surviving detach is the SQL twins' exposure too; SetRepoUpstream refuses a detaching row with ErrAlreadyExists until a detach retry completes it. SQL-parity divergence, accepted: a midway-crashed s3wal detach leaves the row visible (and claim-refusing) until the next detach call resumes it, where SQL's atomic delete would have left it fully live.
Without a sink there is no write-through in the topology (hook processes only read the fence and never detach), so the single-CAS delete is complete on its own.
func (*S3WAL) DeleteWebhook ¶
DeleteWebhook removes the org's webhook; ErrNotFound when the org is unknown or the ID does not belong to it. Recovery class PRED on the semantic effect: webhook IDs are random and never reused, so an absent ID proves the deletion holds.
func (*S3WAL) FinishPush ¶
func (s *S3WAL) FinishPush(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
FinishPush is FinishSync on the push_* lane: holder-guarded completion, per-direction counters, force mandate cleared, ErrClaimLost for stolen claims. Same counter-advancement recovery predicate as FinishSync.
func (*S3WAL) FinishSync ¶
func (s *S3WAL) FinishSync(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
FinishSync completes the run claimed by holder: increments RunCount (and SuccessCount when success), clears Syncing and the lease, returns the updated row; SyncPending is NOT touched. ErrNotFound / ErrClaimLost per the EventingStore contract. With a sink wired, the manifest fence token is cleared (owner-guarded) BEFORE the meta row is finalized; a clear failure aborts the finish (retryable — the meta claim stays held until retried or expired, and the manifest token keeps fencing only this holder's writes).
Recovery class PRED on counter advancement: our finish landed iff the claim is no longer ours AND both counters advanced by exactly our increment from the claimed snapshot. A thief that stole our expired claim and finished with the SAME success bit is indistinguishable — and produces the identical terminal row, so reporting success is harmless; a thief with a different bit fails the predicate and re-runs into the SQL twins' late-finisher classification (ErrClaimLost / ErrNotFound).
func (*S3WAL) GetGitCredential ¶
GetGitCredential returns the credential by ID; ErrNotFound when absent.
func (*S3WAL) GetRepoByID ¶
GetRepoByID returns the live repo with the given RepoID; ErrNotFound when absent or soft-deleted.
func (*S3WAL) GetRepoByURL ¶
GetRepoByURL returns the live repo with the given url; ErrNotFound when absent or soft-deleted.
func (*S3WAL) GetRepoUpstream ¶
GetRepoUpstream returns the row; ErrNotFound when detached/never linked.
func (*S3WAL) GetWebhook ¶
GetWebhook returns the webhook with the given ID regardless of org (delivery jobs carry only the ID); ErrNotFound when absent.
func (*S3WAL) GitCredentialForRepo ¶
GitCredentialForRepo returns the repo's credential; ErrNotFound when none is stored (public upstream).
func (*S3WAL) GitCredentialMeta ¶
GitCredentialMeta returns the row with Password "" (the password field is never read — recovery for undecryptable rows, see the interface comment). ErrNotFound when absent.
func (*S3WAL) InitSchema ¶
InitSchema proves the store's conditional-write fidelity before serving (plan §2.6) — there is no DDL, the document is created lazily by the first write. Unlike ref.S3WAL.InitSchema it does NOT require an HTTP Date witness: this backend runs no §2.4 TTL expiry verdicts (claim leases compare caller-written absolute stamps under the injected clock, the SQLite twin's semantics), so no destructive verdict ever consults the store clock. Idempotent.
func (*S3WAL) ListRepos ¶
func (s *S3WAL) ListRepos(ctx context.Context, org string, params ListParams) (ListResult, error)
ListRepos pages live repos for the org, keyset-ordered by (created_at DESC, repo_id DESC) with the shared opaque cursor (cursor.go). The Q filter is a literal case-insensitive substring match; folding is Unicode-wide here (strings.ToLower) where SQLite folds ASCII only and PG folds per locale — an accepted divergence of the same kind the SQL twins already document against each other.
func (*S3WAL) ListWebhooks ¶
ListWebhooks returns the org's webhooks in registration order (created_at, id). ErrNotFound when the org is unknown; undecryptable rows degrade to Secret == "" (see the EventingStore contract).
func (*S3WAL) PublicKeysForOrg ¶
PublicKeysForOrg returns the org's registered PEM public key blocks in registration order — served from the in-memory snapshot, never a synchronous store read (PR #168 security P2; see the orgKeys field comment for the flood argument). Unknown orgs — including flooded unverified iss values — answer ErrNotFound from memory. Freshness: same-process writes are visible immediately (mutate installs the snapshot before returning); foreign writes within s3walKeyRefreshInterval. Cold start (nil snapshot) blocks on one real read: production serving populates the snapshot during bootstrap (UpsertOrgWithKeys runs before the listener starts), and Open must stay I/O-free (ADR-12), so this path only serves CLI/hook processes.
func (*S3WAL) PurgeRepoMeta ¶
PurgeRepoMeta — see the PG twin: deletes the repo's credential and upstream rows and stamps purged_at, guarded on the row still being soft-deleted; a missing or already-stamped row is an idempotent no-op; a live row aborts with ErrRepoNotDeleted before anything is touched. Recovery class ID: every effect is a guarded delete/stamp, so a re-run on a fresh snapshot converges on the same terminal state.
func (*S3WAL) PurgeableRepos ¶
func (s *S3WAL) PurgeableRepos(ctx context.Context, deletedBefore time.Time, after PurgeCandidate, limit int) ([]PurgeCandidate, error)
PurgeableRepos — see the PG twin. deleted_at is stored as unixnano; ordering and the keyset cursor match the SQL row-value comparison.
func (*S3WAL) RemoveSigningKey ¶
RemoveSigningKey deletes the signing key with the given ID. ErrNotFound when the org is unknown or the ID does not belong to it. Recovery class PRED on the semantic effect: IDs are never reused, so an absent ID proves the deletion holds.
func (*S3WAL) RepoPurgeState ¶
RepoPurgeState — see the PG twin.
func (*S3WAL) RequestPush ¶
RequestPush raises the outbound pending level (issue #90); levels only coalesce upward. ErrNotFound when detached. Recovery class ID: the MAX-merge is monotone.
func (*S3WAL) RequestSync ¶
RequestSync marks a pending sync request (issue #79 protocol); ErrNotFound when detached. Recovery class ID: the flag is a monotone upsert, re-applied harmlessly.
func (*S3WAL) SetRepoUpstream ¶
func (s *S3WAL) SetRepoUpstream(ctx context.Context, up RepoUpstream) (RepoUpstream, error)
SetRepoUpstream inserts the upstream row (CreatedAt assigned by the store; counters/flags/leases zeroed). ErrAlreadyExists when the repo already has one; ErrNotFound for deleted or unknown repos (live-repo guard, issue #52). Recovery class PRED: the per-run stamp plus the full identity fields prove OUR insert (rows are created at most once per repo, so a matching row under our stamp cannot predate this call).
func (*S3WAL) SigningKeysForOrg ¶
SigningKeysForOrg returns the org's registered signing keys in registration order; empty slice when none are registered.
func (*S3WAL) SoftDeleteRepo ¶
SoftDeleteRepo sets the deletion stamp and returns the affected repo. ErrNotFound when no row exists; ErrAlreadyDeleted when already soft-deleted. Recovery class PRED: the per-run stamp proves OUR delete landed; a row deleted under a different stamp re-runs into the ErrAlreadyDeleted classification.
func (*S3WAL) TryBeginPush ¶
func (s *S3WAL) TryBeginPush(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, bool, string, error)
TryBeginPush is TryBeginSync on the push_* lane (issue #90): claims the outbound run under a lease, consumes PushPending whole, and inherits a dead claim's persisted force mandate on steal. There is no legacy adoption and no write-through — the outbound claim protects upstream refs, which no local CASRef fences. Recovery class PRED: the (holder, expiry-stamp) pair proves OUR claim, as in TryBeginSync.
func (*S3WAL) TryBeginSync ¶
func (s *S3WAL) TryBeginSync(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, string, error)
TryBeginSync atomically claims a sync run under a lease, consuming sync_pending; an EXPIRED claim is stolen in the same CAS (the read-check-write atomicity the SQL twins buy with row locks is the object CAS here). claimed=false with the current row when an unexpired run is in flight; ErrNotFound when detached. No legacy adoption lane exists (see the file comment).
Recovery class PRED for the claim write: the (holder, expiry-stamp) pair is unique to this attempt — holder tokens are per-run and the expiry is this call's clock reading — so finding both on the fresh snapshot proves OUR claim landed (holder equality alone would not, the Phase 1b same-field-rewrite lesson).
With a SyncClaimSink wired, the §2.8 write-through runs after the meta claim commits — see the file comment for the ordering and fail-closed argument. A STEAL additionally clears the expired victim's manifest token BEFORE the meta steal CAS commits (owner-guarded, so a victim concurrently finishing makes it a no-op): otherwise a thief crashing after its meta commit but before its own fence install would leave the superseded victim's token authorizing an expired-but-alive victim's writes — the one place the manifest fence lagged the SQL row fence, which excludes the victim atomically with the steal. The claim fn refuses to commit a steal whose victim it cannot prove pre-cleared (errStealVictimUncleared aborts the CAS with nothing written; the caller clears the observed victim and retries), so on every committed steal the clear is already durable. A crash between the clear and the steal CAS strands the victim fenceless with its expired claim intact: fail-closed (its writes reject; its FinishSync clear no-ops; the expired lease was already stealable — nothing regresses). A clear failure aborts the attempt retryably with nothing consumed. The fresh-claim path pays no extra sink call, and the same-holder re-claim (per-run tokens make it test-only) is handled uniformly: the holder's old token is cleared and the post-commit write-through reinstalls it.
func (*S3WAL) UpdateGitCredential ¶
func (s *S3WAL) UpdateGitCredential(ctx context.Context, id, username, password string) (GitCredential, error)
UpdateGitCredential replaces username/password; ErrNotFound when absent; ErrReservedSecret for reserved-prefix passwords. Recovery class PRED: with encryption on, the stored envelope is unique per call (fresh GCM nonce), so field equality proves OUR write; with plaintext storage an identical competing update is indistinguishable and identical in effect.
func (*S3WAL) UpdateRepoDefaultBranch ¶
func (s *S3WAL) UpdateRepoDefaultBranch(ctx context.Context, org, url, oldBranch, newBranch string) (bool, error)
UpdateRepoDefaultBranch CASes the live repo's default branch (Store contract). Recovery class PRED on the semantic effect: branch == new proves the swap holds (an ambiguous PUT superseded by a competitor that wrote the SAME transition is reported won=true although SQL would grant exactly one of the two — an accepted divergence: the terminal state is identical and the corner requires a lost response).
func (*S3WAL) UpsertOrgWithKeys ¶
func (s *S3WAL) UpsertOrgWithKeys(ctx context.Context, name string, pemPublicKeys []string) (Org, error)
UpsertOrgWithKeys idempotently creates the org and replaces its registered PEM public keys. Recovery class ID: a re-run on a fresh snapshot re-applies the same terminal state, and the ID mint is re-derived per run (an org our landed write created is found, never re-minted).
type S3WALConvertOptions ¶ added in v0.19.0
type S3WALConvertOptions struct {
// Force converts even when a create without an upstream link is unfinished.
Force bool
}
S3WALConvertOptions changes how ConvertS3WALMeta treats unfinished creates on --to 1.
type S3WALMetaConversion ¶ added in v0.19.0
type S3WALMetaConversion struct {
// Missing is true when the bucket has no metadata document. Nothing
// is written: the first write of the serving binary creates one.
Missing bool
// From and To are the stored versions before and after. From == To
// means the document already had the target version (no write).
From, To int
// SeqBefore and SeqAfter are the stored document sequence numbers.
SeqBefore, SeqAfter int64
// Before and After count the rows of the source and the result.
Before, After S3WALMetaCounts
// RepairedSyncCreates (--to 1) lists the pending sync creates whose
// first-sync request the conversion recorded, by repo ID.
RepairedSyncCreates []string
// ForcedCreates (--to 1 with Force) lists the pending creates without
// an upstream link that now read as finished.
ForcedCreates []S3WALPendingCreate
}
S3WALMetaConversion reports one ConvertS3WALMeta call.
func ConvertS3WALMeta ¶ added in v0.19.0
func ConvertS3WALMeta(ctx context.Context, cs s3wal.ConditionalStore, to int, opts S3WALConvertOptions) (S3WALMetaConversion, error)
ConvertS3WALMeta rewrites the bucket's metadata document to version to (1 or 2). It applies the zero-value rule of ADR-51: to 2 changes only the version; to 1 drops the four v2 fields and sets sync_pending on every lane whose pull claim is set, so v1 boot recovery of any release restarts that pull. It also sets sync_pending for every unfinished sync create, which v1 then completes with its first sync, and refuses with *S3WALPendingCreatesError (nothing written) while a create without an upstream link is unfinished, unless opts.Force. After the write it re-reads the document and checks it equals the expected conversion of the source. Re-running on a document at the target version writes nothing.
type S3WALMetaCounts ¶ added in v0.19.0
type S3WALMetaCounts struct {
Orgs, Repos, Webhooks, Credentials, Upstreams int
}
S3WALMetaCounts are the row counts of one metadata document.
type S3WALPendingCreate ¶ added in v0.19.0
type S3WALPendingCreate struct {
RepoID, URL string
}
S3WALPendingCreate is a live repo whose create had not finished.
func (S3WALPendingCreate) String ¶ added in v0.19.0
func (c S3WALPendingCreate) String() string
String renders the create for an operator's terminal. The URL comes from a token's repo claim, so it is quoted: control bytes and escape sequences print escaped and cannot restyle or forge the output.
type S3WALPendingCreatesError ¶ added in v0.19.0
type S3WALPendingCreatesError struct {
Creates []S3WALPendingCreate
}
S3WALPendingCreatesError refuses --to 1 while creates without an upstream link (forks, copies, sync creates whose link was removed) are unfinished: v1 stores no create state, so on a v1 release each one reads as a finished, possibly empty repository. Nothing was written.
func (*S3WALPendingCreatesError) Error ¶ added in v0.19.0
func (e *S3WALPendingCreatesError) Error() string
type SQLite ¶
type SQLite struct {
// contains filtered or unexported fields
}
SQLite implements Backend on one local SQLite file via the pure-Go modernc.org/sqlite driver (cgo-free cross-compiled single binaries).
func OpenSQLite ¶
OpenSQLite opens sqlite:///abs/path/db.sqlite. The path is everything after "sqlite://" (relative paths are tolerated; --data-dir always derives absolute ones). Parent directories are created; the DB file itself is created by the driver on first open.
func (*SQLite) AddSigningKey ¶
AddSigningKey registers a commit-signing key for the org and returns the stored row. Format validation (SSH line / armored PGP block) is the caller's responsibility per the Store contract; the text is stored verbatim.
func (*SQLite) AdoptUpstreamDefaultBranch ¶ added in v0.19.0
func (s *SQLite) AdoptUpstreamDefaultBranch(ctx context.Context, repoID, branch string) (bool, error)
AdoptUpstreamDefaultBranch (meta.EventingStore contract) in one BEGIN IMMEDIATE transaction.
func (*SQLite) ClaimDuePush ¶
func (*SQLite) ClaimDueSync ¶
func (*SQLite) CompleteRepoCreate ¶
CompleteRepoCreate moves a pending repo to complete (meta.Store contract).
func (*SQLite) CompleteSyncRepoCreate ¶
func (s *SQLite) CompleteSyncRepoCreate(ctx context.Context, org, repoID string) (CreateOutcome, error)
CompleteSyncRepoCreate is CompleteRepoCreate for a sync create (issue #388): when the row is pending it also records the first-sync request (sync_pending, which the local schedule's sweep runs) in the same write. A complete or deleted row records nothing.
func (*SQLite) CreateGitCredential ¶
func (s *SQLite) CreateGitCredential(ctx context.Context, cred GitCredential) (GitCredential, error)
CreateGitCredential stores cred (ID/CreatedAt assigned by the store). ErrAlreadyExists when the repo already has a credential; ErrReservedSecret when the password starts with the reserved envelope prefix.
func (*SQLite) CreateLinkedRepo ¶
func (s *SQLite) CreateLinkedRepo(ctx context.Context, org, url, defaultBranch string, up RepoUpstream, cred *GitCredential) (Repo, error)
CreateLinkedRepo runs CreateRepo, CreateGitCredential and SetRepoUpstream in one BEGIN IMMEDIATE transaction (meta.EventingStore contract).
func (*SQLite) CreateRepo ¶
CreateRepo inserts a live repo with a server-generated RepoID. ErrAlreadyExists when (org, url) is taken — even by a soft-deleted row.
func (*SQLite) CreateRepoWithState ¶
func (s *SQLite) CreateRepoWithState(ctx context.Context, org, url, defaultBranch string, state CreateState) (Repo, error)
CreateRepoWithState is CreateRepo with an explicit create state.
func (*SQLite) CreateWebhook ¶
CreateWebhook inserts a webhook with a server-generated ID. ErrNotFound when the org is unknown; ErrReservedSecret when the secret starts with the reserved envelope prefix.
func (*SQLite) DeleteGitCredential ¶
DeleteGitCredential removes the credential; ErrNotFound when absent.
func (*SQLite) DeleteRepoUpstream ¶
DeleteRepoUpstream detaches the repo; ErrNotFound when already detached.
func (*SQLite) DeleteWebhook ¶
DeleteWebhook removes the org's webhook; ErrNotFound when the org is unknown or the ID does not belong to it.
func (*SQLite) FinishPush ¶
func (s *SQLite) FinishPush(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
FinishPush is FinishSync on the push_* lane: holder-guarded completion, per-direction counters, ErrClaimLost for stolen claims.
func (*SQLite) FinishPushOutcome ¶
func (s *SQLite) FinishPushOutcome(ctx context.Context, repoID, holder string, o PushOutcome) (RepoUpstream, error)
func (*SQLite) FinishSync ¶
func (s *SQLite) FinishSync(ctx context.Context, repoID, holder string, success bool) (RepoUpstream, error)
FinishSync completes the run claimed by holder: increments RunCount (and SuccessCount when success), clears Syncing and the claim lease, returns the updated row. ErrNotFound when the repo was detached mid-run — or when no run is in flight: the `AND syncing` guard keeps a stale finisher (detached mid-run, then re-attached) from polluting the fresh row's counters. ErrClaimLost when the row is claimed by a DIFFERENT holder: the caller's expired claim was stolen (issue #86), and the late finisher must not clear the thief's claim or touch the counters.
func (*SQLite) FinishSyncOutcome ¶
func (s *SQLite) FinishSyncOutcome(ctx context.Context, repoID, holder string, o SyncOutcome) (RepoUpstream, error)
func (*SQLite) GetGitCredential ¶
GetGitCredential returns the credential by ID; ErrNotFound when absent.
func (*SQLite) GetRepoByID ¶
GetRepoByID returns the live repo with the given RepoID; ErrNotFound when absent or soft-deleted.
func (*SQLite) GetRepoByURL ¶
GetRepoByURL returns the live repo with the given url; ErrNotFound when absent or soft-deleted.
func (*SQLite) GetRepoUpstream ¶
GetRepoUpstream returns the row; ErrNotFound when detached/never linked.
func (*SQLite) GetWebhook ¶
GetWebhook returns the webhook with the given ID regardless of org (delivery jobs carry only the ID); ErrNotFound when absent.
func (*SQLite) GitCredentialForRepo ¶
GitCredentialForRepo returns the repo's credential; ErrNotFound when none is stored (public upstream).
func (*SQLite) GitCredentialMeta ¶
GitCredentialMeta returns the row with Password "" (the password column is never read — see the interface comment: recovery for undecryptable rows). ErrNotFound when absent.
func (*SQLite) InitSchema ¶
InitSchema creates the metadata tables (idempotent). No advisory lock (unlike PG): SQLite's file lock + busy_timeout serialize concurrent startups, and CREATE TABLE IF NOT EXISTS is race-free under that lock.
func (*SQLite) ListLocalWork ¶
func (*SQLite) ListRepos ¶
func (s *SQLite) ListRepos(ctx context.Context, org string, params ListParams) (ListResult, error)
ListRepos pages live repos for the org, keyset-ordered by (created_at DESC, repo_id DESC).
PG's `url ILIKE ?` becomes `lower(url) LIKE lower(?) ESCAPE '\'`. SQLite's lower() only folds ASCII, so non-ASCII urls match case-sensitively here while PG ILIKE folds per locale — an accepted divergence for local mode.
func (*SQLite) ListWebhooks ¶
ListWebhooks returns the org's webhooks in registration order. ErrNotFound when the org is unknown.
func (*SQLite) PublicKeysForOrg ¶
PublicKeysForOrg returns the org's registered PEM public key blocks in registration order.
func (*SQLite) PurgeRepoMeta ¶
PurgeRepoMeta — see the PG twin. The inTx write transaction (BEGIN IMMEDIATE) serializes the guard read with the deletes.
func (*SQLite) PurgeableRepos ¶
func (s *SQLite) PurgeableRepos(ctx context.Context, deletedBefore time.Time, after PurgeCandidate, limit int) ([]PurgeCandidate, error)
PurgeableRepos — see the PG twin. deleted_at is stored as unixnano.
func (*SQLite) RemoveSigningKey ¶
RemoveSigningKey deletes the signing key with the given ID. ErrNotFound when the org is unknown or the ID does not belong to it.
func (*SQLite) RepoPurgeState ¶
func (s *SQLite) RepoPurgeState(ctx context.Context, repoID string) (state PurgeState, found bool, err error)
RepoPurgeState — see the PG twin.
func (*SQLite) RequestPush ¶
RequestPush raises the outbound pending level (issue #90); levels only coalesce upward so force is never downgraded. ErrNotFound when detached.
func (*SQLite) RequestPushNow ¶
func (*SQLite) RequestSync ¶
RequestSync marks a pending sync request; ErrNotFound when detached. See the EventingStore contract (eventing.go) for the pending-flag protocol (issue #79).
func (*SQLite) SetRepoUpstream ¶
func (s *SQLite) SetRepoUpstream(ctx context.Context, up RepoUpstream) (RepoUpstream, error)
SetRepoUpstream inserts the upstream row (CreatedAt assigned by the store; counters/syncing/sync_pending/claim lease zeroed). ErrAlreadyExists when the repo already has one.
func (*SQLite) SigningKeysForOrg ¶
SigningKeysForOrg returns the org's registered signing keys in registration order; empty slice when none are registered.
func (*SQLite) SoftDeleteRepo ¶
SoftDeleteRepo sets deleted_at and returns the affected repo. ErrNotFound when no row exists; ErrAlreadyDeleted when already soft-deleted.
func (*SQLite) TryBeginPush ¶
func (s *SQLite) TryBeginPush(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, bool, string, error)
TryBeginPush is TryBeginSync on the push_* lane (issue #90): BEGIN IMMEDIATE read-check-write on Go wall time (single-host topology). No legacy adoption exists here — push claims only ever come from #90-aware binaries, so an expired lease is always provably dead. A steal inherits the dead claim's persisted force mandate (push_claim_force), so a crash between claim and finish cannot downgrade an accepted forced request.
func (*SQLite) TryBeginSync ¶
func (s *SQLite) TryBeginSync(ctx context.Context, repoID, holder string, lease time.Duration) (RepoUpstream, bool, string, error)
TryBeginSync atomically claims a sync run (syncing false -> true) under a lease and consumes sync_pending (the claimed run's fetch postdates every request marked before the claim). An EXPIRED claim is stolen in the same guarded UPDATE (issue #86: the only recovery for a holder that died mid-run). The BEGIN IMMEDIATE transaction makes the read-check-write atomic against concurrent claimers; local mode is a single-host topology, so Go wall time is a consistent clock source. claimed=false with the current row when an unexpired run is in flight; ErrNotFound when detached.
func (*SQLite) UpdateGitCredential ¶
func (s *SQLite) UpdateGitCredential(ctx context.Context, id, username, password string) (GitCredential, error)
UpdateGitCredential replaces username/password; ErrNotFound when absent; ErrReservedSecret when the password starts with the reserved envelope prefix.
func (*SQLite) UpdateRepoDefaultBranch ¶
func (s *SQLite) UpdateRepoDefaultBranch(ctx context.Context, org, url, oldBranch, newBranch string) (bool, error)
UpdateRepoDefaultBranch CASes the live repo's default branch (Store contract). One UPDATE carries the whole CAS; zero rows affected then splits inside the same write transaction (BEGIN IMMEDIATE) into "no live repo" (ErrNotFound) and "default branch changed concurrently" (won=false).
func (*SQLite) UpsertOrgWithKeys ¶
func (s *SQLite) UpsertOrgWithKeys(ctx context.Context, name string, pemPublicKeys []string) (Org, error)
UpsertOrgWithKeys idempotently creates the org and replaces its registered PEM public keys (startup bootstrap in single-org mode).
PG locks the org row (SELECT ... FOR UPDATE) so concurrent upserts serialize; here _txlock=immediate makes the whole transaction a write transaction from BEGIN, and SQLite allows only one writer at a time, so the DELETE/INSERT replacement is last-writer-wins by construction.
type SigningKey ¶
SigningKey is one registered commit-signing key for an org (docs/reference/authentication.md verify-sig; the OSS stand-in for the upstream dashboard key registry). KeyText is either a single SSH public key line ("ssh-ed25519 AAAA...") or an ASCII-armored PGP public key block; callers validate the format before AddSigningKey — the store persists the text verbatim.
type StaleDocumentError ¶ added in v0.19.0
type StaleDocumentError struct {
Got int64 // the fetched document's seq
Seen int64 // the seq of the document this process last saw
}
StaleDocumentError reports a fetched metadata document that does not continue the one this process last saw (plan §2.3 stale-read watermark): its seq is lower, or it is a different document at the same seq, or this process already saw such a regression. The bytes come from the bucket, so this is not an internal invariant: the store served a stale read, or the object was replaced while this process ran (issue #488). The process keeps its view and refuses to read or write through the fetched document, but it must not crash: the org-key refresher meets this on its next tick.
func (*StaleDocumentError) Error ¶ added in v0.19.0
func (e *StaleDocumentError) Error() string
type Store ¶
type Store interface {
// UpsertOrgWithKeys idempotently creates the org and replaces its
// registered PEM public keys (startup bootstrap in single-org mode).
UpsertOrgWithKeys(ctx context.Context, name string, pemPublicKeys []string) (Org, error)
// PublicKeysForOrg returns the org's registered PEM public key blocks
// for JWT verification.
PublicKeysForOrg(ctx context.Context, name string) ([]string, error)
// OrgNames returns every org row's name, ordered by name. Org rows are
// retained forever (single-org mode never deletes them), so the serve
// command uses this at startup to warn about rows that no longer match
// the configured org (issue #47).
OrgNames(ctx context.Context) ([]string, error)
// CreateRepo inserts a live repo with a server-generated RepoID.
// ErrAlreadyExists when (org, url) is taken — even by a soft-deleted row.
// The row's CreateState is CreateStateLegacy.
CreateRepo(ctx context.Context, org, url, defaultBranch string) (Repo, error)
// CreateRepoWithState is CreateRepo with an explicit CreateState
// (pending or complete).
CreateRepoWithState(ctx context.Context, org, url, defaultBranch string, state CreateState) (Repo, error)
// CompleteRepoCreate moves a pending repo to complete. A soft-deleted
// repo returns CreateDeleted, even if it was complete. ErrNotFound when
// no row exists. On a legacy row it returns an error.
CompleteRepoCreate(ctx context.Context, org, repoID string) (CreateOutcome, error)
// GetRepoByURL returns the live repo with the given url;
// ErrNotFound when absent or soft-deleted.
GetRepoByURL(ctx context.Context, org, url string) (Repo, error)
// GetRepoByID returns the live repo with the given server-generated
// RepoID; ErrNotFound when absent or soft-deleted. Used by the git
// smart-HTTP path, whose URLs address repos by repo_id (docs/explanation/git-protocol-and-namespaces.md) while
// JWT `repo` claims carry the url (docs/reference/authentication.md).
GetRepoByID(ctx context.Context, org, repoID string) (Repo, error)
// UpdateRepoDefaultBranch compare-and-swaps the live repo's default
// branch from oldBranch to newBranch (openapi.yaml PATCH repo: the
// update is "a transactional compare-and-swap" against the value the
// caller read). won=false (no error) when the live row's default
// branch is no longer oldBranch — someone else changed it first;
// ErrNotFound when no live repo matches. A won swap also clears the
// upstream's DefaultBranchOmitted: an explicit choice wins (#599).
UpdateRepoDefaultBranch(ctx context.Context, org, url, oldBranch, newBranch string) (won bool, err error)
// SoftDeleteRepo sets deleted_at and returns the affected repo.
// ErrNotFound when no row exists; ErrAlreadyDeleted when already
// soft-deleted.
SoftDeleteRepo(ctx context.Context, org, url string) (Repo, error)
// ListRepos pages live repos for the org per ListParams.
ListRepos(ctx context.Context, org string, params ListParams) (ListResult, error)
// AddSigningKey registers a commit-signing key for the org and returns
// the stored row (ID for later removal). keyText must be non-blank; see
// SigningKey for the accepted formats (callers validate before adding).
AddSigningKey(ctx context.Context, org, keyText string) (SigningKey, error)
// SigningKeysForOrg returns the org's registered signing keys in
// registration order. Empty slice when none are registered — the
// verify-sig hook reads this fresh from the store on every push and
// rejects on an empty keyring.
SigningKeysForOrg(ctx context.Context, org string) ([]SigningKey, error)
// RemoveSigningKey deletes the signing key with the given ID.
// ErrNotFound when the org is unknown or the ID does not belong to it.
RemoveSigningKey(ctx context.Context, org string, id int64) error
// EventingStore is the phase-5 webhook / git-credential / repo-upstream
// persistence surface (eventing.go).
EventingStore
}
Store is the metadata persistence contract. All org-scoped methods key by org name (the JWT `iss` claim); implementations resolve it internally and return ErrNotFound for unknown orgs.
type SyncClaimSink ¶
type SyncClaimSink interface {
// SetSyncClaim installs holder as repoID's fence token, superseding
// any previous token (the caller won the meta claim lease).
SetSyncClaim(ctx context.Context, repoID, holder string) error
// ClearSyncClaim removes repoID's fence token iff it still belongs
// to holder.
ClearSyncClaim(ctx context.Context, repoID, holder string) error
}
SyncClaimSink mirrors inbound sync-claim transitions into the ref authority so CASRef's §2.8 fence reads the holder from the object it CASes. ref.S3WAL implements it (SetSyncClaim/ClearSyncClaim); SQL modes run without one (nil sink = no-op) because their fence reads the shared repo_upstreams row directly.
type SyncOutcome ¶
type SyncOutcome struct {
Kind LaneOutcome
Success bool // Done: counts a successful run
Delay time.Duration // Restore: due = now + Delay
}
SyncOutcome describes how a pull run ends (Done or Restore).
type Webhook ¶
type Webhook struct {
ID string // server-generated "wh_<16 hex>"
OrgID int64
URL string
Secret string
CreatedAt time.Time // UTC
}
Webhook is one registered delivery endpoint (docs/reference/webhooks.md). Secret is the HMAC signing key — plaintext at this interface; at rest it is stored plaintext by default and envelope-encrypted when the store carries a secrets codec (issue #54, OPENSTORAGE_SECRETS_KEY; see secrets.go). Never log Secret.