Documentation
¶
Overview ¶
commitgraph.go refreshes the split commit-graph of a cache repo after a full hydrate (issue #307, ADR-47). The graph is rebuildable cache state: read-side history walks (`git log -- <path>`, `--topo-order`) use it when present and fall back to parsing commit objects for anything it does not cover, so every failure here is logged and dropped, never returned to a request.
Git LFS Batch API (docs/explanation/git-protocol-and-namespaces.md): the storage host serves the standard LFS discovery path POST /{repo}.git/info/lfs/objects/batch on the same JWT-authenticated remote — no separate LFS server or config. Object bytes live in the BlobStore under blob.LFSKey(repoID, oid); transfers use the "basic" adapter.
DOWNLOADS use presigned S3 URLs when the backend supports them, falling back to proxying through this server (fileblob local mode) via GET /{repo}.git/info/lfs/objects/{oid}. Reads are never quota-gated.
UPLOADS always go through the verified proxy transfer (PUT /{repo}.git/info/lfs/objects/{oid}, issue #51): a presigned PUT binds neither content length nor digest with the current signer (gocloud.dev SignedURLOptions cannot sign Content-Length), so only the server-verified path can anchor the storage-quota ledger and the sha256 content address. Upload admission is a durable reservation (ref.ReservePackBytes) charged at batch time on the DECLARED size and re-charged at PUT time on the request's ACTUAL Content-Length (the declared-size-lie gate); a verified upload settles the reservation into the usage counter via ref.RegisterLFSObject, which is also the registry the LFS orphan sweep classifies against.
Package-level helper for throttled (429) responses. Upstream's OpenAPI documents TWO headers on every 429: Retry-After and RateLimit (issue #150). RateLimit is the current IETF draft shape (draft-ietf-httpapi-ratelimit-headers, §4): a structured-field service limit Item — a quoted quota-policy name with parameters r (available quota) and t (effective window in seconds). On a throttled response r is by definition 0, and t mirrors Retry-After's delay-seconds: the draft (§6) requires Retry-After to reference a point no earlier than the end of the effective window, and equal values are the exact boundary (the draft's own throttled-response example, appendix B.1.4, does the same).
The policy name is opaque to clients: upstream deliberately advertises no RateLimit-Policy field ("the gateway's capacity is dynamic"), so the identifier here is the draft's canonical example name "default".
This helper is the single source of truth for the wire shape across both serving surfaces: the LFS busy/backpressure 429s in this package and the api package's rate-limiter middleware and busy handlers (the api package imports githttp; the reverse edge would cycle).
Package githttp is the git smart HTTP surface (docs/explanation/git-protocol-and-namespaces.md): stateless HTTP nodes with Postgres as ref authority, S3/MinIO as immutable pack storage, and local disk as a rebuildable cache. Authentication follows docs/reference/authentication.md: customer-signed JWTs verified against the issuing org's registered keys.
Index ¶
- Constants
- Variables
- func BaseRepo(repo string) string
- func EnsureCacheRepo(cacheDir, repo string) (string, error)
- func EnsureCacheRepoCtx(ctx context.Context, cacheDir, repo string) (repoPath string, retErr error)
- func HeadTarget(defaultBranch string) string
- func HoldRef(ctx context.Context, cacheDir, repo, refName string) (release func(), err error)
- func Hydrate(ctx context.Context, st ref.RefStore, bl blob.BlobStore, ...) (repoPath string, refsErr error, err error)
- func HydratePacks(ctx context.Context, st ref.RefStore, bl blob.BlobStore, repoPath, repo string) error
- func MaterializePack(ctx context.Context, bl blob.BlobStore, repoPath string, p ref.Pack) error
- func NewHookLogHandler(stderr io.Writer, opts *slog.HandlerOptions) slog.Handler
- func Refuse(err error) error
- func RepoPath(cacheDir, repo string) string
- func RunHook(ctx context.Context, args []string) error
- func SetRateLimited(h http.Header, retryAfterSecs int)
- func ShownToPusher(err error) bool
- func ValidRepoName(repo string) bool
- func ValidateCacheDir(cacheDir string) error
- func WaitForPushPublication(ctx context.Context, cacheDir, repo string) error
- type Server
Constants ¶
const ( EnvRepo = "OPENSTORAGE_GIT_REPO" // EnvSyncClaim carries the loopback sync push's claim-holder token // (auth.Claims.SyncClaim, issue #86): the update hook refuses the ref // CAS when the repo's sync claim is no longer held by this holder, // fencing a stale run's in-flight force push out of the ref store. EnvSyncClaim = "OPENSTORAGE_SYNC_CLAIM" EnvRefsPolicy = "OPENSTORAGE_GIT_REFS_POLICY" EnvOps = "OPENSTORAGE_GIT_OPS" // EnvScopes carries the pushing token's scope list (JSON array) into // the hook subprocess. The update hook itself never consults it; the // hook-side forward_on_push poke does (issue #90): auto-forwarding // exports every local head/tag upstream, so a write-only token's push // must not trigger it (the same git:read gate the explicit // push-upstream route enforces). EnvScopes = "OPENSTORAGE_GIT_SCOPES" // EnvOrg names the org whose signing keys verify-sig loads. Shared with // the serve config on purpose: the server sets it explicitly for hooks. EnvOrg = "OPENSTORAGE_ORG" // EnvPushHold carries the absolute path of this push's hold file // (issue #170, hydrate.go push-hold machinery). pre-receive reserves // every ref of the push in it before any update-hook CAS can run; // concurrent full syncs skip held refs so they cannot race // receive-pack's own local update of a ref it just committed, and // D/F-conflicting concurrent pushes serialize on the reservation. // Minted per request by the server and stripped from inherited env // like the other per-push inputs. EnvPushHold = "OPENSTORAGE_GIT_PUSH_HOLD" // EnvAtomic marks the push `--atomic` (issue #28). githooks(5) exposes // no atomic indicator, so the HTTP layer sniffs the wire capability // (atomic.go) and sets this; pre-receive then evaluates the policy // across ALL commands and denies before publication when any command // is denied — git would abort every ref on the first update-hook // denial anyway. EnvAtomic = "OPENSTORAGE_GIT_ATOMIC" EnvTestMode = "OPENSTORAGE_TEST_MODE" EnvCrash = "OPENSTORAGE_TEST_CRASH" EnvHookDelay = "OPENSTORAGE_TEST_HOOK_DELAY_MS" // EnvPushDrainMS shrinks the post-copy drain read deadline // (pushDrainTimeout) so the e2e lane can prove hook phases outlive it // (issue #141's receive-pack sibling); ignored outside test mode. EnvPushDrainMS = "OPENSTORAGE_TEST_PUSH_DRAIN_MS" // EnvCASDrop/EnvCASDropHard hold a marker-file path; the first update // hook to create the marker lets its CAS statement commit, then reports // a synthetic connection-drop error (issue #2's ambiguous outcome). // The hard variant also fails the recovery recheck read, as a truly // dead connection would. Later invocations pass through untouched. EnvCASDrop = "OPENSTORAGE_TEST_CAS_DROP" EnvCASDropHard = "OPENSTORAGE_TEST_CAS_DROP_HARD" // EnvUpdateBarrier holds a handshake-file path prefix; the first update // hook to create <path>.ready pauses AFTER its authority CAS commits // and before the hook exits — i.e. before receive-pack performs its own // local ref update — until the test creates <path>.release (issue #170: // this is exactly the window a concurrent hydrate can full-sync the // just-committed ref into the shared cache repo). Later invocations // pass through untouched. EnvUpdateBarrier = "OPENSTORAGE_TEST_UPDATE_BARRIER" // EnvUpdateBarrierEach is EnvUpdateBarrier's per-command variant: a // handshake-path PREFIX at which EVERY update hook pauses after its // committed authority CAS, each under a ref-derived suffix (issue // #186 finding 5: the coalescing e2e lane interleaves advertisements // BETWEEN a fanout's hook CASes, which the one-shot barrier cannot // stage). EnvUpdateBarrierEach = "OPENSTORAGE_TEST_UPDATE_BARRIER_EACH" // EnvPreReceiveBarrier holds a handshake-file path prefix; the first // pre-receive to create <path>.ready pauses AFTER pack publication // and BEFORE the deferred hold reservation (Codex round-23 security // P2: exactly the window in which a concurrent same-name or // D/F-relative push must be able to proceed) until the test creates // <path>.release. Later invocations pass through untouched. EnvPreReceiveBarrier = "OPENSTORAGE_TEST_PRERECEIVE_BARRIER" // CrashHeader lets tests request an EnvCrash injection per push; the // server refuses it outside test mode. CrashHeader = "X-Openstorage-Test-Crash" )
Env names used to route request context from the HTTP handler into the git hooks (which re-exec this binary). OPENSTORAGE_TEST_* fault-injection knobs are honored only when OPENSTORAGE_TEST_MODE=1.
const EnvHookErrors = "OPENSTORAGE_GIT_HOOK_ERRORS"
EnvHookErrors carries the path of this push's hook-errors file. git relays a hook's stderr and its error to the pusher, and a store error names the database host, user and name, or the S3 endpoint and bucket. So the hooks write such errors to this file instead, and serve logs them for the operator after receive-pack ends. serve creates the file in the push's private state directory and strips the variable from inherited env like the other per-push inputs.
const EnvPackQuota = "OPENSTORAGE_GIT_PACK_QUOTA"
EnvPackQuota carries the resolved per-repo pack-bytes quota from the serve process into the pre-receive hook (set only when the quota is enabled).
const EnvPackReservationTTL = "OPENSTORAGE_GIT_PACK_RESERVATION_TTL"
EnvPackReservationTTL carries the resolved reservation TTL (issue #29, a Go duration string) from the serve process into the pre-receive hook. Set alongside EnvPackQuota; the hook falls back to defaultReservationTTL when it is absent (a serve binary predating the ledger), so quota enforcement never silently weakens on skew.
const EnvUsageFloor = "OPENSTORAGE_GIT_USAGE_FLOOR"
EnvUsageFloor carries the repo's UNCOUNTED-bytes admission floor (the packs + LFS listing minus the usage counter at scan start — Codex round 44) from the serve process's shared single-flight cache into the pre-receive hook (Codex round 21): the hook is a fresh process per push, so a hook-side listing cannot be cached across pushes and re-scans both prefixes every time — ~100 pages per push at the LFS object cap. Set alongside EnvPackQuota as "<bytes>:<unix mint time>" (Codex round 30): the value is minted at receive-pack spawn but the hook runs only after the client's transfer, whose duration the CLIENT controls, so the hook self-lists when the floor has aged past hookFloorMaxAge instead of trusting it — beyond the reservation TTL, orphaned uploads whose reservations expired are visible only to a listing.
const EnvVerifySigBudget = "OPENSTORAGE_GIT_VERIFY_SIG_BUDGET"
EnvVerifySigBudget carries the resolved per-push verified-commit budget (issue #59) from the serve process into the hooks. Set only when the budget is enabled (> 0); malformed values fail closed like EnvPackQuota.
const EnvVerifySpawnLog = "OPENSTORAGE_TEST_VERIFY_SPAWN_LOG"
EnvVerifySpawnLog is a test-mode-only knob (EnvCASDrop pattern): when set, every actual `git verify-commit` spawn appends its sha to the named file, so tests can assert exact spawn counts across the pre-receive gate and every update hook of one push.
const EnvVerifyState = "OPENSTORAGE_GIT_VERIFY_STATE"
EnvVerifyState carries the per-push verify-sig state file path from the serve process into the hooks. The file is created empty by the server before receive-pack starts and removed when the request ends; hooks append verdict lines as they verify. Absent (a serve binary predating issue #59, or a bare hook invocation) the hooks fall back to an in-process cache — correctness is unchanged, only cross-process reuse is lost.
const MinPackReservationTTL = usageCacheTTL + UsageFloorHandoffMargin
MinPackReservationTTL is the exclusive lower bound serve's config enforces on OPENSTORAGE_PACK_RESERVATION_TTL (Codex rounds 36+42 on #51): the reservation TTL must STRICTLY exceed the usage-floor cache lifetime plus the floor-to-reservation handoff margin. CachedUsageFloor feeds LFS and REST admission directly with a listing up to usageCacheTTL stale, and the subsequent reserve call takes up to the margin; with a TTL at or below their sum, an orphaned upload's reservation (published, registration failed) can expire while a floor listed BEFORE the publish still drives an admission — the writer then sees neither the orphaned bytes nor a live reservation and is admitted into headroom that no longer exists. The hook path tolerates such TTLs by self-listing (hookUsageFloor's maxAge goes non-positive), but the direct cache consumers have no fresher source, so the configuration is rejected at startup instead.
const UsageFloorHandoffMargin = 30 * time.Second
UsageFloorHandoffMargin is the freshness headroom the reservation TTL must leave BEYOND the floor cache lifetime (Codex round 42 on #51): a floor can be served at the very end of its usageCacheTTL window and then sit while ReservePackBytes waits on the database, so the interval from scan start to the admission snapshot is floor age plus handoff time. Charging reserve calls are deadline-bounded by this margin, which makes the chain airtight: an orphan published after its page was scanned (publish p > scan start t0) had its reservation refreshed at the pre-publish admission, so it gates until p+TTL, while any admission using a floor from t0 snapshots by t0 + usageCacheTTL + margin < p+TTL. The reservation stamps and the snapshot share the DATABASE clock, so only the duration — not absolute clocks — must agree.
Variables ¶
var ErrHeldRefStale = errors.New("a push holds a ref whose cached value is older than the authority")
ErrHeldRefStale is the refsErr of a strict hydrate whose ref sync skipped a push-held ref: the cache keeps a value the authority no longer has (a sync only touches refs whose values differ), and the hold may belong to a push whose CAS fails, so nothing would heal it before the caller acts on the stale value.
var ErrPackMissing = errors.New("githttp: registered pack missing from blob store")
ErrPackMissing is returned (wrapped, match with errors.Is) when a registered pack's blob stays missing after maxHydrateRaces listing passes: the ref store references bytes that do not exist, which is corruption grade — refs may point at unreachable objects.
var ErrPushPublicationUnverified = errors.New("push publication wait: live push holds could not be checked; publication stays held")
ErrPushPublicationUnverified marks a WaitForPushPublication that could not prove every live push hold released: a hold or its tombstones were unreadable, or checking them would exceed the wait's resource bounds (issue #331). Publication must then be treated as still pending.
var ErrRefHeld = errors.New("a concurrent push or ref write holds the ref")
ErrRefHeld marks a push-hold reservation that gave up after pushHoldWaitMax because a concurrent push or ref write still holds a conflicting ref, or the reservation lock. It is transient: the API answers a ref write that fails with it with 503 and Retry-After.
var WrapHookRefStore = func(ctx context.Context, st ref.RefStore) (ref.RefStore, error) { return ref.IdempotentCAS{RefStore: st}, nil }
WrapHookRefStore decorates the ref store the update hook opens for its CAS. Hooks run in a fresh `openstorage hook` subprocess (git execs the installed hook script), so the serve process's wired store cannot reach them; instead the cmd wiring seam — which runs in the hook subprocess too, it is the same binary — replaces this variable at startup with the events.RefStore decorator so git pushes emit push events (docs/reference/webhooks.md) exactly like API writes (that decorator wraps ref.IdempotentCAS itself). The eventless default keeps hooks working before/without eventing wiring, wrapping just the CAS-recovery decorator so ambiguous outcomes still reconcile. An error refuses the ref update before its CAS (issues #445, #466: the metadata document is in a format this binary cannot write events to, or corrupt). When the hook cannot wire its eventing, the cmd wiring returns a store that refuses every update that could record forward intent and passes the rest, still through ref.IdempotentCAS (cmd/openstorage intentlessRefStore, ADR-33's amendment).
Functions ¶
func BaseRepo ¶
BaseRepo strips the optional namespace suffix: JWTs carry the base repo id in the `repo` claim and are valid for its namespaces (docs/explanation/git-protocol-and-namespaces.md URL scheme).
func EnsureCacheRepo ¶
EnsureCacheRepo idempotently creates the bare cache repo with the server config and hooks that re-exec this binary (no shell/psql deps in hooks).
Config and hooks are verified on every call, not only at init time: a crash between `git init` and hook installation (or a deleted/tampered hook file) must never leave a repo whose receive-pack silently runs without the hooks that own S3 upload, PG pack/ref registration, and policy enforcement. The hook script also embeds this binary's path, which changes across deployments.
The steady-state verification is memoized (issue #93): the full pass — two `git config --get` forks plus per-hook ReadFile/Stat — used to run on every request and dominated the hydration fast path (~93% of it, PR #97 baseline). Every mutator of the verified state is our own code (`git init` and the `git config` writes below rewrite config via lockfile + rename; ensureHook rewrites hooks via temp + rename; nothing in this codebase deletes cache repos), so a fingerprint of the exact file states the last full pass in THIS process verified proves them unchanged without forking — see cacheRepoMemo.
func EnsureCacheRepoCtx ¶
EnsureCacheRepoCtx is EnsureCacheRepo honoring ctx in its lock wait and git subprocesses (init, config): the repack fold's materialization runs the setup under the fold's wall-clock budget, and a stalled subprocess or filesystem must not outlive it.
func HeadTarget ¶
HeadTarget qualifies a repo's recorded default branch into the HEAD symref target its cache repo advertises (openapi.yaml PATCH repo: `main` becomes `refs/heads/main`; a value that already starts with `refs/` is used verbatim).
func HoldRef ¶ added in v0.20.0
HoldRef takes a push hold on one ref of repo's cache for a write that changes the ref outside receive-pack: a REST write on the local schedule (ADR-50) holds the ref from before it records its forward intent until its authority CAS returns. While the hold is live, WaitForPushPublication blocks and a strict sync fetch refuses the ref, so a push run that claims the intent cannot snapshot the ref before the CAS. A conflicting hold is waited out as for a push (pushHoldWaitMax), then HoldRef fails with an error that matches ErrRefHeld. release stops the refresh and removes the hold; the TTL machinery covers a process that dies first.
func Hydrate ¶
func Hydrate(ctx context.Context, st ref.RefStore, bl blob.BlobStore, cacheDir, repo, defaultBranch string) (repoPath string, refsErr error, err error)
Hydrate rebuilds the cache repo state for repo from PG + S3 (the stateless node property): ensure repo, hydrate packs, then sync refs. On the receive path a concurrent receive-pack may hold ref locks while we sync; retry briefly, and let the caller decide whether a residual failure is fatal (upload path) or tolerable staleness (receive path, where git re-validates old SHAs itself).
Fast path (issue #4): when the store generation AND the HEAD symref target derived from defaultBranch (issue #18) match the local marker written by the last fully successful hydrate — and the marker is within its TTL — the packs, refs, and HEAD are provably unchanged and Hydrate returns after one point-read, skipping ListPacks, per-pack stats, the paged ListRefsAfter scan, and the for-each-ref/update-ref/ symbolic-ref subprocesses. EnsureCacheRepo still runs unconditionally: hook and config repair must stay verified on every request (the hook script embeds this binary's path, which changes across deployments), and it touches no DB state. After the first full pass its verification is memoized fingerprint stats (issue #93), so the fast path as a whole forks zero subprocesses.
defaultBranch is the repo's default branch as resolved from the meta row for THIS request. A PATCH /repo landing after that read can transiently lose to this hydrate's older value (each full sync records the HEAD it actually set, so the marker never lies), and the next request — which resolves the row fresh — repairs HEAD with one full sync.
Full syncs serialize per repo on <repoPath>/codestorage.hydrate.lock: two overlapping full hydrates spanning a mutation could otherwise interleave so the OLDER ref sync lands after the newer one while the newer marker survives — and the marker would then short-circuit reads onto stale refs for up to a TTL window. Under the lock, each full sync re-reads the generation before listing refs and publishes its marker before releasing, so marker and refs advance monotonically together. The marker read stays lock-free (temp+rename keeps it atomic): the idle fast path must not serialize on an exclusive lock.
func HydratePacks ¶
func HydratePacks(ctx context.Context, st ref.RefStore, bl blob.BlobStore, repoPath, repo string) error
HydratePacks downloads and indexes every registered pack that is not already present in the cache repo. Packs are self-contained (index-pack --fix-thin ran on the receiving node before upload), so each is independently indexable.
Concurrent hydrators (serve requests, re-exec'd hooks) race on the same cache dir, so each pack is materialized under a per-pack flock: download to a uniquely-named temp file, index the temp, then rename pack+idx into place. Only our own temp names are ever removed, and "idx appeared while waiting for the lock" counts as success.
A download hitting blob.ErrNotFound restarts the pass on a fresh ListPacks (bounded by maxHydrateRaces): the pack was folded away by a concurrent repack and the new listing carries its replacement. Restarts are cheap and idempotent — hydrated packs are skipped by the .idx probe. Transport errors keep failing fast; on bound exhaustion the hydrate fails with ErrPackMissing rather than looping.
func MaterializePack ¶
MaterializePack materializes ONE registered pack into repoPath — the bounded, cancelable alternative to HydratePacks for callers that must not touch any other pack (the issue-#1 repack fold selects its input under byte/object budgets BEFORE downloading; hydrating the whole repo first would defeat them). Safe against concurrent hydrators of the same cache repo: identical per-pack lock, temp naming, and rename ordering. A download hitting blob.ErrNotFound propagates (wrapped, match with errors.Is): for a fold it means a concurrent repack already replaced the snapshotted rows.
func NewHookLogHandler ¶ added in v0.20.0
NewHookLogHandler returns the slog handler of a hook process. stderr (which reaches the pusher) gets each record with only the attributes that pusherAttr allows. The hook-errors file (EnvHookErrors) gets the whole record. Without that file, as for a hook that runs outside a push, the other attributes are dropped: they never fall back to stderr.
func Refuse ¶ added in v0.20.0
Refuse marks err as a refusal whose whole text the pusher may read. The caller vouches that the text names no store or transport detail: build it from fixed text, ref names, SHAs and numbers, never from a store error. errors.Is and errors.As still see err.
func RunHook ¶
RunHook dispatches `openstorage hook <pre-receive|update> [args...]`, invoked by git via the scripts EnsureCacheRepo installs. Arguments and stdin come from outside this process, so malformed input must produce an error (stderr message + non-zero exit denies the push), never a panic. git shows the pusher the returned error, so an error that is not a refusal (ShownToPusher) becomes a fixed line, and its text goes to the push's hook-errors file for the operator (hookFailure).
func SetRateLimited ¶
SetRateLimited stamps the two throttling headers carried by every HTTP-level 429 the server emits. retryAfterSecs is the delay-seconds value (>= 1, the Retry-After granularity).
func ShownToPusher ¶ added in v0.20.0
ShownToPusher reports whether RunHook returns err to git, which shows it to the pusher, unchanged: err is a refusal or one of pusherSafeSentinels, or it wraps only errors that are. A wrapper adds only its own format text, so a wrapper must not format a store error with %v next to a %w.
func ValidRepoName ¶
ValidRepoName guards against path traversal via the URL repo element.
func ValidateCacheDir ¶
ValidateCacheDir rejects cache locations too deep for the push-hold bookkeeping this package materializes beside each repository (Codex round-25 P2): the deepest auxiliary path — a hold's unreserved-marker sidecar entry — needs more room than git's own files, so a cache root that leaves hydration and git ref locking working can still fail every push with an opaque mid-push error. Validated once, where the operator's choice is accepted (serve boot), against the target OS PATH_MAX with a generous repo-component budget (repo directory names are server-generated and short); the per-request mint stays the defensive backstop for anything that slips through.
func WaitForPushPublication ¶
WaitForPushPublication blocks until no live push hold in repo's cache still names an unreleased ref: every accepted authority update that a hold protected has published its cache ref (or its push has exited). A caller that has already claimed its durable intent (issue #266) uses it to avoid acting on a cache ref an in-flight push has not yet published. Nothing on main calls it yet.
Result contract:
- nil: no live, unreleased held name remained at the last poll.
- ctx.Err() (unwrapped) on cancellation or deadline.
- otherwise an error wrapping ErrPushPublicationUnverified: a hold was unreadable or exceeded the resource bounds. Publication was NOT proven. The caller must not start the protected work, and must keep its durable intent, force flag, and retry scheduling exactly as for any failed wait; a later retry re-checks from scratch. The case is also logged here.
Bounds (defaultPublicationLimits): holds are streamed, never collected or sorted. Each poll reads a hold's tombstone sidecar into a bounded set, then its lines until the first live name. Bytes are charged as they are read, so file growth after the listing cannot exceed the budget (at most one probe byte past it), and each record is charged before it is retained. A record longer than pushRefPathMax fails closed. Verdicts are cached per hold by (size, mtime) of the hold and its sidecar, so only changed holds are reread.
Types ¶
type Server ¶
type Server struct {
Store ref.RefStore
Blob blob.BlobStore
CacheDir string
// Verifier authenticates docs/reference/authentication.md JWTs (multi-algorithm, iss-resolved
// org keys, exp/sub checks). Any Verify failure maps to 401.
Verifier auth.Verifier
// Org is the single organization this node serves. A valid token from a
// *different* registered org must not grant access (403); unknown orgs
// already fail key resolution inside Verifier (401).
Org string
// TestMode gates fault-injection knobs (crash header). Only ever set
// from OPENSTORAGE_TEST_MODE=1; the crash header is refused otherwise.
TestMode bool
// ResolveRepo maps a base path element (docs/explanation/git-protocol-and-namespaces.md: repo_id, but ADR-21
// accepts name-style urls too) to its live repo record. ok=false when
// no live repo matches — both handlers then answer 404 (repos must be
// created via POST /api/v1/repos first). The record's RepoID is the
// only storage key (repo-key-split fix); it also lets tokens carrying
// either the url or the repo_id claim (docs/reference/authentication.md) authenticate. When
// nil, every path element is unresolvable.
ResolveRepo func(ctx context.Context, base string) (meta.Repo, bool)
// MaybeRepack, when set, runs after every receive-pack POST with the
// resolved storage key (issue #1): pre-receive registers pushed packs
// strictly before receive-pack returns, so this is the git surface's
// post-RegisterPacks repack-threshold trigger. It must be quick
// (a pack-count read plus at most one job enqueue) and must never
// block on the fold itself. Nil disables triggering.
MaybeRepack func(ctx context.Context, repo string)
// PackQuotaBytes is the per-repo pack-bytes quota (issue #21, see
// quota.go); <= 0 disables quota enforcement. Set by the serve wiring
// from the resolved config, like ResolveRepo.
PackQuotaBytes int64
// PackReservationTTL bounds how long a crashed push's quota
// reservation gates admission (issue #29, ref/ledger.go). Set by the
// serve wiring; <= 0 leaves the hook on defaultReservationTTL.
PackReservationTTL time.Duration
// BaseURL is the externally reachable base URL (scheme://host, no
// trailing slash) LFS proxy transfer hrefs are built on (issue #51:
// uploads always proxy, so behind a reverse proxy the href must be
// the canonical external URL — the backend request's Host header may
// be internal). Set by the serve wiring from $OPENSTORAGE_BASE_URL
// (or the bound listener); when empty, hrefs fall back to the
// request's Host and X-Forwarded-Proto.
BaseURL string
// MaxPushBytes caps the DECODED bytes one receive-pack request may
// stream into git (issue #55, pushcap.go); <= 0 disables the cap. Set
// by the serve wiring (default: a small multiple of the pack quota).
MaxPushBytes int64
// VerifySigBudget caps how many commits one push may signature-verify
// (issue #59, verifystate.go); <= 0 disables the budget. Set by the
// serve wiring.
VerifySigBudget int
// LFSStagingBytes bounds the aggregate bytes of proxy LFS uploads
// staging on this replica at once (issue #51, Codex round 4): uploads
// stage their full Content-Length under $TMPDIR before verification,
// and the per-repository quota is neither node-wide nor guaranteed
// enabled, so without this budget concurrent distinct-key uploads
// could exhaust a replica's disk. Set by the serve wiring
// ($OPENSTORAGE_LFS_STAGING_BYTES, default 10 GiB); <= 0 disables the
// bound (the operator's explicit opt-out).
LFSStagingBytes int64
// LFSV020WriteCompat keeps the v0.2.0 LFS read surface alive during a
// rolling upgrade (ADR-41, Codex round 80): old replicas derive only
// the legacy single-key spelling, so a mixed-version fleet would 404
// objects that exist only canonically. When set (serve wiring,
// $OPENSTORAGE_LFS_V020_WRITE_COMPAT), every verified publication
// also copies the bytes to the legacy key — rowless copies the orphan
// sweep reclaims after the flag is dropped post-roll. Off by default:
// the canonical spelling is the secure one, and the compat copies are
// exactly as forgeable as everything v0.2.0 served.
LFSV020WriteCompat bool
// contains filtered or unexported fields
}
Server is one stateless HTTP node: all durable truth lives in PG (refs, pack registry) and S3 (pack bytes); CacheDir is rebuildable at any time.
func (*Server) CachedUsageFloor ¶
CachedUsageFloor returns repo's UNCOUNTED-bytes admission floor — the listing minus the usage counter at that listing's scan start, clamped at zero (Codex round 44: ReservePackBytes SUMS this floor with the live counter, so settlements after the scan count exactly once; the old max() combination hid them whenever pre-registry bytes coexisted with fresh settlements). Serves a ≤usageCacheTTL-stale cached scan and coalesces concurrent misses into one flight (issue #51, Codex round 15: fresh per-transfer listings amplify — a 100-object batch cost ~201 full prefix scans). It feeds every admission surface: LFS transfers here, the pre-receive hook via its spawn env (EnvUsageFloor), and REST pack writers via api.Config.UsageFloor (Codex round 21 — per-push self-listings in those paths re-amplify at the LFS object cap). Staleness is safe FOR THE FLOOR: bytes registered since the scan are in the live counter, in-flight uploads in their reservations, regardless of this value; the floor exists for pre-ledger bytes (fixed) and orphaned uploads, whose admission visibility this cache delays by at most its TTL — a bounded stretch of the ledger's documented orphan-window overshoot. Unlike peekUsage (observational, never blocks), admission needs a value: a cold entry lists inline, concurrent cold callers share the leader's flight.
func (*Server) CachedUsageFloorGen ¶
CachedUsageFloorGen is CachedUsageFloor plus the shared floor generation the served value was validated against — the token admission callers forward into ReservePackBytes' transactional recheck (Codex round 60).
func (*Server) Handler ¶
Handler routes the smart-HTTP and LFS endpoints per repo (docs/explanation/git-protocol-and-namespaces.md):
GET /{repo}.git/info/refs?service=git-{upload|receive}-pack
POST /{repo}.git/git-{upload|receive}-pack
POST /{repo}.git/info/lfs/objects/batch
GET /{repo}.git/info/lfs/objects/{oid} (proxy transfer, local mode)
PUT /{repo}.git/info/lfs/objects/{oid} (proxy transfer, local mode)