api

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 12, 2026 License: Apache-2.0 Imports: 40 Imported by: 0

Documentation

Overview

Named-repo route aliases /api/repos/{repo_name}/... (docs/README, openapi.yaml): a thin routing layer over the unchanged /api/v1 handlers, identical bodies by construction. Contract (frozen by the phase-6 foundation):

  • One alias per repo-scoped v1 route, registered with the SAME auth flavor and scopes as its v1 twin. Org-scoped routes (GET/POST /api/v1/repos list+create, webhooks) get NO alias.
  • Handler shape: decode r.PathValue("repo_name") (Go 1.22 mux matches the escaped path, so "pierre%2Fexample" stays one segment) → meta.ResolveRepoKey for both the path name and the token's repo claim → unknown name or RepoID mismatch ⇒ 404 (error envelope on read routes, result envelope on write routes) → rewrite a cloned *http.Request to the v1 path (query preserved) and re-dispatch through mux. Double JWT verification is accepted.
  • Every alias is registered through rl.limit(flavor) BEFORE auth, exactly like the v1 routes. The re-dispatch runs the v1 chain again, so the rewritten request carries the rateLimitExempt context marker: the v1 twin's limiter passes it through and each aliased request is charged exactly one token (unlike the accepted double JWT verification, double-charging would halve the effective alias rate).

Package api implements the SDK-compatible HTTP API surface (docs/reference/http-api.md): repo CRUD under /api/v1 with the {"error": "..."} envelope, JWT bearer auth (docs/reference/authentication.md), and panic-recovery/request-log middleware.

The package depends only on the frozen phase-1 contracts meta.Store and auth.Verifier; wiring concrete implementations is the serve command's job.

Synchronous repo forking (docs/explanation/domain-model.md invariant 12, docs/reference/http-api.md POST /repos, phase-5 workstream E):

POST /api/v1/repos  repo:write  base_repo {provider: "code"|"code.storage",
                                operation: "fork", auth: {token}} -> one-shot fork
POST /api/v1/repos  repo:write  base_repo {name, ref?, auth: {token}}
                                (no provider/operation: upstream's copy form) -> the same fork

A fork is a deterministic, independent clone performed inside the request (ADR-31): the fork-point closure is packed from the source's materialized cache, published under the NEW RepoID's storage keys (ADR-21: RepoID is the only storage key), and the fork's default branch is CAS-created. No repo_upstreams row is written — a fork is one-shot and fully independent of its source.

api.New (api.go) wires this dispatcher on POST /api/v1/repos with the base_repo-sync-aware creator from handlers_sync.go ((*syncServer).handleCreateRepo) as the fallback; fork_test.go builds the same stack with the frozen (*server).handleCreateRepo as the fallback.

Core git:read endpoints (docs/archive/phases/07-implementation-roadmap.md p3 phases 2a/2b): refs, commits, files, and conditional/range file streaming per docs/reference/http-api.md + docs/explanation/domain-model.md invariants 2 (root commits), 3 (unborn branch 404), 9 (commits newest-first), and 10 (annotated tags listed with the peeled commit sha).

Notes read endpoints (docs/reference/http-api.md "Notes"): GET /repos/notes returns the note attached to one object, GET /repos/notes/refs enumerates notes refs from the ref authority. Custom notes refs are always enabled on this server (the write surface accepts them), so the upstream "custom notes refs disabled" 400 has no equivalent here. Helpers carry an "rnotes" prefix (package convention: sibling endpoint files must not collide).

Read-endpoint handlers, query group (docs/archive/phases/07-implementation-roadmap.md p3 phases 2c/2d): commit and branch diffs (GET /repos/diff, GET /repos/branches/diff), blame, grep, and tar.gz archive streaming. Contracts: docs/reference/http-api.md, docs/explanation/domain-model.md (invariants + the "consumers concatenate files[].raw" rule), docs/upstream/openapi.yaml.

All git plumbing goes through internal/gitread over a cache repo materialized per request; handlers own parameter validation, filtering, cursoring, and wire envelopes.

Streaming write endpoints (docs/archive/phases/07-implementation-roadmap.md p4 W2; contract docs/explanation/streaming-commit-protocols.md + docs/reference/http-api.md):

POST /api/v1/repos/commit-pack    — NDJSON: metadata + blob_chunk frames
POST /api/v1/repos/diff-commit    — NDJSON: metadata + diff_chunk frames
POST /api/v1/repos/restore-commit — plain JSON

NDJSON bodies are parsed INCREMENTALLY (docs/explanation/streaming-commit-protocols.md: chunked bodies without Content-Length, uploads can exceed memory): one bounded line at a time, with blob bytes streamed straight into `git hash-object -w --stdin`.

Write pipeline (docs plan decision #5): materialize the cache repo (read-only through alternates), create objects in a per-request scratch odb, then publish in hook order — pack-objects, blob upload, RegisterPacks, CASRef — so a lost CAS race leaves only an unreferenced immutable pack.

Status mapping (plan decision #2): expected_head_sha mismatch -> 412 precondition_failed; CAS race lost at publish time -> 409 conflict; root commits report old_sha = 40 zeros (docs/explanation/domain-model.md invariant 2). Success is 200 (decision #3). Failures carry the docs/explanation/streaming-commit-protocols.md result envelope; the auth middleware's plain {"error"} bodies on 401/403 are SDK-compatible (the SDK falls back to the error envelope + numeric status mapping).

Ephemeral handling: `ephemeral`/`ephemeral_base` address refs under auth.EphemeralRefName (refs/namespaces/ephemeral/...) in the BASE repo storage key — the convention the frozen p3 read surface already uses (GET /repos/branches?ephemeral=true, gitread.ResolveRef), so API-written ephemeral commits are immediately visible to reads and merge previews. Ref policies match against that namespaced name, mirroring the git hook's `+ephemeral` quirk.

All helpers carry a wc ("write commit") prefix to stay collision-free with sibling slice files in this package. Route wiring: api.New (owned by the foundation slice) must call registerWriteCommitRoutes — tests mount it on their own mux until that one-line hookup lands.

Merge write endpoints (docs/archive/phases/07-implementation-roadmap.md p4; contract docs/reference/http-api.md + openapi):

POST /api/v1/repos/merge          — merge source into target (git:write)
GET  /api/v1/repos/merge/preview  — read-only merge preview (git:read)

Merge engine (docs/archive/phases/13-phase4-implementation-notes.md ADR-23): `git merge-base` + `git merge-tree --write-tree -z` (requires git >= 2.38, probed at first use) running in the per-request scratch odb over the materialized cache repo, so every object the merge creates stays request-local until publish. The merge endpoint publishes in the hook order via the landed wcPublish pipeline (pack-objects over the new-tip reachability frontier -> blob upload -> RegisterPacks -> CASRef); a lost CAS race strands only an unreferenced immutable pack. The preview endpoint never publishes anything — the scratch odb is deleted with the request (docs/explanation/domain-model.md invariant 6).

Status mapping (docs/archive/phases/13-phase4-implementation-notes.md ADR-22): a stale expected_target_sha is a 412 `precondition_failed`; merge conflicts, ff_only non-fast-forwards, unrelated histories without allow_unrelated_histories, and CAS races lost at publish time are 409 `conflict` (merge conflicts additionally carry conflict_paths + merge_base_sha, docs/reference/http-api.md).

Fast-forwards move the ref without publishing: refs only ever name objects already durable in registered packs (the source tip was a ref), so there is nothing new to upload. Reachability-based packing publishes ephemeral->persistent promotions for free; `promoted_commits` reports the number of source commits newly reachable from the persistent namespace when the source is ephemeral (fast-forward counts them all, merge-commit/squash outcomes exclude the server-created commit itself).

Ref policies mirror the git update hook against the STORED target ref name (ephemeral targets match their refs/namespaces/ephemeral/... name): no-push denies every update; no-force-push denies non-fast-forwards via merge-base --is-ancestor (merge outcomes are fast-forward by construction — the old tip is a parent); verify-sig fails closed for the unsigned server-created merge/squash commits but passes fast-forwards, whose commits were already verified when pushed; unknown ops fail closed.

All helpers carry a wm ("write merge") prefix to stay collision-free with sibling slice files; shared plumbing (wcResolveRepo, wcNewScratch, wcPublish, wcRefName, validation helpers, mergeBaseSHA) is reused from the landed slices. Route wiring: api.New (owned by the foundation slice) must call registerWriteMergeRoutes — tests mount it on their own mux until that one-line hookup lands.

Branch, tag, and notes write endpoints (docs/archive/phases/07-implementation-roadmap.md p4 W3, docs/reference/http-api.md): POST /repos/branches/create, DELETE /repos/branches, POST/DELETE /repos/tags, POST/DELETE /repos/notes — with default-branch delete protection (docs/explanation/domain-model.md invariant 4, ephemeral exemption per invariant 7), lightweight-tag peeling (invariant 10), notes add/append CAS via expected_notes_ref_sha (invariant 1), and ref-policy enforcement on every write (invariant 5). Aliased request-body fields (issue #17) accept both the current upstream spelling and the deprecated one, the current name winning on conflict.

Routes are registered by api.New via registerWriteRefRoutes; the tests in write_refs_test.go build their own mux through the same helper.

Ephemeral namespace: consistent with the frozen read surface (handleListBranches, gitread.ResolveRef), ephemeral refs are stored under the base RepoID storage key with the auth.EphemeralRefName prefix (refs/namespaces/ephemeral/refs/heads/...). Ref policies match the stored ref name verbatim, which reproduces the hook's ephemeral-namespace quirk.

All helpers carry a "wref" prefix to stay collision-free with sibling write-endpoint slices in this package (same convention as the core* read helpers).

Pre-auth rate limiting (docs/reference/http-api.md: gateway-level 429). This file holds the frozen phase-6 contract: RateLimitConfig on api.Config, a *rateLimiter built once in New, and (*rateLimiter).limit(flavor) applied at every REST route registration site (v1 + named-repo aliases + repo-urls) BEFORE the auth middleware.

Admission is a per-client-IP token bucket (hand-rolled, x/time/rate semantics: float tokens, monotonic refill, burst cap) in a lazy map with a mutex and a periodic stale-entry sweep. The key is the RemoteAddr host — X-Forwarded-For is trusted only from peers inside the operator's explicit TrustedProxies allowlist (issue #30); by default (empty allowlist) it is deliberately ignored. Limiting is per node, in memory: nodes are stateless and per-node limiting is the documented "basic" scope. 429-before-auth means unauthenticated clients consume rate budget keyed by IP — accepted for the basic scope (plan risk note).

GET /api/repo-urls/{repo_id} (docs/reference/http-api.md, openapi get_repo_url_by_id — note: NO /v1 prefix): resolve an internal repository ID to the repository URL clients put in the JWT `repo` claim. Org-level org:read auth; the response is exactly {"repo_id", "repo_name", "url"} (openapi: additionalProperties false; url is the deprecated mirror of repo_name).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func New

func New(cfg Config) (http.Handler, error)

New builds the full HTTP handler: recover -> request log -> router, with per-route JWT auth (scope + org-level enforcement per docs/reference/authentication.md). Invalid operator-supplied config (RateLimit) returns an error instead of panicking (docs/archive/phases/15-phase6-implementation-notes.md deferred #4); nil required dependencies are still programming errors and assert.

func NewGitRateLimit

func NewGitRateLimit(cfg RateLimitConfig) (func(http.Handler) http.Handler, error)

NewGitRateLimit returns the pre-auth rate-limit middleware for the git smart-HTTP surface (issue #21): the same admission policy and config knob as the REST limiter, applied by the serve wiring around the whole githttp handler. Bucket state is intentionally separate from the REST limiter's — git transfers are far heavier per request, so one surface exhausting its budget must not starve the other. The configured burst is raised to gitMinBurst so a single protocol exchange can always complete. 429 responses carry the Retry-After + RateLimit pair and the git surface's plain {"error"} envelope (git clients surface the status line, not the body). The zero config yields the identity middleware; invalid config is a startup error, mirroring newRateLimiter.

func RegisterWebhookRoutes

func RegisterWebhookRoutes(mux *http.ServeMux, verifier auth.Verifier, org string, store events.WebhookStore, guard events.TargetGuard, logger *slog.Logger, rl *rateLimiter)

RegisterWebhookRoutes mounts the webhook CRUD routes on mux. All three routes are org-level (docs/reference/authentication.md: `repo` claim == "org") and require repo:write, with the plain {"error"} envelope on failures — these are management routes, not docs/explanation/streaming-commit-protocols.md ref-write routes, so requireAuth (not requireAuthWrite) is correct.

Types

type Config

type Config struct {
	Store    meta.Store
	Verifier auth.Verifier
	Logger   *slog.Logger
	// Org is the single served org (ADR-8), the same value the serve
	// command passes to githttp.NewServer. When set, every REST route
	// rejects verified tokens whose `iss` differs with 403 (issue #47),
	// mirroring the git surface's served-org gate. The zero value disables
	// the gate (matching this Config's other zero-value-disables knobs);
	// it exists ONLY for this repo's multi-org integration harness — the
	// serve command always sets Org.
	Org string
	// HTTPBaseURL is the externally reachable git base URL (scheme://host),
	// used to build the `http_url` in the POST /repos response, e.g.
	// "https://selfhost.open.storage".
	HTTPBaseURL string
	Fallback    http.Handler

	// RefStore/BlobStore/CacheDir back the git:read endpoints (docs/archive/phases/07-implementation-roadmap.md p3):
	// handlers materialize the repo cache via gitread.Materialize(RefStore,
	// BlobStore, CacheDir, ...) and run read plumbing over it. The serve
	// command passes the same instances the githttp fallback uses.
	RefStore  ref.RefStore
	BlobStore blob.BlobStore
	CacheDir  string

	// SyncQueue schedules upstream sync runs (docs/archive/phases/07-implementation-roadmap.md p5): base_repo
	// creates and pull-upstream enqueue through it. The serve command
	// passes the jobs seam (River on pg, in-process in local mode); tests
	// may pass a recorder/no-op.
	SyncQueue upsync.Queue

	// RateLimit enables the pre-auth per-client-IP token bucket on every
	// REST route (docs/reference/http-api.md gateway-level 429; see ratelimit.go). The zero
	// value disables limiting. The serve command plumbs it from
	// OPENSTORAGE_RATE_RPS / OPENSTORAGE_RATE_BURST.
	RateLimit RateLimitConfig

	// MaybeRepack, when set, runs after every successful RegisterPacks on
	// a write pipeline (issue #1): the serve command passes the repack
	// trigger's threshold check, which enqueues at most one unique-by-repo
	// background fold. It must be quick and never block on the fold. Nil
	// disables triggering.
	MaybeRepack func(ctx context.Context, repo string)

	// PackQuotaBytes is the per-repo pack-bytes quota (issue #21), the same
	// value the git surface enforces at pre-receive: REST pack writers
	// (commit-pack, merge, notes, fork via publishPack) publish under the
	// same packs/ prefix and must not bypass it. <= 0 disables enforcement.
	// The serve command plumbs it from OPENSTORAGE_PACK_QUOTA_BYTES.
	PackQuotaBytes int64

	// PackReservationTTL bounds how long a crashed writer's quota
	// reservation gates admission (issue #29, ref/ledger.go). <= 0 falls
	// back to a safe one-hour default. The serve command plumbs it from
	// OPENSTORAGE_PACK_RESERVATION_TTL.
	PackReservationTTL time.Duration

	// UsageFloor, when set, resolves the pack-admission usage floor for a
	// repo. The serve command wires the git server's shared single-flight
	// cached listing (githttp.Server.CachedUsageFloor), so REST pack
	// writers do not re-list the repo's pack and LFS prefixes on every
	// write (issue #51, Codex round 21 — per-write listings amplify at
	// the LFS object cap). Nil falls back to a fresh listing per write.
	// Returns the floor plus the shared floor generation it was
	// validated against (forwarded into ReservePackBytes' transactional
	// recheck — Codex round 60).
	UsageFloor func(ctx context.Context, repo string) (int64, int64, error)

	// WebhookAllowPrivate relaxes the webhook SSRF guard's private-range
	// denial (issue #53) for both the registration-time courtesy check
	// here and — plumbed separately into the delivery workers — the
	// authoritative dial-time enforcement. The zero value keeps the guard
	// strict (the safe default). The serve command plumbs it from
	// OPENSTORAGE_WEBHOOK_ALLOW_PRIVATE.
	WebhookAllowPrivate bool

	// SyncPullAllowPrivate mirrors the resolved pull-lane sync egress
	// policy (issue #139) for the registration-time courtesy check on
	// base_repo upstream_host; the authoritative dial-time enforcement is
	// the sync worker's vetting proxy, plumbed separately. The serve
	// command resolves it from OPENSTORAGE_SYNC_ALLOW_PRIVATE, whose
	// transitional unset default keeps it TRUE (0.6.x compatibility), so
	// unlike the webhook knob the zero value here is the STRICT policy.
	SyncPullAllowPrivate bool
}

Config wires the API service. Store, Verifier, HTTPBaseURL, RefStore, BlobStore, and CacheDir are required; Logger defaults to slog.Default(); Fallback (optional) receives every request not matching an /api/v1 route — the serve command mounts the git smart-HTTP handler there.

type RateLimitConfig

type RateLimitConfig struct {
	// RPS is the sustained refill rate per client IP.
	RPS float64
	// Burst is the token-bucket depth; must be >= 1 when RPS > 0.
	Burst int
	// TrustedProxies is the reverse-proxy allowlist for client identity
	// (issue #30). When the TCP peer is inside one of these prefixes, the
	// bucket key becomes the rightmost X-Forwarded-For hop that is not
	// itself allowlisted — the client address the nearest trusted proxy
	// saw. Empty (the default) keeps RemoteAddr-only keying: a spoofable
	// key would let any client mint fresh buckets at will, so forwarded
	// headers from unlisted peers are never consulted. Operator contract:
	// list ONLY proxies that set/append X-Forwarded-For themselves; a
	// listed proxy that forwards client-supplied values verbatim
	// reintroduces exactly that bucket-minting spoof. IPv4-mapped IPv6
	// prefixes (::ffff:a.b.c.d/96+) are normalized to IPv4 at
	// construction; mapped prefixes below /96 are a config error.
	TrustedProxies []netip.Prefix
}

RateLimitConfig enables per-client-IP rate limiting when RPS > 0; the zero value disables limiting entirely (the middleware becomes the identity). Limiting is per node, in memory: nodes are stateless and per-node limiting is the documented "basic" scope.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL