s3gateway

package
v0.44.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 15, 2026 License: Apache-2.0 Imports: 44 Imported by: 0

Documentation

Overview

Package s3gateway is the funcd S3-protocol frontend over the blob.Bucket substrate (ADR-0080), with in-platform identity via funcd-managed per-principal SigV4 keypairs resolved by an in-process versitygw auth.IAMService (ADR-0085).

AuthN: every in-platform Function and Lake engine gets a DETERMINISTIC keypair derived from a node-local master secret over its (kind, namespace, name) (ADR-0175) — funcd injects it into the sandbox env; the worker never chooses its secret and cannot derive a peer's. AuthZ: every backend op is a PEP on the cedar PDP (s3::read / s3::write), exactly the spec.bindings.blob binding-as-grant model of slice 2.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DecodeIdentityAccess

func DecodeIdentityAccess(access string) (ns, name string, ok bool)

DecodeIdentityAccess maps an Identity access key back to its (namespace, name). ok=false for a non-Identity or malformed key. Used by the store-backed ExternalKeys to resolve the Identity + secret.

func IdentityAccessKey

func IdentityAccessKey(ns, name string) string

IdentityAccessKey returns the stable SigV4 access key id for a user-assigned Identity (ADR-0135): "FUNCID" + base32-noPad-upper(ns + "\x00" + name). Deterministic in (ns, name); the SECRET is NOT derived (it is generated + stored in the Identity's owned Secret, so it is rotatable). Because the prefix is not "FUNCD", the Function decodeAccess skips it and the gateway resolves it via ExternalKeys.

func LoadOrCreateMaster

func LoadOrCreateMaster(masterSecretFile, dataDir string) ([]byte, error)

LoadOrCreateMaster loads the node S3 master secret from file (ADR-0085), or, when the path is empty, from / generates+persists 0600 at <dataDir>/s3gateway/master.key. With neither, the secret is generated in memory and never written. The secret is NEVER logged. A supplied masterSecretFile that does not exist is an error.

func MasterPath added in v0.10.0

func MasterPath(masterSecretFile, dataDir string) string

MasterPath is where the node master secret lives (ADR-0204 Decision 7): masterSecretFile when set, else <dataDir>/s3gateway/master.key; "" with neither (the master stays in memory).

func MigrateMaster added in v0.10.0

func MigrateMaster(masterSecretFile, dataDir string, gatewayOn bool, logger *slog.Logger) error

MigrateMaster moves a working-directory master of an earlier gateway-off node to its ADR-0204 Decision 7 location before LoadOrCreateMaster runs. Only a gateway-off node with no masterSecretFile migrates: a working-directory key is copied 0600 to <dataDir>/s3gateway/master.key when that file is absent, and a different key there is fault.Invalid. Otherwise a working-directory key that is not the master's file is ignored with a warning. With neither masterSecretFile nor dataDir the master stays in memory and nothing is checked.

func WriteMaster added in v0.29.0

func WriteMaster(path string, secret []byte) error

WriteMaster replaces the node master secret at path, as a restore installs it or rolls it back (ADR-0204).

Types

type Deps

type Deps struct {
	// BucketFor resolves (namespace, Bucket name) → the substrate bucket view: fault.NotFound when no such Bucket
	// exists for that namespace (⇒ NoSuchBucket), any other error when the lookup failed (⇒ InternalError).
	BucketFor func(ns v1.NamespaceName, bucket string) (blob.Bucket, error)
	// Buckets lists a namespace's Bucket resources; HeadBucket and ListBuckets answer
	// from it, keeping only the Buckets the caller is bound to.
	Buckets func(ctx context.Context, ns v1.NamespaceName) ([]v1.Bucket, error)
	// PDP is the cedar authorizer; every backend op is a PEP on it (s3::read/s3::write).
	PDP authz.Authorizer
	// Master is the node master secret keypairs are derived from (ADR-0085). Required.
	Master []byte
	// KeyPeriod and KeySalt make derived pairs rotate (ADR-0270); a zero KeyPeriod keeps the fixed pair. KeyNow is
	// the clock (nil: time.Now).
	KeyPeriod time.Duration
	KeySalt   []byte
	KeyNow    func() time.Time
	// External is the optional external-keypair store (the untrusted SigV4 edge); nil ⇒
	// in-platform only.
	External ExternalKeys
	// Listen is the node-private bind address (e.g. 127.0.0.1:9000).
	Listen string
	// Catalogs is a namespace's view of the Lake catalog store, served under v1.LakeCatalogBucket to the Lake that owns
	// the location a key names (ADR-0293); nil serves none. CatalogMaxBytes is lake.catalogMaxSize in bytes.
	Catalogs        func(ns v1.NamespaceName) blob.Bucket
	CatalogMaxBytes int64
	// LakeAt names the Lake of ns located on bucket/prefix, ok=false when none is: a catalog's delete is refused while
	// one is (ADR-0293). nil refuses every catalog delete.
	LakeAt func(ctx context.Context, ns v1.NamespaceName, bucket, prefix string) (v1.ObjectName, bool, error)
	// MaxUploadBytes caps a single buffered (multipart/put) object; 0 ⇒ defaultMaxUpload. It also sizes the
	// multipart budget (ADR-0188): buffered parts plus assembled copies stay within 3 × MaxUploadBytes daemon-wide,
	// and one principal's parts within MaxUploadBytes.
	MaxUploadBytes int64
	// Logger is the server logger; nil ⇒ slog.Default(). The master secret is NEVER logged.
	Logger *slog.Logger
	// Held is the namespace hold under which a write into the caller's namespace is refused (ADR-0290 Decision 6);
	// nil ⇒ hold.NeverCall.
	Held hold.CallGate
}

Deps configures the s3gateway server (ADR-0080 + ADR-0085).

func (Deps) Deriver added in v0.25.0

func (d Deps) Deriver() Deriver

Deriver is the key-pair deriver the gateway checks with; the injection of worker pairs uses the same (ADR-0270).

type Deriver added in v0.25.0

type Deriver struct {
	Master []byte
	Salt   []byte
	Period time.Duration
	Now    func() time.Time // nil: time.Now
}

Deriver derives and checks in-platform key pairs (ADR-0270). A positive Period gives a pair that holds the time window it was derived in and a per-start Salt; a zero Period gives the fixed pair of DeriveKeypair (funcdctl dev).

func (Deriver) Keypair added in v0.25.0

func (d Deriver) Keypair(kind v1.Kind, ns, name string) Keypair

Keypair is the identity's pair for the current window.

func (Deriver) RenewalAge added in v0.25.0

func (d Deriver) RenewalAge(kind v1.Kind, ns, name string) time.Duration

RenewalAge is the worker age, between 80% and 90% of Period, at which the identity's worker is replaced; 0 with a zero Period (never).

func (Deriver) Secret added in v0.25.0

func (d Deriver) Secret(access string) (kind v1.Kind, ns, name, secret string, ok bool)

Secret decodes access and derives its secret. With a Period it accepts only a four-part key whose window is the current or the previous one; with a zero Period only a three-part key. ok is false otherwise.

type ExternalKeys

type ExternalKeys interface {
	Lookup(access string) (secret string, namespace string, ok bool)
}

ExternalKeys is the funcd store of issued external scoped keypairs — the one untrusted edge (ADR-0085). It resolves an access key to its secret + scoped namespace for the S3Identity principal. Lifecycle UX (issue/rotate/revoke) is a follow-on; this read-only seam is all the gateway needs.

type Keypair

type Keypair struct{ AccessKey, SecretKey string }

Keypair is a principal's deterministic S3 credentials (ADR-0085). Pure value; no I/O.

func DeriveKeypair

func DeriveKeypair(master []byte, kind v1.Kind, ns, name string) Keypair

DeriveKeypair returns the stable per-(kind, namespace, name) keypair (ADR-0085, ADR-0175):

  • AccessKey = "FUNCD" + base32-noPad-upper(code + "\x00" + ns + "\x00" + name), code "F" for a Function and "L" for a Lake — decodable via decodeAccess; not secret (knowing it grants nothing).
  • SecretKey = base64( HMAC-SHA256(master, "s3:" + kind + ":" + ns + "/" + name) ) — the full kind is in the MAC input, so a Function cannot compute the secret of a same-named Lake.

Pure and deterministic: the same (master, kind, ns, name) always yields the same keypair, across daemon restarts. No storage, no rotation.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is the funcd S3 gateway: a versitygw s3api.New server over the funcd backend + in-process IAM, bound to a node-private TCP listener (ADR-0085). It is built via s3api.New (NOT embedgw.RunVersityGW) so the custom IAM is injectable.

func New

func New(d Deps) (*Server, error)

New builds the gateway server (ADR-0085). It validates Deps, constructs the in-process IAM + backend, and wires them through s3api.New with a random per-process root account (used by nothing external) so versitygw's required-root invariant is satisfied without granting anyone root. It does NOT bind a listener until Run is called.

func (*Server) Addr

func (s *Server) Addr() string

Addr returns the configured node-private listen address (host:port).

func (*Server) Close

func (s *Server) Close() error

Close gracefully shuts the server down (idempotent).

func (*Server) Ready

func (s *Server) Ready() <-chan struct{}

Ready returns a channel closed once the listener is bound and accepting (ADR-0085): callers (and tests) wait on it instead of a fixed sleep.

func (*Server) Run

func (s *Server) Run(ctx context.Context) (err error)

Run binds the node-private listener and serves until ctx is done (ADR-0085). It is opt-in: the daemon only calls it when s3gateway.enabled. Run blocks; callers run it in a goroutine. On ctx cancellation it triggers a graceful ShutDown.

func (*Server) Wait added in v0.2.3

func (s *Server) Wait(ctx context.Context) error

Wait blocks until the listener is bound and accepting (nil), Run has returned first (its error), or ctx is done (ctx's error). versitygw binds the address only when Run serves it, so a taken port surfaces here.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL