provisioner

package
v0.0.0-...-d67005b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 22, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DropDatabase

func DropDatabase(ctx context.Context, adminDSN, dbName string) error

DropDatabase removes dbName on the Postgres server addressed by adminDSN. Idempotent: returns nil when the database is already absent (3D000). Forces disconnection of any active sessions on the target DB so DROP DATABASE can't hang waiting for clients to drain — necessary at duckling teardown time because the per-tenant Lakekeeper pod may still be alive when the drop runs (the k8s teardown is fire-and-forget and the operator's reconciliation lag means connections linger).

Reassigns ownership to CURRENT_USER before the drop so the admin role can issue DROP DATABASE even when it doesn't own the target. EnsureRole runs ALTER DATABASE ... OWNER TO <role>, which means a non-superuser admin (e.g. the ducklingexample master on a shared RDS) wouldn't otherwise have permission — 42501 must be owner of database. GRANT role-membership first so CURRENT_USER inherits the necessary privileges to ALTER OWNER (which itself requires being a member of the new owner role on Postgres 14+).

Caller must connect via a privileged DSN against a different database than dbName (the admin DSN's path is OK to be `postgres`).

func DropRole

func DropRole(ctx context.Context, adminDSN, role string) error

DropRole removes role on the Postgres server addressed by adminDSN. Idempotent: returns nil when the role is already absent. Best-effort REASSIGN/DROP OWNED first so any object the role owns (e.g. grants on the maintenance DB, default privileges) doesn't block DROP ROLE with 2BP01 ("role cannot be dropped because some objects depend on it").

Requires role-membership in `role` for REASSIGN OWNED + DROP OWNED to run (Postgres 14+); the GRANT is best-effort because if `role` is already gone the GRANT itself fails. The caller's admin must either be a superuser or already a member of role; on RDS we explicitly GRANT the membership first since the admin is neither.

Caller must connect via a privileged DSN.

func EnsureDatabase

func EnsureDatabase(ctx context.Context, adminDSN, dbName string) error

EnsureDatabase creates dbName on the Postgres server addressed by adminDSN if it does not already exist. Idempotent: returns nil when the database is already present. Caller owns the DSN's credential lifetime.

CREATE DATABASE cannot run in a transaction and Postgres does not support CREATE DATABASE IF NOT EXISTS, so we probe pg_database first and only fire CREATE DATABASE when the row is missing. There is a TOCTOU race against concurrent callers; we handle the duplicate_database SQLSTATE (42P04) as a benign collision rather than an error.

func EnsureRole

func EnsureRole(ctx context.Context, adminDSN, role, password, ownedDB string) error

EnsureRole creates a login role with the given password, or rotates the password if the role already exists. Used by the Lakekeeper provisioner to make sure Lakekeeper's pod can connect with the credentials stored in its K8s Secret — a freshly-created database has no users by default.

On a re-run with the same password the ALTER ROLE is a no-op for Postgres internals. On a re-run with a different password we explicitly rotate; callers must keep the password in their Secret in sync with whatever was last passed here. The Lakekeeper provisioner achieves this by reading the existing Secret on every run (resolveOrGenerateSecret) rather than regenerating, so the same password threads through to EnsureRole.

Grants the role ALL PRIVILEGES on the named database. Cluster admin permissions are not granted.

func ProbeMetadataStore

func ProbeMetadataStore(ctx context.Context, endpoint, user, password, database, sslMode string) error

ProbeMetadataStore does a real end-to-end connect to the metadata Postgres and runs SELECT 1 to confirm the path is reachable from a CP pod.

Purpose: AWS Aurora reports a cluster as Available before the cluster's DNS record has propagated to in-cluster CoreDNS, and even further before pgbouncer's resolver picks it up. If the provisioner flips the warehouse to state=ready on AWS-Available alone, worker activations during the next 3-5 minutes hit "DuckLake migration check failed → connect to metadata store: DNS lookup failed". This probe closes that window — we only flip to ready when the actual path the workers will use actually works.

`endpoint` is "<host>[:<port>]"; missing port defaults to 5432. `sslMode` should be:

  • "disable" when probing through the duckling's pgbouncer (the CR's pgbouncer pod is configured for plaintext between worker and pooler and brings TLS to RDS itself)
  • "require" when probing the metadata RDS endpoint directly (Aurora requires TLS; AWS issues the cert from the AWS-RDS-Root CA chain)

The caller (provisioner controller) decides which path to probe based on the warehouse's pgbouncer.enabled toggle — ducklings can run with or without the pgbouncer pooler and the chosen path must match.

Types

type APIError

type APIError struct {
	Status int
	Method string
	Path   string
	Body   string
}

APIError is returned for non-2xx responses. Status holds the HTTP code; Body holds the raw response body (often a JSON error envelope from Lakekeeper that we don't bother unmarshalling).

func (*APIError) Error

func (e *APIError) Error() string

type Controller

type Controller struct{}

Controller is a stub for non-Kubernetes builds.

func NewController

func NewController(_ *configstore.ConfigStore, _ time.Duration) (*Controller, error)

NewController returns an error on non-Kubernetes builds since it requires K8s API access.

func (*Controller) Run

func (c *Controller) Run(_ context.Context)

Run is a no-op stub.

type CreateWarehouseRequest

type CreateWarehouseRequest struct {
	WarehouseName     string                     `json:"warehouse-name"`
	ProjectID         string                     `json:"project-id,omitempty"`
	StorageProfile    WarehouseStorageProfile    `json:"storage-profile"`
	StorageCredential WarehouseStorageCredential `json:"storage-credential"`
}

CreateWarehouseRequest is the body of POST /management/v1/warehouse.

type LakekeeperClient

type LakekeeperClient struct {
	// contains filtered or unexported fields
}

LakekeeperClient is a thin HTTP client for the Lakekeeper management + catalog REST surface that the provisioner needs to drive: bootstrap the server, create the per-org warehouse, and check ready/bootstrapped state.

One client instance addresses one Lakekeeper deployment. Per-org Lakekeeper deployments get their own client instances built ad-hoc by the provisioner.

Authentication: a static bearer token can be set via WithBearer. In the current allowall + NetworkPolicy deployment model the token is unused; the field is kept so the OIDC follow-up (PR3) can plug in without changing callers.

func NewLakekeeperClient

func NewLakekeeperClient(baseURL string) *LakekeeperClient

func (*LakekeeperClient) Bootstrap

func (c *LakekeeperClient) Bootstrap(ctx context.Context) error

Bootstrap is the one-shot init of a fresh Lakekeeper server. Returns nil if the server was already bootstrapped (Lakekeeper responds 409 in that case; we treat it as success).

func (*LakekeeperClient) EnsureWarehouse

func (c *LakekeeperClient) EnsureWarehouse(ctx context.Context, req CreateWarehouseRequest) (*Warehouse, error)

EnsureWarehouse creates the warehouse if it doesn't exist, otherwise returns the existing one. Match is by warehouse-name within the default project.

Idempotent under concurrent callers: if two callers both observe an empty list and both POST, the second POST will 409 and we re-list to return the winner. Callers that need stronger ordering should hold a per-org lock outside this method.

func (*LakekeeperClient) Info

func (c *LakekeeperClient) Info(ctx context.Context) (*ServerInfo, error)

Info fetches the server info. Returns ErrServerNotReady if the endpoint is reachable but the server reports itself as not yet ready (rare; treated as transient by the caller).

func (*LakekeeperClient) WithBearer

func (c *LakekeeperClient) WithBearer(token string) *LakekeeperClient

WithBearer sets the bearer token used on subsequent requests. The token must be a single-line string (no CR/LF) — Go's http.Header.Set won't error on construction but http.Client.Do will reject malformed headers at send.

NOT safe to call concurrently with in-flight requests. Build the client once, configure it, then share read-only across goroutines. PR3 will add proper rotation primitives when OIDC refresh lands.

func (*LakekeeperClient) WithHTTPClient

func (c *LakekeeperClient) WithHTTPClient(hc *http.Client) *LakekeeperClient

type ServerInfo

type ServerInfo struct {
	Version          string `json:"version"`
	Bootstrapped     bool   `json:"bootstrapped"`
	ServerID         string `json:"server-id"`
	AuthzBackend     string `json:"authz-backend"`
	DefaultProjectID string `json:"default-project-id"`
}

ServerInfo is the subset of GET /management/v1/info the provisioner cares about. Bootstrapped flips to true after the first POST /management/v1/bootstrap.

type Warehouse

type Warehouse struct {
	ID          string `json:"id"`
	WarehouseID string `json:"warehouse-id"`
	Name        string `json:"name"`
	ProjectID   string `json:"project-id"`
	Status      string `json:"status"`
}

Warehouse is the subset of the create/list response we use.

type WarehouseStorageCredential

type WarehouseStorageCredential struct {
	Type               string `json:"type"`            // "s3"
	CredentialType     string `json:"credential-type"` // "access-key" or "aws-system-identity"
	AWSAccessKeyID     string `json:"aws-access-key-id,omitempty"`
	AWSSecretAccessKey string `json:"aws-secret-access-key,omitempty"`
}

WarehouseStorageCredential supports access-key creds (dev/MinIO) and instance-profile / IRSA-style creds (prod AWS, no static key).

type WarehouseStorageProfile

type WarehouseStorageProfile struct {
	Type                 string `json:"type"` // "s3"
	Bucket               string `json:"bucket"`
	KeyPrefix            string `json:"key-prefix"`
	Endpoint             string `json:"endpoint,omitempty"`     // e.g. http://minio:9000; omit for real AWS
	STSEndpoint          string `json:"sts-endpoint,omitempty"` // optional
	Region               string `json:"region"`
	PathStyleAccess      bool   `json:"path-style-access"`
	Flavor               string `json:"flavor"` // "s3-compat" for MinIO, "aws" for AWS
	STSEnabled           bool   `json:"sts-enabled"`
	RemoteSigningEnabled bool   `json:"remote-signing-enabled"`
	// STSRoleARN is the IAM role Lakekeeper assumes to mint vended (scoped,
	// short-lived) S3 credentials for clients. Lakekeeper requires it for the
	// AWS flavor when sts-enabled. Empty for s3-compat (MinIO). We set it to
	// the per-org duckling role, which the Lakekeeper pod already runs as via
	// EKS Pod Identity — so it assumes itself (the role trusts its own ARN).
	STSRoleARN string `json:"sts-role-arn,omitempty"`
}

WarehouseStorageProfile is the subset of fields we send for an S3 / S3-compat warehouse. The Lakekeeper API accepts more; we only set what we need.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL