Documentation
¶
Index ¶
- func DropDatabase(ctx context.Context, adminDSN, dbName string) error
- func DropRole(ctx context.Context, adminDSN, role string) error
- func EnsureDatabase(ctx context.Context, adminDSN, dbName string) error
- func EnsureRole(ctx context.Context, adminDSN, role, password, ownedDB string) error
- func ProbeMetadataStore(ctx context.Context, endpoint, user, password, database, sslMode string) error
- type APIError
- type Controller
- type CreateWarehouseRequest
- type LakekeeperClient
- func (c *LakekeeperClient) Bootstrap(ctx context.Context) error
- func (c *LakekeeperClient) EnsureWarehouse(ctx context.Context, req CreateWarehouseRequest) (*Warehouse, error)
- func (c *LakekeeperClient) Info(ctx context.Context) (*ServerInfo, error)
- func (c *LakekeeperClient) WithBearer(token string) *LakekeeperClient
- func (c *LakekeeperClient) WithHTTPClient(hc *http.Client) *LakekeeperClient
- type ServerInfo
- type Warehouse
- type WarehouseStorageCredential
- type WarehouseStorageProfile
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DropDatabase ¶
DropDatabase removes dbName on the Postgres server addressed by adminDSN. Idempotent: returns nil when the database is already absent (3D000). Forces disconnection of any active sessions on the target DB so DROP DATABASE can't hang waiting for clients to drain — necessary at duckling teardown time because the per-tenant Lakekeeper pod may still be alive when the drop runs (the k8s teardown is fire-and-forget and the operator's reconciliation lag means connections linger).
Reassigns ownership to CURRENT_USER before the drop so the admin role can issue DROP DATABASE even when it doesn't own the target. EnsureRole runs ALTER DATABASE ... OWNER TO <role>, which means a non-superuser admin (e.g. the ducklingexample master on a shared RDS) wouldn't otherwise have permission — 42501 must be owner of database. GRANT role-membership first so CURRENT_USER inherits the necessary privileges to ALTER OWNER (which itself requires being a member of the new owner role on Postgres 14+).
Caller must connect via a privileged DSN against a different database than dbName (the admin DSN's path is OK to be `postgres`).
func DropRole ¶
DropRole removes role on the Postgres server addressed by adminDSN. Idempotent: returns nil when the role is already absent. Best-effort REASSIGN/DROP OWNED first so any object the role owns (e.g. grants on the maintenance DB, default privileges) doesn't block DROP ROLE with 2BP01 ("role cannot be dropped because some objects depend on it").
Requires role-membership in `role` for REASSIGN OWNED + DROP OWNED to run (Postgres 14+); the GRANT is best-effort because if `role` is already gone the GRANT itself fails. The caller's admin must either be a superuser or already a member of role; on RDS we explicitly GRANT the membership first since the admin is neither.
Caller must connect via a privileged DSN.
func EnsureDatabase ¶
EnsureDatabase creates dbName on the Postgres server addressed by adminDSN if it does not already exist. Idempotent: returns nil when the database is already present. Caller owns the DSN's credential lifetime.
CREATE DATABASE cannot run in a transaction and Postgres does not support CREATE DATABASE IF NOT EXISTS, so we probe pg_database first and only fire CREATE DATABASE when the row is missing. There is a TOCTOU race against concurrent callers; we handle the duplicate_database SQLSTATE (42P04) as a benign collision rather than an error.
func EnsureRole ¶
EnsureRole creates a login role with the given password, or rotates the password if the role already exists. Used by the Lakekeeper provisioner to make sure Lakekeeper's pod can connect with the credentials stored in its K8s Secret — a freshly-created database has no users by default.
On a re-run with the same password the ALTER ROLE is a no-op for Postgres internals. On a re-run with a different password we explicitly rotate; callers must keep the password in their Secret in sync with whatever was last passed here. The Lakekeeper provisioner achieves this by reading the existing Secret on every run (resolveOrGenerateSecret) rather than regenerating, so the same password threads through to EnsureRole.
Grants the role ALL PRIVILEGES on the named database. Cluster admin permissions are not granted.
func ProbeMetadataStore ¶
func ProbeMetadataStore(ctx context.Context, endpoint, user, password, database, sslMode string) error
ProbeMetadataStore does a real end-to-end connect to the metadata Postgres and runs SELECT 1 to confirm the path is reachable from a CP pod.
Purpose: AWS Aurora reports a cluster as Available before the cluster's DNS record has propagated to in-cluster CoreDNS, and even further before pgbouncer's resolver picks it up. If the provisioner flips the warehouse to state=ready on AWS-Available alone, worker activations during the next 3-5 minutes hit "DuckLake migration check failed → connect to metadata store: DNS lookup failed". This probe closes that window — we only flip to ready when the actual path the workers will use actually works.
`endpoint` is "<host>[:<port>]"; missing port defaults to 5432. `sslMode` should be:
- "disable" when probing through the duckling's pgbouncer (the CR's pgbouncer pod is configured for plaintext between worker and pooler and brings TLS to RDS itself)
- "require" when probing the metadata RDS endpoint directly (Aurora requires TLS; AWS issues the cert from the AWS-RDS-Root CA chain)
The caller (provisioner controller) decides which path to probe based on the warehouse's pgbouncer.enabled toggle — ducklings can run with or without the pgbouncer pooler and the chosen path must match.
Types ¶
type APIError ¶
APIError is returned for non-2xx responses. Status holds the HTTP code; Body holds the raw response body (often a JSON error envelope from Lakekeeper that we don't bother unmarshalling).
type Controller ¶
type Controller struct{}
Controller is a stub for non-Kubernetes builds.
func NewController ¶
func NewController(_ *configstore.ConfigStore, _ time.Duration) (*Controller, error)
NewController returns an error on non-Kubernetes builds since it requires K8s API access.
type CreateWarehouseRequest ¶
type CreateWarehouseRequest struct {
WarehouseName string `json:"warehouse-name"`
ProjectID string `json:"project-id,omitempty"`
StorageProfile WarehouseStorageProfile `json:"storage-profile"`
StorageCredential WarehouseStorageCredential `json:"storage-credential"`
}
CreateWarehouseRequest is the body of POST /management/v1/warehouse.
type LakekeeperClient ¶
type LakekeeperClient struct {
// contains filtered or unexported fields
}
LakekeeperClient is a thin HTTP client for the Lakekeeper management + catalog REST surface that the provisioner needs to drive: bootstrap the server, create the per-org warehouse, and check ready/bootstrapped state.
One client instance addresses one Lakekeeper deployment. Per-org Lakekeeper deployments get their own client instances built ad-hoc by the provisioner.
Authentication: a static bearer token can be set via WithBearer. In the current allowall + NetworkPolicy deployment model the token is unused; the field is kept so the OIDC follow-up (PR3) can plug in without changing callers.
func NewLakekeeperClient ¶
func NewLakekeeperClient(baseURL string) *LakekeeperClient
func (*LakekeeperClient) Bootstrap ¶
func (c *LakekeeperClient) Bootstrap(ctx context.Context) error
Bootstrap is the one-shot init of a fresh Lakekeeper server. Returns nil if the server was already bootstrapped (Lakekeeper responds 409 in that case; we treat it as success).
func (*LakekeeperClient) EnsureWarehouse ¶
func (c *LakekeeperClient) EnsureWarehouse(ctx context.Context, req CreateWarehouseRequest) (*Warehouse, error)
EnsureWarehouse creates the warehouse if it doesn't exist, otherwise returns the existing one. Match is by warehouse-name within the default project.
Idempotent under concurrent callers: if two callers both observe an empty list and both POST, the second POST will 409 and we re-list to return the winner. Callers that need stronger ordering should hold a per-org lock outside this method.
func (*LakekeeperClient) Info ¶
func (c *LakekeeperClient) Info(ctx context.Context) (*ServerInfo, error)
Info fetches the server info. Returns ErrServerNotReady if the endpoint is reachable but the server reports itself as not yet ready (rare; treated as transient by the caller).
func (*LakekeeperClient) WithBearer ¶
func (c *LakekeeperClient) WithBearer(token string) *LakekeeperClient
WithBearer sets the bearer token used on subsequent requests. The token must be a single-line string (no CR/LF) — Go's http.Header.Set won't error on construction but http.Client.Do will reject malformed headers at send.
NOT safe to call concurrently with in-flight requests. Build the client once, configure it, then share read-only across goroutines. PR3 will add proper rotation primitives when OIDC refresh lands.
func (*LakekeeperClient) WithHTTPClient ¶
func (c *LakekeeperClient) WithHTTPClient(hc *http.Client) *LakekeeperClient
type ServerInfo ¶
type ServerInfo struct {
Version string `json:"version"`
Bootstrapped bool `json:"bootstrapped"`
ServerID string `json:"server-id"`
AuthzBackend string `json:"authz-backend"`
DefaultProjectID string `json:"default-project-id"`
}
ServerInfo is the subset of GET /management/v1/info the provisioner cares about. Bootstrapped flips to true after the first POST /management/v1/bootstrap.
type Warehouse ¶
type Warehouse struct {
ID string `json:"id"`
WarehouseID string `json:"warehouse-id"`
Name string `json:"name"`
ProjectID string `json:"project-id"`
Status string `json:"status"`
}
Warehouse is the subset of the create/list response we use.
type WarehouseStorageCredential ¶
type WarehouseStorageCredential struct {
Type string `json:"type"` // "s3"
CredentialType string `json:"credential-type"` // "access-key" or "aws-system-identity"
AWSAccessKeyID string `json:"aws-access-key-id,omitempty"`
AWSSecretAccessKey string `json:"aws-secret-access-key,omitempty"`
}
WarehouseStorageCredential supports access-key creds (dev/MinIO) and instance-profile / IRSA-style creds (prod AWS, no static key).
type WarehouseStorageProfile ¶
type WarehouseStorageProfile struct {
Type string `json:"type"` // "s3"
Bucket string `json:"bucket"`
KeyPrefix string `json:"key-prefix"`
Endpoint string `json:"endpoint,omitempty"` // e.g. http://minio:9000; omit for real AWS
STSEndpoint string `json:"sts-endpoint,omitempty"` // optional
Region string `json:"region"`
PathStyleAccess bool `json:"path-style-access"`
Flavor string `json:"flavor"` // "s3-compat" for MinIO, "aws" for AWS
STSEnabled bool `json:"sts-enabled"`
RemoteSigningEnabled bool `json:"remote-signing-enabled"`
// STSRoleARN is the IAM role Lakekeeper assumes to mint vended (scoped,
// short-lived) S3 credentials for clients. Lakekeeper requires it for the
// AWS flavor when sts-enabled. Empty for s3-compat (MinIO). We set it to
// the per-org duckling role, which the Lakekeeper pod already runs as via
// EKS Pod Identity — so it assumes itself (the role trusts its own ARN).
STSRoleARN string `json:"sts-role-arn,omitempty"`
}
WarehouseStorageProfile is the subset of fields we send for an S3 / S3-compat warehouse. The Lakekeeper API accepts more; we only set what we need.