cork

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 18, 2026 License: Apache-2.0 Imports: 59 Imported by: 0

Documentation

Overview

The artifact archive hardening in this file is adapted from ArmyCyberInstitute/cmgr at commit bdbf839: https://github.com/ArmyCyberInstitute/cmgr

The original project is available under the Apache License 2.0. This file is a modified implementation for this fork, also distributed under cmgr's Apache License 2.0.

Adapted from ArmyCyberInstitute/cmgr bef2185 (#60)

The artifact resource bounds in this file are adapted from ArmyCyberInstitute/cmgr at commit bdbf839, which introduced configurable limits alongside its artifact hardening work: https://github.com/ArmyCyberInstitute/cmgr

The original project is available under the Apache License 2.0. This file is a reduced implementation covering only the artifact limits, also distributed under cmgr's Apache License 2.0.

The per-challenge seccomp profile handling in this file is adapted from ArmyCyberInstitute/cmgr at commit a5a3bda: https://github.com/ArmyCyberInstitute/cmgr

The original project is available under the Apache License 2.0. This file is a reduced implementation for this fork, also distributed under cmgr's Apache License 2.0. Upstream additionally supports a `legacy` mode and named `tweaks` applied through an OCI runtime interceptor. Neither is adopted here: this fork already applies the historical cmgr profile by default, so `legacy` would be a no-op, and a complete profile does not require a runtime interceptor because Docker accepts one directly.

Index

Examples

Constants

View Source 
const (
	BuildPlaneLocal    string = "local"
	BuildPlaneExternal string = "external"
)

The two values of BUILD_PLANE_ENV, and what BuildPlane reports.

View Source 
const (
	DB_ENV                string = "CORK_DB"
	DIR_ENV               string = "CORK_DIR"
	ARTIFACT_DIR_ENV      string = "CORK_ARTIFACT_DIR"
	REGISTRY_ENV          string = "CORK_REGISTRY"
	REGISTRY_USER_ENV     string = "CORK_REGISTRY_USER"
	REGISTRY_TOKEN_ENV    string = "CORK_REGISTRY_TOKEN"
	REGISTRY_CERT_DIR_ENV string = "CORK_REGISTRY_CERT_DIR"
	LOGGING_ENV           string = "CORK_LOGGING"
	IFACE_ENV             string = "CORK_INTERFACE"
	PORTS_ENV             string = "CORK_PORTS"
	// CONCURRENT_LAUNCHES_ENV was the one setting named only as a literal
	// where it is read; it is a constant so the list of what a build plane
	// ignores can name it (orchestratorOnlySettings).
	CONCURRENT_LAUNCHES_ENV string = "CORK_CONCURRENT_LAUNCHES"
	DISK_QUOTA_ENV          string = "CORK_ENABLE_DISK_QUOTAS"
	PRUNE_AGE_ENV           string = "CORK_PRUNE_AGE"
	DB_WAL_ENV              string = "CORK_DB_WAL"

	// Worker tunables (see workerTiming in workers.go).
	WORKER_POLL_INTERVAL_ENV   string = "CORK_WORKER_POLL_INTERVAL"
	WORKER_POLL_TIMEOUT_ENV    string = "CORK_WORKER_POLL_TIMEOUT"
	WORKER_MAX_MISSES_ENV      string = "CORK_WORKER_MAX_MISSES"
	WORKER_CONTROL_TIMEOUT_ENV string = "CORK_WORKER_CONTROL_TIMEOUT"
	WORKER_PULL_TIMEOUT_ENV    string = "CORK_WORKER_PULL_TIMEOUT"
	WORKER_LAUNCH_WAIT_ENV     string = "CORK_WORKER_LAUNCH_WAIT"

	DYNAMIC_INSTANCES int = -1
	LOCKED            int = -2
)
View Source 
const BASE_PINS_ENV = "CORK_BASE_PINS"
View Source 
const BUILD_PLANE_ENV string = "CORK_BUILD_PLANE"

BUILD_PLANE_ENV selects where challenge images are built: 'local' (the default) on the docker daemon this process reaches through DOCKER_HOST, from the challenge tree in CORK_DIR; or 'external', where something else builds, pushes to the registry and hands the finished builds to this daemon, which then has no docker daemon and no challenge tree of its own.

View Source 
const PURGE_AFTER_PUSH_ENV string = "CORK_PURGE_AFTER_PUSH"

Purging the builder's local images after a build is pushed.

The orchestrator builds every challenge image and pushes it to the registry; workers pull from there. Nothing then reads the builder's own copy, so keeping it means the image store grows with the whole fleet -- challenges times seeds times retained generations -- on a volume it shares with the challenge checkout, the database and the artifact bundles.

What makes this safe is the move to BuildKit. Under the legacy builder the layer cache WAS untagged images, so reclaiming image space destroyed the shared apt and pip layers and the next build re-ran every install in the fleet. BuildKit keeps its cache in a separate store that image removal cannot reach, which was measured: after removing every cmgr.managed image, overlay2 was unchanged and the next build downloaded nothing. (That measurement predates the inline cache: a build now does resolve its cache refs against the registry. The conclusion it supports -- that removing images does not destroy the layer cache -- is unaffected.) Only `docker builder prune` drops the bases, which is builder.gc's business, not this file's.

Nothing else needs the local copy either:

  • a build's images are read exactly twice, both inside executeBuild: by the artifact extraction container, and by the push;
  • launches call ensureImages, which pulls from the registry for whichever daemon hosts the instance -- including the local one, so an instance that falls back to local placement (no workers registered) still works;
  • the retention paths only ever REMOVE images and already tolerate their absence (pruneReplacedImages and destroyImages both log a missing tag at debug and continue).

The "builder" host's image is purged along with the rest, and it is the one exception to "every image is in the registry": it is never pushed and never launched (startContainers skips it), existing only for the extraction container, which has already run by the time this is called. So dropping it is free today -- but it cannot be recovered by a pull, only by a rebuild. The rollback sketched at the extraction container in docker.go would re-run extraction against the PrevChecksum image without building, and would have to reckon with that.

Registry mode only, and that is a hard gate rather than a default: without a registry the images are never pushed, ensureImages does not pull, and the builder's copy is the only one there is. Purging there would break every launch.

View Source 
const (
	// WORKER_SERVERNAME is the TLS name every worker's dockerd certificate is
	// issued for (SAN DNS:academy-docker-worker). Workers share one server
	// cert so they can be cloned without reprovisioning; the client pins this
	// name instead of verifying against the dialed IP.
	WORKER_SERVERNAME = "academy-docker-worker"
)

Variables

View Source 
var (
	// ErrWorkerBusy: no slot on the instance's daemon within the wait.
	// Retryable (corkd answers 503 with Retry-After).
	ErrWorkerBusy = errors.New("worker is busy")
	// ErrWorkerDown: the instance's worker went down between placement and
	// the launch stage. Retryable: placement now skips it.
	ErrWorkerDown = errors.New("worker went down")
	// ErrPullTimeout: an image pull exceeded the pull timeout. Retryable: the
	// next attempt may land on a worker that already holds the image.
	ErrPullTimeout = errors.New("image pull timed out")
)
View Source 
var (
	ErrAllWorkersOverloaded = errors.New("all workers are overloaded")
	ErrAllWorkersDown       = errors.New("no workers are reachable")
)

Selection failures, distinguished so corkd can map them to 503 vs 500.

View Source 
var ErrChallengeHasBuilds = errors.New("the challenge still has builds")

ErrChallengeHasBuilds refuses to remove a challenge with builds on record: they go with the schema that wants them (DeleteSchema) or with Destroy, never silently with the challenge. corkd reports it as 409.

View Source 
var ErrDatabaseBusy = errors.New("database busy")

ErrDatabaseBusy: SQLite's write lock was not obtained within its busy timeout, which only a burst of concurrent launches brings about. Retryable (corkd answers 503 with Retry-After): the same request goes through moments later.

View Source 
var ErrExternalBuildPlane = errors.New("the build plane is external: this daemon does not build")

ErrExternalBuildPlane answers every operation that needs the local daemon or the challenge tree when the build plane is external: an update and its dry run, a manual build, the base image pins, and a schema converge that wants a build nobody has handed over. corkd reports it as 409: the request is well-formed, this daemon is just not the one that builds.

View Source 
var ErrHandOverInvalid = errors.New("invalid hand-over")

ErrHandOverInvalid is a hand-over refused for what it says: a payload that contradicts itself, or a build whose content checksum is not what its inputs give. Nothing was written. corkd reports it as 400.

View Source 
var ErrLocalBuildPlane = errors.New("the build plane is local: this daemon builds its own challenges and takes no hand-over")

ErrLocalBuildPlane refuses a hand-over on a local build plane: that daemon builds its own challenges from its tree, and a build handed to it would be rebuilt over by the next update. corkd reports it as 409.

View Source 
var ErrNoWorkers = errors.New("no workers are registered")

ErrNoWorkers refuses a launch when no worker is registered. On a local build plane that launch runs on the local daemon; an external one has none. A fleet with no worker at all is a failure of the fleet, not a passing state like every worker being overloaded, so unlike ErrAllWorkersOverloaded it is not retryable: the platform surfaces it, and a player's retry costs less than a retry loop holding its launch workers.

View Source 
var ErrNotInRegistry = errors.New("not in the registry")

ErrNotInRegistry is a hand-over refused for what the registry says: an image it names is not served under its tag, so no worker could pull it. Nothing was written. corkd reports it as 409.

Functions

func Getenv 

func Getenv(name string) string

Getenv is os.Getenv with LookupEnv's fallback: the empty string when neither name is set.

func LookupEnv 

func LookupEnv(name string) (string, bool)

LookupEnv is os.LookupEnv for a setting: the CORK_ name, then its CMGR_ name. Names outside the CORK_ prefix (DOCKER_CERT_PATH, say) are looked up as they are.

func SeccompProfiles 

func SeccompProfiles(challenge *ChallengeMetadata) map[string]string

SeccompProfiles is the resolved text of every seccomp profile a challenge declares, keyed by the filename it declares it under. It is how a profile reaches a daemon that has no challenge directory to read it from.

The text lives in the unexported effectiveProfile, filled in by resolve() from the tree, so it cannot travel in the payload's own JSON -- which is how a hand-over came to record that a challenge had a profile while every container it launched ran under the embedded default instead. Returned keyed by filename rather than by host because that is what the declaration names, and a challenge that uses one profile on several hosts should send it once.

func Version 

func Version() string

Returns the version string associated with the build (results of `git describe --tags`) or "unknown" if it was not set at build time.

Types

type BasePin 

type BasePin struct {
	Ref    string `json:"ref"`
	Digest string `json:"digest"`
	// InUse is the number of challenge Dockerfiles whose FROM lines name this
	// reference; a pin nothing uses is a candidate for removal.
	InUse int `json:"in_use"`
}

BasePin is one entry of the pin map, as reported by the API.

type BuildId 

type BuildId int64

type BuildMetadata 

type BuildMetadata struct {
	Id BuildId `json:"id"`

	Flag       string            `json:"flag"`
	LookupData map[string]string `json:"lookup_data,omitempty"`

	Seed   int    `json:"seed"`
	Format string `json:"format"`
	// Checksum identifies the content this build's images were produced from:
	// a CRC-32 over the challenge's source checksum and the flag format (see
	// contentChecksum). It is set when the images are built, so after a source
	// change it intentionally differs from the value derived from the
	// challenge's current metadata until the build is rebuilt.
	Checksum uint32 `json:"checksum,omitempty"`
	// PrevChecksum is the generation Checksum displaced on the last rebuild
	// (0 = none): its images are retained as the rollback target. A future
	// `rollback` operation would swap it with Checksum, re-extract /challenge
	// from that image (see executeBuild's extraction step), and restart
	// instances. Same-row rollback is format- and seed-stable by construction.
	PrevChecksum uint32 `json:"prev_checksum,omitempty" db:"prevchecksum"`
	// SourceChecksum is the challenge source generation (ChallengeMetadata.
	// SourceChecksum) this build's images were produced from; 0 until the
	// build has been built. A build is current exactly when it equals the
	// challenge's recorded source checksum, so a rebuild that failed — the
	// challenge row already carries the new generation, the build still
	// serves the old one — is visible as the difference rather than lost.
	// It is deliberately the source generation and not Checksum: base-image
	// pins are folded into Checksum and a pin refresh must not make every
	// build look stale.
	SourceChecksum uint32              `json:"source_checksum,omitempty" db:"sourcechecksum"`
	Images         []Image             `json:"images"`
	HasArtifacts   bool                `json:"has_artifacts"`
	LastSolved     int64               `json:"last_solved"`
	Challenge      ChallengeId         `json:"challenge_id"`
	Instances      []*InstanceMetadata `json:"instances,omitempty"`

	Schema        string `json:"schema"`
	InstanceCount int    `json:"instance_count"`
}

type BuildSpecification 

type BuildSpecification struct {
	Seeds         []int `json:"seeds"          yaml:"seeds"`
	InstanceCount int   `json:"instance_count" yaml:"instance_count"`
}

type ChallengeId 

type ChallengeId string

type ChallengeMetadata 

type ChallengeMetadata struct {
	Id               ChallengeId         `json:"id"`
	Name             string              `json:"name,omitempty"`
	Namespace        string              `json:"namespace"`
	ChallengeType    string              `json:"challenge_type"`
	Description      string              `json:"description,omitempty"`
	Details          string              `json:"details,omitempty"`
	Hints            []string            `json:"hints,omitempty"`
	SourceChecksum   uint32              `json:"source_checksum"`
	MetadataChecksum uint32              `json:"metadata_checksum"`
	Path             string              `json:"path"`
	Templatable      bool                `json:"templatable,omitempty"`
	PortMap          map[string]PortInfo `json:"port_map,omitempty"`
	Hosts            []HostInfo          `json:"hosts"`
	MaxUsers         int                 `json:"max_users,omitempty"`
	Category         string              `json:"category,omitempty"`
	Points           int                 `json:"points,omitempty"`
	Tags             []string            `json:"tags,omitempty"`
	Attributes       map[string]string   `json:"attributes,omitempty"`
	ChallengeOptions ChallengeOptions    `json:"challenge_options,omitempty"`

	SolveScript bool `json:"solve_script,omitempty"`

	// DeliveryType classifies what a player receives and therefore what cork
	// must stand up at runtime. It is derived (deriveDeliveryType), never stored
	// or author-written, so it cannot disagree with the port map or challenge
	// type it is computed from. Always emitted (no omitempty) so consumers can
	// distinguish "service" from absent-on-old-cmgr. Instances are launched only
	// for "service" challenges; consumers should treat an empty instance pool as
	// expected exactly when this is not "service".
	DeliveryType DeliveryType `json:"delivery_type" db:"-"`

	Builds []*BuildMetadata `json:"builds,omitempty"`
}

func (*ChallengeMetadata) NeedsInstance 

func (cm *ChallengeMetadata) NeedsInstance() bool

NeedsInstance reports whether running instances are part of delivering this challenge. Only "service" challenges need them; artifact-only and flag-only challenges are fully delivered by their builds. An unset DeliveryType (metadata that never passed through the loader or a database read) counts as needing instances, so a missed derivation can never silently skip instance management.

type ChallengeOptions 

type ChallengeOptions struct {
	NetworkOptions   `yaml:",inline"`
	ContainerOptions `yaml:",inline"`
	Overrides        map[string]ContainerOptions `json:"overrides,omitempty" yaml:"overrides"`
}

type ChallengeUpdates 

type ChallengeUpdates struct {
	Added      []*ChallengeMetadata `json:"added"`
	Refreshed  []*ChallengeMetadata `json:"refreshed"`
	Updated    []*ChallengeMetadata `json:"updated"`
	Stale      []*ChallengeMetadata `json:"stale"`
	Removed    []*ChallengeMetadata `json:"removed"`
	Unmodified []*ChallengeMetadata `json:"unmodified"`
	Errors     []error              `json:"errors"`
}

ChallengeUpdates is DetectChanges' verdict on each challenge, one bucket per challenge: Added (on disk, not in the database), Updated (source changed: every build rebuilt), Refreshed (metadata only: re-persisted), Stale (nothing changed on disk, but a build still serves an earlier generation because its last rebuild failed), Unmodified, Removed. Updated outranks Refreshed outranks Stale. Whatever a failed rebuild left behind is rebuilt on the Refreshed and Stale paths alike, so a Stale verdict is only ever given when there is nothing else to report.

type ContainerOptions 

type ContainerOptions struct {
	Init            bool            `json:"init,omitempty"            yaml:"init"`
	Cpus            string          `json:"cpus,omitempty"            yaml:"cpus"`
	Memory          string          `json:"memory,omitempty"          yaml:"memory"`
	Ulimits         []string        `json:"ulimits,omitempty"         yaml:"ulimits"`
	PidsLimit       int64           `json:"pidslimit,omitempty"       yaml:"pidslimit"`
	ReadonlyRootfs  bool            `json:"readonlyrootfs,omitempty"  yaml:"readonlyrootfs"`
	DroppedCaps     []string        `json:"droppedcaps,omitempty"     yaml:"droppedcaps"`
	NoNewPrivileges bool            `json:"nonewprivileges,omitempty" yaml:"nonewprivileges"`
	DiskQuota       string          `json:"diskquota,omitempty"       yaml:"diskquota"`
	CgroupParent    string          `json:"cgroupparent,omitempty"    yaml:"cgroupparent"`
	CapImmutable    bool            `json:"capimmutable,omitempty"    yaml:"cap_immutable"`
	Seccomp         *SeccompOptions `json:"seccomp,omitempty"    yaml:"seccomp,omitempty"`
}

type DeliveryType 

type DeliveryType string

DeliveryType is the intrinsic runtime shape of a challenge: what the player receives and what infrastructure must exist for it. It is orthogonal to the schema-level deployment policy (instance_count: persistent vs on-demand), which only applies to "service" challenges. 

const (
	// DeliveryService: the challenge publishes at least one port; a running
	// instance is required for players to interact with it.
	DeliveryService DeliveryType = "service"
	// DeliveryArtifactOnly: no published ports; the artifacts produced at build
	// time are the entire challenge and no instance is ever launched.
	DeliveryArtifactOnly DeliveryType = "artifact_only"
	// DeliveryFlagOnly: intentionally no ports and no artifacts; the build runs
	// only to generate the flag and lookup data (e.g. multiple-choice options)
	// for a bare submission prompt. No instance is ever launched. Declared via
	// the "flag-only" challenge type (that literal and the embedded
	// flag-only.Dockerfile name must stay in sync with deriveDeliveryType).
	DeliveryFlagOnly DeliveryType = "flag_only"
)

type HandOver 

type HandOver struct {
	// Challenge as the build plane describes it, in the shape GET /state
	// gives, with Builds the builds being handed over: seed, format, schema
	// and instance count as the schema names them there, the flag and
	// lookup data, the images with their exposed ports, whether the build
	// published artifacts, and the content and source checksums the build
	// was stamped with. A build's id is adopted as delivered -- the build
	// plane's bundle is named by it and the platform addresses the artifact
	// files by the id this daemon reports, so the two numberings are one.
	// Instances, last-solved times and the rollback generation are this
	// daemon's own and are ignored.
	Challenge *ChallengeMetadata `json:"challenge"`
	// PinFingerprint is the base image pin fingerprint the builds were made
	// under, 0 with pinning off: an input to the content checksum this
	// daemon recomputes for every build, and one it does not otherwise have
	// (the pins live on the build plane).
	PinFingerprint uint32 `json:"pin_fingerprint"`
	// SeccompProfiles is the text of every seccomp profile the challenge
	// declares, keyed by the filename it declares it under. The resolved
	// text is read from the challenge directory, which this daemon does not
	// have, and it is not in the challenge's own JSON -- so without it a
	// challenge that asked for a narrower syscall set would be recorded as
	// having a profile and run under the embedded default instead.
	SeccompProfiles map[string]string `json:"seccomp_profiles,omitempty"`
}

HandOver is what PUT /challenges/{id} carries.

type HostInfo 

type HostInfo struct {
	Name   string `json:"name"`
	Target string `json:"target,omitempty"`
}

type Image 

type Image struct {
	Id    ImageId  `json:"id"`
	Host  string   `json:"host"`
	Ports []string `json:"exposed_ports"`
	Build BuildId  `json:"build"`
}

type ImageId 

type ImageId int64

type InstanceId 

type InstanceId int64

type InstanceMetadata 

type InstanceMetadata struct {
	Id          InstanceId     `json:"id"`
	IsFinalized bool           `json:"-" db:"is_finalized"`
	Ports       map[string]int `json:"ports,omitempty"`
	Containers  []string       `json:"containers"`
	LastSolved  int64          `json:"last_solved"`
	CreatedAt   *time.Time     `json:"created_at" db:"created_at"`
	Build       BuildId        `json:"build_id"`
	// Worker is the IP of the docker worker hosting this instance; empty means
	// the local (env-configured) daemon. Included in the JSON so API consumers
	// get the host along with the ports.
	Worker string `json:"worker,omitempty" db:"worker"`
	// WorkerPublic is the worker's player-facing address (IP or hostname) —
	// what consumers should hand to users, since Worker is the private
	// orchestration address. Resolved from the worker registry at read time
	// (GetInstanceMetadata), never stored; falls back to Worker when the
	// worker has no public address configured.
	WorkerPublic string `json:"worker_public,omitempty" db:"-"`
}

type LogLevel 

type LogLevel int
const (
	DISABLED LogLevel = iota
	ERROR
	WARN
	INFO
	DEBUG
)

func LogLevelFromEnv 

func LogLevelFromEnv(fallback LogLevel) (LogLevel, error)

LogLevelFromEnv reads CORK_LOGGING, falling back to the level given when it is unset or unreadable. Unreadable rather than fatal on purpose: a typo in a log level is no reason to refuse to start a daemon, and the complaint it returns is logged once the logger exists.

This is the whole of that setting's effect, and for a while it had none: the constant was declared and read nowhere, corkd hardcoded INFO and offered no flag, so there was no way to raise an orchestrator's logging at all -- while `corkd --help` went on documenting it.

func (LogLevel) String 

func (l LogLevel) String() string

String names a level the way CORK_LOGGING spells it.

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

func NewManager 

func NewManager(logLevel LogLevel, options ...ManagerOption) *Manager

Creates a new instance of the challenge manager validating the appropriate environment variables in the process. A return value of `nil` indicates a fatal error occurred during intitialization. The options say what the calling process is, which its environment does not get to contradict (see ManagerOption).

func (*Manager) AddWorker 

func (m *Manager) AddWorker(ip, public string) error

AddWorker registers the worker at the given private IP, with an optional player-facing public address (IP or hostname; empty means players are given the private IP). Workers are keyed by private IP; re-adding an existing one (e.g. to recover a sticky-down worker) tears down its old connection and poller, starts fresh, and replaces the public address with the one given. Either way the box is reconciled before it takes placements: see reconcileWorker.

func (*Manager) BasePinFingerprint 

func (m *Manager) BasePinFingerprint() uint32

BasePinFingerprint is the identity contribution of the pins in force here: 0 with pinning off. It is an input to every build's content checksum (contentChecksum), and the one input a daemon taking a hand-over cannot derive for itself, the pins living where the builds are made, so the build plane states it and the daemon recomputes with it (HandOver).

func (*Manager) Build 

func (m *Manager) Build(challenge ChallengeId, seeds []int, flagFormat string) ([]*BuildMetadata, error)

Templates out a "challenge" and generates concrete images, flags, and lookup values for the seeds provided which is called a "build" and returns a list of identifiers that can be used to reference the build in other API functions. This function may take a significant amount of time because it will implicitly download base docker images and build the artifacts.

NOTE: if `CORK_REGISTRY` is specified, the generation this build displaces is offered to BuildKit as a cache source and the image produced carries inline cache metadata, so a builder whose local cache was reclaimed recovers the shared layers from the registry rather than re-running every install. See cacheRefsFor and BUILDER.md. (This replaced a per-challenge "freeze" image, which is gone.)

func (*Manager) BuildPlane 

func (m *Manager) BuildPlane() string

BuildPlane reports the configured build plane, BuildPlaneLocal or BuildPlaneExternal.

func (*Manager) ContentChecksum 

func (m *Manager) ContentChecksum(sourceChecksum uint32, format string, pinFingerprint uint32, challengeType string) uint32

ContentChecksum is the identity a build has under the inputs given: its source generation, its flag format, a base image pin fingerprint and the challenge type whose template it is built from. It is what a hand-over states and what the daemon taking one recomputes (see checkHandOver), so a build plane can ask the same question of its own builds before it hands them over -- a build made under other pins carries an identity these inputs do not give, and would be refused.

func (*Manager) CreateSchema 

func (m *Manager) CreateSchema(schema *Schema) []error

Uses the schema as a definition of builds and instances that should be created/started. Prevents management of those builds and instances from other API calls unless explicitly allowed by the schema. This call is likely to be extremely time and resource intensive as it will start creating all of the requested builds immediately and not return until complete.

Schema operations share updateMu with rebuilds (UpdateWithOptions): a converge builds, stops and launches instances of the same builds an update rebuilds and restarts, and the two interleaved would work over each other's instances just as two updates would. The lock is taken here, at the API boundary, and never inside: the unlocked helpers below are what the operations call each other through.

func (*Manager) DeleteSchema 

func (m *Manager) DeleteSchema(name string, retire bool) error

Tears down all instances and builds belonging to the schema. Serialized with rebuilds and other schema operations; see CreateSchema. DeleteSchema takes a schema off this daemon. retire says what becomes of the registry tags its builds carry: true removes them, which is what a removal means; false leaves them, which is what a migration to another orchestrator means -- the content is changing hands, not ending, and the tag is content-addressed so the orchestrator taking it resolves the same one.

func (*Manager) Destroy 

func (m *Manager) Destroy(build BuildId) error

Destroys the assoicated "build".

func (*Manager) DetectChanges 

func (m *Manager) DetectChanges(fp string) *ChallengeUpdates

Traverses the entire directory and captures all valid challenge descriptions it comes across. In general, it will continue even when it encounters errors (permission, poorly formatted JSON, etc.) in order to give the as much feedback as possible to the caller. However, it will fail fast on two challenges with the same name and namespace.

This function does not have any side-effects on the database or built/running challenge state, but changes that it detects will effect new builds. It is important to resolve any issues/errors it raises before making any other API calls for affected challenges. Failure to follow this guidance could result in inconsistencies in deployed challenges.

func (*Manager) DumpState 

func (m *Manager) DumpState(challenges []ChallengeId) ([]*ChallengeMetadata, error)

func (*Manager) EnableWorkerPlacement 

func (m *Manager) EnableWorkerPlacement()

EnableWorkerPlacement turns on worker selection for new instances. Only corkd calls this; without it, configured workers are still routable for operations on their existing instances, but new instances stay local.

func (*Manager) GetBuildMetadata 

func (m *Manager) GetBuildMetadata(build BuildId) (*BuildMetadata, error)

func (*Manager) GetChallengeMetadata 

func (m *Manager) GetChallengeMetadata(challenge ChallengeId) (*ChallengeMetadata, error)

func (*Manager) GetDockerfile 

func (m *Manager) GetDockerfile(challengeType string) []byte

Returns a byte array with the contents of the Dockerfile associated with `challengeType` (if it exists). If the challenge type does not exist, then an empty array is returned.

func (*Manager) GetInstanceMetadata 

func (m *Manager) GetInstanceMetadata(instance InstanceId) (*InstanceMetadata, error)

func (*Manager) GetSchemaState 

func (m *Manager) GetSchemaState(name string) ([]*ChallengeMetadata, error)

Returns the fully-nested metadata for the schema from challenges to the associated builds which belong to the schema through to the instances currently running (to include dynamic instances).

func (*Manager) HandOverChallenge 

func (m *Manager) HandOverChallenge(id ChallengeId, handOver *HandOver, options UpdateOptions) (*ChallengeUpdates, error)

HandOverChallenge records a challenge and the builds handed over with it, verifying first and writing only then. The verdict comes back the way an update reports one: the challenge in exactly one of Added, Updated, Refreshed, Stale and Unmodified, as recorded, its builds with their ids; and in Errors whatever could not be committed once the writes had begun, per build, as a rebuild reports its failures. An error instead is a refusal that wrote nothing: ErrLocalBuildPlane, ErrHandOverInvalid or ErrNotInRegistry, or the registry or the database not answering.

A build whose row already serves the generation delivered is left as it is, instances included, so handing the same thing over twice changes nothing -- the guarantee an update gives an unmodified challenge.

func (*Manager) ListBasePins 

func (m *Manager) ListBasePins() ([]BasePin, error)

ListBasePins reports the current pins alongside how many challenges use each.

func (*Manager) ListChallenges 

func (m *Manager) ListChallenges() []*ChallengeMetadata

Obtains a list of challenges with minimal version information filled into the metadata object.

Example

Iterates over all of the discovered challenges 

mgr := NewManager(WARN)

for _, c := range mgr.ListChallenges() {
	fmt.Printf("%s (%s)", c.Id, c.Name)
}

func (*Manager) ListSchemas 

func (m *Manager) ListSchemas() ([]string, error)

Lists all schemas as currently defined in the database.

func (*Manager) ListWorkers 

func (m *Manager) ListWorkers() ([]WorkerInfo, error)

ListWorkers returns every configured worker with its health and the number of instances currently recorded on it.

func (*Manager) Prune 

func (m *Manager) Prune() error

func (*Manager) RefreshBasePins 

func (m *Manager) RefreshBasePins() ([]BasePin, error)

RefreshBasePins resolves every base reference the challenge directory uses to the digest the registry serves right now and persists the result. This is the one moment cork consults a mutable tag; builds never do.

Existing pins are re-resolved too, so a refresh is how a base is moved.

func (*Manager) RelocateArtifactBundles 

func (m *Manager) RelocateArtifactBundles(schema string) (int, error)

RelocateArtifactBundles moves a schema's existing bundles into the directory its destination now names, and reports how many it moved.

It exists because giving a schema a destination changes nothing a build is made of. Identity is source, flag format, pins and template; a schema is never hashed. So adding `destination:` to a schema whose builds are all complete leaves generateBuilds short-circuiting on buildsComplete, executeBuild never runs, no bundle is promoted, and the files stay where they were -- while SetArtifactNamespaces has already made the new directory and every message says the deploy succeeded. The artifact server goes on publishing the old prefix and the new one stays empty.

Only out of the artifact directory itself, and for that the same reason the prune has: a bundle is named "<build id>.tar.gz" and an id is unique only within one plane's database, so a file of that name under ANOTHER destination is not evidence of anything and is left alone. The artifact directory itself is where a schema's bundles were written before it had a destination, which is the case this exists for. A destination that changes from one name to another goes through migrate-schema, which drops the rows and rebuilds.

func (*Manager) RemoveChallenge 

func (m *Manager) RemoveChallenge(id ChallengeId) error

RemoveChallenge takes a challenge off the catalogue: on an external build plane the counterpart of the hand-over that put it there, for a challenge the build plane no longer has. It has to have no builds on record: those go with the schema that wants them (DeleteSchema) or with Destroy, never silently with the challenge. A local build plane accepts it too, though there the tree decides: the next update finds the challenge again unless it is gone from the tree as well.

func (*Manager) RemoveWorker 

func (m *Manager) RemoveWorker(ip string) error

RemoveWorker purges the worker: its instance rows (cascading port and container assignments) and its registry entry are deleted, and its poller is stopped. Containers still running on a live worker are not touched now: the box is expected to be on its way out. Should it ever be re-added, reconcileWorker removes them then. This is the path for a box that is terminated and recreated rather than rebooted or repaired: remove the old entry, worker-add the new box.

func (*Manager) SearchChallenges 

func (m *Manager) SearchChallenges(tags []string) []*ChallengeMetadata

Obtains a list of challenges which match on all of the given tags. If no tags are passed, then it returns the same results as `ListChallenges`. Wildcards are allowed as either '*' or '%' and the search is ASCII case insensitive.

func (*Manager) SetArtifactNamespaces 

func (m *Manager) SetArtifactNamespaces(bySchema map[string]string) error

SetArtifactNamespaces says which directory under the artifact directory each schema's bundles belong in: schema name to namespace, which a build plane fills in from every schema's destination. A schema named here with an empty namespace keeps the artifact directory itself -- a single-host deployment, or a schema with no destination. A schema not named here at all is a different case, and artifactDirForBuild treats it as one.

This is what keeps one build plane's bundles sorted for the several orchestrators it builds for: what publishes artifacts to players watches one namespace and uploads it under one prefix, so a challenge's files land where that destination's players look for them. It is also why a namespace is the destination's name exactly, with nothing prepended: the name an operator wrote in CORK_DESTINATIONS is the name on disk, and the prefix a deployment wants is a destination named that way rather than a convention cork invented.

Keyed on the schema rather than set around each build for a reason that cost a defect to find: an update rebuilds every build of a challenge whose source moved, and those rebuilds happen before any schema is converged and span every schema on the plane at once. A "current namespace" moved between schemas would send all of them to whichever directory happened to be set, while a build row already carries the schema it belongs to.

Called once, before anything is built, and never concurrently with a build. Nothing else calls it: an orchestrator holds no bundles.

func (*Manager) SetWorkerDown 

func (m *Manager) SetWorkerDown(ip string) error

SetWorkerDown marks a worker down administratively, taking it out of placement without touching its registry entry or instance records (unlike RemoveWorker, which purges both). Intended for a box about to be taken out of service, for a reboot, a repair or its termination: down is sticky — the poller treats it as terminal — so recovery is AddWorker on the same IP, which rebuilds the connection and poller, or RemoveWorker and AddWorker of the replacement.

Note this also switches that worker's instances to the DB-only stop path (see stopInstance): their containers are no longer torn down over docker. The worker-add that recovers the box removes what those stops left behind.

func (*Manager) Start 

func (m *Manager) Start(build BuildId, envVars map[string]string) (InstanceId, error)

Creates a running "instance" of the given build and returns its identifier on success otherwise an error. The optional envVars map provides extra environment variables to inject into the instance's containers. Keys in envVars should already include the "CMGR_" prefix. Pass nil if no additional environment variables are needed.

func (*Manager) Stop 

func (m *Manager) Stop(instance InstanceId) error

Stops the running "instance".

func (*Manager) Update 

func (m *Manager) Update(fp string) *ChallengeUpdates

This will update the global system state based off the changes that are detected by a call to `DetectChanges`. Specifically, in addition to updating challenge metadata (new and existing) it will rebuild and, if successful restart, existing challenges and then remove the metadata for challenges that can no longer be found. Challenges that have not been modified should not be affected.

In the presence of errors, this function will do addition and updates as best it can in order to preserve a consistent system state. If a build fails, the build keeps its previous generation (row, images, instances) while the challenge metadata is already updated; the challenge is then reported as Stale by every later DetectChanges, and rebuilt by every later update, until a rebuild succeeds. Additionally, in the presence of errors it will not perform any removals of challenge metadata (removing a built challenge is considered an error).

func (*Manager) UpdateSchema 

func (m *Manager) UpdateSchema(schema *Schema) []error

Updates the definition of the schema internally and then converges to the new definition. Certain updates are more expensive than others. In particular, updating the flag format will cause a complete rebuild of the state. Serialized with rebuilds and other schema operations; see CreateSchema.

func (*Manager) UpdateWithOptions 

func (m *Manager) UpdateWithOptions(fp string, options UpdateOptions) *ChallengeUpdates

UpdateWithOptions is identical to Update but takes explicit options; Update is equivalent to calling this with the zero-value UpdateOptions.

func (*Manager) WorkersConfigured 

func (m *Manager) WorkersConfigured() bool

WorkersConfigured reports whether any workers exist; used by corkd to decide between the legacy single-host gate and worker placement.

type ManagerOption 

type ManagerOption func(*Manager)

ManagerOption tells NewManager what the process it is being built for is, as against what the environment it is being built in says. What a binary is, no variable should be able to contradict.

func AsBuildPlane 

func AsBuildPlane() ManagerOption

AsBuildPlane marks a manager as a build plane that serves nothing, which is what cork-build is: it builds and pushes, and it never takes a tag back out of the challenge registry.

The registry is shared and write-once, and deciding a tag is spent means knowing every row that still names its content. A build plane cannot know. Its database is bookkeeping -- thrown away and re-derived at will, and carrying only what this build plane itself built -- while the builds it no longer names are still being served by an orchestrator that was handed them. Left to decide for itself it would untag, on a dropped seed or a changed flag format, images a running event still pulls. Retiring them is the orchestrator's to do, whose own schema converge releases those same builds against the database that does know what references them.

type NetworkOptions 

type NetworkOptions struct{}

type PortInfo 

type PortInfo struct {
	Host string `json:"host"`
	Port int    `json:"port"`
}

type Schema 

type Schema struct {
	Name       string                             `json:"name"        yaml:"name"`
	FlagFormat string                             `json:"flag_format" yaml:"flag_format"`
	Challenges map[ChallengeId]BuildSpecification `json:"challenges"  yaml:"challenges"`
	// Destination names the orchestrator this schema is served by, as a
	// short name a build plane resolves against its configured
	// destinations rather than an address (see cork-build). A field and
	// not a header comment: a schema may be json, which has no comments,
	// and where an event is served is part of what the schema is.
	//
	// Nothing here hashes a schema -- a build's identity is its source,
	// flag format, base pins and template (contentChecksum) -- so this
	// changes no build that already exists. It is meaningless to a daemon,
	// which is told what it serves rather than choosing; an orchestrator
	// ignores it.
	Destination string `json:"destination,omitempty" yaml:"destination,omitempty"`
}

type SeccompOptions 

type SeccompOptions struct {
	Profile     string `json:"profile,omitempty"      yaml:"profile"`
	ProfileHash string `json:"profile_hash,omitempty" yaml:"-"`
	// contains filtered or unexported fields
}

SeccompOptions selects the seccomp policy applied to a challenge's runtime containers. A challenge that sets no profile keeps cork's embedded policy.

type UnknownIdentifierError 

type UnknownIdentifierError struct {
	Type string
	Name string
}

func (*UnknownIdentifierError) Error 

func (e *UnknownIdentifierError) Error() string

type UpdateOptions 

type UpdateOptions struct {
	// PruneOldImages untags the image generation a rebuild displaces from its
	// rollback slot: each build keeps its new images plus the generation it
	// just replaced (the rollback target, see BuildMetadata.PrevChecksum), and
	// the one pushed beyond that is untagged after all of the challenge's builds
	// are processed. Only generations displaced by THIS update are removed —
	// generations orphaned by earlier updates that ran without this flag are
	// not tracked and are not reclaimed here. Off by default: a tag can be
	// referenced outside this cork database (e.g. another cork instance on the
	// same docker daemon that built identical content), and cork cannot see
	// those references.
	PruneOldImages bool
}

UpdateOptions adjusts the behavior of `UpdateWithOptions`.

type WorkerInfo 

type WorkerInfo struct {
	IP        string `json:"ip"`
	Public    string `json:"public"`
	Health    string `json:"health"`
	Instances int    `json:"instances"`
}

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.