Help us shape the future of Go! Take the 2026 Go Developer Survey

hosts

package
v0.17.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: AGPL-3.0 Imports: 44 Imported by: 0

Documentation

Overview

Package hosts is the install's cluster: the hosts that have joined it and the join tokens they join with (R-414, R-415, docs/design/notes-cluster.md).

It models membership, roles and liveness. It never chooses where a workload runs: that stays the runtime adapter's (R-256), and nothing here may rank hosts for one (R-010).

Index

Constants

View Source
const (
	JoinedFile        = "joined.json"
	HostCertFile      = "host.crt"
	HostKeyFile       = "host.key"
	AuthorityCertFile = "ca.crt"
	// HostClientCertFile and HostClientKeyFile are what the host checks the
	// others' agents with (R-421); PeersFile is the cluster's hosts as the
	// host last heard them (PeerSync).
	HostClientCertFile = "host-client.crt"
	HostClientKeyFile  = "host-client.key"
	PeersFile          = "peers.json"
	// HostNextKeyFile is the key a renewal not yet answered and written asks
	// a certificate for (HostRenewal.nextKey, R-421).
	HostNextKeyFile = "host-next.key"
	// JoinKeyFile is the key a join not yet finished asks a certificate for,
	// kept so a join tried again asks for the same one and replaces its own
	// pending join rather than being taken for another machine of the same
	// name (R-415, Joiner.Join).
	JoinKeyFile = "join.key"
)

Files in the cluster directory.

View Source
const (
	DefaultTokenLifetime = 24 * time.Hour
	MaxTokenLifetime     = 30 * 24 * time.Hour
	MaxTokenUses         = 100
)

Join token limits (R-415).

View Source
const (
	AuthorityPath = "/cluster/authority"
	JoinPath      = "/cluster/join"
	// PeersPath lists the cluster's hosts to a host that has joined, which
	// presents its own certificate (Service.Peers, R-421).
	PeersPath = "/cluster/peers"
	// RenewPath issues a joined host new certificates before its own expire,
	// to the host presenting the one it holds (Service.Renew, R-421).
	RenewPath = "/cluster/renew"
)

Paths on the cluster listener.

View Source
const ConvertAction = "cluster.runtime.convert"

ConvertAction is the audit action of turning the runtime into docker-hosts.

View Source
const DefaultAuthorityPath = "/var/lib/pando/cluster-ca.pem"

DefaultAuthorityPath is where the cluster's certificate authority lives unless server.cluster_authority_path says otherwise: beside the token key. A control host is given a copy when it joins (R-415, R-417).

View Source
const DefaultClusterDir = "/var/lib/pando/cluster"

DefaultClusterDir is where a joined host keeps what it was given (R-415), unless server.cluster_dir says otherwise.

View Source
const DefaultClusterPort = 7444

DefaultClusterPort is where a control host's cluster listener answers unless server.cluster_addr says otherwise (R-415): the port after the host agent's (hostagent.DefaultPort), which a host holding both roles also listens on.

View Source
const DefaultJoinCheckTimeout = 30 * time.Second

DefaultJoinCheckTimeout is how long a join's checks may take before the join is refused (R-415 [P]).

View Source
const InvalidField = "(invalid)"

InvalidField stands in for a name or version that failed CheckJoinRequest in the audit log, which keeps what a caller who has not joined sent only when it is something a host could be called (R-347: the log cannot be trimmed for three months).

View Source
const ListenerCommonName = "pando-cluster-listener"

ListenerCommonName and listenerURI name the cluster listener's certificate (R-415). Every host holds a certificate from the same authority, so chaining to it says only "a member of this cluster": a host that checked no more would take any member, an app host included, for the listener, and hand it a join token's secret. Only listenerCert issues a certificate carrying both, and only a holder of the authority's key, a control host, can run it; a host's own certificate is named for the host (hostagent.ServerName), and a probe's, the collector's and the proxy's carry other subjects.

View Source
const RevokedFile = "revoked.json"

RevokedFile, in the cluster directory, is every host removed from the cluster as this host last heard it (Members).

Variables

This section is empty.

Functions

func CheckAgentAddress

func CheckAgentAddress(address string) error

CheckAgentAddress refuses an agent address the runtime cannot dial (R-416): it must be host:port, a DNS name or an IP address and a port number. One that is not would break the runtime's host list for every host.

func CheckAppRoleJoin

func CheckAppRoleJoin(runtimeKind string, roles []string) error

CheckAppRoleJoin refuses an app-role join where the runtime cannot take one (R-418): Kubernetes and Incus already span machines, so a host joins them for the control role only.

func CheckHostName

func CheckHostName(name string) error

CheckHostName refuses a name a host cannot have (R-415): one the Docker hosts runtime would not take (runtimeHostName, its own rule), since a joined app host is written into its host list under it and one bad name stops every later sync of that list (R-418), or installHostName, which the runtime's own entry for the install's machine may already have. Which names the runtime's host list holds is checked once the token is (Runtime.CheckJoin).

func CheckJoinRequest

func CheckJoinRequest(req JoinRequest) error

CheckJoinRequest refuses what a joining host sent when Pando cannot record or use it (R-415): every check that needs no token, so that a refusal says nothing about the cluster.

func ControlHostCert

func ControlHostCert(dir string, a *Authority, name string, roles []string, now time.Time) (tls.Certificate, string, error)

ControlHostCert is the certificate a control host answers the cluster's liveness checks with (R-421), and its fingerprint, which ControlPeers records as the host's cert_sha256. It is host.crt and host.key in dir: what a joined control host was issued at its join. When they are missing, unreadable, issued for another name or due for renewal, a new one is issued from the authority, which a control host holds, and written there: the install's first host, which never joined, gets its own this way, and keeps it across restarts so its pin does not change at each start.

func ControlRoleNeedsEveryVerb

func ControlRoleNeedsEveryVerb(roles []string) bool

ControlRoleNeedsEveryVerb reports whether minting a token for roles hands over every key the install has: a control host receives the secrets key and the token key at join (R-415, R-417).

func Encode

func Encode(address, authorityHash string, tokenSecret secret.Value) string

Encode is the string a joining host is given: pj1.<base64url address>.<authority hash>.<token id>.<secret>.

func InstallHost

func InstallHost(h state.Host) bool

InstallHost reports whether h is the install's own first host: the machine the install started on, recorded at its first start (RecordInstallHost) rather than brought in by a join token.

func InstallHostName

func InstallHostName(hostname string) string

InstallHostName is the first label of hostname, lower case, with anything a DNS label cannot hold turned into a hyphen: "Build-01.example.com" is build-01. It is cut to 40 characters, the most the docker-hosts runtime takes, since the runtime names the machine as its row does (Runtime's localName, R-421). A name that leaves nothing usable is "control".

func JoinedTLS

func JoinedTLS(authorityPEM []byte, cert tls.Certificate) (*tls.Config, error)

JoinedTLS is how a joined host connects to the cluster listener: its own certificate, and the listener's checked against the cluster authority the host was given (ca.crt), whatever name the join token's address uses.

func ListenerAddress

func ListenerAddress(address string) (string, error)

ListenerAddress is address as a join token names it: the cluster listener's, https://<host>:<port>, with the default port when it has none (R-415). The listener serves TLS only, so plain http is refused.

func NormalizeRoles

func NormalizeRoles(roles []string) ([]string, error)

NormalizeRoles checks roles and returns them sorted and without repeats.

func PingAddress

func PingAddress(address string, port int) string

PingAddress is address, host:port, with its port replaced by port: where a control host answers pings, on its cluster listener, at the machine it recorded. An address without a port is taken as a host.

func PinnedTLS

func PinnedTLS(hash string) *tls.Config

PinnedTLS is the TLS configuration a host connects to the cluster listener with, joining or joined (R-415). It trusts exactly one authority, the one whose public key hashes to hash, and refuses the server before anything is sent unless the server's certificate chains to it and is the listener's (isListener). The server's name is not checked: the token's address may be an IP address or a name the listener's certificate does not carry, and the pinned authority with the listener's identity is the trust, as in kubeadm's discovery.

func RecordInstallHost

func RecordInstallHost(ctx context.Context, store InstallHostStore, info InstallHostInfo) (state.Host, bool, error)

RecordInstallHost records the install's own machine as its first host, with both roles, unless a first host is recorded already. It reports whether it recorded one. A host that joined a cluster never calls it: it was recorded by its join.

func RenewingControlHostCert

func RenewingControlHostCert(dir string, a *Authority, name string, roles []string, now func() time.Time) (*hostagent.RenewingCert, error)

RenewingControlHostCert is ControlHostCert held by a running control host, issued again, and written over the old one, once it is due (R-421): the cluster listener answers pings with what it holds at each handshake.

func WritePeers

func WritePeers(path string, list []Peer) error

WritePeers keeps list at path, for ReadPeers, replacing what it held: the peer list a host probes when it cannot ask for a fresh one (R-421).

Types

type AdapterStore

type AdapterStore interface {
	List(ctx context.Context) ([]state.AdapterConfig, error)
	Save(ctx context.Context, c state.AdapterConfig) (state.Replaced, error)
}

AdapterStore is the slice of state.Adapters the runtime sync uses.

type AuditWriter

type AuditWriter interface {
	Write(ctx context.Context, e audit.Event) error
}

AuditWriter appends to the audit log.

type Authority

type Authority struct {
	Cert *x509.Certificate
	Key  crypto.Signer
}

Authority is the cluster's certificate authority. A host's certificate is issued from it at join, and a joining host checks it against the hash in its join token before sending anything (R-415).

func LoadOrCreateAuthority

func LoadOrCreateAuthority(path string) (*Authority, error)

LoadOrCreateAuthority reads the authority at path, generating it on first use. Written by keyfile.Create, so two replicas starting on an empty shared volume end with one authority, as with the token key.

func (*Authority) Authorities

func (a *Authority) Authorities() hostagent.Authorities

Authorities is the authority as the host agent's package takes it.

func (*Authority) CertPEM

func (a *Authority) CertPEM() []byte

CertPEM is the authority's certificate, without its key: what every host is given to check the others' certificates against.

func (*Authority) Hash

func (a *Authority) Hash() string

Hash is the SHA-256 of the authority's public key, hex: what a join token carries, as kubeadm's discovery hash does.

func (*Authority) IssueHost

func (a *Authority) IssueHost(csrPEM []byte, name string, roles []string, now time.Time) (certPEM []byte, sha string, err error)

IssueHost signs a joining host's certificate request (R-415) as the host agent expects a host's certificate (hostagent.Authorities.IssueHost, R-416): named pando-agent.<name>.invalid, its roles as organizational units, for server and client authentication. Only the request's key is taken from it; every other field is the authority's choice. It is valid from now; a host renews it before it expires (Service.Renew, R-421).

func (*Authority) IssueHostClient

func (a *Authority) IssueHostClient(hostID string, now time.Time) (certPEM []byte, keyPEM secret.Value, err error)

IssueHostClient issues the certificate the host hostID checks the other hosts with (hostagent.Authorities.IssueHostClient, R-421), as PEM: it carries the host's ID, which is who its pings are from.

func (*Authority) ListenerTLS

func (a *Authority) ListenerTLS(names []string, now time.Time) (*tls.Config, error)

ListenerTLS is the TLS configuration of the cluster listener (R-415): a certificate issued from the authority now, for names, sent with the authority's own so a joining host can check the chain against its token's hash before it sends anything. A client certificate is checked against the authority when one is presented: a joining host has none yet, and a host that has joined presents its own for the peer list (PeersPath). The certificate is issued again while the listener runs, before it expires.

func (*Authority) PEM

func (a *Authority) PEM() ([]byte, error)

PEM is the authority as the runtime's agent_authority credential holds one: its certificate, then its key.

type BackupDestinations

type BackupDestinations interface {
	Default(c api.Category) (string, bool)
	Backup(ref string) (api.BackupAdapter, bool)
}

BackupDestinations is the slice of *api.Registry the check reads.

type ControlKeys

type ControlKeys struct {
	DatabaseURL secret.Value
	// SecretsKey is the local secrets adapter's key, empty when the install
	// uses another secrets adapter. SecretsKeyPath is where that adapter's
	// configuration, which the joining host reads from the shared database,
	// says it lives.
	SecretsKey     secret.Value
	SecretsKeyPath string
	TokenKey       secret.Value
	// Authority is the cluster authority's certificate and key, PEM.
	Authority secret.Value
}

ControlKeys are what a control host is handed at join and an app host never is (R-415, R-417): the database's address and the keys that protect what is in it against a dump (R-190, R-063).

type ControlPeers

type ControlPeers struct {
	Store HostPublisher
	Self  string
	Port  int
	Pin   string
	// PinOf, when set, is the fingerprint of the certificate the host
	// answers with now, in place of Pin: the certificate is renewed while
	// the host runs (R-421).
	PinOf  func() string
	Logger *zap.Logger
}

ControlPeers is a control host's peer list (R-421), read from the database each round. Reading it also keeps the host's own row saying where it answers pings, its cluster listener at Port, and the fingerprint of the certificate it answers with, Pin, in the row's ping columns. The row's address and pin stay the agent's, which the runtime reaches the host through (R-416): a host with both roles runs apps through its agent and answers pings on its listener. Written whenever it differs, so a write that failed is made the next round.

A joined control host answers pings with the certificate its join wrote, which its agent also serves when it holds the app role, and renews it while it runs (ControlHostCert). So when that pin is not yet the row's agent pin, it becomes it and the old one is kept as the one before (RotateHostCert): the runtime accepts both (state.Host.Pins) until the agent serves the new one. The install's own host has no agent of its own.

func (*ControlPeers) Peers

func (c *ControlPeers) Peers(ctx context.Context) ([]Peer, error)

Peers lists every host, this one with where it answers now.

type CredentialStore

type CredentialStore interface {
	Put(ctx context.Context, adapterID, field string, v secret.Value) error
}

CredentialStore is the slice of state.AdapterCredentials it uses.

type Decoded

type Decoded struct {
	Address       string
	AuthorityHash string
	Secret        secret.Value
}

Decoded is a join token taken apart.

func Decode

func Decode(token string) (Decoded, error)

Decode takes a join token apart, without checking it with anyone.

type EveryVerbChecker

type EveryVerbChecker interface {
	CheckHoldsEveryInstallVerb(ctx context.Context, p authz.Principal, act, because string) error
}

EveryVerbChecker refuses a principal that does not hold every install verb (authz.Authorizer.CheckHoldsEveryInstallVerb, R-080a).

type HostCerts

type HostCerts struct {
	Host  *hostagent.RenewingCert
	Probe *hostagent.RenewingCert
}

HostCerts are a joined host's certificates while it runs (R-421): its own, which its agent serves and which it presents to the cluster listener, and its probe certificate, which it checks the other hosts with. Both are renewed together from the cluster listener once its own is due, written over the files the join left (JoinFiles), and taken by the next handshake.

type HostLister

type HostLister interface {
	Hosts(ctx context.Context) ([]state.Host, error)
}

HostLister lists the cluster's hosts.

type HostPublisher

type HostPublisher interface {
	HostLister
	// Revoked is every host removed from the cluster, whose pings a control
	// host refuses (R-415, Members).
	Revoked(ctx context.Context) (state.Revoked, error)
	SetHostPing(ctx context.Context, hostID, address, certSHA256 string) error
	RotateHostCert(ctx context.Context, hostID, next string) error
}

HostPublisher is the slice of state.Cluster a control host keeps its own row current with.

type HostRenewal

type HostRenewal struct {
	// Address is the cluster listener, https://host:port: the join token's.
	Address string
	// Dir is the cluster directory the join wrote to.
	Dir  string
	Name string
	// AuthorityPEM is the cluster authority's certificate (ca.crt).
	AuthorityPEM []byte
	// HTTP makes the client a renewal connects with; nil is one with a
	// one-minute timeout.
	HTTP func(*tls.Config) *http.Client
	// Now is the clock; nil is time.Now.
	Now func() time.Time
}

HostRenewal is how a joined host renews its certificates.

func (*HostRenewal) Certs

func (r *HostRenewal) Certs(host, probe tls.Certificate) *HostCerts

Certs holds host and probe, renewing them from the cluster when host is due.

func (*HostRenewal) Renew

func (r *HostRenewal) Renew(ctx context.Context, current tls.Certificate) (host, probe tls.Certificate, err error)

Renew asks the cluster for new certificates, presenting current, for a new key, checks what it is given, and writes it to Dir. It returns the host's new certificate and its new probe certificate.

type InstallHostInfo

type InstallHostInfo struct {
	// Hostname is the machine's; the host is named by its first label.
	Hostname string
	// Address is where its agent is reached, host:port.
	Address  string
	Platform string
	Version  string
}

InstallHostInfo is what the first host is recorded with.

type InstallHostStore

type InstallHostStore interface {
	RecordFirstHost(ctx context.Context, h state.Host) (state.Host, bool, error)
}

InstallHostStore records the first host: state.Cluster.

type JoinFiles

type JoinFiles struct {
	// Dir holds the host's certificate and key, the authority's
	// certificate, the database address for a control host, and joined.json,
	// written last, which says the join is done.
	Dir string
	// DatabaseURLPath is where config.Load reads a joined control host's
	// database address from, ahead of database.url.
	DatabaseURLPath string
	// TokenKeyPath and AuthorityPath are server.token_key_path and
	// server.cluster_authority_path: a control host's keys replace what its
	// own first start made there.
	TokenKeyPath  string
	AuthorityPath string
}

JoinFiles is where a joining host writes what it was given.

func (JoinFiles) Read

func (f JoinFiles) Read() (Joined, bool, error)

Read returns the join this Pando made, or ok false when it has not joined.

type JoinReadiness

type JoinReadiness interface {
	ReadyToJoin(ctx context.Context, roles []string) error
}

JoinReadiness says whether the cluster can take a host with roles now, such as a second host needing a backup destination off the host first (R-417). Optional; a join asks it before anything else, and a refusal spends no use of the token.

type JoinRequest

type JoinRequest struct {
	Secret   secret.Value
	Name     string
	CSR      []byte
	Platform string
	Version  string
	// Address is where the joining host's agent listens, host:port (R-416).
	Address string
}

JoinRequest is what a joining host sends on the cluster listener (R-415).

func (JoinRequest) AuditDetail

func (req JoinRequest) AuditDetail() map[string]any

AuditDetail is what the audit log records of a join that was refused.

type JoinResult

type JoinResult struct {
	Host state.Host
	// Certificate is the host's own, issued from the authority; Authority is
	// the authority's certificate. Both PEM.
	Certificate []byte
	Authority   []byte
	// HostClientCert and HostClientKey are the certificate, and its key, the
	// host checks the others' agents with (R-421): subject pando-host, client
	// authentication only, which an agent answers with a ping and nothing
	// else. Issued here because only a holder of the authority's key can
	// issue one. PEM.
	HostClientCert []byte
	HostClientKey  secret.Value
	// NetworkPool is the runtime's app network range, which the host's agent
	// forwards into and nowhere else (R-416). Empty is the runtime's default.
	NetworkPool string
	// Keys is set for a host with the control role, and only then.
	Keys *ControlKeys
}

JoinResult is what a host that joined is given.

func (JoinResult) Wire

func (r JoinResult) Wire() WireJoinResult

Wire is the result as the listener sends it.

type Joined

type Joined struct {
	HostID   string    `json:"host_id"`
	Name     string    `json:"name"`
	Roles    []string  `json:"roles"`
	Cluster  string    `json:"cluster"`
	JoinedAt time.Time `json:"joined_at"`
	// NetworkPool is the runtime's app network range when the host joined,
	// which its agent forwards into (R-416). Empty is the runtime's default.
	NetworkPool string `json:"network_pool,omitempty"`
}

Joined is what joined.json records: that this Pando is a host of a cluster, and as what.

func (Joined) AppOnly

func (j Joined) AppOnly() bool

AppOnly reports whether the host holds the app role and not the control role: it runs as a host agent and serves no API (R-414).

type Joiner

type Joiner struct {
	// Fresh reports whether this Pando has had an account or an app; a Pando
	// that has cannot join (R-415).
	Fresh func(ctx context.Context) (accounts, apps bool, err error)

	Files JoinFiles

	// Version and Platform are this binary's; AgentAddress is where this
	// host's agent will listen, host:port (R-416).
	Version      string
	Platform     string
	AgentAddress string

	// HTTP makes the client a join connects with, from the pinned TLS
	// configuration. Nil is a client with a one-minute timeout.
	HTTP func(*tls.Config) *http.Client

	// DefaultName is the host's name when the join gives none: this
	// machine's hostname.
	DefaultName string

	// Restart starts Pando again once a join made at setup has been answered,
	// into the cluster: as a replica for the control role, as a host agent
	// for the app role alone.
	Restart func()

	// SetupToken checks the setup token a join from the setup screen presents
	// (state.Users.CheckSetupToken): the one POST /setup needs, so that being
	// first to a fresh Pando is not enough to join it to a cluster of one's
	// own (R-046, R-415). Nil refuses every such join.
	SetupToken func(ctx context.Context, token secret.Value) error

	Now func() time.Time
}

Joiner is the joining side of R-415: a fresh Pando taking a join token, checking the cluster's authority against it, and being made a host.

func (*Joiner) Join

func (j *Joiner) Join(ctx context.Context, token, name string) (Joined, error)

Join joins the cluster token names, as name, and writes what it was given (JoinFiles). It does not restart Pando; the caller does, once it has answered.

func (*Joiner) Setup

func (j *Joiner) Setup(ctx context.Context, token, name string) (Joined, error)

Setup is the join a fresh Pando's setup makes (R-415): SetupWithToken and PANDO_JOIN_TOKEN, which needs no setup token, since setting it takes the host already. It joins, then asks for the restart that takes this Pando into the cluster.

func (*Joiner) SetupWithToken

func (j *Joiner) SetupWithToken(ctx context.Context, setupToken secret.Value, token, name string) (Joined, error)

SetupWithToken is the join POST /setup/join makes (R-415): refused unless setupToken is this Pando's setup token, checked before anything else.

type KeySource

type KeySource struct {
	DatabaseURL  secret.Value
	TokenKeyPath string
	// SecretsKeyPath is where the local secrets adapter keeps its key, or ""
	// when the install uses another.
	SecretsKeyPath func(context.Context) string
}

KeySource reads the keys a control host is handed (R-415). Read at each join rather than held, so what is handed over is what is on disk now.

type Members

type Members struct {
	// contains filtered or unexported fields
}

Members is the cluster's hosts as a host last heard them, by ID, and every host removed from it: a ping from a host not among them, or from one removed, is refused (R-415, hostagent.Agent.Member).

What is revoked is kept for good, here and in RevokedFile, which a host reads before it hears any list (LoadRevoked): a removed host's probe certificate may still be within its validity, its ID never comes back (a host that joins again is a new host), and a host that restarts while the control plane is away, or that reads an older peer list, still refuses it. Revocations come with the peer list, every round (PeerSync on an app host, the database on a control host), so a removal reaches every host's answers within a probe interval of being made.

func (*Members) Admits

func (m *Members) Admits(hostID, certSHA256 string) bool

Admits reports whether a ping from hostID, sent with the certificate whose fingerprint is certSHA256, is answered: the host is a member and neither it nor the certificate was revoked (R-415).

func (*Members) Has

func (m *Members) Has(hostID string) bool

Has reports whether hostID is in the cluster as last heard and was never removed. Before any list is heard every host not removed is taken, so a host that has just started answers the others rather than being reported unreached by all of them.

func (*Members) KeepRevoked

func (m *Members) KeepRevoked(path string, r state.Revoked) error

KeepRevoked adds r to what is refused and, when that added anything or the last write failed, writes everything refused to path, a RevokedFile, for the next start.

func (*Members) LoadRevoked

func (m *Members) LoadRevoked(path string)

LoadRevoked adds what path, a RevokedFile, holds. A file that is missing or unreadable adds nothing: the next peer list brings it again.

func (*Members) Revoke

func (m *Members) Revoke(r state.Revoked) bool

Revoke adds r to what is refused, and reports whether that added anything. Nothing is ever taken back.

func (*Members) Revoked

func (m *Members) Revoked() state.Revoked

Revoked is everything refused, for RevokedFile.

func (*Members) Track

func (m *Members) Track(peers func(context.Context) ([]Peer, error)) func(context.Context) ([]Peer, error)

Track is peers, remembering each list it returns.

type MintRequest

type MintRequest struct {
	Roles     []string
	MaxUses   int
	ExpiresIn time.Duration
	Address   string
}

MintRequest is what a join token is minted from. Zero values take the defaults.

type Minted

type Minted struct {
	Token state.JoinToken `json:"token"`
	Join  secret.Value    `json:"join"`
}

Minted is a join token and the string a joining host is given. Join is present exactly once, here.

type Peer

type Peer struct {
	ID      string `json:"id"`
	Name    string `json:"name"`
	Address string `json:"address"`
	// Control is that the host holds the control role, so its Address is
	// a cluster listener: where an app host asks for this list again when
	// the one in its join token is gone (PeerSync).
	Control bool `json:"control,omitempty"`
	// Lost is that the host is marked lost (R-421): it is left out of the
	// cluster's size on every host, as it is on the leader.
	Lost bool `json:"lost,omitempty"`
}

Peer is a host as another host's agent probes it.

func LivenessPeers

func LivenessPeers(list []state.Host) []Peer

LivenessPeers is the hosts that take part in the cluster's liveness checks: every host, whatever its roles, the install's own first host included (R-421). An app host answers on its agent; a control host on its cluster listener (ControlPeers, its row's ping address), so every host counted in the cluster's size is also watched, and none is reported unreached for answering nowhere.

func ReadPeers

func ReadPeers(path string) ([]Peer, error)

ReadPeers reads a peer list written by PeerSync.

type PeerList

type PeerList struct {
	Hosts   []Peer        `json:"hosts"`
	Revoked state.Revoked `json:"revoked"`
}

PeerList is what GET /cluster/peers answers a host with: the hosts it probes, and every host removed from the cluster, whose pings it refuses from then on (R-415, Members).

type PeerSync

type PeerSync struct {
	// Address is the cluster listener, https://host:port: the join token's.
	Address string
	TLS     *tls.Config
	File    string
	// HTTP is the client; nil is one with a short timeout, since a round
	// waits for it.
	HTTP *http.Client
	// Members, when set, is given every host the list says was removed,
	// and keeps them in RevokedFile beside File (R-415).
	Members *Members
}

PeerSync keeps a joined host's peer list current: every call fetches it from the cluster listener at Address and writes it to File, and when the listener cannot be reached returns what File last held (R-421: the control plane being away never stops an app). When Address does not answer, every control host in the list File holds is asked in turn, so a host whose join token named a control host that has since gone still hears of the cluster's changes.

func (*PeerSync) Peers

func (p *PeerSync) Peers(ctx context.Context) ([]Peer, error)

Peers is the peer list, fresh when a control host answers.

type PolicyLoader

type PolicyLoader interface {
	Load(ctx context.Context) (policy.Document, error)
}

PolicyLoader reads host policy as it is now.

type Readiness

type Readiness struct {

	// Routing is the adapter registry, for the check that a second control
	// host's edge can share certificates with this one's (O-68). Nil skips
	// it.
	Routing RoutingAdapters
	// contains filtered or unexported fields
}

Readiness checks an install is ready for a second host to join.

func NewReadiness

func NewReadiness(backups BackupDestinations, policy PolicyLoader, registry planner.InstallRegistries) *Readiness

NewReadiness returns the check. backups is the adapter registry, policy the host policy store, and registry the install's image registry (*imageregistry.Service).

func (*Readiness) ReadyToJoin

func (r *Readiness) ReadyToJoin(ctx context.Context, roles []string) error

ReadyToJoin refuses a join, whatever its roles, while backups, audit archives or app images would stay on this host alone (R-417), and a control host's while its edge could not share this one's certificates (O-68). Every reason is said at once, so one round of changes is enough.

type Removed

type Removed struct {
	Host state.Host `json:"host"`
	// Rotate names the keys a removed control host was handed at its join
	// (R-415), which removing it does not take back.
	Rotate []string `json:"rotate,omitempty"`
	Notice string   `json:"notice,omitempty"`
}

Removed is a host removed from the cluster, and what its removal leaves to be done by hand.

type RenewResult

type RenewResult struct {
	Host           state.Host
	Certificate    []byte
	HostClientCert []byte
	HostClientKey  secret.Value
}

RenewResult is what a host that renewed is given: its new certificate, for the key its request proved, and a new probe certificate carrying its ID.

func (RenewResult) Wire

func (r RenewResult) Wire() WireRenewResult

Wire is the renewal as the listener sends it.

type RestartSignal

type RestartSignal interface {
	RequestRestart(ctx context.Context) error
}

RestartSignal asks every replica to restart, the way saved adapter configuration starts running (R-253): state.Replicas.

type RoutingAdapters

type RoutingAdapters interface {
	ByCategory(c api.Category) []string
	Routing(ref string) (api.RoutingAdapter, bool)
}

RoutingAdapters is the slice of *api.Registry the certificate check reads.

type Runtime

type Runtime struct {
	Hosts       HostLister
	Adapters    AdapterStore
	Credentials CredentialStore
	Restart     RestartSignal
	Audit       AuditWriter

	// Authority is the cluster's: the agents' mutual TLS is the cluster's,
	// handed to the runtime as its agent_authority credential (R-416).
	Authority *Authority

	// LocalName is the name the install's own machine takes in the host list
	// when it becomes the control host and first app host (R-418), when it
	// has no row in cluster_hosts to take it from (localName). Empty is
	// "control". It names the machine's volumes from then on, so it does not
	// change.
	LocalName string

	// LocalAgentAddress is where the control host's agent is reached, as
	// host:port: this machine's address and the agent port.
	LocalAgentAddress string

	// Joined is set on a control host that joined the cluster: it is not the
	// install's own machine, so a conversion it makes takes that machine's
	// agent address from the machine's cluster_hosts row, not from
	// LocalAgentAddress (R-416).
	Joined bool

	// Declared reports whether the configuration file declares the adapter
	// with this ID (R-271), which a stored row then does not override.
	Declared func(id string) bool
}

Runtime keeps the runtime adapter's host list in line with the cluster's app hosts. The server calls Sync when the host list changes, and CheckJoin before an app-role join.

func (*Runtime) CheckJoin

func (r *Runtime) CheckJoin(ctx context.Context, name string, roles []string) error

CheckJoin refuses an app-role join the runtime cannot take, before the host is let in: a runtime spanning machines, a Docker runtime that cannot be turned into a multi-host one, or a name the runtime's host list already gives a host Pando did not add (R-415, R-418), which would stop every later Sync.

func (*Runtime) NetworkPool

func (r *Runtime) NetworkPool(ctx context.Context) string

NetworkPool is the default runtime's app network range (network_pool), the range a joining app host's agent forwards into (R-416), or "" for the runtime's own default, or when it cannot be read: the join that asks has checked the runtime already (CheckJoin).

func (*Runtime) Sync

func (r *Runtime) Sync(ctx context.Context) (SyncResult, error)

Sync writes the cluster's app hosts into the runtime adapter's host list, converting a single-host docker runtime first (R-418), and asks every replica to restart when anything changed. Running it again with the same hosts changes nothing.

type RuntimeHost

type RuntimeHost struct {
	Name             string `json:"name"`
	Endpoint         string `json:"endpoint,omitempty"`
	CertSHA256       string `json:"cert_sha256,omitempty"`
	AgentAddress     string `json:"agent_address,omitempty"`
	Control          bool   `json:"control,omitempty"`
	TotalCPUMillis   int    `json:"total_cpu_millis,omitempty"`
	TotalMemoryBytes int64  `json:"total_memory_bytes,omitempty"`
	TotalDiskBytes   int64  `json:"total_disk_bytes,omitempty"`
}

RuntimeHost is one entry of the docker-hosts runtime's host list, in its configuration's own field names.

func RuntimeHosts

func RuntimeHosts(hosts []state.Host) ([]RuntimeHost, error)

RuntimeHosts is the cluster's app hosts as docker-hosts entries, in the order they joined: each reached through its agent at the address it joined with, which must present the certificate it joined with: the agent's address and pin, never where a control host answers pings (R-416, R-421), so a host with both roles runs apps through its agent. The install's own machine (InstallHost) is the control entry already in the list and is left out. A joined host is never left out for its name: one named as the control entry is refused by mergeHosts, not dropped.

type Service

type Service struct {
	Store     Store
	Authority *Authority

	// Address is where a joining host reaches this install's cluster
	// listener, used when a token is minted without one:
	// https://<host>:<cluster port>.
	Address string

	// Runtime keeps the runtime adapter's host list in line with the
	// cluster's app hosts (runtime.go). A join checks Runtime.CheckJoin
	// before it lets an app host in. Nil in tests that do not need it.
	Runtime *Runtime

	// OnChange runs after a host joins or is removed: the server syncs the
	// runtime's host list there (Runtime.Sync) and logs what fails, which
	// does not undo the change. Nil does nothing.
	OnChange func(ctx context.Context)

	// Version is this server's, which a joining host must match (R-415 [P]).
	Version string

	// Keys are read for a host joining with the control role.
	Keys KeySource

	// Readiness, when set, is asked before a join spends a use of its token.
	Readiness JoinReadiness

	// JoinCheckTimeout bounds the checks a join makes before it spends its
	// token's use (readiness, the runtime, the keys); zero is
	// DefaultJoinCheckTimeout.
	JoinCheckTimeout time.Duration

	// IsSelf picks the host serving this process's requests, which Remove
	// refuses. Nil refuses none but the install's own first host.
	IsSelf func(state.Host) bool

	Now func() time.Time
}

Service mints join tokens and lists and removes hosts. The caller checks install.cluster.manage; Mint itself checks every install verb for a token carrying the control role (R-080a), so every surface that mints one holds to it (R-261).

func (*Service) CertRevoked

func (s *Service) CertRevoked(ctx context.Context, certDER []byte) (bool, error)

CertRevoked reports whether certDER belonged to a host removed from the cluster (R-415), which nothing accepts again.

func (*Service) Changed

func (s *Service) Changed(ctx context.Context)

Changed reports a change to the host list (OnChange).

func (*Service) ControlCount

func (s *Service) ControlCount(ctx context.Context) (int, error)

ControlCount is how many hosts hold the control role, at least one: the install's own first host does, also before it is recorded (R-414). Each runs an edge (R-417).

func (*Service) Count

func (s *Service) Count(ctx context.Context) (int, error)

Count is how many hosts the cluster has, this install's own included: it records itself at its first start (RecordInstallHost, R-414). An install is never fewer than one host, also before that record is made.

func (*Service) DefaultRoles

func (s *Service) DefaultRoles(ctx context.Context) ([]string, error)

DefaultRoles is the roles a token minted without any gets [P]: both while the cluster has fewer than three hosts, the app role after.

func (*Service) Hosts

func (s *Service) Hosts(ctx context.Context) ([]state.Host, error)

Hosts lists the cluster's hosts.

func (*Service) Join

func (s *Service) Join(ctx context.Context, req JoinRequest) (JoinResult, error)

Join admits a host (R-415). It refuses, spending no use of the token, a request Pando cannot use, a host at another version, and whatever Readiness or the runtime refuses; otherwise it issues the host's certificates, then spends one use and records the host, pending, and answers with what the token's roles need — the keys only for the control role. The cluster's version is said only to a host whose token is good: a caller without one learns nothing about the cluster.

Every slow step runs before the token is spent, with no database connection held and within JoinCheckTimeout, so a check that hangs keeps no other join waiting on the token's row and holds none of the pool; the spend and the record are one short transaction (state.Cluster.Join). The host is listed once it presents the certificate it was issued (Service.Peers, which the joining host asks right after writing its files); until then a join of the same name and key replaces it, and one of the same name and another key, another machine's, is refused.

func (*Service) Mint

Mint creates a join token for p, who has install.cluster.manage. A token for the control role also needs every install verb, which az checks (R-415, R-080a); a nil az refuses one.

func (*Service) PeerList

func (s *Service) PeerList(ctx context.Context, certDER []byte) (PeerList, error)

PeerList is Peers, with what is revoked.

func (*Service) Peers

func (s *Service) Peers(ctx context.Context, certDER []byte) ([]Peer, error)

Peers is the peer list, for the host whose verified certificate is certDER: one a host of the cluster holds (cluster_hosts.cert_sha256, or the one before its latest renewal, or the one a control host answers pings with), never a removed host's, so a removed host no longer hears where the others are (R-415). A host presenting the certificate it renewed to is using it, so the one before is no longer its own (R-421). A host whose join is pending presenting the certificate its join issued has finished joining: it is listed from here on (R-415).

func (*Service) Remove

func (s *Service) Remove(ctx context.Context, hostID string) (Removed, error)

Remove removes a host from the cluster (R-414). The install's own first host and the host serving the request are refused: the first would be recorded again under a new ID at its next start, leaving its apps' homes naming a host that is gone, and the second would cut off the request. Every certificate the host held is refused from then on (R-415). A control host knows the installation's keys, which removing it does not take back; the result names them.

func (*Service) Renew

func (s *Service) Renew(ctx context.Context, certDER, csrPEM []byte) (RenewResult, error)

Renew issues the host presenting certDER a new certificate for the key csrPEM proves, with the same name and roles, and a new probe certificate, and records the new certificate as the host's (R-421). The one presented stays the host's until it presents the new one (Peers).

A certificate is renewed once (state.Cluster.RenewHostCert): a renewal presenting one already renewed — another made at the same time, or the same one tried again after its answer was lost — is answered with the certificate the first was, when it asks for the same key, and refused otherwise. So the host ends up holding the certificate it presented or the one it was answered with, and the cluster pins both.

func (*Service) Revoke

func (s *Service) Revoke(ctx context.Context, tokenID string) (state.JoinToken, error)

Revoke revokes a join token.

func (*Service) ThisHost

func (s *Service) ThisHost(list []state.Host) string

ThisHost is the ID of the host serving this process's requests, among list, or "" when it is not among them.

func (*Service) Tokens

func (s *Service) Tokens(ctx context.Context) ([]state.JoinToken, error)

Tokens lists every join token, newest first.

type Store

type Store interface {
	CreateJoinToken(ctx context.Context, roles []string, maxUses int, createdBy string, expiresAt time.Time) (state.IssuedJoinToken, error)
	JoinTokens(ctx context.Context) ([]state.JoinToken, error)
	RevokeJoinToken(ctx context.Context, tokenID string) (state.JoinToken, error)
	Hosts(ctx context.Context) ([]state.Host, error)
	RemoveHost(ctx context.Context, hostID string) (state.Host, error)
	// A join (R-415): the token checked without spending it, then spent
	// with the host recorded pending, which the host confirms.
	CheckJoinToken(ctx context.Context, presented secret.Value) (state.JoinToken, error)
	Join(ctx context.Context, presented secret.Value, h state.Host) (state.JoinToken, state.Host, error)
	ConfirmJoin(ctx context.Context, hostID string) (bool, error)
	// The host's certificates (R-415, R-421): which host holds one, pending
	// or not, renewing them, and refusing a removed host's.
	HostByCert(ctx context.Context, sha string) (state.Host, bool, error)
	RenewHostCert(ctx context.Context, hostID, presented, next string, nextPEM []byte) (state.Host, bool, error)
	ConfirmHostCert(ctx context.Context, hostID, current string) (bool, error)
	CertRevoked(ctx context.Context, sha string) (bool, error)
	Revoked(ctx context.Context) (state.Revoked, error)
}

Store is the slice of state.Cluster the service uses.

type SyncResult

type SyncResult struct {
	AdapterID string `json:"adapter_id,omitempty"`
	Converted bool   `json:"converted"`
	Changed   bool   `json:"changed"`
}

SyncResult is what a Sync did.

type WireJoinRequest

type WireJoinRequest struct {
	Secret   string `json:"secret"`
	Name     string `json:"name"`
	CSR      string `json:"csr"`
	Platform string `json:"platform"`
	Version  string `json:"version"`
	Address  string `json:"address"`
}

WireJoinRequest is POST /cluster/join's body.

func (WireJoinRequest) Request

func (w WireJoinRequest) Request() JoinRequest

Request is the body as core reads it.

type WireJoinResult

type WireJoinResult struct {
	Host           state.Host `json:"host"`
	Certificate    string     `json:"certificate"`
	Authority      string     `json:"authority"`
	HostClientCert string     `json:"host_client_certificate,omitempty"`
	HostClientKey  string     `json:"host_client_key,omitempty"`
	NetworkPool    string     `json:"network_pool,omitempty"`
	Keys           *WireKeys  `json:"keys,omitempty"`
}

WireJoinResult is POST /cluster/join's answer.

type WireKeys

type WireKeys struct {
	DatabaseURL      string `json:"database_url"`
	SecretsKey       string `json:"secrets_key,omitempty"`
	SecretsKeyPath   string `json:"secrets_key_path,omitempty"`
	TokenKey         string `json:"token_key"`
	ClusterAuthority string `json:"cluster_authority"`
}

WireKeys is a control host's keys on the wire. Key files are base64, since a key is bytes.

type WireRenewRequest

type WireRenewRequest struct {
	CSR string `json:"csr"`
}

WireRenewRequest is POST /cluster/renew's body: a certificate request for the host's next key.

type WireRenewResult

type WireRenewResult struct {
	Host           state.Host `json:"host"`
	Certificate    string     `json:"certificate"`
	HostClientCert string     `json:"host_client_certificate"`
	HostClientKey  string     `json:"host_client_key"`
}

WireRenewResult is POST /cluster/renew's answer. HostClientKey is the probe certificate's key, sent only over the mutually authenticated connection.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL