deviceaccess

package
v1.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 21, 2026 License: Apache-2.0 Imports: 33 Imported by: 0

Documentation

Overview

Package deviceaccess is the device access service: the one component on a lab node that holds the container runtime socket and opens a shell (a console) or moves a file in a lab device on behalf of the L7 proxy.

Path of a console: participant, L7 proxy (authorization), this service on the node of the device, a shell in the device.

The Cilium policy lets only the proxy pods reach the service, and a request is authenticated by what it carries, not by where it comes from: the console grant the tenant's platform signed (see Auth and internal/grant), verified with the tenant's access public key. The service holds no key of its own; the keys are public data mirrored into a ConfigMap it mounts. A request must agree with its grant (client, group, lab, device), the pod must carry the name of the grant's group, and the stream ends at the grant's exp. The proxy is not trusted with the container: a request names a lab device (namespace, lab, device) and the service finds the container itself from what the runtime shows (see Resolve), copies the process spec from that container (see ProcessFor) and refuses everything that is not a lab device with a console. It has no Kubernetes API rights.

Index

Constants

View Source
const (

	// MaxCommandItems and MaxCommandItem bound the terminal command the lab author set.
	MaxCommandItems = 32
	MaxCommandItem  = 4096
)
View Source
const (
	StreamPath = wire.StreamPath
	HealthPath = "/healthz"
)

StreamPath (wire.StreamPath) is where the proxy opens a stream, a WebSocket. HealthPath answers 200 while the service runs.

Variables

View Source
var (
	// ErrNotFound: no lab device matches, or the pod behind the name is not a lab device at all (the VPN, the gateway, a system pod).
	ErrNotFound = errors.New("not a running lab device")
	// ErrDisabled: the device exists but its lab did not ask for a console.
	ErrDisabled = errors.New("the device has no console")
	// ErrAmbiguous: more than one running container answers to the name. The service refuses rather than guess.
	ErrAmbiguous = errors.New("more than one container answers to the device")
)

Refusals of Resolve. The service tells a caller only the short code, not which one of the checks failed in detail.

View Source
var ErrBadSettings = errors.New("invalid terminal settings")

ErrBadSettings is returned for a command or user the service will not run.

Functions

func CheckSettings

func CheckSettings(command []string, user string) ([]string, error)

CheckSettings validates the terminal command and user and returns the command to run (the default shell when empty).

func CommandHash

func CommandHash(command []string) string

CommandHash is a short fingerprint of a command line for the session log (see wire.CommandHash).

func ListenChecker

func ListenChecker(path string, gid, peerUID int) (net.Listener, error)

ListenChecker makes the checker socket: a unix socket at path, mode 0660, group gid (the group the checker pod runs as; the service pod is a member of it, which is what lets a user without capabilities hand a file it owns to that group). A connection whose peer is not the checker's user (peerUID, read from SO_PEERCRED) is closed at once, whatever the file permissions say.

func NewHTTPServer

func NewHTTPServer(addr string, h http.Handler) *http.Server

NewHTTPServer returns the HTTP server for Handler with the timeouts a listener open to the pod network needs. An upgraded connection leaves these timeouts behind; the service bounds a stream with its own deadlines.

func ProcessFor

func ProcessFor(c *specs.Process, a ExecArgs) (*specs.Process, error)

ProcessFor builds the process an exec runs, the way the CRI plugin does for `kubectl exec`: a copy of the process block of the container (user, capabilities, no_new_privs, seccomp and the rest of the container's restrictions, the environment base, working directory, rlimits), then only the arguments, the terminal, TERM and, when asked, the user are set. The capabilities are never built from scratch, because the runtime does not clamp them to the container's: a set written here would be exactly what the process gets. Everything else of the container's process block (no_new_privs, the AppArmor profile, the SELinux label, rlimits, the OOM score adjustment) is carried over by the copy.

Types

type Auth

type Auth struct {
	// Keys finds the access key of (tenant, key id): the keys the operator mirrors into the service's ConfigMap (grant.DirKeys).
	Keys grant.KeyLookup
	// MaxTTL is the longest exp - iat accepted (the chart's terminal.grantMaxTTL); zero means grant.DefaultMaxTTL.
	MaxTTL time.Duration
	// Now is the clock; nil means time.Now.
	Now func() time.Time
}

Auth checks the grant of an open request. Check is the one place where an open request is authenticated; it runs before anything else is done with the request. The grant is the console grant the tenant's platform signed (internal/grant); the service verifies it with the tenant's access public key and holds no key of its own, so a rooted node cannot make one.

func NewAuth

func NewAuth(keys grant.KeyLookup, maxTTL time.Duration) *Auth

NewAuth returns an Auth over a key lookup.

func (*Auth) Check

func (a *Auth) Check(r wire.OpenRequest) (grant.Claims, error)

Check authenticates a request: a console grant for this service (aud laboratory-device-access), valid now, lived no longer than MaxTTL, and the open request agrees with it: client with sub, group with group_id, lab, and device with dev. It returns the claims; the stream ends at their exp. A failure never says which check failed.

type CheckerLimits

type CheckerLimits struct {
	// MaxSeconds is the ceiling of the time one probe run may take, whatever the request says.
	MaxSeconds int
	// MaxConcurrent is the most probe runs at once on this node.
	MaxConcurrent int
}

CheckerLimits are the values of the checker socket.

type Config

type Config struct {
	Runtime Runtime
	// Auth authenticates every open request. It is required: a service without one refuses every request.
	Auth   *Auth
	Limits Limits
	Log    logr.Logger
	// Tap, when set, sees every byte of a console in both directions (see stream.pass). Nothing sets it today.
	Tap Tap
}

Config builds a Service.

type Conn

type Conn interface {
	// Read returns the next message and whether it is a text message (the control JSON) or a binary one (a frame).
	Read(ctx context.Context) (text bool, msg []byte, err error)
	// WriteText sends a control message; WriteBinary sends a frame.
	WriteText(ctx context.Context, msg []byte) error
	WriteBinary(ctx context.Context, msg []byte) error
}

Conn is one stream with the proxy: whole messages, no ordering between the two directions. The WebSocket adapter is in server.go; tests use an in-memory one.

type ContainerInfo

type ContainerInfo struct {
	ID     string
	Labels map[string]string
}

ContainerInfo is what the container runtime shows of a container: its id and its labels. Nothing else is trusted to find a device.

type Containerd

type Containerd struct {
	// contains filtered or unexported fields
}

Containerd is the Runtime over the native containerd client: the exec is task.Exec with FIFOs in a host directory, not the CRI Exec (its stream server listens on the host's loopback, out of reach of a pod without host networking) and not the API server's pods/exec (the load would fall on the API server). Nothing opens a network port: the whole path is the unix socket and the FIFOs.

The directory of the FIFOs must be the same path in the host and in the pod: the client makes the FIFOs, and the runtime's shim, which runs on the host, opens them by the path in the request.

func DialContainerd

func DialContainerd(sock, namespace, fifoDir string) (*Containerd, error)

DialContainerd connects to the runtime's socket. namespace is the containerd namespace of the kubelet's CRI (k8s.io).

func (*Containerd) Close

func (c *Containerd) Close() error

Close closes the connection.

func (*Containerd) ListByLabels

func (c *Containerd) ListByLabels(ctx context.Context, match map[string]string) ([]ContainerInfo, error)

func (*Containerd) Running

func (c *Containerd) Running(ctx context.Context, id string) (bool, error)

func (*Containerd) Spec

func (c *Containerd) Spec(ctx context.Context, id string) (*specs.Spec, error)

func (*Containerd) Start

func (c *Containerd) Start(ctx context.Context, id string, proc *specs.Process, stdio IO) (Exec, error)

func (*Containerd) Sweep

func (c *Containerd) Sweep(ctx context.Context) (execs, fifos int, err error)

Sweep removes what a previous run of the service left: execs still running in devices (found by the journal) and stale FIFOs. It is called once at start, before the first stream.

type Device

type Device struct {
	ContainerID string
	PodUID      string
	// TerminalSpec is the digest of the terminal command and user the operator stamped (names.LabelTerminalSpec); empty when there is none.
	TerminalSpec string
}

Device is a lab device container.

func Resolve

func Resolve(ctx context.Context, rt Runtime, t Target) (Device, error)

Resolve finds the container of a lab device, and only that.

How "this container is a lab device" is known, without the Kubernetes API: the pod labels are on the SANDBOX container of the pod in the runtime (the kubelet hands them to the CRI plugin as the labels of the sandbox, not of the app containers). The operator stamps the UIDs of the Lab and of the Device on the pod, and nothing else makes a pod with them: the platform's label prefix is stripped from every user label, and the VPN, the gateway and the system pods carry neither. So a sandbox with both UIDs, the right namespace, lab and device is a lab device. Labels are never read from the app container: the CRI copies the labels of the image onto it, and the image of a device is written by a participant.

The device container is then the running container of the same pod (the same pod UID) whose name is the device name; the other containers of the pod (init containers, a helper) never match.

func ResolveLabDevice

func ResolveLabDevice(ctx context.Context, rt Runtime, t Target) (Device, error)

ResolveLabDevice is Resolve for a caller that needs no console on the device: the checker socket runs the probe in any lab device. It is the same white list (a sandbox with both UIDs of the operator and no component label, then the running container of that pod named like the device) without the console label.

type Direction

type Direction int

Direction is which way bytes of a stream go.

const (
	// ToDevice is what the participant types.
	ToDevice Direction = iota
	// FromDevice is what the device prints.
	FromDevice
)

func (Direction) String

func (d Direction) String() string

type Exec

type Exec interface {
	// Wait delivers the exit status once, then closes.
	Wait() <-chan ExitStatus
	// Resize changes the size of the terminal of a TTY exec.
	Resize(ctx context.Context, cols, rows uint16) error
	// Close kills the process, deletes it from the runtime and removes its FIFOs. It is safe to call more than once and from any
	// state; the caller always calls it, whatever way the stream ended.
	Close(ctx context.Context) error
}

Exec is a process started in a container.

type ExecArgs

type ExecArgs struct {
	Args     []string
	Terminal bool
	// Cols and Rows are the initial terminal size (Terminal only).
	Cols, Rows uint16
	// User, when set, replaces the user of the container's process.
	User *ExecUser
}

ExecArgs is everything the service chooses for an exec. Everything else comes from the container.

type ExecUser

type ExecUser struct {
	UID    uint32   `json:"uid"`
	GID    uint32   `json:"gid"`
	Groups []uint32 `json:"groups,omitempty"`
	Home   string   `json:"home,omitempty"`
}

ExecUser is a user of the device, resolved inside it (see the probe's resolve-user): the runtime is given numbers only.

type ExitStatus

type ExitStatus struct {
	Code uint32
	Err  error
}

ExitStatus is how an exec ended.

type IO

type IO struct {
	Stdin    io.Reader
	Stdout   io.Writer
	Stderr   io.Writer
	Terminal bool
}

IO is where the standard streams of an exec go. The runtime copies between its FIFOs and these; the service owns the other end of every one of them. Stderr is unused (and may be nil) when Terminal is set: a terminal has one output.

type Limits

type Limits struct {
	// PerClient, PerLab and PerNode bound the consoles open at once: of one client of one group, in one lab, and on this node.
	PerClient, PerLab, PerNode int
	// Idle closes a console that no byte crossed for that long. Lifetime closes any console at that age.
	Idle, Lifetime time.Duration
	// WriteTimeout bounds every write towards the proxy or the device: a peer that does not take bytes for that long is cut.
	WriteTimeout time.Duration
	// Cleanup bounds the time the clean-up of an exec may take.
	Cleanup time.Duration
	// KeyRecheck is how often an open console asks again whether the key its grant was verified with still exists; a console ends (access-lost)
	// when it is gone at two checks in a row. Zero means DefaultLimits().KeyRecheck.
	KeyRecheck time.Duration
}

Limits are the chart values of the service. A zero limit is no limit, except the timeouts, which have defaults.

func DefaultLimits

func DefaultLimits() Limits

DefaultLimits are the values of the chart defaults.

type PeerCheck

type PeerCheck func(c net.Conn) error

PeerCheck decides whether the process on the other end of an accepted unix connection may use the checker socket.

type Runtime

type Runtime interface {
	// ListByLabels returns the containers whose labels contain every given pair.
	ListByLabels(ctx context.Context, match map[string]string) ([]ContainerInfo, error)
	// Running reports whether the container has a running task.
	Running(ctx context.Context, id string) (bool, error)
	// Spec returns the OCI runtime spec of the container.
	Spec(ctx context.Context, id string) (*specs.Spec, error)
	// Start runs proc in the container and starts it. The process spec is used as it is.
	Start(ctx context.Context, id string, proc *specs.Process, stdio IO) (Exec, error)
}

Runtime is the part of the container runtime the service uses. The real one is the native containerd client (containerd_linux.go); tests use a fake. There is no method that takes a command or a container name from outside: the callers of this interface have already decided what to run.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service serves consoles.

func New

func New(c Config) *Service

New returns a Service.

func (*Service) CheckerHandler

func (s *Service) CheckerHandler(lim CheckerLimits) http.Handler

CheckerHandler serves the checker socket. Mount it on a unix listener whose peer is checked (see ListenChecker).

func (*Service) Handler

func (s *Service) Handler() http.Handler

Handler is the listener of the service: the stream endpoint and the health endpoint. It is plain HTTP on the pod network; the Cilium policy of the chart lets only the L7 proxy pods connect.

func (*Service) MetricsHandler

func (s *Service) MetricsHandler() http.Handler

MetricsHandler serves the metrics of the service.

func (*Service) Registry

func (s *Service) Registry() *prometheus.Registry

Registry is the metrics registry of the service.

func (*Service) Serve

func (s *Service) Serve(ctx context.Context, conn Conn)

Serve handles one stream: the open request is the first text message of conn. It returns when the stream is over and everything it started is cleaned up.

func (*Service) Shutdown

func (s *Service) Shutdown(ctx context.Context) error

Shutdown stops new streams, ends the running ones and waits until their execs are removed (or ctx ends).

type StreamInfo

type StreamInfo struct {
	Kind, Namespace, Lab, Device, Client string
}

StreamInfo says whose bytes a Tap sees.

type Tap

type Tap func(info StreamInfo, dir Direction, p []byte)

Tap sees every byte of a stream. It is the one place where recording a console can be attached later; nothing implements it today, and the bytes must not be kept, logged or copied anywhere until the owner decides on recording.

type Target

type Target struct {
	Namespace, Lab, Device string
	// GroupName, when not empty, is the name of the LabGroup (names.EncodeName of the platform's group id) the pod must be stamped with
	// (names.LabelGroupName). The console always sets it; the checker socket, which has no group, does not.
	GroupName string
	// Tenant, when not empty, is the tenant (the issuer of the grant) the pod must be stamped with (names.LabelTenant, taken from the LabGroup
	// by the operator). A pod without the label never matches.
	Tenant string
}

Target names a device. It is all a caller says about the container it wants.

Directories

Path Synopsis
Package wire is the framing of a console stream.
Package wire is the framing of a console stream.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL