Documentation
¶
Overview ¶
Package deviceaccess is the device access service: the one component on a lab node that holds the container runtime socket and opens a shell (a console) or moves a file in a lab device on behalf of the L7 proxy.
Path of a console: participant, L7 proxy (authorization), this service on the node of the device, a shell in the device.
The Cilium policy lets only the proxy pods reach the service, and a request is authenticated by what it carries, not by where it comes from: the console grant the tenant's platform signed (see Auth and internal/grant), verified with the tenant's access public key. The service holds no key of its own; the keys are public data mirrored into a ConfigMap it mounts. A request must agree with its grant (client, group, lab, device), the pod must carry the name of the grant's group, and the stream ends at the grant's exp. The proxy is not trusted with the container: a request names a lab device (namespace, lab, device) and the service finds the container itself from what the runtime shows (see Resolve), copies the process spec from that container (see ProcessFor) and refuses everything that is not a lab device with a console. It has no Kubernetes API rights.
Index ¶
- Constants
- Variables
- func CheckSettings(command []string, user string) ([]string, error)
- func CommandHash(command []string) string
- func ListenChecker(path string, gid, peerUID int) (net.Listener, error)
- func NewHTTPServer(addr string, h http.Handler) *http.Server
- func ProcessFor(c *specs.Process, a ExecArgs) (*specs.Process, error)
- type Auth
- type CheckerLimits
- type Config
- type Conn
- type ContainerInfo
- type Containerd
- func (c *Containerd) Close() error
- func (c *Containerd) ListByLabels(ctx context.Context, match map[string]string) ([]ContainerInfo, error)
- func (c *Containerd) Running(ctx context.Context, id string) (bool, error)
- func (c *Containerd) Spec(ctx context.Context, id string) (*specs.Spec, error)
- func (c *Containerd) Start(ctx context.Context, id string, proc *specs.Process, stdio IO) (Exec, error)
- func (c *Containerd) Sweep(ctx context.Context) (execs, fifos int, err error)
- type Device
- type Direction
- type Exec
- type ExecArgs
- type ExecUser
- type ExitStatus
- type IO
- type Limits
- type PeerCheck
- type Runtime
- type Service
- func (s *Service) CheckerHandler(lim CheckerLimits) http.Handler
- func (s *Service) Handler() http.Handler
- func (s *Service) MetricsHandler() http.Handler
- func (s *Service) Registry() *prometheus.Registry
- func (s *Service) Serve(ctx context.Context, conn Conn)
- func (s *Service) Shutdown(ctx context.Context) error
- type StreamInfo
- type Tap
- type Target
Constants ¶
const ( // MaxCommandItems and MaxCommandItem bound the terminal command the lab author set. MaxCommandItems = 32 MaxCommandItem = 4096 )
const ( StreamPath = wire.StreamPath HealthPath = "/healthz" )
StreamPath (wire.StreamPath) is where the proxy opens a stream, a WebSocket. HealthPath answers 200 while the service runs.
Variables ¶
var ( // ErrNotFound: no lab device matches, or the pod behind the name is not a lab device at all (the VPN, the gateway, a system pod). ErrNotFound = errors.New("not a running lab device") // ErrDisabled: the device exists but its lab did not ask for a console. ErrDisabled = errors.New("the device has no console") // ErrAmbiguous: more than one running container answers to the name. The service refuses rather than guess. ErrAmbiguous = errors.New("more than one container answers to the device") )
Refusals of Resolve. The service tells a caller only the short code, not which one of the checks failed in detail.
var ErrBadSettings = errors.New("invalid terminal settings")
ErrBadSettings is returned for a command or user the service will not run.
Functions ¶
func CheckSettings ¶
CheckSettings validates the terminal command and user and returns the command to run (the default shell when empty).
func CommandHash ¶
CommandHash is a short fingerprint of a command line for the session log (see wire.CommandHash).
func ListenChecker ¶
ListenChecker makes the checker socket: a unix socket at path, mode 0660, group gid (the group the checker pod runs as; the service pod is a member of it, which is what lets a user without capabilities hand a file it owns to that group). A connection whose peer is not the checker's user (peerUID, read from SO_PEERCRED) is closed at once, whatever the file permissions say.
func NewHTTPServer ¶
NewHTTPServer returns the HTTP server for Handler with the timeouts a listener open to the pod network needs. An upgraded connection leaves these timeouts behind; the service bounds a stream with its own deadlines.
func ProcessFor ¶
ProcessFor builds the process an exec runs, the way the CRI plugin does for `kubectl exec`: a copy of the process block of the container (user, capabilities, no_new_privs, seccomp and the rest of the container's restrictions, the environment base, working directory, rlimits), then only the arguments, the terminal, TERM and, when asked, the user are set. The capabilities are never built from scratch, because the runtime does not clamp them to the container's: a set written here would be exactly what the process gets. Everything else of the container's process block (no_new_privs, the AppArmor profile, the SELinux label, rlimits, the OOM score adjustment) is carried over by the copy.
Types ¶
type Auth ¶
type Auth struct {
// Keys finds the access key of (tenant, key id): the keys the operator mirrors into the service's ConfigMap (grant.DirKeys).
Keys grant.KeyLookup
// MaxTTL is the longest exp - iat accepted (the chart's terminal.grantMaxTTL); zero means grant.DefaultMaxTTL.
MaxTTL time.Duration
// Now is the clock; nil means time.Now.
Now func() time.Time
}
Auth checks the grant of an open request. Check is the one place where an open request is authenticated; it runs before anything else is done with the request. The grant is the console grant the tenant's platform signed (internal/grant); the service verifies it with the tenant's access public key and holds no key of its own, so a rooted node cannot make one.
func (*Auth) Check ¶
Check authenticates a request: a console grant for this service (aud laboratory-device-access), valid now, lived no longer than MaxTTL, and the open request agrees with it: client with sub, group with group_id, lab, and device with dev. It returns the claims; the stream ends at their exp. A failure never says which check failed.
type CheckerLimits ¶
type CheckerLimits struct {
// MaxSeconds is the ceiling of the time one probe run may take, whatever the request says.
MaxSeconds int
// MaxConcurrent is the most probe runs at once on this node.
MaxConcurrent int
}
CheckerLimits are the values of the checker socket.
type Config ¶
type Config struct {
Runtime Runtime
// Auth authenticates every open request. It is required: a service without one refuses every request.
Auth *Auth
Limits Limits
Log logr.Logger
// Tap, when set, sees every byte of a console in both directions (see stream.pass). Nothing sets it today.
Tap Tap
}
Config builds a Service.
type Conn ¶
type Conn interface {
// Read returns the next message and whether it is a text message (the control JSON) or a binary one (a frame).
Read(ctx context.Context) (text bool, msg []byte, err error)
// WriteText sends a control message; WriteBinary sends a frame.
WriteText(ctx context.Context, msg []byte) error
WriteBinary(ctx context.Context, msg []byte) error
}
Conn is one stream with the proxy: whole messages, no ordering between the two directions. The WebSocket adapter is in server.go; tests use an in-memory one.
type ContainerInfo ¶
ContainerInfo is what the container runtime shows of a container: its id and its labels. Nothing else is trusted to find a device.
type Containerd ¶
type Containerd struct {
// contains filtered or unexported fields
}
Containerd is the Runtime over the native containerd client: the exec is task.Exec with FIFOs in a host directory, not the CRI Exec (its stream server listens on the host's loopback, out of reach of a pod without host networking) and not the API server's pods/exec (the load would fall on the API server). Nothing opens a network port: the whole path is the unix socket and the FIFOs.
The directory of the FIFOs must be the same path in the host and in the pod: the client makes the FIFOs, and the runtime's shim, which runs on the host, opens them by the path in the request.
func DialContainerd ¶
func DialContainerd(sock, namespace, fifoDir string) (*Containerd, error)
DialContainerd connects to the runtime's socket. namespace is the containerd namespace of the kubelet's CRI (k8s.io).
func (*Containerd) ListByLabels ¶
func (c *Containerd) ListByLabels(ctx context.Context, match map[string]string) ([]ContainerInfo, error)
type Device ¶
type Device struct {
ContainerID string
PodUID string
// TerminalSpec is the digest of the terminal command and user the operator stamped (names.LabelTerminalSpec); empty when there is none.
TerminalSpec string
}
Device is a lab device container.
func Resolve ¶
Resolve finds the container of a lab device, and only that.
How "this container is a lab device" is known, without the Kubernetes API: the pod labels are on the SANDBOX container of the pod in the runtime (the kubelet hands them to the CRI plugin as the labels of the sandbox, not of the app containers). The operator stamps the UIDs of the Lab and of the Device on the pod, and nothing else makes a pod with them: the platform's label prefix is stripped from every user label, and the VPN, the gateway and the system pods carry neither. So a sandbox with both UIDs, the right namespace, lab and device is a lab device. Labels are never read from the app container: the CRI copies the labels of the image onto it, and the image of a device is written by a participant.
The device container is then the running container of the same pod (the same pod UID) whose name is the device name; the other containers of the pod (init containers, a helper) never match.
func ResolveLabDevice ¶
ResolveLabDevice is Resolve for a caller that needs no console on the device: the checker socket runs the probe in any lab device. It is the same white list (a sandbox with both UIDs of the operator and no component label, then the running container of that pod named like the device) without the console label.
type Exec ¶
type Exec interface {
// Wait delivers the exit status once, then closes.
Wait() <-chan ExitStatus
// Resize changes the size of the terminal of a TTY exec.
Resize(ctx context.Context, cols, rows uint16) error
// Close kills the process, deletes it from the runtime and removes its FIFOs. It is safe to call more than once and from any
// state; the caller always calls it, whatever way the stream ended.
Close(ctx context.Context) error
}
Exec is a process started in a container.
type ExecArgs ¶
type ExecArgs struct {
Args []string
Terminal bool
// Cols and Rows are the initial terminal size (Terminal only).
Cols, Rows uint16
// User, when set, replaces the user of the container's process.
User *ExecUser
}
ExecArgs is everything the service chooses for an exec. Everything else comes from the container.
type ExecUser ¶
type ExecUser struct {
UID uint32 `json:"uid"`
GID uint32 `json:"gid"`
Groups []uint32 `json:"groups,omitempty"`
Home string `json:"home,omitempty"`
}
ExecUser is a user of the device, resolved inside it (see the probe's resolve-user): the runtime is given numbers only.
type IO ¶
IO is where the standard streams of an exec go. The runtime copies between its FIFOs and these; the service owns the other end of every one of them. Stderr is unused (and may be nil) when Terminal is set: a terminal has one output.
type Limits ¶
type Limits struct {
// PerClient, PerLab and PerNode bound the consoles open at once: of one client of one group, in one lab, and on this node.
PerClient, PerLab, PerNode int
// Idle closes a console that no byte crossed for that long. Lifetime closes any console at that age.
Idle, Lifetime time.Duration
// WriteTimeout bounds every write towards the proxy or the device: a peer that does not take bytes for that long is cut.
WriteTimeout time.Duration
// Cleanup bounds the time the clean-up of an exec may take.
Cleanup time.Duration
// KeyRecheck is how often an open console asks again whether the key its grant was verified with still exists; a console ends (access-lost)
// when it is gone at two checks in a row. Zero means DefaultLimits().KeyRecheck.
KeyRecheck time.Duration
}
Limits are the chart values of the service. A zero limit is no limit, except the timeouts, which have defaults.
func DefaultLimits ¶
func DefaultLimits() Limits
DefaultLimits are the values of the chart defaults.
type PeerCheck ¶
PeerCheck decides whether the process on the other end of an accepted unix connection may use the checker socket.
type Runtime ¶
type Runtime interface {
// ListByLabels returns the containers whose labels contain every given pair.
ListByLabels(ctx context.Context, match map[string]string) ([]ContainerInfo, error)
// Running reports whether the container has a running task.
Running(ctx context.Context, id string) (bool, error)
// Spec returns the OCI runtime spec of the container.
Spec(ctx context.Context, id string) (*specs.Spec, error)
// Start runs proc in the container and starts it. The process spec is used as it is.
Start(ctx context.Context, id string, proc *specs.Process, stdio IO) (Exec, error)
}
Runtime is the part of the container runtime the service uses. The real one is the native containerd client (containerd_linux.go); tests use a fake. There is no method that takes a command or a container name from outside: the callers of this interface have already decided what to run.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service serves consoles.
func (*Service) CheckerHandler ¶
func (s *Service) CheckerHandler(lim CheckerLimits) http.Handler
CheckerHandler serves the checker socket. Mount it on a unix listener whose peer is checked (see ListenChecker).
func (*Service) Handler ¶
Handler is the listener of the service: the stream endpoint and the health endpoint. It is plain HTTP on the pod network; the Cilium policy of the chart lets only the L7 proxy pods connect.
func (*Service) MetricsHandler ¶
MetricsHandler serves the metrics of the service.
func (*Service) Registry ¶
func (s *Service) Registry() *prometheus.Registry
Registry is the metrics registry of the service.
type StreamInfo ¶
type StreamInfo struct {
Kind, Namespace, Lab, Device, Client string
}
StreamInfo says whose bytes a Tap sees.
type Tap ¶
type Tap func(info StreamInfo, dir Direction, p []byte)
Tap sees every byte of a stream. It is the one place where recording a console can be attached later; nothing implements it today, and the bytes must not be kept, logged or copied anywhere until the owner decides on recording.
type Target ¶
type Target struct {
Namespace, Lab, Device string
// GroupName, when not empty, is the name of the LabGroup (names.EncodeName of the platform's group id) the pod must be stamped with
// (names.LabelGroupName). The console always sets it; the checker socket, which has no group, does not.
GroupName string
// Tenant, when not empty, is the tenant (the issuer of the grant) the pod must be stamped with (names.LabelTenant, taken from the LabGroup
// by the operator). A pod without the label never matches.
Tenant string
}
Target names a device. It is all a caller says about the container it wants.