imageship

package
v0.0.0-...-6f76ffe Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package imageship moves locally-held docker images onto a remote docker host over SSH, without a registry and without the docker CLI on the machine doing the shipping.

It is the transport behind `sc image ship` and behind the vm delivery path's `imageDelivery: ssh-tar` (P5.1). The shape is the operator's own description of the delivery they wanted: *«собрать образ в ci/cd, залить его прямо архивом на виртуалку и импортировать в локальный docker images»* — so: `ImageSave` stream → compress → `docker load` on the far end, one pipe, no temporary file on either side.

Several images per ship, rather than one call per image, because `docker save` deduplicates shared layers across the images it is given: a stack's own image and the third-party images beside it usually share a base, and an app image and the Caddy in front of it both ship a distroless or alpine root. One stream also means one `docker load`, one SSH connection and one compressor warm-up.

See forge/docs/design/2026-10-13/sc-russia-market-readiness/compose-on-vm.md.

Index

Constants

View Source
const (
	// CompressionZstd is the default: ~3x faster than gzip at a better ratio, which
	// on a 400 MiB image over a link to another country is the difference between a
	// deploy and a coffee break.
	CompressionZstd = "zstd"
	// CompressionGzip is for a docker engine old enough to predate zstd support in
	// its load path (before moby 23). It fails with an opaque "unexpected EOF" from
	// `docker load` rather than naming the compression, which is why this escape
	// hatch exists at all.
	CompressionGzip = "gzip"
	CompressionNone = "none"
)

The value of Options.Compression.

`docker load` detects the compression from the stream itself (the daemon runs it through moby/go-archive's DecompressStream), so none of these need a matching binary on the machine — not even zstd.

View Source
const DefaultSshTimeout = 30 * time.Second

Variables

This section is empty.

Functions

This section is empty.

Types

type DockerAPI

type DockerAPI interface {
	ImageInspect(ctx context.Context, ref string) (image.InspectResponse, error)
	ImageSave(ctx context.Context, refs []string) (io.ReadCloser, error)
	// ImagePull fetches ref into the local docker. Only called when Options.Pull is
	// set, and only for a reference the local docker does not already have.
	ImagePull(ctx context.Context, ref string) error
}

DockerAPI is the slice of moby's client that shipping needs. It is an interface so the logic above it can be tested without a docker daemon — the arch gate and the skip-if-present decision are the parts worth pinning, and neither needs real bytes.

type ImageResult

type ImageResult struct {
	Image string
	// ImageID is the image's config digest — the identity `docker load` preserves,
	// and therefore the one worth comparing across the two hosts.
	ImageID string
	// Skipped reports that the machine already held this image id under this
	// reference and this image contributed nothing to the transfer.
	Skipped bool
	// Pulled reports that the image was absent locally and Options.Pull fetched it.
	Pulled bool
	// Platform is `os/arch`, as the arch gate read it off the image.
	Platform string
}

ImageResult is the outcome for one image of a ship.

type LocalDocker

type LocalDocker struct {
	// contains filtered or unexported fields
}

LocalDocker adapts moby's client to DockerAPI. It talks to whatever DOCKER_HOST says, so a CI runner with the daemon on a socket and a developer with a remote context both work, and no docker CLI has to be installed next to `sc`.

func NewLocalDocker

func NewLocalDocker() (*LocalDocker, error)

func (*LocalDocker) Close

func (d *LocalDocker) Close() error

func (*LocalDocker) ImageInspect

func (d *LocalDocker) ImageInspect(ctx context.Context, ref string) (image.InspectResponse, error)

func (*LocalDocker) ImagePull

func (d *LocalDocker) ImagePull(ctx context.Context, ref string) error

ImagePull fetches ref into the local docker and waits for it to finish.

No RegistryAuth is passed, and that is a real limit rather than an oversight: the engine API takes credentials per call and does NOT read ~/.docker/config.json (that is the CLI's doing), so there is no honest way to reuse a login from here. A pull of a private third-party image therefore fails with the registry's own 401. The way through is to pull it with the docker CLI before `sc image ship` — Options.Pull only fetches what is ABSENT, so an image the runner already holds is shipped as it stands.

Wait, not a drained io.Reader: the engine reports a failed pull inside the JSON message stream with a 200 on the HTTP response, so a pull whose stream is merely discarded looks successful and the failure resurfaces as "image is still not inspectable" one call later.

func (*LocalDocker) ImageSave

func (d *LocalDocker) ImageSave(ctx context.Context, refs []string) (io.ReadCloser, error)

type Options

type Options struct {
	// Images are the local references to ship (`ghcr.io/org/app:2026.10.16-abc1234`).
	// They are validated against security.ValidateImageRef by the caller; that
	// validation is also what makes them safe to interpolate into a remote shell
	// command, since the pattern admits no shell metacharacter.
	Images      []string
	Compression string
	// Force re-ships even when the machine already holds this exact image id.
	Force bool
	// Pull fetches an image the local docker does not have before saving it.
	//
	// This is what makes shipping usable for images SC does not build — a `mongo:7`
	// in someone's compose file, or the ingress Caddy — where the runner is the host
	// with a route to the registry and the machine is the one without. It pulls only
	// what is ABSENT: an image already in the local docker is shipped as it stands,
	// so a tag that moved upstream is not re-fetched on every deploy. For a pinned
	// third-party tag that is the behaviour you want; for a moving tag the ship is
	// as stale as the runner's cache, and `docker pull` before `sc image ship` is
	// the way to say otherwise.
	Pull bool
	// AllowArchMismatch downgrades the architecture gate to a warning. `docker load`
	// accepts a foreign-arch image happily and the failure surfaces much later, as a
	// container that exits with "exec format error".
	AllowArchMismatch bool
	// Log receives progress lines. Optional.
	Log func(format string, args ...any)
}

type Remote

type Remote interface {
	// Run executes cmd and returns its stdout with surrounding whitespace trimmed.
	// stdin may be nil. An exit status other than 0 is an error.
	Run(ctx context.Context, cmd string, stdin io.Reader) (string, error)
	// Describe names the target for log lines and error messages. It must never
	// include a key, a password or anything else that cannot be printed.
	Describe() string
}

Remote runs commands on the machine being shipped to.

type Result

type Result struct {
	// Images is one entry per requested image, in the order given (after
	// deduplication).
	Images []ImageResult
	// RemotePlatform is the machine's `os/arch`, as the arch gate compared against.
	RemotePlatform string
	// Bytes and Duration cover the single stream that carried every non-skipped
	// image, so they are not attributable per image: `docker save` shares layers
	// across them, which is the point of shipping them together.
	BytesRaw        int64
	BytesCompressed int64
	Duration        time.Duration
}

func Ship

func Ship(ctx context.Context, docker DockerAPI, remote Remote, opts Options) (*Result, error)

Ship copies opts.Images to the remote docker host. It is idempotent per image: a machine that already holds an image id under the same reference is left alone, and a ship where that is true of every image transfers nothing.

func (*Result) Shipped

func (r *Result) Shipped() []string

Shipped is the references that were actually transferred.

func (*Result) SkippedAll

func (r *Result) SkippedAll() bool

SkippedAll reports that the machine already held every requested image and nothing was transferred.

type SshRemote

type SshRemote struct {
	// contains filtered or unexported fields
}

func DialSSH

func DialSSH(ctx context.Context, t SshTarget) (*SshRemote, error)

DialSSH opens the connection. The caller closes it.

func (*SshRemote) Close

func (r *SshRemote) Close() error

func (*SshRemote) Describe

func (r *SshRemote) Describe() string

func (*SshRemote) Run

func (r *SshRemote) Run(ctx context.Context, cmd string, stdin io.Reader) (string, error)

Run executes cmd on the machine. stdin, when given, is streamed to the command — which is how a whole image gets there without a file at either end.

type SshTarget

type SshTarget struct {
	Address string
	User    string
	Port    int
	// PrivateKey is the key material in PEM form, not a path. The caller decides
	// where it came from — a file for `sc image ship`, a `${secret:...}` for the
	// provisioning tier — and this package never logs it.
	PrivateKey         string
	PrivateKeyPassword string

	// HostKey pins the machine's public key, as one authorized_keys-style line
	// (`ssh-ed25519 AAAA…`). Preferred in CI, where known_hosts is empty and a
	// freshly created machine has never been seen before.
	HostKey string
	// KnownHostsFile is consulted when HostKey is empty. Defaults to
	// ~/.ssh/known_hosts.
	KnownHostsFile string
	// InsecureSkipHostKeyCheck accepts whatever key the far end presents. It has to
	// be asked for explicitly: without a host key, SSH authenticates the channel but
	// nothing authenticates the *machine*, and the deploy key plus every secret in
	// the compose env goes to whoever answered on that address.
	InsecureSkipHostKeyCheck bool

	Timeout time.Duration
	// Log receives the one warning this type can emit. Optional.
	Log func(format string, args ...any)
}

SshTarget is everything needed to reach one machine. It is the live-connection twin of vm.VmHost (the declarative form in a parent stack's `hosts:`).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL