fusefile

package
v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 2, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

package fusefile is the canonical authoring format for a fuse environment. a Fusefile is parsed and compiled client-side into the orchestrator wire (CreateEnvironmentRequest); the orchestrator never sees a Fusefile.

Index

Constants

View Source
const (
	// ReasonCacheDisabled means the Fusefile has no `cache: {enabled: true}`.
	ReasonCacheDisabled = "caching is not enabled"
	// ReasonStepOptedOut means the step set `cache: false`.
	ReasonStepOptedOut = "step opted out with cache: false"
	// ReasonParentUncacheable means an earlier step is uncacheable, so this
	// step has no known parent layer to build on.
	ReasonParentUncacheable = "an earlier step is uncacheable"
	// ReasonGPU means the environment requests a GPU. GPU environments run on
	// the qemu backend, whose snapshot endpoints hard-501 because a vfio
	// device cannot be checkpointed, so there is nowhere to store a layer.
	ReasonGPU = "gpu environments cannot be snapshotted"
)

reasons a step is not cacheable. these are stable strings so callers (the `--plan` renderer, and per-step hit/miss reporting) can branch on them instead of matching prose.

View Source
const MaxFilesBytes = 64 << 10

MaxFilesBytes caps the total decoded size of Fusefile.Files.

The binding constraint is not the 1 MiB create body (api.MaxEnvironmentBodyBytes) but the way the script is delivered: the orchestrator runs it as `sh -lc <script>` and the host agent joins that into one shell word before handing it to ssh, so the whole script ends up as a single argv element. Linux caps one argument at MAX_ARG_STRLEN, 128 KiB, and overflowing it fails the boot with a bare E2BIG rather than anything an author could act on.

64 KiB of files becomes roughly 88 KiB of base64 once wrapped, which leaves around 40 KiB for the prelude, setup, and run. Exceeding the cap is a compile error rather than a runtime failure, so the author learns at `fuse up` time which file pushed them over.

View Source
const MinIdleTimeout = time.Minute

MinIdleTimeout is the smallest meaningful resources.idle_timeout. Idle expiry is detected on the orchestrator's reconcile loop (30s default) and requires two consecutive observations, so anything shorter than this promises a precision the loop cannot deliver.

Variables

This section is empty.

Functions

func BaseKey added in v0.11.0

func BaseKey(image string, files []File) string

BaseKey derives the root of the chain from the base image reference and the file block. An empty image means the host's baked base rootfs, which still gets a distinct key rather than an empty one so the first step's preimage is never ambiguous.

Files belong here rather than in any step's key because they are written before the first setup step runs, so they are part of the state that step builds on. Leaving them out would let an edited `files:` entry serve a layer baked against the old contents.

func KindSupportsMIG added in v0.7.0

func KindSupportsMIG(kind string) bool

KindSupportsMIG reports whether a GPU kind can plausibly run a MIG profile. An empty kind (unknown at request time) and any unrecognized kind return true so the scheduler's per-device MIGCapable flag remains the source of truth; only a kind on the known non-MIG denylist returns false. Matching is case-insensitive and substring-based so "NVIDIA V100" and "v100-sxm2" both resolve.

func ResolveFiles added in v0.12.0

func ResolveFiles(f *Fusefile, baseDir string) error

ResolveFiles reads every Files entry that names a Source into its Content, resolving Source relative to baseDir (the Fusefile's directory).

It is separate from Compile so Compile stays a pure function of the Fusefile: only this step touches the filesystem, and a caller that already holds the content (a test, or a future non-file authoring path) can skip it. Callers that never set Source can skip it too; Compile validates either way.

func ValidExposeName added in v0.14.0

func ValidExposeName(s string) bool

ValidExposeName reports whether s is a well-formed expose[].as name.

func ValidGPUProfile added in v0.7.0

func ValidGPUProfile(s string) bool

ValidGPUProfile reports whether s is a well-formed MIG profile name ("1g.10gb", "2g.20gb", ...). Shared with the API layer's request validation so raw SDK callers are held to the same vocabulary as Fusefile authors.

func ValidLabel added in v0.11.0

func ValidLabel(s string) bool

ValidLabel reports whether s is a well-formed placement label key or value. Shared with the API layer so raw SDK callers and host registration are held to the same vocabulary as Fusefile authors.

func Validate added in v0.12.0

func Validate(f *Fusefile) error

Validate reports every structural rule violation in f, joined into a single error.

Types

type Cache added in v0.11.0

type Cache struct {
	Enabled bool `yaml:"enabled,omitempty"`
}

Cache is the top-level opt-in for the setup layer cache. caching is off by default: a layer is a rootfs captured mid-provisioning, so opting in is a deliberate act.

type Compiled

type Compiled struct {
	Spec            ResourceSpec
	ManifestJSON    []byte
	StartupScript   string
	RequiredSecrets []string
	Expose          []ExposeSpec

	// BuildScript is the setup phase alone, for `fuse build` to run through
	// the exec path (600s) rather than the startup-script path (30s).
	// StartupScript still carries setup+run, so `fuse up` is unchanged.
	BuildScript string

	// RunScript is the run command alone, for a boot whose setup phase is
	// already baked into a seed rootfs (`fuse up --from-build`). Replaying
	// setup there would redo the work the artifact exists to skip.
	RunScript string

	// StartupTimeoutSeconds carries Fusefile.StartupTimeout to the wire.
	// Zero means the author did not ask for one and the orchestrator's
	// default applies.
	StartupTimeoutSeconds int64
}

Compiled is the result of compiling a Fusefile: the resource spec, the manifest json to upload to the guest, the startup script to run, the secrets the environment needs at create time, and any ports to expose.

func Compile

func Compile(f *Fusefile) (*Compiled, error)

Compile turns the human-friendly Fusefile.Resources into a ResourceSpec.

type EnvValue

type EnvValue struct {
	Value  string `yaml:"value,omitempty"`
	Secret string `yaml:"secret,omitempty"`
}

EnvValue is either a literal value or a secret reference. exactly one is set.

type Expose

type Expose struct {
	Port int    `yaml:"port"`
	As   string `yaml:"as,omitempty"`
}

Expose publishes a guest port to the outside world.

type ExposeSpec

type ExposeSpec struct {
	Port int
	As   string
}

ExposeSpec requests that a guest port be published as a reachable endpoint. Mirrors Fusefile's Expose entries one-for-one.

type File added in v0.12.0

type File struct {
	// Path is where the file lands in the guest. A relative path resolves
	// against the workspace.
	Path string `yaml:"path"`
	// Source is a path on the authoring machine, relative to the Fusefile.
	Source string `yaml:"source,omitempty"`
	// Content is a literal file body.
	Content string `yaml:"content,omitempty"`
	// Mode is an octal permission string, e.g. "0755". Empty leaves the
	// mode to the guest's umask.
	Mode string `yaml:"mode,omitempty"`
}

File is one file materialized inside the guest before setup runs.

Exactly one of Source (a path on the authoring machine, resolved relative to the Fusefile's directory by ResolveFiles) and Content (a literal) is set. Content is what the compiler emits; Source is sugar that ResolveFiles reads into Content, which keeps Compile a pure function of the Fusefile.

This is a transport for config and small code, not for model weights or datasets: entries are base64-encoded into the startup script, which travels in the create request body, so MaxFilesBytes caps their total size. Fetch large artifacts from inside `setup` instead.

type Fusefile

type Fusefile struct {
	Version   int                `yaml:"version"`
	Image     string             `yaml:"image,omitempty"`
	Resources Resources          `yaml:"resources,omitempty"`
	Placement Placement          `yaml:"placement,omitempty"`
	Cache     Cache              `yaml:"cache,omitempty"`
	Files     []File             `yaml:"files,omitempty"`
	Setup     []Step             `yaml:"setup,omitempty"`
	Services  map[string]Service `yaml:"services,omitempty"`
	Run       string             `yaml:"run,omitempty"`
	Workspace string             `yaml:"workspace,omitempty"`
	Expose    []Expose           `yaml:"expose,omitempty"`
	Secrets   []string           `yaml:"secrets,omitempty"`

	// StartupTimeout bounds the generated startup script (setup + run) as a
	// go duration, e.g. "45s". Empty means the orchestrator's default. The
	// orchestrator rejects a value above its configured ceiling rather than
	// clamping it, so an author always knows what bound they got.
	//
	// This is not the knob for a genuinely long setup phase: the ceiling is
	// bounded by the control plane's HTTP write timeout because create is
	// synchronous. Bake long work into an image with `fuse build` instead.
	StartupTimeout string `yaml:"startup_timeout,omitempty"`
}

Fusefile is the v1 authoring contract.

func Decode added in v0.12.0

func Decode(data []byte) (*Fusefile, error)

Decode decodes a Fusefile from yaml with a strict decoder without validating it. Parse is the normal entry point; Decode is separate so a caller that wants to report structural and compile problems together (`fuse validate`) can run Validate and Compile over the same decoded file.

func Parse

func Parse(data []byte) (*Fusefile, error)

Parse decodes a Fusefile from yaml bytes using a strict decoder (unknown fields are rejected) and then validates the result. It returns the parsed Fusefile only if it is structurally valid.

type HealthCheck added in v0.13.0

type HealthCheck struct {
	Test     []string `yaml:"test,omitempty"`
	Interval string   `yaml:"interval,omitempty"`
	Timeout  string   `yaml:"timeout,omitempty"`
	Retries  int      `yaml:"retries,omitempty"`
}

HealthCheck is a compose-native container healthcheck. Interval and Timeout are go durations (e.g. "10s").

type InputDigester added in v0.11.0

type InputDigester func(patterns []string) (string, error)

InputDigester hashes a step's declared `inputs` patterns and returns the digest to fold into that step's key. It is supplied by the caller because this package deliberately does no filesystem IO: only the CLI knows the directory the Fusefile was read from. It must return "" for an empty pattern list.

type LayerKey added in v0.11.0

type LayerKey struct {
	// Index is the step's position in Fusefile.Setup.
	Index int
	// Script is the shell fragment this step emits, hashed byte for byte.
	Script string
	// Key is the step's content-addressed layer key, hex sha256. Empty when
	// the step is not cacheable.
	Key string
	// ParentKey is the key this step was chained onto: the previous step's
	// key, or the base key for the first step. Empty when not cacheable.
	ParentKey string
	// InputsDigest is the digest of the step's declared inputs, "" when the
	// step declares none.
	InputsDigest string
	// Cacheable reports whether this step can produce or restore a layer.
	Cacheable bool
	// Reason explains why Cacheable is false; empty when it is true.
	Reason string
}

LayerKey is the derived cache identity of one setup step.

func LayerKeys added in v0.11.0

func LayerKeys(f *Fusefile, inputs InputDigester, opts LayerOptions) ([]LayerKey, error)

LayerKeys derives the layer key of every setup step, in order.

The key is a chained hash, so it invalidates in exactly one direction: edit step N and steps N..end all get new keys, while steps before N are untouched. That is also why a miss cascades: step N+1's key is defined in terms of step N's, so a step whose parent is unknown cannot be keyed at all and neither can anything after it.

layer_key(i) = sha256(
    "fusefile-layer/v1\n" +
    parent_key(i) + "\n" +   // layer_key(i-1), or the base key for i == 0
    step_script(i) + "\n" +  // the fragment the step emits, byte for byte
    inputs_digest(i) + "\n" +
    workspace + "\n" +       // setup runs after `cd <workspace>`
    arch)

Deliberately excluded: secret values (a layer must not be keyed on material it is not allowed to bake in, so a step that reads secrets opts out instead), the task id, every `resources` field except the arch (none of them change the filesystem), and `run:` (it is the entrypoint, not a layer).

LayerKeys is pure apart from the digester it is handed, so it can be tested without touching a disk.

type LayerOptions added in v0.11.0

type LayerOptions struct {
	// BaseKey identifies the base rootfs the first step builds on. Empty
	// means derive it from Fusefile.Image via BaseKey.
	//
	// Ideally this is a digest of the resolved rootfs bytes; the host agent
	// does not expose one yet (it resolves `image` to a file by name with no
	// content hash), so today it is a digest of the image reference. That is
	// a weaker binding: republishing the same tag with new contents does not
	// invalidate. Wiring a real rootfs digest is backend work.
	BaseKey string

	// Arch is the architecture of the host the layer will be built on. An
	// ext4 rootfs is not portable across architectures, so a layer built on
	// arm64 must never be served to an amd64 boot.
	Arch string
}

LayerOptions carries the two key components that are not derivable from the Fusefile itself.

type Placement added in v0.11.0

type Placement struct {
	// Host pins the environment to an exact host id. A pin is not an
	// override: the pinned host still has to be active, run the right
	// backend, and fit the request.
	Host string `yaml:"host,omitempty"`

	// Labels must all match the host's operator-declared labels (AND).
	Labels map[string]string `yaml:"labels,omitempty"`
}

Placement constrains which host in a self-hosted fleet may run the environment. Every field is a hard gate, not a preference: a request that matches no host is rejected immediately, never queued.

Region is deliberately absent; it lives under Resources for historical reasons and a second spelling of the same selector would be worse than the block split.

type ResourceSpec

type ResourceSpec struct {
	CPUs      int32
	RamMB     int32
	StorageGB int32
	Region    string
	// Arch restricts scheduling to hosts of this CPU architecture ("amd64",
	// "arm64"). Not yet authorable in a Fusefile; carried for wire parity
	// so compile output mirrors the orchestrator's spec shape.
	Arch              string
	MaxRuntimeSeconds int64
	// IdleTimeoutSeconds destroys the environment after this many seconds
	// with no exec and no attach session. Zero means no idle expiry.
	IdleTimeoutSeconds int64
	Image              string
	GPUs               int32
	GPUKind            string
	GPUProfile         string
	// HostID pins the environment to an exact host id (placement.host).
	HostID string
	// Labels are the placement label selectors (placement.labels); every
	// pair must match the host's declared labels.
	Labels map[string]string
}

ResourceSpec mirrors internal/api.ResourceSpec field-for-field so the CLI can copy Compiled.Spec into the sdk/api spec without importing the api package.

Image selects the VM's base rootfs at create time (a name resolved by the firecracker host agent to a pre-baked rootfs file; see FUSEFILE_PLAN.md Phase 7). It lives here, not in the manifest json, because rootfs selection happens at Provider.Create — before the guest boots and long before the manifest is ever uploaded to it.

type Resources

type Resources struct {
	CPUs    VCPUs  `yaml:"cpus,omitempty"`
	GPU     int    `yaml:"gpu,omitempty"`      // device count: whole GPUs, or MIG instances when gpu_profile is set
	GPUKind string `yaml:"gpu_kind,omitempty"` // optional match, e.g. "a100"
	// GPUProfile requests fractional GPU allocation: a MIG profile in
	// nvidia mig-parted vocabulary (e.g. "1g.10gb", "2g.20gb"). When set,
	// `gpu` counts MIG instances of this profile rather than whole
	// devices (decision D5). Empty means whole-device allocation.
	GPUProfile string `yaml:"gpu_profile,omitempty"`
	Memory     string `yaml:"memory,omitempty"` // e.g. "2GB", "2G", "2GiB", "512MB"
	// Disk sizes the guest root disk, e.g. "10GB". It is the preferred
	// spelling; Storage is a permanent alias kept for files written before
	// the rename. Setting both is allowed only if they mean the same size.
	Disk       string `yaml:"disk,omitempty"`
	Storage    string `yaml:"storage,omitempty"`     // alias for Disk, never removed
	Region     string `yaml:"region,omitempty"`      // schedules only onto a host registered in this region; empty matches any
	MaxRuntime string `yaml:"max_runtime,omitempty"` // go duration
	// IdleTimeout destroys the environment after this long with no exec
	// and no attach session. Go duration. "idle" means exactly that: no
	// control-plane activity. In-guest CPU or network traffic is not
	// observed. Empty means no idle expiry.
	IdleTimeout string `yaml:"idle_timeout,omitempty"` // go duration
}

Resources is the human-friendly hardware spec; compiled to ResourceSpec.

type Service

type Service struct {
	Image string              `yaml:"image"`
	Ports []int               `yaml:"ports,omitempty"`
	Env   map[string]EnvValue `yaml:"env,omitempty"`

	// Command, Restart, HealthCheck, and DependsOn map one-to-one onto their
	// compose-native counterparts; the guest's `docker compose up` (decision
	// D1) is what interprets them, so no new runtime semantics are
	// introduced here. They govern the compose container, not the VM: a
	// failing healthcheck marks the service unhealthy inside the guest, it
	// does not signal the orchestrator that the environment is dead.
	Command []string `yaml:"command,omitempty"`
	// Restart must be one of the compose-native policies: "no", "always",
	// "on-failure", "unless-stopped".
	Restart     string       `yaml:"restart,omitempty"`
	HealthCheck *HealthCheck `yaml:"healthcheck,omitempty"`
	DependsOn   []string     `yaml:"depends_on,omitempty"`
}

Service is one in-vm service; compiled to manifest.services and a compose unit.

type Step added in v0.11.0

type Step struct {
	Run    string   `yaml:"run"`
	Inputs []string `yaml:"inputs,omitempty"`
	Cache  *bool    `yaml:"cache,omitempty"`

	// Workdir scopes this one step to a directory. a relative path resolves
	// against Fusefile.Workspace, since every step starts there. the step is
	// emitted as a subshell, so the directory change does not leak into the
	// next step; only the directory is scoped, not the shell (see the note on
	// setupScripts). empty means the step runs in the workspace, unchanged.
	Workdir string `yaml:"workdir,omitempty"`
}

Step is one setup step. it accepts two yaml forms: a bare scalar ("apt-get update -qq"), equivalent to {run: ...} and unchanged from v1's list of strings; and a mapping ({run: npm ci, inputs: [package.json]}), which adds inputs, cache, and workdir.

Cache is a pointer so "unset" is distinguishable from an explicit "cache: false": a step that reads secrets or writes outside the rootfs must opt out, and an unset field must not be read as an opt-out.

func (*Step) UnmarshalYAML added in v0.11.0

func (s *Step) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML decodes either a bare scalar (the legacy form, kept working byte for byte) or the mapping form.

type VCPUs added in v0.14.0

type VCPUs int32

VCPUs is a whole vCPU count. It accepts a yaml integer or a whole-valued float, so `cpus: 2` and `cpus: 2.0` both mean two vCPUs, and rejects a genuine fraction: firecracker takes an integer vcpu_count, qemu an integer -smp, and neither host agent writes a cgroup quota, so there is nothing in the stack that could honor `cpus: 0.5`.

func (*VCPUs) UnmarshalYAML added in v0.14.0

func (c *VCPUs) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML decodes the vCPU count through float64 so a whole-valued float is accepted and a fraction is rejected with a reason.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL