Documentation
¶
Overview ¶
package fusefile is the canonical authoring format for a fuse environment. a Fusefile is parsed and compiled client-side into the orchestrator wire (CreateEnvironmentRequest); the orchestrator never sees a Fusefile.
Index ¶
- Constants
- func BaseKey(image string, files []File) string
- func KindSupportsMIG(kind string) bool
- func ResolveFiles(f *Fusefile, baseDir string) error
- func ValidExposeName(s string) bool
- func ValidGPUProfile(s string) bool
- func ValidLabel(s string) bool
- func Validate(f *Fusefile) error
- type Cache
- type Compiled
- type EnvValue
- type Expose
- type ExposeSpec
- type File
- type Fusefile
- type HealthCheck
- type InputDigester
- type LayerKey
- type LayerOptions
- type Placement
- type ResourceSpec
- type Resources
- type Service
- type Step
- type VCPUs
Constants ¶
const ( // ReasonCacheDisabled means the Fusefile has no `cache: {enabled: true}`. ReasonCacheDisabled = "caching is not enabled" // ReasonStepOptedOut means the step set `cache: false`. ReasonStepOptedOut = "step opted out with cache: false" // ReasonParentUncacheable means an earlier step is uncacheable, so this // step has no known parent layer to build on. ReasonParentUncacheable = "an earlier step is uncacheable" // ReasonGPU means the environment requests a GPU. GPU environments run on // the qemu backend, whose snapshot endpoints hard-501 because a vfio // device cannot be checkpointed, so there is nowhere to store a layer. ReasonGPU = "gpu environments cannot be snapshotted" )
reasons a step is not cacheable. these are stable strings so callers (the `--plan` renderer, and per-step hit/miss reporting) can branch on them instead of matching prose.
const MaxFilesBytes = 64 << 10
MaxFilesBytes caps the total decoded size of Fusefile.Files.
The binding constraint is not the 1 MiB create body (api.MaxEnvironmentBodyBytes) but the way the script is delivered: the orchestrator runs it as `sh -lc <script>` and the host agent joins that into one shell word before handing it to ssh, so the whole script ends up as a single argv element. Linux caps one argument at MAX_ARG_STRLEN, 128 KiB, and overflowing it fails the boot with a bare E2BIG rather than anything an author could act on.
64 KiB of files becomes roughly 88 KiB of base64 once wrapped, which leaves around 40 KiB for the prelude, setup, and run. Exceeding the cap is a compile error rather than a runtime failure, so the author learns at `fuse up` time which file pushed them over.
const MinIdleTimeout = time.Minute
MinIdleTimeout is the smallest meaningful resources.idle_timeout. Idle expiry is detected on the orchestrator's reconcile loop (30s default) and requires two consecutive observations, so anything shorter than this promises a precision the loop cannot deliver.
Variables ¶
This section is empty.
Functions ¶
func BaseKey ¶ added in v0.11.0
BaseKey derives the root of the chain from the base image reference and the file block. An empty image means the host's baked base rootfs, which still gets a distinct key rather than an empty one so the first step's preimage is never ambiguous.
Files belong here rather than in any step's key because they are written before the first setup step runs, so they are part of the state that step builds on. Leaving them out would let an edited `files:` entry serve a layer baked against the old contents.
func KindSupportsMIG ¶ added in v0.7.0
KindSupportsMIG reports whether a GPU kind can plausibly run a MIG profile. An empty kind (unknown at request time) and any unrecognized kind return true so the scheduler's per-device MIGCapable flag remains the source of truth; only a kind on the known non-MIG denylist returns false. Matching is case-insensitive and substring-based so "NVIDIA V100" and "v100-sxm2" both resolve.
func ResolveFiles ¶ added in v0.12.0
ResolveFiles reads every Files entry that names a Source into its Content, resolving Source relative to baseDir (the Fusefile's directory).
It is separate from Compile so Compile stays a pure function of the Fusefile: only this step touches the filesystem, and a caller that already holds the content (a test, or a future non-file authoring path) can skip it. Callers that never set Source can skip it too; Compile validates either way.
func ValidExposeName ¶ added in v0.14.0
ValidExposeName reports whether s is a well-formed expose[].as name.
func ValidGPUProfile ¶ added in v0.7.0
ValidGPUProfile reports whether s is a well-formed MIG profile name ("1g.10gb", "2g.20gb", ...). Shared with the API layer's request validation so raw SDK callers are held to the same vocabulary as Fusefile authors.
func ValidLabel ¶ added in v0.11.0
ValidLabel reports whether s is a well-formed placement label key or value. Shared with the API layer so raw SDK callers and host registration are held to the same vocabulary as Fusefile authors.
Types ¶
type Cache ¶ added in v0.11.0
type Cache struct {
Enabled bool `yaml:"enabled,omitempty"`
}
Cache is the top-level opt-in for the setup layer cache. caching is off by default: a layer is a rootfs captured mid-provisioning, so opting in is a deliberate act.
type Compiled ¶
type Compiled struct {
Spec ResourceSpec
ManifestJSON []byte
StartupScript string
RequiredSecrets []string
Expose []ExposeSpec
// BuildScript is the setup phase alone, for `fuse build` to run through
// the exec path (600s) rather than the startup-script path (30s).
// StartupScript still carries setup+run, so `fuse up` is unchanged.
BuildScript string
// RunScript is the run command alone, for a boot whose setup phase is
// already baked into a seed rootfs (`fuse up --from-build`). Replaying
// setup there would redo the work the artifact exists to skip.
RunScript string
// StartupTimeoutSeconds carries Fusefile.StartupTimeout to the wire.
// Zero means the author did not ask for one and the orchestrator's
// default applies.
StartupTimeoutSeconds int64
}
Compiled is the result of compiling a Fusefile: the resource spec, the manifest json to upload to the guest, the startup script to run, the secrets the environment needs at create time, and any ports to expose.
type EnvValue ¶
type EnvValue struct {
Value string `yaml:"value,omitempty"`
Secret string `yaml:"secret,omitempty"`
}
EnvValue is either a literal value or a secret reference. exactly one is set.
type ExposeSpec ¶
ExposeSpec requests that a guest port be published as a reachable endpoint. Mirrors Fusefile's Expose entries one-for-one.
type File ¶ added in v0.12.0
type File struct {
// Path is where the file lands in the guest. A relative path resolves
// against the workspace.
Path string `yaml:"path"`
// Source is a path on the authoring machine, relative to the Fusefile.
Source string `yaml:"source,omitempty"`
// Content is a literal file body.
Content string `yaml:"content,omitempty"`
// Mode is an octal permission string, e.g. "0755". Empty leaves the
// mode to the guest's umask.
Mode string `yaml:"mode,omitempty"`
}
File is one file materialized inside the guest before setup runs.
Exactly one of Source (a path on the authoring machine, resolved relative to the Fusefile's directory by ResolveFiles) and Content (a literal) is set. Content is what the compiler emits; Source is sugar that ResolveFiles reads into Content, which keeps Compile a pure function of the Fusefile.
This is a transport for config and small code, not for model weights or datasets: entries are base64-encoded into the startup script, which travels in the create request body, so MaxFilesBytes caps their total size. Fetch large artifacts from inside `setup` instead.
type Fusefile ¶
type Fusefile struct {
Version int `yaml:"version"`
Image string `yaml:"image,omitempty"`
Resources Resources `yaml:"resources,omitempty"`
Placement Placement `yaml:"placement,omitempty"`
Cache Cache `yaml:"cache,omitempty"`
Files []File `yaml:"files,omitempty"`
Setup []Step `yaml:"setup,omitempty"`
Services map[string]Service `yaml:"services,omitempty"`
Run string `yaml:"run,omitempty"`
Workspace string `yaml:"workspace,omitempty"`
Expose []Expose `yaml:"expose,omitempty"`
Secrets []string `yaml:"secrets,omitempty"`
// StartupTimeout bounds the generated startup script (setup + run) as a
// go duration, e.g. "45s". Empty means the orchestrator's default. The
// orchestrator rejects a value above its configured ceiling rather than
// clamping it, so an author always knows what bound they got.
//
// This is not the knob for a genuinely long setup phase: the ceiling is
// bounded by the control plane's HTTP write timeout because create is
// synchronous. Bake long work into an image with `fuse build` instead.
StartupTimeout string `yaml:"startup_timeout,omitempty"`
}
Fusefile is the v1 authoring contract.
func Decode ¶ added in v0.12.0
Decode decodes a Fusefile from yaml with a strict decoder without validating it. Parse is the normal entry point; Decode is separate so a caller that wants to report structural and compile problems together (`fuse validate`) can run Validate and Compile over the same decoded file.
type HealthCheck ¶ added in v0.13.0
type HealthCheck struct {
Test []string `yaml:"test,omitempty"`
Interval string `yaml:"interval,omitempty"`
Timeout string `yaml:"timeout,omitempty"`
Retries int `yaml:"retries,omitempty"`
}
HealthCheck is a compose-native container healthcheck. Interval and Timeout are go durations (e.g. "10s").
type InputDigester ¶ added in v0.11.0
InputDigester hashes a step's declared `inputs` patterns and returns the digest to fold into that step's key. It is supplied by the caller because this package deliberately does no filesystem IO: only the CLI knows the directory the Fusefile was read from. It must return "" for an empty pattern list.
type LayerKey ¶ added in v0.11.0
type LayerKey struct {
// Index is the step's position in Fusefile.Setup.
Index int
// Script is the shell fragment this step emits, hashed byte for byte.
Script string
// Key is the step's content-addressed layer key, hex sha256. Empty when
// the step is not cacheable.
Key string
// ParentKey is the key this step was chained onto: the previous step's
// key, or the base key for the first step. Empty when not cacheable.
ParentKey string
// InputsDigest is the digest of the step's declared inputs, "" when the
// step declares none.
InputsDigest string
// Cacheable reports whether this step can produce or restore a layer.
Cacheable bool
// Reason explains why Cacheable is false; empty when it is true.
Reason string
}
LayerKey is the derived cache identity of one setup step.
func LayerKeys ¶ added in v0.11.0
func LayerKeys(f *Fusefile, inputs InputDigester, opts LayerOptions) ([]LayerKey, error)
LayerKeys derives the layer key of every setup step, in order.
The key is a chained hash, so it invalidates in exactly one direction: edit step N and steps N..end all get new keys, while steps before N are untouched. That is also why a miss cascades: step N+1's key is defined in terms of step N's, so a step whose parent is unknown cannot be keyed at all and neither can anything after it.
layer_key(i) = sha256(
"fusefile-layer/v1\n" +
parent_key(i) + "\n" + // layer_key(i-1), or the base key for i == 0
step_script(i) + "\n" + // the fragment the step emits, byte for byte
inputs_digest(i) + "\n" +
workspace + "\n" + // setup runs after `cd <workspace>`
arch)
Deliberately excluded: secret values (a layer must not be keyed on material it is not allowed to bake in, so a step that reads secrets opts out instead), the task id, every `resources` field except the arch (none of them change the filesystem), and `run:` (it is the entrypoint, not a layer).
LayerKeys is pure apart from the digester it is handed, so it can be tested without touching a disk.
type LayerOptions ¶ added in v0.11.0
type LayerOptions struct {
// BaseKey identifies the base rootfs the first step builds on. Empty
// means derive it from Fusefile.Image via BaseKey.
//
// Ideally this is a digest of the resolved rootfs bytes; the host agent
// does not expose one yet (it resolves `image` to a file by name with no
// content hash), so today it is a digest of the image reference. That is
// a weaker binding: republishing the same tag with new contents does not
// invalidate. Wiring a real rootfs digest is backend work.
BaseKey string
// Arch is the architecture of the host the layer will be built on. An
// ext4 rootfs is not portable across architectures, so a layer built on
// arm64 must never be served to an amd64 boot.
Arch string
}
LayerOptions carries the two key components that are not derivable from the Fusefile itself.
type Placement ¶ added in v0.11.0
type Placement struct {
// Host pins the environment to an exact host id. A pin is not an
// override: the pinned host still has to be active, run the right
// backend, and fit the request.
Host string `yaml:"host,omitempty"`
// Labels must all match the host's operator-declared labels (AND).
Labels map[string]string `yaml:"labels,omitempty"`
}
Placement constrains which host in a self-hosted fleet may run the environment. Every field is a hard gate, not a preference: a request that matches no host is rejected immediately, never queued.
Region is deliberately absent; it lives under Resources for historical reasons and a second spelling of the same selector would be worse than the block split.
type ResourceSpec ¶
type ResourceSpec struct {
CPUs int32
RamMB int32
StorageGB int32
Region string
// Arch restricts scheduling to hosts of this CPU architecture ("amd64",
// "arm64"). Not yet authorable in a Fusefile; carried for wire parity
// so compile output mirrors the orchestrator's spec shape.
Arch string
MaxRuntimeSeconds int64
// IdleTimeoutSeconds destroys the environment after this many seconds
// with no exec and no attach session. Zero means no idle expiry.
IdleTimeoutSeconds int64
Image string
GPUs int32
GPUKind string
GPUProfile string
// HostID pins the environment to an exact host id (placement.host).
HostID string
// Labels are the placement label selectors (placement.labels); every
// pair must match the host's declared labels.
Labels map[string]string
}
ResourceSpec mirrors internal/api.ResourceSpec field-for-field so the CLI can copy Compiled.Spec into the sdk/api spec without importing the api package.
Image selects the VM's base rootfs at create time (a name resolved by the firecracker host agent to a pre-baked rootfs file; see FUSEFILE_PLAN.md Phase 7). It lives here, not in the manifest json, because rootfs selection happens at Provider.Create — before the guest boots and long before the manifest is ever uploaded to it.
type Resources ¶
type Resources struct {
CPUs VCPUs `yaml:"cpus,omitempty"`
GPU int `yaml:"gpu,omitempty"` // device count: whole GPUs, or MIG instances when gpu_profile is set
GPUKind string `yaml:"gpu_kind,omitempty"` // optional match, e.g. "a100"
// GPUProfile requests fractional GPU allocation: a MIG profile in
// nvidia mig-parted vocabulary (e.g. "1g.10gb", "2g.20gb"). When set,
// `gpu` counts MIG instances of this profile rather than whole
// devices (decision D5). Empty means whole-device allocation.
GPUProfile string `yaml:"gpu_profile,omitempty"`
Memory string `yaml:"memory,omitempty"` // e.g. "2GB", "2G", "2GiB", "512MB"
// Disk sizes the guest root disk, e.g. "10GB". It is the preferred
// spelling; Storage is a permanent alias kept for files written before
// the rename. Setting both is allowed only if they mean the same size.
Disk string `yaml:"disk,omitempty"`
Storage string `yaml:"storage,omitempty"` // alias for Disk, never removed
Region string `yaml:"region,omitempty"` // schedules only onto a host registered in this region; empty matches any
MaxRuntime string `yaml:"max_runtime,omitempty"` // go duration
// IdleTimeout destroys the environment after this long with no exec
// and no attach session. Go duration. "idle" means exactly that: no
// control-plane activity. In-guest CPU or network traffic is not
// observed. Empty means no idle expiry.
IdleTimeout string `yaml:"idle_timeout,omitempty"` // go duration
}
Resources is the human-friendly hardware spec; compiled to ResourceSpec.
type Service ¶
type Service struct {
Image string `yaml:"image"`
Ports []int `yaml:"ports,omitempty"`
Env map[string]EnvValue `yaml:"env,omitempty"`
// Command, Restart, HealthCheck, and DependsOn map one-to-one onto their
// compose-native counterparts; the guest's `docker compose up` (decision
// D1) is what interprets them, so no new runtime semantics are
// introduced here. They govern the compose container, not the VM: a
// failing healthcheck marks the service unhealthy inside the guest, it
// does not signal the orchestrator that the environment is dead.
Command []string `yaml:"command,omitempty"`
// Restart must be one of the compose-native policies: "no", "always",
// "on-failure", "unless-stopped".
Restart string `yaml:"restart,omitempty"`
HealthCheck *HealthCheck `yaml:"healthcheck,omitempty"`
DependsOn []string `yaml:"depends_on,omitempty"`
}
Service is one in-vm service; compiled to manifest.services and a compose unit.
type Step ¶ added in v0.11.0
type Step struct {
Run string `yaml:"run"`
Inputs []string `yaml:"inputs,omitempty"`
Cache *bool `yaml:"cache,omitempty"`
// Workdir scopes this one step to a directory. a relative path resolves
// against Fusefile.Workspace, since every step starts there. the step is
// emitted as a subshell, so the directory change does not leak into the
// next step; only the directory is scoped, not the shell (see the note on
// setupScripts). empty means the step runs in the workspace, unchanged.
Workdir string `yaml:"workdir,omitempty"`
}
Step is one setup step. it accepts two yaml forms: a bare scalar ("apt-get update -qq"), equivalent to {run: ...} and unchanged from v1's list of strings; and a mapping ({run: npm ci, inputs: [package.json]}), which adds inputs, cache, and workdir.
Cache is a pointer so "unset" is distinguishable from an explicit "cache: false": a step that reads secrets or writes outside the rootfs must opt out, and an unset field must not be read as an opt-out.
type VCPUs ¶ added in v0.14.0
type VCPUs int32
VCPUs is a whole vCPU count. It accepts a yaml integer or a whole-valued float, so `cpus: 2` and `cpus: 2.0` both mean two vCPUs, and rejects a genuine fraction: firecracker takes an integer vcpu_count, qemu an integer -smp, and neither host agent writes a cgroup quota, so there is nothing in the stack that could honor `cpus: 0.5`.