Documentation
¶
Overview ¶
Package module executes detached module processes on behalf of the host.
Every rule the protocol states is enforced here rather than assumed, because a module is a separate program -- possibly buggy, possibly hostile, and in this architecture deliberately written against an independent implementation of the contract.
Three donor defects are fixed by construction, all observed in the PicoClaw prototype this host replaces:
- stdout and stderr are captured SEPARATELY. The donor used CombinedOutput, so any module diagnostic corrupted the JSON stream it was parsing.
- output is BOUNDED. The donor buffered unbounded child output into memory and echoed malformed payloads back into model context.
- the whole PROCESS TREE is killed. The donor's context cancellation killed only the direct child, orphaning grandchildren such as ffmpeg.
Index ¶
- Constants
- func ApplyGrants(d *modproto.Descriptor, req *modproto.Request, policy GrantPolicy)
- func DescribeInstalled(ctx context.Context, home, id string) (*modproto.Descriptor, error)
- func GrantBinaries(d *modproto.Descriptor, req *modproto.Request) (missing []string)
- func Install(ctx context.Context, home, binaryPath string) (string, error)
- func ModulesDir(home string) string
- func NewRequestID() (string, error)
- func Remove(home, id string) error
- func ResolveArtifact(req *modproto.Request, a modproto.Artifact) (string, error)
- func ResolveBinaries(declared []string) (resolved map[string]string, missing []string)
- func StageArtifact(req *modproto.Request, a modproto.Artifact, storeDir string) (string, error)
- func WithinRoot(root, path string) bool
- type GrantPolicy
- type PartialInstall
- type Result
- type Runner
Constants ¶
const ( // DefaultDeadline is the Runner's own fallback for a caller that sets no // deadline at all. It is DELIBERATELY NOT the invocation default -- see // DefaultInvokeDeadlineMS, which is what every production caller passes and // what the contract documents. // // Two numbers, and the difference is not an oversight. This one predates // the measurement: it was chosen when a module invocation was a small // bounded thing, before a real Remotion render was observed at 1m55s // against a 120s budget. Every production path -- CLI, agent, cockpit -- // sets DeadlineMS explicitly, so this fallback is reached only by a caller // inside this repo that constructed a Request by hand, which today means // tests. // // Kept at 60s rather than raised to match, because the two answer // different questions. A hand-built Request with no deadline is a caller // who has not thought about bounds; giving it the render budget would make // a forgotten field cost three minutes before anyone notices. The // invocation default is measured against real work; this is a guard // against absent-mindedness. // // If a production path ever reaches this, that is the bug -- not the // value. TestEveryProductionInvocationSetsItsOwnDeadline pins that. DefaultDeadline = 60 * time.Second DefaultMaxOutputBytes = 1 << 20 // 1 MiB DefaultMaxStderrBytes = 64 << 10 // DefaultInvokeDeadlineMS is how long a capability may run when the caller // does not say. // // One constant, because three callers each had their own: the CLI allowed // 120s while the agent and the cockpit allowed 180s, so a render that // worked through the browser failed through `handoff` with // "command_timeout: node was cancelled or timed out" -- which reads as a // broken module rather than a shorter leash. // // Video rendering is the case that sets the floor. It is a wall-clock // budget rather than a promise: a module that needs longer should return a // job handle, not hold the host. // // The floor is measured, not chosen: the same render takes ~17s on an idle // machine and was observed at 2m32s while a frontend build, a test run and // the gateway competed for the same cores. A budget set from the idle case // would kill real work whenever the machine is busy, which is exactly when // a person is most likely to be waiting on it. DefaultInvokeDeadlineMS = 180000 )
Defaults applied when a caller does not specify bounds. They are deliberately modest: a module needing more must say so, so that "unbounded" is never the accidental default.
const EnvSubprocessAllow = "FACET_STUDIO_SUBPROCESS_ALLOW"
EnvSubprocessAllow names the executables the host may authorize, overriding the default of "everything the module declared".
It exists because a module choosing among several declared binaries picks the first the host GRANTS, and the host cannot tell a signed-in CLI from a merely installed one -- `copilot --version` exits 0 on a machine where every real request fails to authenticate. Narrowing the grant is the one lever the host legitimately has: it is a statement about what this machine authorizes, not a guess about which CLI works.
Unset means unchanged: every declared name is authorized, as before. Setting it is the user saying which of their CLIs they are actually signed in to.
Variables ¶
This section is empty.
Functions ¶
func ApplyGrants ¶
func ApplyGrants(d *modproto.Descriptor, req *modproto.Request, policy GrantPolicy)
ApplyGrants fills req.Grants from what the module DECLARED, intersected with what the host policy authorizes.
This is the per-invocation half of "installing a module grants nothing". Authority is conferred here, on this call, and only for names the module declared up front -- the same rule GrantBinaries applies to executables.
Two properties are load-bearing:
- A grant is not consent. Authorizing a paid provider says the host will let the module reach it; whether to spend money on this particular call is a separate human decision the module still has to ask for. Collapsing the two is how an unpriced call gets made because someone once ticked a box.
- An empty grant list means "nothing authorized", and that is different from a module being invoked outside any host mediation. A module that receives an explicit empty list must fail closed rather than assume it was called directly.
Subprocess IS set here, and the comment that used to say otherwise was wrong in a way that cost a journey. The reasoning -- that req.Binaries carries the resolved paths, so naming them again would be a second source of truth -- sounded right and left Grants.Subprocess permanently empty. A module reads the GRANT to decide whether it may shell out and the PATH to know what to run; supplying only the path handed it an executable with no permission to use it, and Midden refused with "no AI CLI was granted" one line after the host printed "binaries: 3/3 resolved".
The two are not duplicates: one is authority, the other is location.
func DescribeInstalled ¶
DescribeInstalled returns the descriptor of one installed module.
func GrantBinaries ¶
func GrantBinaries(d *modproto.Descriptor, req *modproto.Request) (missing []string)
GrantBinaries fills req.Binaries from the module's declared subprocess names, intersected with what the host could actually resolve.
This is the per-invocation half of "installing a module grants nothing": authority to execute a binary is conferred here, on this call, and only for names the module declared up front.
func Install ¶
Install registers a module binary with this host.
Registration is a HOST action, deliberately. A module never writes into the host's state directory and never needs to know its layout: it hands the host a path and the host decides everything else. That keeps the confinement rule intact -- a module that could write into host state could grant itself authority -- and it means a module installer only has to find `facet-studio` on PATH.
The order matters:
- describe and VALIDATE before copying, so a binary that does not speak the protocol is refused rather than installed and discovered broken later;
- derive the module ID from the DESCRIPTOR, never from the filename or the caller, so a binary cannot be installed under a name it does not claim;
- copy into a host-chosen path;
- re-describe the INSTALLED copy, so what was verified is what will run.
It is idempotent: installing over an existing module replaces it, which is how upgrade works.
func ModulesDir ¶
ModulesDir is where installed modules live under the host state root.
func NewRequestID ¶
NewRequestID generates the host-side correlation ID a module must echo verbatim. It is host-generated so correlation never depends on trusting a module to be unique.
func Remove ¶
Remove uninstalls a module and deletes its binary.
A module's own state under <home>/state/<id>/ is deliberately left alone: uninstalling should not destroy a user's work, and reinstalling should find it again. Removing state is a separate, explicit action.
func ResolveArtifact ¶
ResolveArtifact turns a module-reported artifact into an absolute host path, refusing anything that escapes its declared root.
Resolution is done with EvalSymlinks where possible, so a symlink pointing outside the root cannot smuggle a path past a purely lexical check.
func ResolveBinaries ¶
ResolveBinaries turns a module's DECLARED subprocess names into absolute paths for one invocation.
Modules run with no inherited environment, so there is no PATH for them to search. The host resolves on their behalf, which is deliberately stronger than handing over a minimal PATH: a search can resolve to something that was never declared -- a shadowing entry, or a second binary with the same name in a supplied directory -- whereas an absolute path is an identity rather than a query. It also makes Permissions.Subprocess load-bearing: a binary the module did not declare is never supplied, so it cannot be run.
Only names in `declared` are ever looked up. A name absent from the descriptor is not resolvable by asking, which is what keeps the capability list and the execution authority in agreement.
A declared binary that cannot be resolved is OMITTED rather than mapped to an empty string, so "not supplied" and "supplied as nothing" stay distinguishable -- the same rule that keeps unknown cost from becoming zero. The unresolved names are returned so the host can tell the user which requirement is unmet instead of letting the module fail opaquely.
func StageArtifact ¶
StageArtifact copies an artifact a module produced into the host's own artifact store and verifies its digest at the moment of transfer.
NOT THE COMPOSITION MECHANISM, and this comment used to say it was.
Chaining two modules is ordinary tool calling: the agent invokes one capability, receives an artifact path in the tool result, and passes it as an argument to the next. The reasoning driver handles that without host help, which is why nothing in the agent loop calls this function and nothing should be built to make it do so.
What this exists for is the narrower case where a consumer needs a file under a root it actually holds. Root names are scoped to one invocation and are not portable: a file one module writes under its own rw root cannot be named by a root another was given, and neither should learn the other's layout. When that matters, the host copies the artifact somewhere neutral and supplies THAT as a read-only root.
Its one caller today is the `handoff` CLI command, which prints numbered steps to demonstrate the transfer. That is a demonstration, not a product path.
Verifying here rather than on read is deliberate: it is the one moment the host has both the producer's claim and the bytes, so a mismatch is caught before anything downstream can treat unverified content as provenanced.
The digest checked here is Artifact.Digest, which answers "did these bytes survive the copy". It is NOT any digest a module carries inside its own payload: a Midden seed, for instance, has an evidence_digest covering evidence.jsonl alone, deliberately excluding prose so that regenerating a brief does not invalidate a consumer's provenance claim. Two values, two questions. Comparing one against the other would be a bug rather than a mismatch, so the host reads only the artifact's own digest and never looks inside the payload.
A directory artifact -- a seed bundle is one -- is copied whole, and the digest is checked against the file the artifact actually points at.
func WithinRoot ¶
WithinRoot reports whether path is inside root, treating both as cleaned absolute-ish paths.
Exported because three other places were doing this with strings.HasPrefix, which is wrong in a way that matters: "/srv/modules/acme-evil" has the prefix "/srv/modules/acme" and is a different directory. A module declaring "../acme-evil/overlay.md" escaped its own tree and the check accepted it.
One implementation, so a fix here cannot leave a copy behind.
Types ¶
type GrantPolicy ¶
type GrantPolicy struct {
// Network, Credentials and PaidProviders are the names the host will
// authorize when a module declares them.
Network []string
Credentials []string
PaidProviders []string
// Subprocess is the set of executable NAMES the host will authorize.
//
// It is separate from Request.Binaries, which carries the resolved absolute
// PATHS. A module reads the grant to decide whether it MAY shell out, and
// the path to know what to run -- so filling only Binaries left a module
// with a path it was not authorized to use. Midden's evidence.extract
// refused with "no AI CLI was granted for this invocation" one line after
// the host printed "binaries: 3/3 resolved"; both were true, about
// different fields.
Subprocess []string
// Publish is separate because it is a single capability rather than a set,
// and because publishing is the one effect a user almost always wants to
// approve per act rather than per session.
Publish bool
}
GrantPolicy is what the HOST is willing to authorize for an invocation.
It is the host's half of the grant: a module declares what it may need, the policy says what the host will actually allow, and the grant is the intersection. Neither side alone decides -- a module cannot widen its own authority by declaring more, and the host cannot confer authority for something the module never declared and therefore cannot be expecting.
The zero value authorizes nothing, which is the correct default for a host that has not made a decision yet.
func GrantAll ¶
func GrantAll() GrantPolicy
GrantAll authorizes everything a module declared.
This is for the trusted single-user desktop case this host was built for: the person running the cockpit installed the module deliberately, and the approval that matters happens at the call, in front of them, not in a policy table they never wrote. Consent and cost gates still apply -- a grant says "you may reach this provider", never "you may spend this money".
func GrantNothing ¶
func GrantNothing() GrantPolicy
GrantNothing authorizes nothing while still being an explicit decision.
type PartialInstall ¶
PartialInstall reports a module that installed and runs, but whose declared knowledge could not be copied.
It is an error type rather than a silent warning because the consequence is invisible at runtime: the agent behaves as though the module documented nothing, with no signal that anything is missing.
func (*PartialInstall) Error ¶
func (e *PartialInstall) Error() string
type Result ¶
type Result struct {
Envelope *modproto.Envelope
// Stderr is advisory diagnostics only: bounded, surfaced to humans, and
// never parsed for control flow or protocol data.
Stderr string
// Truncated reports that output hit its bound. The envelope is then
// unusable, since a truncated JSON document cannot be trusted even if it
// happens to parse.
Truncated bool
Duration time.Duration
ExitCode int
// Warnings are host-side advisory findings about this invocation, distinct
// from the module's own Envelope.Warnings.
Warnings []string
}
Result is one completed module invocation, including the evidence needed to explain what happened when it went wrong.
type Runner ¶
type Runner struct {
// Binary is the absolute path to the module executable.
Binary string
// ModuleID is the expected module ID; a descriptor claiming a different
// one is rejected, so a binary cannot impersonate another module.
ModuleID string
MaxOutputBytes int
MaxStderrBytes int
}
Runner executes one installed module binary.
func (*Runner) Describe ¶
Describe runs `<binary> module describe --json` and returns a validated descriptor.
func (*Runner) Invoke ¶
func (r *Runner) Invoke(ctx context.Context, d *modproto.Descriptor, req *modproto.Request) (*Result, error)
Invoke runs one capability and returns a fully validated envelope.
The descriptor is required rather than optional: without it the host cannot check reported execution against declared effects, which is the check that keeps an unpriced call from rendering as free and a writing capability from routing around approval.
Source Files