projectseed

package
v0.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package projectseed carries a cloop project into a sandbox that had to fetch its own source tree.

The gap this closes

`cloop run` inside a sandbox reads `.cloop/` from the directory it is standing in. When the executor shares the hub's filesystem that directory is the hub's own project and everything works. When it does not — a remote agent, a Kubernetes Pod — the workspace is a *git clone*, and a clone of a source repository contains no cloop project. The run exited on its first line with "no cloop project found (run 'cloop init' first)" against a checkout that was entirely correct.

Why the legacy state.json shape

This package does not invent a transport format. It writes the one cloop already reads: `.cloop/state.json`, the pre-SQLite layout that state.Load still migrates from on every open (see migrateLegacyIfNeeded). That choice is doing real work:

  • The sandbox side needs no new code at all. The first state.Load in the sandbox finds a state.json and no state.db, migrates, and returns a populated project. The import path is the one every legacy project on disk has exercised for a year, rather than a second deserializer written for this feature and tested only by this feature.
  • It is also the cure for a reused workspace. A workspace kept on an edge device holds the *previous* dispatch's state.db, and a run that read it reported "all tasks complete", or a reset task still failed, without running anything. Write removes that database, so state.Load has nothing to prefer over the seed; see staleDatabaseFiles for why being newer than it was not enough.

What is deliberately not in a seed

Steps. A long-running project's step history is the overwhelming majority of its state — megabytes of captured output — and the sandbox needs none of it: it is about to generate its own. Dropping it is what keeps a seed in the hundreds of kilobytes for a 482-task plan.

WorkDir. The hub's absolute project path means nothing inside a sandbox, and leaving it set is not merely useless — ProjectState.WorkDir is what Save writes back through, so a seed that carried the hub's path would point the sandbox's writes at a directory on another machine. It is cleared here and re-derived on the far side by migrateFromJSON, which fills an empty WorkDir with the directory it is migrating in.

The way back

A seed is a copy, and a run writes its outcomes into the copy. result.go and merge.go are the return half (Task 20339): the device reads back what the run changed once the workload exits, and the hub folds it into its own project. Without them a run could finish its task on a device while the dashboard — which renders the hub's database — showed it pending forever.

Index

Constants

View Source
const MaxDecompressedBytes = 32 << 20 // 32 MiB

MaxDecompressedBytes bounds a seed after inflation.

A compressed cap alone is not a bound: gzip will happily expand a few kilobytes into gigabytes, and the receiving side of this is an edge device whose disk the hub does not manage. This sits well under the 64 MiB ceiling state.Load's own reader applies to a state.json, so a seed that passes here is one the far side will still agree to read.

View Source
const MaxResultInflatedBytes = 16 << 20 // 16 MiB

MaxResultInflatedBytes bounds a result after decompression, for the same reason MaxDecompressedBytes bounds a seed: a compressed ceiling alone is not a bound on what gzip will expand to.

View Source
const ResultFormat = 1

ResultFormat is the version of the Result document this build writes.

Readers accept any version at or above 1 and read the fields they know: every field is optional, so a later format that adds one is still a valid format-1 document to an older hub. Changing the meaning of an existing field needs a new name, not a new number.

View Source
const (
	SeedFileName = "state.json"
)

SeedFileName and seedDir are where a seed lands, relative to the workspace. The receiver owns this choice: see the note in executor/seed.go on why a seed names no path of its own.

Variables

View Source
var ErrNoProject = errors.New("projectseed: no project state")

ErrNoProject reports that the hub has no project state to seed from.

View Source
var ErrNoSeed = errors.New("projectseed: the workload was not sent a project, so there is none to read back")

ErrNoSeed reports that a workload was started without a seed. Such a workload read its project from somewhere the hub can see (a bind mount), or had none; either way there is no dispatch-time copy to measure a change against, and nothing to send back.

Functions

func Build

func Build(st *state.ProjectState) ([]byte, error)

Build serialises a project into the bytes a Spec carries.

The input is the hub's own loaded state. It is copied — not mutated — before the fields that must not travel are cleared, because the caller's pointer is very often the live state a handler is still holding.

func Harvest

func Harvest(dir string, seed []byte, scrub func(string) string) ([]byte, error)

Harvest reads back what a seeded run left in dir and returns it as the compressed bytes of a Result.

dir is the workload's working directory — the one Write placed the seed in — and seed is the exact payload the workload was started with; the change is measured against it. scrub, when non-nil, is applied to every free-text field before anything is encoded, so a credential the workload printed does not cross the wire.

Nothing outside dir/.cloop is read. A `.cloop` or database file that the workload replaced with a symbolic link is refused rather than followed: the agent may be able to read files the workload cannot, and a link is how a workload would ask it to.

func Merge

func Merge(st *state.ProjectState, r *Result, prov Provenance, scrub func(string) string, now time.Time) (MergeReport, Records)

Merge folds r into st, the hub's copy of the project, and returns what it did together with the steps, events and cost rows to record. It performs no I/O; Apply is the caller that loads, merges and saves.

scrub, when non-nil, is applied to every free-text field taken from r — the hub redacts what it leased to the run rather than trusting the device to have done it.

func Write

func Write(dir string, seed []byte) error

Write places a seed into a workspace the caller has already confined.

dir must be the workload's working directory. Nothing in seed influences the path: the file always lands at <dir>/.cloop/state.json, which is the property that keeps a hostile or corrupt seed from being an arbitrary-file-write on whichever host materialised it.

An empty seed is not an error. Most dispatches have none — every executor that shares the hub's filesystem — and a caller that had to test for that itself would eventually forget to.

Types

type Attachment added in v0.0.6

type Attachment struct {
	TaskID int `json:"task_id"`
	// TaskTitle is the title the task had in the seed — or, for a task the
	// run created, as it left it. The hub attaches the file only to the task
	// the run worked on: one the hub deleted or retitled while the run was
	// out, and whose id now belongs to another, does not get it.
	TaskTitle string    `json:"task_title,omitempty"`
	Name      string    `json:"name"`
	Size      int64     `json:"size"`
	SHA256    string    `json:"sha256,omitempty"`
	CreatedAt time.Time `json:"created_at"`
	Data      []byte    `json:"data,omitempty"`
	Left      string    `json:"left,omitempty"`
}

Attachment is one file the run attached to a task (Task 20428).

Data is the file — scrubbed like every other field — when the result had room for it, and empty otherwise, with Left saying why: the hub records the attachment either way, so a person looking at the task learns the file exists and where it is. Size and SHA256 describe Data when it travels, and are the device's account of the file when it does not; the hub checks the one and only reports the other.

type Cost

type Cost struct {
	Timestamp      time.Time `json:"timestamp"`
	TaskID         int       `json:"task_id,omitempty"`
	TaskTitle      string    `json:"task_title,omitempty"`
	Provider       string    `json:"provider,omitempty"`
	Model          string    `json:"model,omitempty"`
	InputTokens    int       `json:"input_tokens,omitempty"`
	OutputTokens   int       `json:"output_tokens,omitempty"`
	ThinkingTokens int       `json:"thinking_tokens,omitempty"`
	EstimatedUSD   float64   `json:"estimated_usd,omitempty"`
}

Cost is one cost-ledger row. It carries no identity: who pays for a run is decided by the hub that dispatched it, never by the device that ran it.

type Event

type Event struct {
	Timestamp time.Time `json:"timestamp"`
	Type      string    `json:"type"`
	TaskID    int       `json:"task_id,omitempty"`
	TaskTitle string    `json:"task_title,omitempty"`
	Step      int       `json:"step"`
	Message   string    `json:"message,omitempty"`
	Details   string    `json:"details,omitempty"`
}

Event is one journal row, in a form that survives JSON: statedb.EventRow has no tags and an ID that means nothing on another database.

type MergeReport

type MergeReport struct {
	// Updated are the tasks whose outcome was taken from the run, and
	// Outcomes their status afterwards, by ID.
	Updated  []int
	Outcomes map[int]pm.TaskStatus
	// Added are the tasks the run created, as numbered on the hub.
	Added []int
	// Renumbered maps a created task's ID on the device to the one it was
	// given here, for the tasks whose ID the hub had already used.
	Renumbered map[int]int
	// Kept are changes the hub declined, one sentence each.
	Kept []string
	// StatusFrom and StatusTo record the project status change, if any.
	StatusFrom, StatusTo string
	// Steps, Events and Costs count what was recorded.
	Steps, Events, Costs int
	// Omitted is what the device said it left out.
	Omitted []string
	// Unreviewed are tasks that came back done without a review although the
	// project's review gate is on (Task 20357): the device's cloop predates
	// the gate, so it ran them — and published their work — unreviewed.
	Unreviewed []int
	// Attached counts the run's files stored on the hub (Task 20428),
	// LeftBehind names, one sentence each, the ones recorded without their
	// bytes — left in the executor's workspace, or refused here — and
	// NotRecorded the ones the hub has no task for.
	Attached    int
	LeftBehind  []string
	NotRecorded []string
	// contains filtered or unexported fields
}

MergeReport says what folding a Result into the hub's project did.

func Apply

func Apply(workDir string, r *Result, prov Provenance, scrub func(string) string) (MergeReport, error)

Apply loads the hub's copy of the project at workDir, merges r into it, and records the steps, events and cost rows the run brought back.

When the plan could not be saved the report is empty, because nothing was merged. An error after the plan was saved — recording the journal or the cost rows — comes with the report of what did land, so the caller can say both.

func (MergeReport) Changed

func (m MergeReport) Changed() bool

Changed reports whether the merge altered the hub's project at all.

func (MergeReport) Summary

func (m MergeReport) Summary() string

Summary renders the report as one journal message.

type Provenance

type Provenance struct {
	// ExecutorID, ExecutorKind and Isolation are where the hub sent the run.
	ExecutorID   string
	ExecutorKind string
	Isolation    string
	// RunID is the execution id the hub minted for the dispatch; it joins the
	// tasks to the lease rows the broker wrote.
	RunID string
	// Identity is who the run's spend is billed to.
	Identity string
}

Provenance is what the hub knows about a dispatch and the device does not get a say in.

type Records

type Records struct {
	Steps  []state.StepResult
	Events []statedb.EventRow
	Costs  []cost.LedgerEntry
}

Records are what a merge produced for the tables beside the plan.

type Result

type Result struct {
	// Format is the document version; see ResultFormat.
	Format int `json:"format"`
	// BaseStep is the step counter the run started from — the seed's
	// CurrentStep. Informational: the hub numbers the run's steps after its
	// own history whatever this says, because its history may have grown
	// while the run was out.
	BaseStep int `json:"base_step"`
	// Status is the project status the run left behind ("complete",
	// "paused", "failed", ...).
	Status string `json:"status,omitempty"`
	// PauseReason explains a "paused" Status, including when a usage cap
	// lifts — which is what lets the hub's auto-resume pick the run back up.
	PauseReason *pausereason.Reason `json:"pause_reason,omitempty"`
	// InputTokens, OutputTokens and EvolveSteps are what the run *added*,
	// rather than the device's totals: the hub's own totals may have moved
	// while the run was out, and adding a delta is correct either way.
	InputTokens  int `json:"input_tokens,omitempty"`
	OutputTokens int `json:"output_tokens,omitempty"`
	EvolveSteps  int `json:"evolve_steps,omitempty"`
	// Tasks are the tasks the run changed or created.
	Tasks []TaskChange `json:"tasks,omitempty"`
	// Steps are the steps the run recorded, in order.
	Steps []state.StepResult `json:"steps,omitempty"`
	// Events are the journal rows the run wrote, oldest first.
	Events []Event `json:"events,omitempty"`
	// Costs are the cost rows the run recorded.
	Costs []Cost `json:"costs,omitempty"`
	// Attachments are the files the run attached to tasks (Task 20428), with
	// their bytes when the result had room for them.
	Attachments []Attachment `json:"attachments,omitempty"`
	// Omitted says what the device left out to fit the transport, one
	// sentence each, so the hub can say so instead of presenting a partial
	// history as a whole one.
	Omitted []string `json:"omitted,omitempty"`
}

Result is what a seeded run changed, as the device reports it.

func DecodeResult

func DecodeResult(data []byte) (*Result, error)

DecodeResult parses the bytes a device sent back, refusing anything a well-behaved device could not have produced. It checks shape and bounds only; what the hub accepts *from* a valid document is decided by Merge.

type TaskChange

type TaskChange struct {
	// Before is the task as the seed had it; nil when the run created it.
	//
	// It travels so the hub can tell its own edits from the run's: a field
	// the hub changed while the run was out no longer matches Before, and the
	// hub's value is kept. It is the device's claim, and the hub treats it as
	// one — see merge.go.
	Before *pm.Task `json:"before,omitempty"`
	// After is the task as the run left it.
	After *pm.Task `json:"after"`
}

TaskChange is one task the run touched.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL