secrets

package
v0.37.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 11, 2026 License: AGPL-3.0 Imports: 13 Imported by: 0

Documentation

Overview

Package secrets materializes user-supplied secrets onto the sandbox pod filesystem with strict validation and TOCTOU-safe permissions.

This package is the single source of truth for secret materialization. Both code paths use it:

  • Boot-time: `workspace-agentd materialize --from /sandbox-cfg/secrets.json` invoked from the runtime entrypoint script before opencode starts.
  • Apply: the agentd secrets pipeline calls Materialize directly with the request body.

Before this package existed, materialization was duplicated across a bash script (entrypoint-common.sh) and an inline Go function inside cmd/workspace-agentd/main.go, both of which suffered from Epic 17 G2: shell-quoted interpolation that broke on a single quote in the value. They also lacked input validation, used non-atomic chmod-after-write (G20), and used naive string-contains checks for path traversal.

The Materialize function:

  • Validates every field of every Secret against an allowlist (var names, key types, hostnames, protocols, mount-path scopes).
  • Writes files using os.OpenFile with O_CREATE|O_EXCL and mode 0600 so permissions are atomic with creation — no window where the file is readable with default umask.
  • Encodes the env-file value using shellquote.Bash (single-quoted with embedded single quotes escaped) so a malicious PLAINTEXT cannot break out of the bash `source` consumer at agentd buildEnvFrom (the entrypoint-opencode.sh consumer was deleted with the entrypoints, design 0053 S3).
  • Resolves mount paths via filepath.Clean + strict prefix containment against the secrets base directory.
  • Returns a typed *MaterializeResult that carries per-secret outcomes (Materialized / Skipped / Failed) along with a redacted reason. The caller decides whether to surface this to the operator (via pod status) or to logs.

Threat-model invariants this package enforces: 

T1 No interpretation of secret values by the shell.
T2 No DELIVERED credential file exists with a mode outside its
   consumer contract, and every delivered file is written by the
   consuming uid (R2b, #1165 — ownership by construction). Residual
   gid-1000-scoped exceptions: (a) agent-config.json ALWAYS 0640
   (uid-2000 sidecar writer, uid-1000 opencode reader, ruled
   uid-1000-readable BY NECESSITY — the embedded MCP Basic header is a
   documented residual); (b) secrets-env 0640 under CROSS_UID (init
   uid-1000 writer, sidecar uid-2000 boot-handoff reader — a platform
   parser with no ownership semantics).
T3 No path staged or delivered outside the staging tree, SecretsBasePath,
   $HOME/.ssh, or AgentConfigPath; the delivery side additionally
   confines every manifest target to its configured roots.
T4 No env-file line that does not round-trip cleanly through `source`.
T5 An invalid secret skips that secret only; the rest still materialize.

See `secrets_test.go` for the regression corpus, including:

  • Single-quote, dollar-sign, backtick, newline injection in PLAINTEXT.
  • Path traversal via "..", URL-encoded "..", absolute paths, symlinks.
  • Hostname injection via " IdentityFile /etc/shadow".
  • Var-name injection via reserved bash names and embedded "=" / ";".

Index

Constants

View Source 
const (
	ModeSSHPrivateKey = 0o600 // ssh: refuses group/private keys only on drift; 0600 is the contract
	ModeSSHConfig     = 0o600 // ssh: owner-must-be-invoking-user, never group/world-writable
	ModeGitCredential = 0o600 // git store helper: no checks today; 0600 by secret semantics
	ModeSecretFile    = 0o600 // arbitrary consumer; user may override via metadata "mode"
)

File-mode contracts (design 0057 R2b): one table, the single place that knows what each file class's consumer requires. Owner-only modes are correct because the DELIVERED files are written by the uid-1000 consumer itself — the CROSS_UID 0640 group-read machinery is obsolete for these classes (it solved readability; the consumers' parsers need ownership, which only the writer's uid can confer).

View Source 
const AgentConfigWriteMode os.FileMode = 0o640

AgentConfigWriteMode is the mode for the materialized agent-config.json (design 0051 §D1): 0640 — written across the uid split (uid-1000 init at boot, uid-2000 sidecar on reload), read by uid-1000 opencode via the pod's shared gid 1000. The T2 exception; see the header comment.

View Source 
const ManifestName = "manifest.json"

ManifestName is the manifest file inside the staging tree. The pull endpoint reads it fresh at request time; its absence means "no file-class secrets" (the quiet empty state, design 0057 law 5).

Variables

View Source 
var ErrPartialFailure = errors.New("secret materialization had partial failures")

ErrPartialFailure is returned by Materialize when at least one secret reached OutcomeFailed. Callers should still consider partially-applied state — files for already-materialized secrets remain on disk.

This sentinel stays a plain error (not *apierrors.APIError) because it lives in pkg/ which cannot import api/internal/ (Go internal-package visibility). It is consumed only by cmd/workspace-agentd (the agent daemon), never by an HTTP handler, so HTTP status mapping is not needed.

Functions

func FormatEnvLine 

func FormatEnvLine(varName, value string) string

FormatEnvLine produces an `export VAR='value'` line suitable for `source` by bash. The single-quote escaping (`'\”`) is the canonical safe form for embedding arbitrary text in a single-quoted shell string.

Consumers MUST read the resulting file via `bash source` (or an equivalent that implements bash's quoting rules), NOT via line-based regex parsing — values may contain literal newlines inside the quoted region, and a naive split-on-newline parser will mangle them.

In this codebase, the consumer is buildEnvFrom() in cmd/workspace-agentd which delegates to a bash subprocess so the source-of-truth parser is bash itself.

func RenderOpencodeMCPServerEntry added in v0.7.0 

func RenderOpencodeMCPServerEntry(srv StagedMCPServer) map[string]any

RenderOpencodeMCPServerEntry renders one MCP server into the opencode config shape per MATERIALIZE-CONTRACT.md. Shared between the AgentConfigWriter (cmd/workspace-agentd/agent_config_writer.go) and the boot-time applyMCPServersToConfig (cmd/workspace-agentd/secrets.go) so both paths produce identical output.

Types

type BatchFile added in v0.26.0 

type BatchFile struct {
	Secrets  []Secret
	Revision *sec.BatchRevision
}

BatchFile is a parsed secrets batch: the materialization entries plus the revision the entries arrived under. Revision is nil for legacy batches (bare arrays and pushes) — no revision anchoring, byte-compat with the pre-US-70.2 behavior.

func LoadBatchFile added in v0.26.0 

func LoadBatchFile(path string) (BatchFile, error)

LoadBatchFile reads and parses a secrets batch file, accepting both wire generations. Envelope detection is by the "entries" key: the v2 API's envelope always carries it; a legacy file is a JSON array.

func ParseBatchFile added in v0.26.0 

func ParseBatchFile(data []byte) (BatchFile, error)

ParseBatchFile parses raw batch bytes into a BatchFile.

type Filesystem 

type Filesystem interface {
	RemoveAll(path string) error
	MkdirAll(path string, perm os.FileMode) error
	Chmod(path string, perm os.FileMode) error
	OpenForCreate(path string, flag int, perm os.FileMode) (io.WriteCloser, error)
	Remove(path string) error
	Rename(oldpath, newpath string) error
}

Filesystem is the minimal interface Materialize needs. Tests inject a fake; production uses RealFS which delegates to os.*.

func RealFS 

func RealFS() Filesystem

RealFS returns the production Filesystem.

type LLMProviderFormatter 

type LLMProviderFormatter func(providers []sec.LLMProviderData) ([]byte, error)

LLMProviderFormatter is a callback that renders staged LLM provider data into the agent-specific config format. Each agent type (opencode, Claude Code, Codex) provides its own implementation.

type MaterializeResult 

type MaterializeResult struct {
	Results []SecretResult `json:"results"`
}

MaterializeResult aggregates outcomes for a Materialize call. The aggregate Error is nil when every secret was Materialized; otherwise it is a sentinel that callers can wrap. Per-secret reasons live on Results so callers can render structured status.

func (*MaterializeResult) Counts 

func (r *MaterializeResult) Counts() (int, int, int)

Counts returns (materialized, skipped, failed).

func (*MaterializeResult) HasFailures 

func (r *MaterializeResult) HasFailures() bool

HasFailures returns true if any secret produced an OutcomeFailed. OutcomeSkipped does not count: skipping a malformed secret is a successful security decision, not a failure.

type Materializer 

type Materializer struct {
	FS    Filesystem
	Paths Paths
	// CrossUID arms the group bits on the tool-consumed rt/* stores
	// (design 0051 US-4b): dirs 0770, files 0640. Set ONLY on the
	// sidecar's reload path, where uid-2000 writes must stay readable by
	// the uid-1000 tools that consume ~/.secrets/*, ~/.ssh/* and
	// ~/.git-credentials via the pod's shared gid 1000 (US-35.7 class C
	// ruling). Unset (single-container, init-container materialize): the
	// pre-US-4b owner-only modes stand.
	CrossUID bool
	// Revision (US-70.2): the delivery revision of the batch being
	// materialized. When set, the published staging manifest carries the
	// "<seq>:<manifestHash>" anchor; nil (push batches, legacy files)
	// keeps the manifest legacy-shaped (byte-compat).
	Revision *sec.BatchRevision
	// contains filtered or unexported fields
}

Materializer holds dependencies for materialization. Construct with NewMaterializer or pass a Materializer{} with field defaults filled in by the caller.

func NewMaterializer 

func NewMaterializer() *Materializer

NewMaterializer returns a Materializer using the production filesystem and paths derived from $HOME.

func (*Materializer) EnrichProviders 

func (m *Materializer) EnrichProviders(fn func([]sec.LLMProviderData) []sec.LLMProviderData)

EnrichProviders applies fn to the staged provider slice, replacing it with the result. Callers use this to inject additional fields (e.g. a live model list fetched from the provider's /models endpoint) after Materialize and before FlushProviders. fn must not be nil.

func (*Materializer) FlushProviders 

func (m *Materializer) FlushProviders(formatter LLMProviderFormatter) error

FlushProviders calls FormatProviders and writes the result to AgentConfigPath. Used by callers that do NOT have an AgentConfigWriter (e.g. the materialize subcommand, which runs as a separate process before agentd starts). Callers inside the agentd process should use FormatProviders + AgentConfigWriter.Rebuild instead.

When formatter is nil, FlushProviders is a no-op (no agent config is written). This allows callers to conditionally skip agent-specific rendering.

func (*Materializer) FormatProviders 

func (m *Materializer) FormatProviders(formatter LLMProviderFormatter) ([]byte, error)

FormatProviders calls the formatter with all staged LLM provider data and returns the formatted bytes WITHOUT writing to disk. Callers that use an external config writer (e.g. AgentConfigWriter in cmd/workspace-agentd) call this instead of FlushProviders so the writer is the sole disk writer.

Returns (nil, nil) when formatter is nil or no providers are staged, matching FlushProviders' no-op semantics. This lets callers unconditionally call FormatProviders → writer.SetProviders without branching.

func (*Materializer) Materialize 

func (m *Materializer) Materialize(secrets []Secret) (*MaterializeResult, error)

Materialize processes secrets and returns per-secret outcomes. The function performs a full reset of the secrets base directory, SSH directory, env file, agent config, and git credentials before applying the new set, matching the existing reload semantics.

func (*Materializer) StagedMCPServers added in v0.7.0 

func (m *Materializer) StagedMCPServers() []StagedMCPServer

StagedMCPServers returns the MCP servers accumulated during Materialize.

func (*Materializer) StagedProviders 

func (m *Materializer) StagedProviders() []sec.LLMProviderData

StagedProviders returns the LLM provider data accumulated during Materialize. Returns nil if no llm-provider secrets were in the batch. This allows callers to use the structured data for direct API injection (e.g., PUT /auth/:providerID) instead of or in addition to file-based config rendering via FlushProviders.

type Outcome 

type Outcome string

Outcome describes what happened to a single secret. 

const (
	OutcomeMaterialized Outcome = "materialized"
	OutcomeSkipped      Outcome = "skipped"
	OutcomeFailed       Outcome = "failed"
)

type Paths 

type Paths struct {
	Home            string // user home (e.g. /home/sandbox)
	SecretsBaseDir  string // secret-file root (/sandbox-runtime/rt/secrets)
	SSHDir          string // SSH config directory (/sandbox-runtime/rt/ssh)
	AgentConfigPath string // opencode config (/sandbox-runtime/agent-config.json)
	SecretsEnvPath  string // env-file (/sandbox-runtime/secrets-env)
	GitCredsPath    string // git-credentials file (/sandbox-runtime/rt/git-credentials)
	// StagingDir is the R2b staging tree: canonical bytes + manifest for
	// file-class credentials, published by the materializer and served by
	// the pull endpoint. The consumed paths above are written ONLY by the
	// uid-1000 supervisor (#1165: ownership by construction).
	StagingDir string // default /sandbox-runtime/staged-secret-files
}

Paths configures filesystem destinations. Defaults match agentd constants; tests override.

func DefaultPaths 

func DefaultPaths(home string) Paths

DefaultPaths returns production paths derived from the agentd package constants and the given home dir.

US-35.7: SSH/git/secrets paths point to /sandbox-runtime/rt/* (tmpfs) to match loadMaterializeConfig() in cmd/workspace-agentd. The $HOME-relative PVC paths are symlinks (created by init container) pointing here.

type Secret 

type Secret struct {
	Type      string            `json:"type"`
	Name      string            `json:"name"`
	Metadata  map[string]string `json:"metadata"`
	Plaintext string            `json:"plaintext"`

	// MetadataInvalid records why the metadata could not be normalized
	// (metadata that is not a JSON object). Materialize reports it
	// per-entry as a Skipped result instead of aborting the whole batch.
	//
	// Wire persistence: round-tripped through the reserved
	// "metadata_invalid" JSON key (MarshalJSON emits it; UnmarshalJSON
	// reads it) so a replayed batch preserves the skip verdict
	// identically — without it the cached entry marshals with
	// "metadata":null, the flag is lost, and a restart materializes the
	// rejected secret with defaults (T5 violation; review N1 on #871).
	MetadataInvalid string `json:"-"`
}

Secret is the materialization-time representation of a credential. Metadata is intentionally kept as a typed map to avoid leaking arbitrary JSON shape into the materializer; unknown keys are ignored.

WIRE COMPATIBILITY (2026-08-15 incident): the injection pipeline writes mcp-server metadata per MATERIALIZE-CONTRACT.md with native JSON types ("args": [...], "timeoutMs": 5000), while this struct historically required map[string]string — one bound MCP server crash-looped every workspace boot ("cannot unmarshal array into ... Secret.metadata"). UnmarshalJSON accepts BOTH shapes: string values pass through; numbers/booleans/arrays/objects are carried JSON-encoded as strings (exactly what the mcp staging branch's json.Unmarshal(argsStr) expects).

func LoadSecretsFile 

func LoadSecretsFile(path string) ([]Secret, error)

LoadSecretsFile reads and parses a legacy secrets.json file (bare array). The materialize path uses LoadBatchFile, which additionally accepts the revisioned envelope; this accessor keeps the []Secret surface the legacy readers (reload-cache replay fixtures) use.

func (Secret) MarshalJSON added in v0.15.7 

func (s Secret) MarshalJSON() ([]byte, error)

MarshalJSON emits the struct fields plus the reserved "metadata_invalid" verdict key when set, so cache round-trips preserve the skip decision. The key is namespaced ("metadata_invalid") so it cannot collide with a legitimate metadata member key; UnmarshalJSON reads the same tag.

func (*Secret) UnmarshalJSON added in v0.15.7 

func (s *Secret) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the dual-shape metadata contract documented on Secret. It never fails on metadata content — malformed metadata sets MetadataInvalid so the entry can be skipped downstream with an accurate reason rather than killing the file parse for every other secret. The reserved metadata_invalid key (emitted by MarshalJSON) restores a persisted verdict so a replayed cache entry skips identically to its first-boot parse.

type SecretResult 

type SecretResult struct {
	Type    string  `json:"type"`
	Name    string  `json:"name"`
	Outcome Outcome `json:"outcome"`
	Reason  string  `json:"reason,omitempty"`
}

SecretResult is the per-secret outcome reported by Materialize. Reason is human-readable but MUST NOT include the secret's plaintext.

type StagedEntry added in v0.26.0 

type StagedEntry struct {
	Target string `json:"target"`
	Mode   int    `json:"mode"`
	File   string `json:"file"`
}

StagedEntry is one manifest row: the delivered target (absolute, confined to the delivery roots), the final mode per the consumer contract, and the staged byte file (relative to the staging dir).

func ReadStagingManifest added in v0.26.0 

func ReadStagingManifest(data []byte) ([]StagedEntry, string, error)

ReadStagingManifest parses manifest bytes of EITHER shape into rows plus the revision anchor ("" for the legacy array).

type StagedMCPServer added in v0.7.0 

type StagedMCPServer struct {
	Name      string
	Transport string
	URL       string
	Command   string
	Args      []string
	TimeoutMs int
	Env       map[string]string
	Headers   map[string]string
}

StagedMCPServer is one materialized MCP server awaiting config rendering.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.