Documentation
¶
Overview ¶
Package jobenvelope defines the single job-envelope schema used by every stage of the TMI Component Platform extraction pipeline. One shape — no discriminator — flows from the monolith through tmi-extractor and tmi-chunk-embed back to the monolith's result-consumer. Input mode is declared per-component (content-ref vs source-locator); the monolith populates the matching Input fields. source-locator fields are RESERVED for the future code extractor and are not exercised by issue #347.
Index ¶
Constants ¶
const ( // MaxReasonCodeLen mirrors the documents.access_reason_code column // width. Enforced at the boundary so an oversize code is rejected up // front instead of erroring inside the database write and redelivering. MaxReasonCodeLen = 64 // MaxReasonDetailLen caps the free-text reason_detail a worker may // attach. SanitizeResult truncates rather than rejects, so an over-long // detail cannot wedge a result delivery into a retry loop. MaxReasonDetailLen = 4096 // MaxResultRefLen bounds output.result_ref. Real refs are short // "<bucket>/<name>" Object Store locators. MaxResultRefLen = 1024 )
Field bounds for the return-path (Result) envelope. Results are published by workers over NATS and cross a trust boundary back into the monolith, which persists the reason fields and later serves them to authenticated clients — so the bounds are enforced here at consume time, not only at the database schema level.
Variables ¶
This section is empty.
Functions ¶
func Validate ¶
Validate checks a Job against the envelope contract. It rejects a missing job_id, a missing content_type, an empty Input, and an Input that sets both the content-ref and source-locator modes at once. It does NOT reject source-locator alone — that mode is RESERVED but schema-valid. SEM@29ce3cc7a3143d57faa07f7e0071ce27b53226e0: validate a job envelope for required fields and mutually exclusive input locators (pure)
func ValidateResult ¶
ValidateResult checks a Result against the envelope contract: job_id present, status one of the known terminal states, reason_code within length and charset bounds, and output.result_ref bounded. It does NOT check reason_detail — free text is normalized by SanitizeResult instead, so a verbose-but-honest worker is truncated rather than dropped. Callers on the consume side should Term (not Nak) on error: an invalid envelope never becomes valid on redelivery. SEM@d056a3ea026249d40d05ab6af7f092a043f72c7a: validate a job result envelope for required fields, status, reason code format, and ref length (pure)
Types ¶
type Duration ¶
Duration is a time.Duration that marshals to and from a JSON string in Go duration syntax (e.g. "60s", "1m30s") rather than a raw nanosecond integer, so the job envelope stays human- and cross-language-readable. SEM@49694f3eccb0f470f5f0308ef283dce1a077a92f: convert time.Duration to/from a human-readable JSON duration string (pure)
func (Duration) MarshalJSON ¶
MarshalJSON renders the duration as a quoted Go duration string. SEM@49694f3eccb0f470f5f0308ef283dce1a077a92f: serialize a Duration as a quoted Go duration string (pure)
func (Duration) Std ¶
Std returns the wrapped value as a standard time.Duration. SEM@49694f3eccb0f470f5f0308ef283dce1a077a92f: convert the custom Duration to a standard time.Duration (pure)
func (*Duration) UnmarshalJSON ¶
UnmarshalJSON parses a quoted Go duration string. It also accepts a bare JSON number, interpreted as nanoseconds, for forward compatibility. SEM@49694f3eccb0f470f5f0308ef283dce1a077a92f: parse a Duration from a quoted Go duration string or raw nanosecond integer (pure)
type Input ¶
type Input struct {
// ObjectRef is the Object Store locator for content-ref mode.
ObjectRef string `json:"object_ref,omitempty"`
// ByteSize is the source payload size in bytes (content-ref mode).
ByteSize int64 `json:"byte_size,omitempty"`
// SourceURL is the fetch target for source-locator mode. RESERVED.
SourceURL string `json:"source_url,omitempty"`
// SourceSecretRef names a secret for authenticated fetch. RESERVED.
SourceSecretRef string `json:"source_secret_ref,omitempty"`
// FetchLimits bounds a source-locator fetch. RESERVED.
FetchLimits *Limits `json:"fetch_limits,omitempty"`
}
Input carries job input. Exactly one mode is populated:
- content-ref: ObjectRef + ByteSize are set; the bytes are already in the JetStream Object Store.
- source-locator: SourceURL (+ optional SourceSecretRef, FetchLimits) is set. RESERVED — not exercised by issue #347.
SEM@fd1c8398b240fc2d88814019a767c2cebb498d27: job input descriptor referencing a payload by object-store ref or source URL (pure)
type Job ¶
type Job struct {
// JobID is the idempotency key — stable across the whole pipeline.
JobID string `json:"job_id"`
// ContentType is the source MIME type, used to route to an extractor.
ContentType string `json:"content_type"`
// Limits are the per-job processing bounds.
Limits Limits `json:"limits"`
// Deadline is the absolute wall-clock deadline for the whole job.
// Nil means no deadline. Producers that enforce a deadline set a
// non-nil, future timestamp.
Deadline *time.Time `json:"deadline,omitempty"`
// Input carries the stage input (by reference).
Input Input `json:"input"`
// Metadata carries small key/value pairs forwarded between stages
// (e.g. document title). Never large payloads.
Metadata map[string]string `json:"metadata,omitempty"`
}
Job is the forward-path envelope a stage consumes. SEM@49694f3eccb0f470f5f0308ef283dce1a077a92f: forward-path pipeline envelope carrying job identity, content type, limits, and input ref (pure)
type Limits ¶
type Limits struct {
// MaxBytes caps the source payload size the worker will read.
MaxBytes int64 `json:"max_bytes"`
// WallClock is the per-job wall-clock budget for the parse/embed stage.
// Serialized as a Go duration string (e.g. "60s").
WallClock Duration `json:"wall_clock"`
}
Limits are the per-job processing bounds carried in the envelope. They mirror the in-code extractor caps so a worker can enforce the same wall even when the controller's cgroup cap differs. SEM@49694f3eccb0f470f5f0308ef283dce1a077a92f: per-job processing bounds for payload size and wall-clock budget (pure)
type Output ¶
type Output struct {
// ResultRef is the Object Store locator for the stage output blob.
ResultRef string `json:"result_ref,omitempty"`
}
Output carries job output by reference. SEM@fd1c8398b240fc2d88814019a767c2cebb498d27: job output descriptor referencing the result blob by object-store ref (pure)
type Result ¶
type Result struct {
// JobID matches the originating Job.
JobID string `json:"job_id"`
// Status is the terminal state.
Status Status `json:"status"`
// ReasonCode is the typed reason on failure (a pkg/extract Reason*
// constant) or empty on success.
ReasonCode string `json:"reason_code,omitempty"`
// ReasonDetail is optional human-readable context (e.g. "slide #42").
ReasonDetail string `json:"reason_detail,omitempty"`
// Output carries the result blob reference on success.
Output Output `json:"output,omitempty"`
}
Result is the return-path envelope published to jobs.result.<job_id>. SEM@fd1c8398b240fc2d88814019a767c2cebb498d27: return-path pipeline envelope reporting terminal status, reason, and output ref (pure)
func SanitizeResult ¶
SanitizeResult returns r with its free-text fields normalized: reason_detail is coerced to valid UTF-8 (so the database write cannot fail on encoding and redeliver), stripped of control characters other than newline and tab (so terminal escape sequences cannot ride a stored detail into a log viewer or client), and truncated to MaxReasonDetailLen bytes on a rune boundary. Call it after ValidateResult on the consume side; truncation instead of rejection keeps an oversize detail from turning into a redelivery loop. SEM@d056a3ea026249d40d05ab6af7f092a043f72c7a: sanitize the free-text reason detail of a job result and return the cleaned copy (pure)
Source Files