Documentation
¶
Overview ¶
Package contentblob stores the repeated parts of a JSON document once, by content, and replaces them in the document with short references.
A turn's durable state repeats itself: every saved step carries the system prompt, the tool descriptions and the whole message history so far. Kept inline, that state grows with the square of the turn's rounds. Compacted, each long string and each run of array elements is stored once, and every later copy is a 64-character hash, so the state grows only by what is new.
Array runs are cut where an element's own hash says so, never by position, so an array that only gained elements at its end yields the same runs as before plus a new one. That is what keeps a growing message history cheap.
Expand(Compact(x)) returns x byte for byte. Callers compare stored JSON with freshly encoded JSON, so a codec that reordered keys or re-escaped text would break them. A reference is replaced by the exact bytes it names, and those bytes may hold references of their own.
Index ¶
Constants ¶
const ( // DefaultCacheBytes bounds the blob bytes one cache keeps in memory. A // workflow replays its whole history on every task, and a store loads the // same contexts again and again, so the same blobs are read repeatedly. DefaultCacheBytes = 64 << 20 )
const ( // MinBlobBytes is the shortest string literal or array run, in encoded // bytes, that is moved out of a document. A reference costs about 90 // bytes, so shorter parts stay inline. MinBlobBytes = 512 )
Variables ¶
This section is empty.
Functions ¶
func Expand ¶
Expand restores a document Compact rewrote, fetching every referenced blob, layer by layer. A document Compact left alone is returned unchanged. A missing blob is an error. fetch must return only blobs whose bytes match their hash; CachedFetch checks each blob once, when it is first loaded.
func IsCompacted ¶
IsCompacted reports whether raw is a document Compact rewrote.
func References ¶
References returns the distinct hashes raw refers to directly, in first-use order. The blobs they name may refer to more.
Types ¶
type Blob ¶
Blob is one stored part of a document, the exact bytes a reference stands for, and the hex SHA-256 of those bytes. The bytes are either one value or a comma-separated run of array elements.
type Cache ¶
type Cache struct {
// contains filtered or unexported fields
}
Cache keeps recently used blobs up to a total byte budget, evicting the least recently used first. A blob's bytes never change for its hash, so a cached blob is always right. It is safe for concurrent use.
func (*Cache) Add ¶
Add keeps data unless it alone exceeds the budget. Callers add only blobs they verified against their hash.
type Converter ¶
type Converter struct {
// contains filtered or unexported fields
}
Converter is a go-workflows payload converter that stores repeated parts once. Workflow inputs and results repeat the turn context in every event, and a compacted payload holds each repeat as a hash.
func NewConverter ¶
NewConverter returns a converter backed by store.
type FetchFunc ¶
FetchFunc returns the stored data for each requested hash. A hash it cannot find is left out of the map.
func CachedFetch ¶
CachedFetch returns a FetchFunc that answers from cache and reads the rest through load, keeping only blobs whose bytes match their hash.