Documentation
¶
Overview ¶
Package vertexcache owns the lifecycle of a single Vertex explicit context cache — Create at agent startup, Refresh on TTL pressure, Delete on session unregister. Callers read the resolved cache name via Name() to plumb it onto GenerateContentConfig.CachedContent per turn.
See core-agent's docs/vertex-context-caching-design.md (https://github.com/go-steer/core-agent/blob/main/docs/vertex-context-caching-design.md) for the v1 scope (system-instruction + tools; not conversation history) and the failure-mode contract (every error path degrades to "no cache for this call," never breaks the session).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CachesClient ¶
type CachesClient interface {
Create(ctx context.Context, model string, config *genai.CreateCachedContentConfig) (*genai.CachedContent, error)
Update(ctx context.Context, name string, config *genai.UpdateCachedContentConfig) (*genai.CachedContent, error)
Delete(ctx context.Context, name string, config *genai.DeleteCachedContentConfig) (*genai.DeleteCachedContentResponse, error)
}
CachesClient is the subset of *genai.Client's Caches field that the manager depends on. Wrapping it in an interface keeps the manager unit-testable without a real Vertex client — the fake in manager_test.go implements the same shape.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager owns one cache handle. Init creates it lazily (called from pkg/providers/gemini on the first GenerateContent that carries a system instruction we can cache); Name returns the resolved name or "" if not yet ready; Refresh extends TTL when it's about to expire; Delete cleans up on shutdown.
Manager is safe for concurrent use. Name() is called from every turn's GenerateContent — the read path is a plain RLock, cheap.
func NewManager ¶
func NewManager(caches CachesClient, model string, opts Options) *Manager
NewManager returns a Manager bound to caches + model. model is the same modelID passed to Caches.Create — must match the model the referencing GenerateContent call uses (Vertex enforces this).
func (*Manager) Delete ¶
Delete releases the Vertex cache. Best-effort — failures are logged but don't propagate. Callers should invoke this on session unregister / daemon shutdown; skipping the call is fine (Vertex reaps expired caches) but leaves the cache resource around until TTL elapses.
Idempotent: safe to call multiple times, and safe to call before Init has landed (skips the RPC if there's no cache to delete).
func (*Manager) Init ¶
Init kicks off cache creation with the given systemInstruction + tools. Non-blocking: returns immediately after a state check; the actual Create RPC runs on a background goroutine so the caller (typically the first GenerateContent call) doesn't wait.
Called at-most-once per outcome — subsequent calls with the manager already initializing / active are no-ops, as are calls made while a failed attempt serves its retry backoff. This is by design: once we've seeded the cache from turn N's config, turn N+1's config should be identical (both come from the same agent's system instruction + tools slice), so re-Init would be wasted RPCs.
func (*Manager) MarkEvicted ¶
MarkEvicted resets the manager to its pre-Init state after learning (via a GenerateContent NOT_FOUND on the cache reference) that Vertex has reaped our cache server-side. Callers who see the eviction — the pkg/providers/gemini wrapper's retry-once path — invoke this so:
- Subsequent Name() calls return "" (the request goes uncached).
- The next non-cached turn can call Init again and get a fresh cache handle instead of the agent running uncached for the rest of its lifetime.
Distinct from stateFailed, which is only reached after the whole retry budget is spent and then stays sticky — a Create that keeps failing signals a persistent problem worth surfacing. Eviction is a normal end-of-TTL event, especially on long-lived daemons whose cache outlives a single session.
Idempotent + safe against races with in-flight Refresh: a refresh that lands after MarkEvicted just sees state != active and no-ops.
func (*Manager) Name ¶
Name returns the resolved cache name if the manager is active, or "" if init hasn't landed yet / failed / been deleted. Callers wire the return value into GenerateContentConfig.CachedContent — an empty value means the request runs uncached, which is always safe.
Name also opportunistically triggers a background Refresh if the active cache is inside its refresh window. The refresh runs asynchronously; the current call still returns the (about-to-expire but still-valid) cache name.
type Options ¶
type Options struct {
// TTL is how long each Create/Update requests the cache live for.
// Vertex caps at 24h; the daemon's default matches the session
// idle timeout (6h). Zero → 6h.
TTL time.Duration
// RefreshThreshold triggers a background Refresh call when Name()
// is read and time-to-expiry drops below this value. Zero → 30min.
RefreshThreshold time.Duration
// DisplayName annotates the cache in Vertex's list view. Optional
// — omitted from Create when empty.
DisplayName string
// Logger receives structured operator-facing warnings when a
// Caches RPC fails. Zero → log.Default() ("mast-vertexcache:"
// prefix included).
Logger *log.Logger
}
Options configures a Manager. Defaults come from the context-caching design doc — TTL 6h, Refresh threshold 30min. Zero values pick the default; callers that want the default for both fields can pass Options{}.
type Status ¶
type Status struct {
Active bool
Failed bool
CacheName string
ExpiresAt time.Time
ExpiresIn time.Duration
}
Status is a snapshot of the manager's state for diagnostic use (e.g. an /internal/cache-status endpoint or a startup log line once init settles). Read-only; each call takes a fresh RLock.
Source Files