vertexcache

package
v0.15.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 14, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package vertexcache owns the lifecycle of a single Vertex explicit context cache — Create at agent startup, Refresh on TTL pressure, Delete on session unregister. Callers read the resolved cache name via Name() to plumb it onto GenerateContentConfig.CachedContent per turn.

See core-agent's docs/vertex-context-caching-design.md (https://github.com/go-steer/core-agent/blob/main/docs/vertex-context-caching-design.md) for the v1 scope (system-instruction + tools; not conversation history) and the failure-mode contract (every error path degrades to "no cache for this call," never breaks the session).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Gone

func Gone(err error) bool

Gone reports whether err is Vertex saying the explicit cache a handle names is no longer there — reaped or expired — as opposed to an RPC merely failing. A true answer means no later call naming the same cache can succeed: drop the handle and create a new one.

One verdict for two callers. The manager sees a dead cache on Caches.Update and the gemini adapter sees it on a GenerateContent that stamped the reference, and Vertex describes the one dead cache differently on each. When the two carried their own predicates, the manager learned the cache was gone and kept handing the name out while the adapter failed to recognise what it was given — mast #325 and core-agent #902 are that bug, found independently. The observed shapes, all for one reaped cache:

Caches.Update    404 NOT_FOUND        Cached content <id> is not found.
GenerateContent  400 INVALID_ARGUMENT Cache content <id> is expired.
GenerateContent  404 NOT_FOUND        Not found: cached content metadata for <id>.

Note "Cache content" without the d in the middle one. The genai SDK exposes no typed error for any of them, so this matches text, and the noun phrase is the discriminator: a plain NOT_FOUND (missing model, wrong region) and a plain 400 (a malformed request, which is what a bare INVALID_ARGUMENT usually is) are left alone. The status is not part of the match, as mast's verdict decided: it is the part that differs between the shapes, while the phrase names the cause in all of them.

func IsBelowCacheMinimum

func IsBelowCacheMinimum(err error) bool

IsBelowCacheMinimum reports whether err says the content Vertex was asked to cache is smaller than the model's minimum cacheable size:

Error 400, Message: The cached content is of 2373 tokens. The minimum
token count to start explicit caching is 4096., Status: INVALID_ARGUMENT

That is the one Create failure time cannot fix — the cached content is the agent's system instruction and tools, fixed for the process's life — so the manager stops instead of running its retry schedule (core-agent #1067). It matches the reason, never the number, which is per-model and has moved before; pairing it with INVALID_ARGUMENT keeps an error that merely mentions token counts from matching.

func IsCachingUnsupported added in v0.13.4

func IsCachingUnsupported(err error) bool

IsCachingUnsupported reports whether err says the model cannot hold an explicit context cache at all:

Error 400, Message: Model `gemini-4-argon` does not support explicit
context caching., Status: INVALID_ARGUMENT

Like IsBelowCacheMinimum it is a Create failure time cannot fix — the model is fixed for the manager's life — so the manager stops instead of running its retry schedule. Such a model may still discount a repeated prefix through implicit caching, which needs no handle and reports its hits as cached tokens on the response. It matches the reason, never the model name, and pairs it with INVALID_ARGUMENT for the same reason IsBelowCacheMinimum does.

Types

type CachesClient

CachesClient is the subset of *genai.Client's Caches field that the manager depends on. Wrapping it in an interface keeps the manager unit-testable without a real Vertex client — the fake in manager_test.go implements the same shape.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager owns one cache handle. Init creates it lazily (called from the gemini adapter on the first GenerateContent that carries a system instruction we can cache); Name returns the resolved name or "" if not yet ready; Refresh extends TTL when it's about to expire; Delete cleans up on shutdown.

Manager is safe for concurrent use. Name() is called from every turn's GenerateContent — the read path is a plain RLock, cheap.

func NewManager

func NewManager(caches CachesClient, model string, opts Options) *Manager

NewManager returns a Manager bound to caches + model. model is the same modelID passed to Caches.Create — must match the model the referencing GenerateContent call uses (Vertex enforces this).

func (*Manager) Delete

func (m *Manager) Delete(ctx context.Context)

Delete releases the Vertex cache. Best-effort — failures are logged but don't propagate. Callers should invoke this on session unregister / daemon shutdown; skipping the call is fine (Vertex reaps expired caches) but leaves the cache resource around until TTL elapses.

Idempotent: safe to call multiple times, and safe to call before Init has landed (skips the RPC if there's no cache to delete).

func (*Manager) Init

func (m *Manager) Init(ctx context.Context, systemInstruction *genai.Content, tools []*genai.Tool)

Init kicks off cache creation with the given systemInstruction + tools. Non-blocking: returns immediately after a state check; the actual Create RPC runs on a background goroutine so the caller (typically the first GenerateContent call) doesn't wait.

Called at-most-once per outcome — subsequent calls with the manager already initializing / active are no-ops, as are calls made while a failed attempt serves its retry backoff. This is by design: once we've seeded the cache from turn N's config, turn N+1's config should be identical (both come from the same agent's system instruction + tools slice), so re-Init would be wasted RPCs.

func (*Manager) MarkEvicted

func (m *Manager) MarkEvicted(reason string)

MarkEvicted resets the manager to its pre-Init state after learning (via a GenerateContent NOT_FOUND on the cache reference) that Vertex has reaped our cache server-side. Callers who see the eviction — the gemini adapter's retry-once path — invoke this so:

  1. Subsequent Name() calls return "" (the request goes uncached).
  2. The next non-cached turn can call Init again and get a fresh cache handle instead of the agent running uncached for the rest of its lifetime.

Distinct from stateFailed, which is only reached after the whole retry budget is spent and then stays sticky — a Create that keeps failing signals a persistent problem worth surfacing. Eviction is a normal end-of-TTL event, especially on long-lived daemons whose cache outlives a single session.

Idempotent + safe against races with in-flight Refresh: a refresh that lands after MarkEvicted just sees state != active and no-ops.

func (*Manager) MarkEvictedName

func (m *Manager) MarkEvictedName(name, reason string)

MarkEvictedName is MarkEvicted for one named cache: it resets only while name is still the active cache. The gemini adapter calls this form, because between stamping a request and hearing that its cache is gone a fresh cache may have replaced it, and a verdict about the old one must not discard the new one.

func (*Manager) Model

func (m *Manager) Model() string

Model is the model this manager's cache is for. A cache serves only the model it was created for.

func (*Manager) Name

func (m *Manager) Name(ctx context.Context) string

Name returns the resolved cache name if the manager is active, or "" if init hasn't landed yet / failed / been deleted. Callers wire the return value into GenerateContentConfig.CachedContent — an empty value means the request runs uncached, which is always safe.

Name also opportunistically triggers a background Refresh if the active cache is inside its refresh window. The refresh runs asynchronously; the current call still returns the (about-to-expire but still-valid) cache name.

func (*Manager) Snapshot

func (m *Manager) Snapshot() Status

Snapshot returns the current manager state. Cheap — RLock-only.

type Options

type Options struct {
	// TTL is how long each Create/Update requests the cache live for.
	// Vertex caps at 24h; the daemon's default matches the session
	// idle timeout (6h). Zero → 6h.
	TTL time.Duration
	// RefreshThreshold triggers a background Refresh call when Name()
	// is read and time-to-expiry drops below this value. Zero → 30min.
	RefreshThreshold time.Duration
	// DisplayName annotates the cache in Vertex's list view. Optional
	// — omitted from Create when empty.
	DisplayName string
	// Logger receives structured operator-facing warnings when a
	// Caches RPC fails. Zero → log.Default(); every line starts
	// "vertexcache:".
	Logger *log.Logger
}

Options configures a Manager. Defaults come from the context-caching design doc — TTL 6h, Refresh threshold 30min. Zero values pick the default; callers that want the default for both fields can pass Options{}.

type Status

type Status struct {
	Active    bool
	Failed    bool
	CacheName string
	ExpiresAt time.Time
	ExpiresIn time.Duration
}

Status is a snapshot of the manager's state for diagnostic use (e.g. an /internal/cache-status endpoint or a startup log line once init settles). Read-only; each call takes a fresh RLock.

func (Status) String

func (s Status) String() string

String is a short human-readable form used by the startup log line ("context cache: active (cache_id=..., ttl=5h59m)").

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL