Documentation
¶
Overview ¶
Package vertexcache owns the lifecycle of a single Vertex explicit context cache — Create at agent startup, Refresh on TTL pressure, Delete on session unregister. Callers read the resolved cache name via Name() to plumb it onto GenerateContentConfig.CachedContent per turn.
See docs/vertex-context-caching-design.md for the v1 scope (system-instruction + tools; not conversation history) and the failure-mode contract (every error path degrades to "no cache for this call," never breaks the session).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CachesClient ¶
type CachesClient interface {
Create(ctx context.Context, model string, config *genai.CreateCachedContentConfig) (*genai.CachedContent, error)
Update(ctx context.Context, name string, config *genai.UpdateCachedContentConfig) (*genai.CachedContent, error)
Delete(ctx context.Context, name string, config *genai.DeleteCachedContentConfig) (*genai.DeleteCachedContentResponse, error)
}
CachesClient is the subset of *genai.Client's Caches field that the manager depends on. Wrapping it in an interface keeps the manager unit-testable without a real Vertex client — the fake in manager_test.go implements the same shape.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager owns one cache handle. Init creates it lazily (called from pkg/models/gemini on the first GenerateContent that carries a system instruction we can cache); Name returns the resolved name or "" if not yet ready; Refresh extends TTL when it's about to expire; Delete cleans up on shutdown.
Manager is safe for concurrent use. Name() is called from every turn's GenerateContent — the read path is a plain RLock, cheap.
func NewManager ¶
func NewManager(caches CachesClient, model string, opts Options) *Manager
NewManager returns a Manager bound to caches + model. model is the same modelID passed to Caches.Create — must match the model the referencing GenerateContent call uses (Vertex enforces this).
func (*Manager) Delete ¶
Delete releases the Vertex cache. Best-effort — failures are logged but don't propagate. Callers should invoke this on session unregister / daemon shutdown; skipping the call is fine (Vertex reaps expired caches) but leaves the cache resource around until TTL elapses.
Idempotent: safe to call multiple times, and safe to call before Init has landed (skips the RPC if there's no cache to delete).
func (*Manager) Init ¶
Init kicks off cache creation with the given systemInstruction + tools. Non-blocking: returns immediately after a state check; the actual Create RPC runs on a background goroutine so the caller (typically the first GenerateContent call) doesn't wait.
Called at-most-once — subsequent calls with the manager already initializing / active are no-ops. This is by design: once we've seeded the cache from turn N's config, turn N+1's config should be identical (both come from the same agent's system instruction + tools slice), so re-Init would be wasted RPCs.
func (*Manager) MarkEvicted ¶
MarkEvicted resets the manager to its pre-Init state after learning (via a GenerateContent NOT_FOUND on the cache reference) that Vertex has reaped our cache server-side. Callers who see the eviction — the pkg/models/gemini wrapper's retry-once path — invoke this so:
- Subsequent Name() calls return "" (the request goes uncached).
- The next non-cached turn can call Init again and get a fresh cache handle instead of the agent running uncached for the rest of its lifetime.
Distinct from stateFailed (which stays sticky — a Create failure at Init time signals a persistent problem worth surfacing). Eviction is a normal end-of-TTL event, especially on long-lived daemons whose cache outlives a single session.
Idempotent + safe against races with in-flight Refresh: a refresh that lands after MarkEvicted just sees state != active and no-ops.
func (*Manager) Name ¶
Name returns the resolved cache name if the manager is active, or "" if init hasn't landed yet / failed / been deleted. Callers wire the return value into GenerateContentConfig.CachedContent — an empty value means the request runs uncached, which is always safe.
Name also opportunistically triggers a background Refresh if the active cache is inside its refresh window. The refresh runs asynchronously; the current call still returns the (about-to-expire but still-valid) cache name.
type Options ¶
type Options struct {
// TTL is how long each Create/Update requests the cache live for.
// Vertex caps at 24h; the daemon's default matches the session
// idle timeout (6h). Zero → 6h.
TTL time.Duration
// RefreshThreshold triggers a background Refresh call when Name()
// is read and time-to-expiry drops below this value. Zero → 30min.
RefreshThreshold time.Duration
// DisplayName annotates the cache in Vertex's list view. Optional
// — omitted from Create when empty.
DisplayName string
// Logger receives structured operator-facing warnings when a
// Caches RPC fails. Zero → log.Default() ("core-agent-vertexcache:"
// prefix included).
Logger *log.Logger
}
Options configures a Manager. Defaults come from docs/vertex-context-caching-design.md — TTL 6h, Refresh threshold 30min. Zero values pick the default; callers that want the default for both fields can pass Options{}.
type Status ¶
type Status struct {
Active bool
Failed bool
CacheName string
ExpiresAt time.Time
ExpiresIn time.Duration
}
Status is a snapshot of the manager's state for diagnostic use (e.g. an /internal/cache-status endpoint or a startup log line once init settles). Read-only; each call takes a fresh RLock.
Source Files