Documentation
¶
Overview ¶
Package vertexcache owns the lifecycle of a single Vertex explicit context cache — Create at agent startup, Refresh on TTL pressure, Delete on session unregister. Callers read the resolved cache name via Name() to plumb it onto GenerateContentConfig.CachedContent per turn.
See core-agent's docs/vertex-context-caching-design.md (https://github.com/go-steer/core-agent/blob/main/docs/vertex-context-caching-design.md) for the v1 scope (system-instruction + tools; not conversation history) and the failure-mode contract (every error path degrades to "no cache for this call," never breaks the session).
Index ¶
- func Gone(err error) bool
- func IsBelowCacheMinimum(err error) bool
- func IsCachingUnsupported(err error) bool
- type CachesClient
- type Manager
- func (m *Manager) Delete(ctx context.Context)
- func (m *Manager) Init(ctx context.Context, systemInstruction *genai.Content, tools []*genai.Tool)
- func (m *Manager) MarkEvicted(reason string)
- func (m *Manager) MarkEvictedName(name, reason string)
- func (m *Manager) Model() string
- func (m *Manager) Name(ctx context.Context) string
- func (m *Manager) Snapshot() Status
- type Options
- type Status
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Gone ¶
Gone reports whether err is Vertex saying the explicit cache a handle names is no longer there — reaped or expired — as opposed to an RPC merely failing. A true answer means no later call naming the same cache can succeed: drop the handle and create a new one.
One verdict for two callers. The manager sees a dead cache on Caches.Update and the gemini adapter sees it on a GenerateContent that stamped the reference, and Vertex describes the one dead cache differently on each. When the two carried their own predicates, the manager learned the cache was gone and kept handing the name out while the adapter failed to recognise what it was given — mast #325 and core-agent #902 are that bug, found independently. The observed shapes, all for one reaped cache:
Caches.Update 404 NOT_FOUND Cached content <id> is not found. GenerateContent 400 INVALID_ARGUMENT Cache content <id> is expired. GenerateContent 404 NOT_FOUND Not found: cached content metadata for <id>.
Note "Cache content" without the d in the middle one. The genai SDK exposes no typed error for any of them, so this matches text, and the noun phrase is the discriminator: a plain NOT_FOUND (missing model, wrong region) and a plain 400 (a malformed request, which is what a bare INVALID_ARGUMENT usually is) are left alone. The status is not part of the match, as mast's verdict decided: it is the part that differs between the shapes, while the phrase names the cause in all of them.
func IsBelowCacheMinimum ¶
IsBelowCacheMinimum reports whether err says the content Vertex was asked to cache is smaller than the model's minimum cacheable size:
Error 400, Message: The cached content is of 2373 tokens. The minimum token count to start explicit caching is 4096., Status: INVALID_ARGUMENT
That is the one Create failure time cannot fix — the cached content is the agent's system instruction and tools, fixed for the process's life — so the manager stops instead of running its retry schedule (core-agent #1067). It matches the reason, never the number, which is per-model and has moved before; pairing it with INVALID_ARGUMENT keeps an error that merely mentions token counts from matching.
func IsCachingUnsupported ¶ added in v0.13.4
IsCachingUnsupported reports whether err says the model cannot hold an explicit context cache at all:
Error 400, Message: Model `gemini-4-argon` does not support explicit context caching., Status: INVALID_ARGUMENT
Like IsBelowCacheMinimum it is a Create failure time cannot fix — the model is fixed for the manager's life — so the manager stops instead of running its retry schedule. Such a model may still discount a repeated prefix through implicit caching, which needs no handle and reports its hits as cached tokens on the response. It matches the reason, never the model name, and pairs it with INVALID_ARGUMENT for the same reason IsBelowCacheMinimum does.
Types ¶
type CachesClient ¶
type CachesClient interface {
Create(ctx context.Context, model string, config *genai.CreateCachedContentConfig) (*genai.CachedContent, error)
Update(ctx context.Context, name string, config *genai.UpdateCachedContentConfig) (*genai.CachedContent, error)
Delete(ctx context.Context, name string, config *genai.DeleteCachedContentConfig) (*genai.DeleteCachedContentResponse, error)
}
CachesClient is the subset of *genai.Client's Caches field that the manager depends on. Wrapping it in an interface keeps the manager unit-testable without a real Vertex client — the fake in manager_test.go implements the same shape.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager owns one cache handle. Init creates it lazily (called from the gemini adapter on the first GenerateContent that carries a system instruction we can cache); Name returns the resolved name or "" if not yet ready; Refresh extends TTL when it's about to expire; Delete cleans up on shutdown.
Manager is safe for concurrent use. Name() is called from every turn's GenerateContent — the read path is a plain RLock, cheap.
func NewManager ¶
func NewManager(caches CachesClient, model string, opts Options) *Manager
NewManager returns a Manager bound to caches + model. model is the same modelID passed to Caches.Create — must match the model the referencing GenerateContent call uses (Vertex enforces this).
func (*Manager) Delete ¶
Delete releases the Vertex cache. Best-effort — failures are logged but don't propagate. Callers should invoke this on session unregister / daemon shutdown; skipping the call is fine (Vertex reaps expired caches) but leaves the cache resource around until TTL elapses.
Idempotent: safe to call multiple times, and safe to call before Init has landed (skips the RPC if there's no cache to delete).
func (*Manager) Init ¶
Init kicks off cache creation with the given systemInstruction + tools. Non-blocking: returns immediately after a state check; the actual Create RPC runs on a background goroutine so the caller (typically the first GenerateContent call) doesn't wait.
Called at-most-once per outcome — subsequent calls with the manager already initializing / active are no-ops, as are calls made while a failed attempt serves its retry backoff. This is by design: once we've seeded the cache from turn N's config, turn N+1's config should be identical (both come from the same agent's system instruction + tools slice), so re-Init would be wasted RPCs.
func (*Manager) MarkEvicted ¶
MarkEvicted resets the manager to its pre-Init state after learning (via a GenerateContent NOT_FOUND on the cache reference) that Vertex has reaped our cache server-side. Callers who see the eviction — the gemini adapter's retry-once path — invoke this so:
- Subsequent Name() calls return "" (the request goes uncached).
- The next non-cached turn can call Init again and get a fresh cache handle instead of the agent running uncached for the rest of its lifetime.
Distinct from stateFailed, which is only reached after the whole retry budget is spent and then stays sticky — a Create that keeps failing signals a persistent problem worth surfacing. Eviction is a normal end-of-TTL event, especially on long-lived daemons whose cache outlives a single session.
Idempotent + safe against races with in-flight Refresh: a refresh that lands after MarkEvicted just sees state != active and no-ops.
func (*Manager) MarkEvictedName ¶
MarkEvictedName is MarkEvicted for one named cache: it resets only while name is still the active cache. The gemini adapter calls this form, because between stamping a request and hearing that its cache is gone a fresh cache may have replaced it, and a verdict about the old one must not discard the new one.
func (*Manager) Model ¶
Model is the model this manager's cache is for. A cache serves only the model it was created for.
func (*Manager) Name ¶
Name returns the resolved cache name if the manager is active, or "" if init hasn't landed yet / failed / been deleted. Callers wire the return value into GenerateContentConfig.CachedContent — an empty value means the request runs uncached, which is always safe.
Name also opportunistically triggers a background Refresh if the active cache is inside its refresh window. The refresh runs asynchronously; the current call still returns the (about-to-expire but still-valid) cache name.
type Options ¶
type Options struct {
// TTL is how long each Create/Update requests the cache live for.
// Vertex caps at 24h; the daemon's default matches the session
// idle timeout (6h). Zero → 6h.
TTL time.Duration
// RefreshThreshold triggers a background Refresh call when Name()
// is read and time-to-expiry drops below this value. Zero → 30min.
RefreshThreshold time.Duration
// DisplayName annotates the cache in Vertex's list view. Optional
// — omitted from Create when empty.
DisplayName string
// Logger receives structured operator-facing warnings when a
// Caches RPC fails. Zero → log.Default(); every line starts
// "vertexcache:".
Logger *log.Logger
}
Options configures a Manager. Defaults come from the context-caching design doc — TTL 6h, Refresh threshold 30min. Zero values pick the default; callers that want the default for both fields can pass Options{}.
type Status ¶
type Status struct {
Active bool
Failed bool
CacheName string
ExpiresAt time.Time
ExpiresIn time.Duration
}
Status is a snapshot of the manager's state for diagnostic use (e.g. an /internal/cache-status endpoint or a startup log line once init settles). Read-only; each call takes a fresh RLock.