Documentation
¶
Overview ¶
Package semanticcache implements a semantic response cache: on PreRequest it serves a cached response when a semantically similar request is found, and on PostResponse it stores successful upstream responses for future hits.
Index ¶
- Constants
- type Option
- type Plugin
- func (p *Plugin) Execute(ctx context.Context, in appplugins.ExecInput) (*appplugins.Result, error)
- func (p *Plugin) MandatoryStages() []policy.Stage
- func (p *Plugin) Name() string
- func (p *Plugin) SupportedModes() []policy.Mode
- func (p *Plugin) SupportedStages() []policy.Stage
- func (p *Plugin) ValidateConfig(settings map[string]any) error
- type SemanticCacheData
Constants ¶
View Source
const PluginName = "semantic_cache"
PluginName is the catalog name used in policy configuration.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Option ¶
type Option func(*Plugin)
Option customizes the plugin.
func WithDimension ¶
WithDimension overrides the vector dimension used to create the index.
type Plugin ¶
type Plugin struct {
// contains filtered or unexported fields
}
Plugin caches responses by request embedding similarity, scoped per registry.
func New ¶
func New( store semantic.Store, locator embeddingfactory.EmbeddingServiceLocator, registry *adapter.Registry, opts ...Option, ) *Plugin
New builds a semantic cache plugin.
func (*Plugin) Execute ¶
func (p *Plugin) Execute(ctx context.Context, in appplugins.ExecInput) (*appplugins.Result, error)
func (*Plugin) MandatoryStages ¶
func (*Plugin) SupportedModes ¶
func (*Plugin) SupportedStages ¶
type SemanticCacheData ¶
type SemanticCacheData struct {
CacheHit bool `json:"cache_hit"`
Similarity float64 `json:"similarity,omitempty"`
Threshold float64 `json:"threshold"`
EmbeddingSize int `json:"embedding_size,omitempty"`
VectorDim int `json:"vector_dimension,omitempty"`
Stored bool `json:"stored"`
Degraded bool `json:"degraded,omitempty"`
DegradedReason string `json:"degraded_reason,omitempty"`
}
SemanticCacheData is the per-invocation trace payload describing the cache lookup/store decision and the similarity scoring that drove it.
Source Files
Click to show internal directories.
Click to hide internal directories.