Documentation
¶
Overview ¶
Package semanticrouter provides fast, embedding-based intent routing, jailbreak detection, and response semantic caching for Genie.
By sitting in front of the main orchestration flow, the semantic router acts as a lightweight gatekeeper. It quickly embeds user requests using a pre-configured vector embedding model (e.g. OpenAI, Gemini) and matches the resulting vector against a set of predefined routes (e.g. "jailbreak", "salutation").
If a route match is found (i.e. above the configured Threshold), the router immediately returns a ClassificationResult (e.g. REFUSE or SALUTATION) without invoking the more expensive, higher-latency front-desk LLM.
Further, it manages Semantic Caching which allows repeated or highly similar queries to immediately bypass generation entirely, fetching previously generated results from the Vector Store.
Index ¶
Constants ¶
const ( RouteJailbreak = "jailbreak" RouteSalutation = "salutation" )
Route types for L1 vector matching
const AgentNamePlaceholder = "{{AGENT_NAME}}"
Variables ¶
This section is empty.
Functions ¶
func GetClassifyPrompt ¶
func GetClassifyPrompt() string
GetClassifyPrompt returns the global classify.txt template value.
Types ¶
type ClassificationResult ¶
type ClassificationResult struct {
Category Category
Reason string // non-empty only for OUT_OF_SCOPE
BypassedLLM bool // true if semantic router (L1) bypassed the LLM completely
}
ClassificationResult carries the category together with an optional reason.
type Config ¶
type Config struct {
// Disabled determines whether semantic routing features are active.
Disabled bool `yaml:"disabled,omitempty" toml:"disabled,omitempty"`
// Threshold for semantic similarity matches (0.0 to 1.0).
// Default is 0.85.
Threshold float64 `yaml:"threshold,omitempty" toml:"threshold,omitempty"`
// EnableCaching enables semantic caching for user LLM responses.
EnableCaching bool `yaml:"enable_caching,omitempty" toml:"enable_caching,omitempty"`
// VectorStore defines the embedding and storage backend used for
// the semantic routing and caching. If empty, uses dummy embedder.
VectorStore vector.Config `yaml:"vector_store,omitempty" toml:"vector_store,omitempty"`
// Routes allows injecting custom semantic routes or extending builtin ones.
Routes []Route `yaml:"routes,omitempty" toml:"routes,omitempty"`
}
Config configures the semantic routing engine.
type IRouter ¶
type IRouter interface {
Classify(ctx context.Context, question, resume string) (ClassificationResult, error)
CheckCache(ctx context.Context, query string) (string, bool)
SetCache(ctx context.Context, query string, response string) error
}
IRouter defines the interface for the semantic router, enabling mocking and testing.
type Route ¶
type Route struct {
Name string `yaml:"name" toml:"name"`
Utterances []string `yaml:"utterances" toml:"utterances"`
}
Route defines a semantic category alongside example utterances.
type Router ¶
type Router struct {
// contains filtered or unexported fields
}
Router provides semantic routing (intent classification), semantic caching, and safety checks using a vector store for fast, embedding-based comparisons and acts as the gatekeeper applying L1 Semantic rules and L2 LLM frontdesk rules.
func New ¶
func New(ctx context.Context, cfg Config, provider modelprovider.ModelProvider) (*Router, error)
New creates a new Semantic Router. It initializes isolated vector stores for caching and routing to prevent collision.
func (*Router) CheckCache ¶
CheckCache looks up the input query in the semantic cache.
func (*Router) Classify ¶
func (r *Router) Classify(ctx context.Context, question, resume string) (ClassificationResult, error)
Classify acts as the unified gatekeeper. L1 Check: Checks semantic vector distance and bypasses LLM if intent matches. L2 Check: Proxies to the LLM-based frontDeskExpert if no L1 matches are found.
The method creates an OTel span ("semanticrouter.classify") that appears as a child of the caller's active span (typically "codeowner.chat"). This ensures classification always shows up in the Langfuse trace hierarchy.
Source Files
Directories