routing

package
v0.260507.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 6, 2026 License: MPL-2.0 Imports: 11 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ContainsService added in v0.260409.1540

func ContainsService(services []*loadbalance.Service, target *loadbalance.Service) bool

ContainsService checks if target exists in services by service ID.

func ExtractRequestContext

func ExtractRequestContext(req interface{}) (*smartrouting.RequestContext, error)

ExtractRequestContext extracts RequestContext from different request types

func FilterActiveServices

func FilterActiveServices(services []*loadbalance.Service) []*loadbalance.Service

FilterActiveServices returns only active services from the input list

func IntersectServices added in v0.260409.1540

func IntersectServices(left, right []*loadbalance.Service) []*loadbalance.Service

IntersectServices keeps services that are present in both lists.

func ResolveSessionID

func ResolveSessionID(c *gin.Context, req interface{}) typ.SessionID

ResolveSessionID returns the best available session identifier from the request. Priority: Anthropic metadata.user_id > X-Tingly-Session-ID header > ClientIP The client IP is always stored in IPBackup as a fallback for rate limiting or logging.

func TestFixtures_helpers

func TestFixtures_helpers(t *testing.T)

Types

type AffinityEntry

type AffinityEntry struct {
	Service   *loadbalance.Service
	MessageID string
	LockedAt  time.Time
	ExpiresAt time.Time
}

AffinityEntry represents a locked service for a session

type AffinityStage

type AffinityStage struct {
	// contains filtered or unexported fields
}

AffinityStage checks if a session has a locked service from previous requests. If found and valid, returns the locked service; otherwise passes to next stage.

func NewAffinityStage

func NewAffinityStage(store AffinityStore, scope string) *AffinityStage

NewAffinityStage creates a new affinity stage with the given store and scope

func (*AffinityStage) Evaluate

func (s *AffinityStage) Evaluate(ctx *SelectionContext, state *selectionState) (*SelectionResult, bool)

Evaluate checks for locked service affinity

func (*AffinityStage) Name

func (s *AffinityStage) Name() string

Name returns the stage identifier

type AffinityStore

type AffinityStore interface {
	Get(ruleUUID, sessionID string) (*AffinityEntry, bool)
	Set(ruleUUID, sessionID string, entry *AffinityEntry)
	CountByService(serviceID string) int // count active sessions locked to this service
}

AffinityStore interface defines operations for session-service affinity

type HealthStage added in v0.260409.1540

type HealthStage struct {
	// contains filtered or unexported fields
}

HealthStage filters unhealthy services from the context. It runs first and narrows ctx.CandidateServices.

func NewHealthStage added in v0.260409.1540

func NewHealthStage(filter *typ.HealthFilter) *HealthStage

NewHealthStage creates a new health stage with the given health filter

func (*HealthStage) Evaluate added in v0.260409.1540

func (s *HealthStage) Evaluate(_ *SelectionContext, state *selectionState) (*SelectionResult, bool)

func (*HealthStage) Name added in v0.260409.1540

func (s *HealthStage) Name() string

type LoadBalancer

type LoadBalancer interface {
	SelectService(rule *typ.Rule) (*loadbalance.Service, error)
	UpdateServiceIndex(rule *typ.Rule, service *loadbalance.Service)
}

LoadBalancer defines the interface for load balancing operations. This avoids importing the server package (which would create circular imports).

type LoadBalancerStage

type LoadBalancerStage struct {
	// contains filtered or unexported fields
}

LoadBalancerStage performs standard load balancing across all rule services. This stage always returns a service (or error), acting as the final fallback.

func NewLoadBalancerStage

func NewLoadBalancerStage(lb LoadBalancer) *LoadBalancerStage

NewLoadBalancerStage creates a new load balancer stage

func (*LoadBalancerStage) Evaluate

func (s *LoadBalancerStage) Evaluate(ctx *SelectionContext, state *selectionState) (*SelectionResult, bool)

Evaluate selects a service using load balancing

func (*LoadBalancerStage) Name

func (s *LoadBalancerStage) Name() string

Name returns the stage identifier

type ProviderResolver

type ProviderResolver interface {
	GetProviderByUUID(uuid string) (*typ.Provider, error)
	SaveCurrentServiceID(ruleUUID string, serviceID string) error
}

ProviderResolver resolves providers by UUID and persists config state.

type SelectionContext

type SelectionContext struct {
	// Rule is the routing rule being evaluated
	Rule *typ.Rule

	// Request is the parsed API request (OpenAI/Anthropic params)
	Request interface{}

	// SessionID is the resolved session identifier for affinity
	// Priority: Anthropic metadata.user_id > X-Tingly-Session-ID header > ClientIP
	SessionID typ.SessionID

	// GinContext provides access to HTTP headers and client info
	GinContext *gin.Context

	// Scenario identifies the request type (openai, anthropic, etc.)
	Scenario typ.RuleScenario

	// MatchedSmartRuleIndex tracks which smart routing rule matched (-1 if none)
	// This is set by SmartRoutingStage and used for smart_rule-scoped affinity
	MatchedSmartRuleIndex int
}

SelectionContext encapsulates all input needed for service selection. It is created once per request and passed through the selection pipeline.

func NewSelectionContext

func NewSelectionContext(
	rule *typ.Rule,
	req interface{},
	c *gin.Context,
	scenario typ.RuleScenario,
) *SelectionContext

NewSelectionContext creates a new selection context with resolved session ID

type SelectionResult

type SelectionResult struct {
	// Service is the selected load-balanced service
	Service *loadbalance.Service

	// FilteredServices contains a narrowed candidate set produced by filter stages.
	// When set, selector updates SelectionContext.CandidateServices with this value.
	FilteredServices []*loadbalance.Service

	// Provider is the resolved provider for the service
	Provider *typ.Provider

	// Source indicates which stage selected this service
	// Values: "affinity", "smart_routing", "load_balancer"
	Source string

	// EvaluatedStages tracks which stages were evaluated (for observability)
	EvaluatedStages []string

	// MatchedSmartRuleIndex is the index of the matched smart routing rule
	// -1 if no smart routing rule matched
	MatchedSmartRuleIndex int
}

SelectionResult represents the output of service selection pipeline. It includes the selected service, provider, and metadata about the selection.

func NewFilterResult added in v0.260409.1540

func NewFilterResult(source string, services []*loadbalance.Service) *SelectionResult

NewFilterResult creates a non-terminal result for filtering stages.

func NewResult

func NewResult(service *loadbalance.Service, source string) *SelectionResult

NewResult creates a new selection result with the given service and source

func (*SelectionResult) AddEvaluatedStage

func (r *SelectionResult) AddEvaluatedStage(stageName string)

AddEvaluatedStage records that a stage was evaluated

type SelectionStage

type SelectionStage interface {
	// Name returns the stage identifier for logging and metrics
	Name() string

	// Evaluate attempts to select a service based on the context.
	// Returns:
	//   - (result, true) if this stage selected a service (stops pipeline)
	//   - (nil, false) if this stage cannot select (continue to next stage)
	Evaluate(ctx *SelectionContext, state *selectionState) (*SelectionResult, bool)
}

SelectionStage represents a single stage in the service selection pipeline. Each stage can evaluate the context and either: - Return a service selection (result, true) - Pass to the next stage (nil, false)

type ServiceSelector

type ServiceSelector struct {
	// contains filtered or unexported fields
}

ServiceSelector is the main entry point for service selection. It orchestrates a pipeline of selection stages and validates the final result.

func NewServiceSelector

func NewServiceSelector(
	cfg ProviderResolver,
	affinity AffinityStore,
	lb LoadBalancer,
) *ServiceSelector

NewServiceSelector creates a new service selector

func (*ServiceSelector) Select

Select is the main entry point for service selection. It picks a pre-built pipeline based on rule configuration and executes it.

func (*ServiceSelector) UpdateServiceIndex

func (s *ServiceSelector) UpdateServiceIndex(rule *typ.Rule, service *loadbalance.Service) error

UpdateServiceIndex updates the current service index for round-robin. This is called from the handler after selection to persist state.

type SimpleSelector

type SimpleSelector struct {
	// contains filtered or unexported fields
}

SimpleSelector provides a simplified API that mimics the old interface but uses the pipeline internally. This makes migration easier.

func NewSimpleSelector

func NewSimpleSelector(selector *ServiceSelector) *SimpleSelector

NewSimpleSelector creates a simplified selector

func (*SimpleSelector) SelectService

func (s *SimpleSelector) SelectService(
	c *gin.Context,
	scenario typ.RuleScenario,
	rule *typ.Rule,
	req interface{},
) (*typ.Provider, *loadbalance.Service, error)

SelectService is a drop-in replacement for DetermineProviderAndModelWithScenario. It handles everything: session resolution, pipeline execution, provider validation.

Migration is simple - just replace the method name:

Before:

provider, service, err := s.DetermineProviderAndModelWithScenario(scenario, rule, req, sessionID)

After:

provider, service, err := s.selector.SelectService(c, scenario, rule, req)

sessionID is automatically resolved and stored in gin context.

func (*SimpleSelector) SelectServiceForEmbeddings added in v0.260507.1

func (s *SimpleSelector) SelectServiceForEmbeddings(
	c *gin.Context,
	scenario typ.RuleScenario,
	rule *typ.Rule,
) (*typ.Provider, *loadbalance.Service, error)

SelectServiceForEmbeddings is a variant of SelectService for embedding requests. Embedding requests don't carry chat-style context, so content-based smart routing is skipped (load balancing, affinity, and health filters still apply).

type SmartRoutingStage

type SmartRoutingStage struct {
	// contains filtered or unexported fields
}

SmartRoutingStage evaluates smart routing rules and returns matched services. If multiple services match, applies load balancing within the matched set.

func NewSmartRoutingStage

func NewSmartRoutingStage(lb LoadBalancer, affinity AffinityStore) *SmartRoutingStage

NewSmartRoutingStage creates a new smart routing stage

func (*SmartRoutingStage) Evaluate

func (s *SmartRoutingStage) Evaluate(ctx *SelectionContext, state *selectionState) (*SelectionResult, bool)

Evaluate evaluates smart routing rules and selects a service

func (*SmartRoutingStage) Name

func (s *SmartRoutingStage) Name() string

Name returns the stage identifier

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL