model

package
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 20, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package model implements the Model Gateway decision chain of the Fenced kernel: versioned model descriptors with price tables, policy checks, incremental token/cost settlement against the task budget ledger, and the hard stop when a reservation is exhausted (tech baseline §12.1; invariant 12). Prompt and completion content never enters kernel telemetry.

Real model invocation: Invoker binds the fenced decision chain (Begin → Settle → Finish) to a provider execution layer. The gateway still owns policy, budget, the call ledger and cost computation; the executor owns the wire. Content flows through and is never persisted — only metadata (usage, cost, provider request id, finish reason) reaches the ledger and the audit receipt.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrModelDenied reports a model reference outside the tenant's policy.
	ErrModelDenied = errors.New("model is not allowed by tenant policy")
	// ErrBudgetExhausted is the hard-stop outcome: the task budget is
	// exhausted and new consumption must stop (invariant 12).
	ErrBudgetExhausted = errors.New("task budget exhausted: new consumption stopped")
)
View Source
var ErrNoProviderExecution = errors.New("model provider has no execution endpoint configured")

ErrNoProviderExecution reports a model whose provider has no execution endpoint configured; the invocation fails closed before any ledger row or budget consumption is opened.

Functions

This section is empty.

Types

type BeginInput

type BeginInput struct {
	TenantID        string
	TaskID          uuid.UUID
	RunID           uuid.UUID
	AttemptID       uuid.UUID
	FencingToken    int64
	AgentVersionRef string
	// ModelRef is the canonical provider/model reference.
	ModelRef       string
	IdempotencyKey string
	// EstimatedInputTokens and MaxOutputTokens define the worst-case call
	// envelope reserved before the provider request is sent. A zero output
	// limit asks the gateway to select a budget-aware bounded default.
	EstimatedInputTokens int64
	MaxOutputTokens      int64
}

BeginInput starts one fenced model invocation.

type BeginResult

type BeginResult struct {
	Call                     store.ModelCall
	Descriptor               store.ModelDescriptor
	EffectiveMaxOutputTokens int64
	PolicyRevision           string
	DenyReasons              []string
}

BeginResult is the outcome of starting an invocation.

type FinishInput

type FinishInput struct {
	TenantID          string
	ModelCallID       uuid.UUID
	ExpectedVersion   int64
	Status            store.ModelCallStatus
	InputTokens       int64
	OutputTokens      int64
	ProviderRequestID string
	FinishReason      string
	UsageCertainty    store.ModelUsageCertainty
	// ProviderName and WireModel are the physical deployment identity the
	// logical modelRef actually resolved to (P2-10). They are recorded in the
	// audit receipt so a call is reproducible against the exact provider route
	// that served it, distinct from the tenant-visible logical reference.
	// ProviderName is a name only — endpoints/credentials never enter a receipt.
	ProviderName string
	WireModel    string
}

FinishInput finalizes a model invocation. Usage is reported by the caller; cost is computed by the gateway from the descriptor's price table (pinned price revision), never trusted from the caller.

type Gateway

type Gateway struct {
	// contains filtered or unexported fields
}

Gateway orchestrates the model call lifecycle: Begin (policy + budget preflight + ledger row), Settle (idempotent incremental settlement with the ledger enforcing the hard stop), Finish (final settlement, terminal state and durable audit receipt). Every settlement carries its own idempotency key derived from the call and sequence, so retries never double-charge.

func NewGateway

func NewGateway(policyChecker PolicyChecker, models ModelStore, budget store.BudgetStore, receipts ReceiptStore) *Gateway

func (*Gateway) Begin

func (g *Gateway) Begin(ctx context.Context, in BeginInput) (BeginResult, error)

Begin validates policy and budget headroom and opens the call ledger row.

func (*Gateway) Finish

func (g *Gateway) Finish(ctx context.Context, call store.ModelCall, in FinishInput) (store.ModelCall, error)

Finish finalizes the invocation: the final usage is settled idempotently, cost is computed from the pinned price table, the ledger row reaches its terminal state, and an audit receipt records the metadata (never content).

func (*Gateway) GetModelCall

func (g *Gateway) GetModelCall(ctx context.Context, tenantID string, callID uuid.UUID) (store.ModelCall, error)

GetModelCall reads one ledger row for fenced transports.

func (*Gateway) Settle

func (g *Gateway) Settle(ctx context.Context, call store.ModelCall, sequence int64, usage Usage) error

Settle appends one idempotent usage step to the ledger. A settlement that would exceed the reservation is rejected and the ledger is marked exhausted: the caller must stop the stream within bounded time. A task without a reservation carries no ceiling and is not gated (mirroring the Tool Gateway).

type InvokeInput

type InvokeInput struct {
	TenantID        string
	TaskID          uuid.UUID
	RunID           uuid.UUID
	AttemptID       uuid.UUID
	FencingToken    int64
	AgentVersionRef string
	ModelRef        string
	IdempotencyKey  string

	Messages        []provider.Message
	Temperature     *float64
	MaxOutputTokens int32
	Stream          bool
	// Tools are the platform-described tool contracts the model may call.
	// Callers resolve them from the AgentVersion's capability grants; the
	// invoker never invents or widens them.
	Tools []provider.ToolDefinition
}

InvokeInput is one fenced real model invocation.

type InvokeOutput

type InvokeOutput struct {
	Call      store.ModelCall
	Content   string
	ToolCalls []provider.ToolCall
}

InvokeOutput carries the terminal ledger row and the completion content.

type Invoker

type Invoker struct {
	// contains filtered or unexported fields
}

Invoker executes real model calls behind the Model Gateway decision chain.

func NewInvoker

func NewInvoker(gateway *Gateway, providers *provider.Registry) *Invoker

NewInvoker binds the decision gateway to a provider registry. A nil registry fails every invocation closed (governance without execution).

func (*Invoker) Invoke

func (inv *Invoker) Invoke(ctx context.Context, in InvokeInput) (InvokeOutput, error)

Invoke performs one non-streaming invocation.

func (*Invoker) InvokeStream

func (inv *Invoker) InvokeStream(ctx context.Context, in InvokeInput, onDelta func(string)) (InvokeOutput, error)

InvokeStream performs one invocation with streaming intent: content deltas are delivered to onDelta as they arrive (nil disables delivery). A descriptor that declares no streaming support transparently falls back to the non-streaming wire call. When the budget ceiling trips mid-stream the provider call is cancelled, the ledger row finishes STOPPED, and ErrBudgetExhausted is returned.

type ModelStore

ModelStore is the model registry and call ledger. The postgres Store satisfies it.

type PolicyChecker

type PolicyChecker interface {
	EvaluateModel(context.Context, string, policy.ModelContext) policy.Decision
}

PolicyChecker decides model calls outside the LLM. The engine is fail-closed: missing tenant data and evaluation errors deny by default.

type ReceiptStore

type ReceiptStore interface {
	WriteRuntimeReceipt(context.Context, store.WriteRuntimeReceiptInput) error
}

ReceiptStore persists the model call outcome for audit. Content is never stored — only metadata.

type Usage

type Usage struct {
	InputTokens  int64
	OutputTokens int64
}

Usage is one settlement step of a streaming call.

Directories

Path Synopsis
Package provider implements the real model execution layer behind the Model Gateway decision chain: an HTTP executor covering two wire dialects.
Package provider implements the real model execution layer behind the Model Gateway decision chain: an HTTP executor covering two wire dialects.
Package tokens provides tokenizer-aware token estimation for model budget reservations.
Package tokens provides tokenizer-aware token estimation for model budget reservations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL