Help us shape the future of Go! Take the 2026 Go Developer Survey

inference

package
v0.0.0-...-c703685 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 30, 2026 License: MIT Imports: 1 Imported by: 2

Documentation

Overview

Package inference describes model exchanges and measured usage without agent control.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Complete

type Complete func(context.Context, Request) (string, error)

Complete returns completed assistant text. The adapter owns credentials, transport, terminal validation, and authoritative usage accounting.

type OutputThroughput

type OutputThroughput struct {
	OutputTokens     uint64 `json:"output_tokens"`
	DurationNanos    uint64 `json:"duration_ns"`
	MeasuredRequests uint64 `json:"measured_requests"`
}

OutputThroughput pairs authoritative output tokens with the elapsed provider request time for exactly the same measured requests. Zero duration is absent timing, not zero throughput. It excludes unmeasured requests from both sums.

func (*OutputThroughput) Add

func (t *OutputThroughput) Add(next OutputThroughput)

func (OutputThroughput) Rate

func (t OutputThroughput) Rate() (float64, bool)

type Request

type Request struct {
	Model, Effort, Instructions, Input string
}

type Usage

type Usage struct {
	// TotalsKnown distinguishes reported zero totals from absent totals.
	TotalsKnown         bool
	InputTokens         uint64
	UncachedInputTokens uint64
	CacheWriteTokens    uint64
	OutputTokens        uint64
	ReasoningTokens     uint64

	// Preserve inconsistent raw categories before normalizing input for existing consumers.
	Inconsistent bool
	// Missing categories remain zero for existing normalized capture consumers,
	// but cannot establish complete thread totals or a reference cost.
	Incomplete bool
	// The terminal provider tier overrides the requested tier for pricing only.
	ServiceTier string
}

Usage carries one provider-authoritative terminal observation. Per-thread accounting and invocation-wide diagnostics consume the same measured facts.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL