inference

package
v0.7.33 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 8, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package inference is the client of a registry's credits sessions: the sessions a run, or a persistent agent, pays for its model with when its policy names model provider agentry. The host opens one with the person's registry sign-in, hands the run only the session's token, renews the session while the run lives, and ends it when the run ends. The token reaches nothing but the registry's model endpoint; the sign-in never leaves the host.

Index

Constants

View Source
const (
	PurposeRun   = "run"
	PurposeAgent = "agent"
)

Purposes a session is opened for.

View Source
const DefaultRenewEvery = 15 * time.Minute

DefaultRenewEvery is how often a session is renewed when the registry names no interval. A session not renewed for an hour ends.

Variables

View Source
var ErrEnded = errors.New("the credits session has ended")

ErrEnded is a session the registry has ended: renewed too late, ended already, or its sign-in revoked.

View Source
var ErrNoCredits = errors.New("the Agentry credits are used up")

ErrNoCredits is a payer whose credits are used up.

View Source
var ErrSignedOut = errors.New("not signed in to the registry")

ErrSignedOut is a client with no sign-in to open or renew a session with.

Functions

This section is empty.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is one registry, as the account its sign-in speaks for.

func New

func New(regURL string, signIn func() (string, error), hc *http.Client) *Client

New is the client of the registry at regURL (no trailing slash). signIn answers the account's registry sign-in at each call, so a renewal made an hour later uses the sign-in as it is then; an empty answer is ErrSignedOut. hc is the HTTP client every request goes through (nil: one with a minute's timeout).

func (*Client) End

func (c *Client) End(ctx context.Context, id string) (View, error)

End settles what is left and ends the session: its token is refused from then on. Ending a session already ended answers its final view.

func (*Client) Open

func (c *Client) Open(ctx context.Context, purpose string) (Opened, error)

Open opens a session for purpose (PurposeRun or PurposeAgent).

func (*Client) Registry

func (c *Client) Registry() string

Registry is the registry's address.

func (*Client) Renew

func (c *Client) Renew(ctx context.Context, id string) (View, error)

Renew settles what the session has used since the last settlement and moves its expiry an hour out. A session the registry has ended is ErrEnded.

func (*Client) WithSignInAdvice added in v0.7.4

func (c *Client) WithSignInAdvice(advice string) *Client

WithSignInAdvice sets what a refused sign-in tells the person to do, for a sign-in that is not this computer's own (a remote runhost's credits token), and returns c.

type Opened

type Opened struct {
	Session    View
	Token      string
	Model      string
	RenewEvery time.Duration
}

Opened is a new session: its token, answered once and never again, the model the registry forwards to, and how often to renew it.

type View

type View struct {
	ID           string    `json:"id"`
	Purpose      string    `json:"purpose"`
	ExpiresAt    time.Time `json:"expiresAt"`
	Ended        bool      `json:"ended"`
	Requests     int64     `json:"requests"`
	InputTokens  int64     `json:"inputTokens"`
	OutputTokens int64     `json:"outputTokens"`
	Credits      int64     `json:"credits"`
}

View is a session as the registry reports it.

Directories

Path Synopsis
Package inferencetest is a fake registry's credits sessions for tests: it opens, renews and ends sessions as a SecureAgentry registry does, and records each call.
Package inferencetest is a fake registry's credits sessions for tests: it opens, renews and ends sessions as a SecureAgentry registry does, and records each call.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL