Documentation
¶
Overview ¶
Package inference is the client of a registry's credits sessions: the sessions a run, or a persistent agent, pays for its model with when its policy names model provider agentry. The host opens one with the person's registry sign-in, hands the run only the session's token, renews the session while the run lives, and ends it when the run ends. The token reaches nothing but the registry's model endpoint; the sign-in never leaves the host.
Index ¶
Constants ¶
const ( PurposeRun = "run" PurposeAgent = "agent" )
Purposes a session is opened for.
const DefaultRenewEvery = 15 * time.Minute
DefaultRenewEvery is how often a session is renewed when the registry names no interval. A session not renewed for an hour ends.
Variables ¶
var ErrEnded = errors.New("the credits session has ended")
ErrEnded is a session the registry has ended: renewed too late, ended already, or its sign-in revoked.
var ErrNoCredits = errors.New("the Agentry credits are used up")
ErrNoCredits is a payer whose credits are used up.
var ErrSignedOut = errors.New("not signed in to the registry")
ErrSignedOut is a client with no sign-in to open or renew a session with.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is one registry, as the account its sign-in speaks for.
func New ¶
New is the client of the registry at regURL (no trailing slash). signIn answers the account's registry sign-in at each call, so a renewal made an hour later uses the sign-in as it is then; an empty answer is ErrSignedOut. hc is the HTTP client every request goes through (nil: one with a minute's timeout).
func (*Client) End ¶
End settles what is left and ends the session: its token is refused from then on. Ending a session already ended answers its final view.
type Opened ¶
Opened is a new session: its token, answered once and never again, the model the registry forwards to, and how often to renew it.
type View ¶
type View struct {
ID string `json:"id"`
Purpose string `json:"purpose"`
ExpiresAt time.Time `json:"expiresAt"`
Ended bool `json:"ended"`
Requests int64 `json:"requests"`
InputTokens int64 `json:"inputTokens"`
OutputTokens int64 `json:"outputTokens"`
Credits int64 `json:"credits"`
}
View is a session as the registry reports it.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package inferencetest is a fake registry's credits sessions for tests: it opens, renews and ends sessions as a SecureAgentry registry does, and records each call.
|
Package inferencetest is a fake registry's credits sessions for tests: it opens, renews and ends sessions as a SecureAgentry registry does, and records each call. |