Documentation
¶
Overview ¶
Package anonymous runs the automation that connects the reviewed anonymous providers, as the gateway's does: it enrolls each profile's provider, discovers what its catalog admits, probes the models with a completion and publishes the exact targets that answered.
Policy and persistence stay product code, behind Hooks: whether the automation runs, enrolling a provider in the product's configuration, the durable claim that checks a provider once a day, the evidence generation, and keeping the results. Catalogs and probes go through the product's Catalog and Invoker, typically its runtime.Runtime, so each provider is read and invoked by its own vertical, with the vertical's anonymous admission and the instance's resilience policy.
It replaces providers.AutoConnectAnonymousProviders and providers.NewAnonymousOpenAICompatibleAdapter, which read catalogs and send probes with their own HTTP client.
Index ¶
Constants ¶
const ( // StatusManaged is Enroll's status for a provider the automation // manages, which a run goes on to check. StatusManaged = "managed" // StatusPassed is a check in which a probe verified inference, and // StatusFailed any other check. StatusPassed = "passed" StatusFailed = "failed" )
Statuses of a result.
const ( FailureAuthenticationRejected = "authentication_rejected" FailureCatalogFailed = "catalog_failed" FailureVerificationFailed = "verification_failed" )
Failure codes of a check, the gateway's.
const ( AuthenticationAccepted = "accepted" AuthenticationRejected = "rejected" AuthenticationUnknown = "unknown" )
Authentication states of a check: whether the provider accepted anonymous access.
const DefaultCheckInterval = 24 * time.Hour
DefaultCheckInterval is how often one provider is checked when Options.CheckInterval is zero: once a day, as the gateway checks.
const DefaultRunInterval = time.Hour
DefaultRunInterval is how often Start runs when it is given no interval: hourly, as the gateway runs.
const OperationVerify = "verify"
OperationVerify is the operation of every check.
Variables ¶
This section is empty.
Functions ¶
func ProbeFailureCode ¶
func ProbeFailureCode(probe core.CompletionProbeEvidence) string
ProbeFailureCode is the failure code the gateway records for a probe: none for a verified probe, else the probe's own code, or FailureVerificationFailed when it has none.
Types ¶
type Catalog ¶
type Catalog interface {
Discover(ctx context.Context, caller core.Caller, instance string) ([]core.ModelInfo, error)
}
Catalog discovers the models an enrolled instance admits to anonymous access, as the instance's vertical admits them. A check reports what it discovered, so an implementation reads the catalog as it is now, not as a cache kept it.
type CatalogFunc ¶
type CatalogFunc func(ctx context.Context, caller core.Caller, instance string) ([]core.ModelInfo, error)
CatalogFunc adapts a function to Catalog.
type Hooks ¶
type Hooks interface {
// Enabled reports whether the automation runs now. A run consults it
// before each provider and again once the provider is claimed, so
// turning the automation off stops a run at the next provider.
Enabled(ctx context.Context) bool
// Enroll makes profile's provider one the product serves and the
// automation manages. It creates a provider that is missing, and never
// changes or enables one configured otherwise. It returns the
// provider's instance and StatusManaged, or a status that says why the
// automation leaves the provider alone, such as a collision.
Enroll(ctx context.Context, profile providers.AnonymousProviderProfile) (providerID, status string)
// Claim claims the check of providerID at at. It succeeds at most once
// every every, across processes, which is how the automation checks a
// provider once a day however often it runs. A product that reads the
// provider again under the claim declines one that changed since
// Enroll.
Claim(ctx context.Context, providerID string, at time.Time, every time.Duration) (bool, error)
// Generation returns the evidence generation a check of providerID
// records under. A check that cannot get one fails with
// FailureEvidenceUnavailable and is not recorded.
Generation(ctx context.Context, providerID string) (int64, error)
// Record keeps a check's result. Its Connect carries the whole
// evidence: the catalog, each probe and the published targets.
Record(ctx context.Context, providerID string, generation int64, result Result) error
}
Hooks is what the product decides and keeps.
type Invoker ¶
type Invoker interface {
Invoke(ctx context.Context, caller core.Caller, instance string, request core.Request) (core.Response, error)
}
Invoker performs one operation on an instance. runtime.Runtime implements it, so a probe reaches the upstream through the instance's vertical and resilience policy, as any request does.
type InvokerFunc ¶
type InvokerFunc func(ctx context.Context, caller core.Caller, instance string, request core.Request) (core.Response, error)
InvokerFunc adapts a function to Invoker.
type Options ¶
type Options struct {
// Profiles are the providers the automation connects. Nil connects
// every anonymous profile of Registry.
Profiles []providers.AnonymousProviderProfile
// Registry is the product's effective registry, which vets every
// profile. Nil uses providers.DefaultRegistry.
Registry *providers.Registry
Catalog Catalog
Invoker Invoker
Hooks Hooks
// Caller is who the checks act for, for Catalog and Invoker. The zero
// Caller is anonymous.
Caller core.Caller
// Probe chooses the models a check probes.
Probe ProbePolicy
// Publish decides which targets a check publishes. Empty publishes
// only targets whose probe verified inference, as the gateway does.
Publish core.TargetPublicationPolicy
// CheckInterval is how often one provider is checked: the every of
// Claim. Zero uses DefaultCheckInterval.
CheckInterval time.Duration
// Now returns the current time, for Claim. Nil uses time.Now.
Now func() time.Time
}
Options configures an Orchestrator. Catalog, Invoker and Hooks are required.
type Orchestrator ¶
type Orchestrator struct {
// contains filtered or unexported fields
}
Orchestrator runs the anonymous-provider automation. It keeps no state between runs, so it is safe for concurrent use.
func New ¶
func New(options Options) (*Orchestrator, error)
New returns an Orchestrator, or an error when a required option is missing or a profile is not one the registry curates for anonymous automation.
func (*Orchestrator) ConnectAll ¶
func (o *Orchestrator) ConnectAll(ctx context.Context) []Result
ConnectAll enrolls and checks every profile now, as the gateway's manual "connect free providers" does: it consults neither Enabled nor Claim.
func (*Orchestrator) RunOnce ¶
func (o *Orchestrator) RunOnce(ctx context.Context) []Result
RunOnce runs the automation once, as the gateway's scheduler does: for each profile, while Enabled holds and ctx lasts, it enrolls the provider and, when the automation manages it and the claim of its daily check succeeds, checks it. It returns a result for every provider it enrolled but leaves alone and every provider it checked, and nil when the automation is off.
func (*Orchestrator) Start ¶
func (o *Orchestrator) Start(ctx context.Context, interval time.Duration, wake <-chan struct{}) (stop func())
Start runs the automation at once, then every interval and whenever wake delivers, until ctx ends or the returned stop runs. stop waits for a run in progress to end and may be called more than once. An interval of zero or less uses DefaultRunInterval, and a nil wake never delivers.
type ProbePolicy ¶
type ProbePolicy int
ProbePolicy chooses which discovered models a check probes.
const ( // ProbeAllDiscovered probes every model the catalog lists once, as the // gateway's automation does, so each model publishes on its own // evidence and no model is verified by a sibling's answer. ProbeAllDiscovered ProbePolicy = iota // ProbeVerificationModel probes only the model // providers.AnonymousVerificationModel selects: the first reviewed // default the catalog lists, or the lexically first model. ProbeVerificationModel )
type Result ¶
type Result struct {
ProviderID string `json:"provider_id"`
RegistryID string `json:"registry_id,omitempty"`
// Status is StatusPassed or StatusFailed for a check.
Status string `json:"status"`
Operation string `json:"operation,omitempty"`
Success bool `json:"success"`
// FailureCode says why a check failed, as one of the Failure codes.
FailureCode string `json:"failure_code,omitempty"`
// Details says why no probe ran: the connect error, whose message
// names the failed operation and never quotes the upstream, or that no
// model was available.
Details string `json:"details,omitempty"`
// VerificationError is set when every probe failed.
VerificationError string `json:"verification_error,omitempty"`
AuthenticationState string `json:"authentication_state,omitempty"`
CatalogEvidence core.CatalogEvidenceStatus `json:"catalog_evidence,omitempty"`
CompletionEvidence core.CompletionProbeStatus `json:"completion_evidence,omitempty"`
// Targets are the published targets and Published counts them. Probed
// counts the probes, and Verified and Failed their outcomes.
Targets []core.Target `json:"targets,omitempty"`
Published int `json:"published,omitempty"`
Probed int `json:"probed,omitempty"`
Verified int `json:"verified,omitempty"`
Failed int `json:"failed,omitempty"`
// Retryable and RetryAfter are the provider health's once probes ran.
Retryable bool `json:"retryable,omitempty"`
RetryAfter time.Duration `json:"retry_after,omitempty"`
// Connect is the connect outcome the result derives from.
Connect core.ProviderConnectResult `json:"-"`
// RecordErr is why the result was not kept: Record's error, or the
// context's when the run stopped during the check.
RecordErr error `json:"-"`
}
Result is one provider's outcome in a run, with the fields of the gateway's automation report. A provider the automation leaves alone has only its ids and the status Enroll gave it.
Source Files