provider

type Cache ¶

type Cache struct {
	TTL   int      `json:"ttl"`
	Marks []string `json:"marks"`
}

Cache is the step record's cache header. Marks are the ids of the records through which the prefix is cached. Anthropic and bedrock put a breakpoint after the system prompt and after the last user block; openai and gemini cache every prefix, so the mark is the last record. TTL is seconds from the step's time: the provider's promise for anthropic and bedrock, a forecast for the rest. The next step's usage.cache_read is the truth.

type Call ¶

type Call struct {
	Name      string          `json:"name"`
	Arguments json.RawMessage `json:"arguments"`
}

Call is the body of a call record.

type Envelope ¶

type Envelope interface {
	Name() string
	Accepts(typ string) bool
	Compile(req Request) (body []byte, cache json.RawMessage, err error)
	Stream(resp io.Reader, emit func(statefile.Record)) (statefile.Header, error)
}

Envelope is one wire shape. Name tags the thinking and step records it produces; only thinking with its own name is replayed. Compile is deterministic and returns the cache markers it placed, which go on the step record. Stream calls emit once per block as the stream completes it and returns the step header with usage. Accepts is the media types the envelope encodes as media; a record of any other type goes as text.

type Provider ¶

type Provider struct {
	URL      string // full endpoint. gemini and bedrock take the model in the path, see Endpoint
	Key      string
	Envelope Envelope
	Headers  func(req *http.Request, key string) // auth and version headers
	Endpoint func(url, model string) string      // nil means URL as is
}

func Anthropic(key string) Provider

func Bedrock(region, key string) Provider

func Gemini(key string) Provider

func OpenAI(key string) Provider

func OpenAIChat(base, key string) Provider

func (p Provider) Step(ctx context.Context, req Request, emit func(statefile.Record)) (statefile.Header, error)

type Request ¶

type Request struct {
	Model   string
	Key     string // session id, the routing key for prefix caches
	Tools   []Tool
	Context []statefile.Record // statefile.Build output
}

type Tool ¶

type Tool struct {
	Name        string
	Description string
	Schema      json.RawMessage // JSON Schema for the arguments object
}

type Turn ¶

type Turn struct {
	Assistant bool
	Records   []statefile.Record
	// contains filtered or unexported fields
}

Turns groups the non-system context into alternating turns in API order: user turns hold results, messages, later system records, summaries, and errors; assistant turns hold the blocks one step closed. The step record ends its assistant turn. A step that closed no blocks, a model that said nothing, is no turn at all: every provider rejects an empty assistant turn, and every provider merges the user turns around it.

func Turns(rest []statefile.Record) []Turn