Documentation
¶
Overview ¶
Package adapter converts jargo's universal conversation context into the request each LLM provider takes.
A provider's wire format is its own: one takes the system prompt as a leading message and another as a field beside the conversation, one names a tool result by an id and another by the call it answers. An adapter is where that translation lives, and it is a plain value the service owns and calls rather than a base the service inherits from. That keeps the conversion a pure function of the conversation, so it can be tested by comparing what went in with what came out, with no endpoint to stand up.
Each provider adapter lives in its own package under this one and satisfies LLMAdapter for its parameter and tool types.
Index ¶
- func CreateLLMSpecificMessage(a Identifier, native any) frames.Message
- func CustomToolsFor[T any](schema frames.ToolsSchema, t frames.AdapterType) ([]T, error)
- func NativeMessage[T any](m frames.Message) (T, error)
- func ToolsForLogging[P, T any](a LLMAdapter[P, T], schema frames.ToolsSchema) []any
- type Base
- func (b *Base) ExtractInitialSystem(system, systemInstruction string, msgs []frames.Message) (string, []frames.Message)
- func (b *Base) RemoveBuiltin(name string) bool
- func (b *Base) ResolveSystemInstruction(fromContext, systemInstruction string, discardContextSystem bool) string
- func (b *Base) SetBuiltin(builtin Builtin)
- func (b *Base) WithBuiltins(schema frames.ToolsSchema) frames.ToolsSchema
- type Builtin
- type ConversionError
- type Identifier
- type LLMAdapter
- type Options
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CreateLLMSpecificMessage ¶
func CreateLLMSpecificMessage(a Identifier, native any) frames.Message
CreateLLMSpecificMessage builds a conversation message written in a's provider's own format, for something the universal conversation has no representation for. Only a's provider is sent it; every other adapter leaves it out.
native must be the message type a's own package defines, which is what its conversion reads back. Anything else is reported as a conversion failure when the conversation is sent.
func CustomToolsFor ¶
func CustomToolsFor[T any](schema frames.ToolsSchema, t frames.AdapterType) ([]T, error)
CustomToolsFor reads the custom tools written for one format back as the tool type that adapter defines, reporting a conversion failure for anything else.
func NativeMessage ¶
NativeMessage reads a provider-native message back as the type that provider's adapter defines, reporting a conversion failure when it holds something else. An adapter calls it on a message it has already established is its own.
func ToolsForLogging ¶
func ToolsForLogging[P, T any](a LLMAdapter[P, T], schema frames.ToolsSchema) []any
ToolsForLogging renders the advertised toolset in the provider's own format, boxed so that a caller which cannot name that provider's tool type can still carry it. It is for a log or a trace, where what matters is showing the toolset the model was actually sent rather than the universal form it was converted from.
It is a function rather than a method because an adapter's tool type is a parameter of its interface, and a service reporting on itself is reached through an interface that cannot name it.
Types ¶
type Base ¶
type Base struct {
// contains filtered or unexported fields
}
Base carries the state an adapter keeps between conversions, and the handling of a system prompt that every provider shares. Embed it in a provider adapter to inherit both.
It is safe for concurrent use: a one-shot inference runs off to the side of the pipeline and so may convert a conversation while a generation is converting another.
func (*Base) ExtractInitialSystem ¶
func (b *Base) ExtractInitialSystem( system, systemInstruction string, msgs []frames.Message, ) (string, []frames.Message)
ExtractInitialSystem reports the system prompt a provider should send beside the conversation, and the messages to send with it.
A conversation carrying nothing but a system prompt is the case this exists for. Sending the prompt on its own would leave the message list empty, which a provider that requires at least one non-system message rejects, so the prompt is sent as a user message instead and no separate instruction goes out. systemInstruction is only read to decide whether that displaced a prompt the caller also gave.
func (*Base) RemoveBuiltin ¶
RemoveBuiltin withdraws the tool registered under name, reporting whether there was one.
func (*Base) ResolveSystemInstruction ¶
func (b *Base) ResolveSystemInstruction( fromContext, systemInstruction string, discardContextSystem bool, ) string
ResolveSystemInstruction settles which system prompt a provider is sent when the conversation carries one and the call was given another.
discardContextSystem says what the provider does with the conversation's own prompt when both are set. A provider that takes the prompt beside the conversation has one field to put it in, so the instruction given for the call replaces it. A provider that takes it as a leading message can carry both, and keeping both is what lets an instruction supplement the conversation's prompt rather than silently replace it; there the returned prompt is empty, because the conversation's own is already in the messages.
func (*Base) SetBuiltin ¶
SetBuiltin adds a tool the service implements itself, replacing any already registered under the same name. It is sent on every request from now on, without the application having to advertise it.
It lives here rather than on the conversation because it belongs to the service: a conversation is shared, and writing the tool into it would offer it to every other service reading that conversation, and edit a context the application owns.
func (*Base) WithBuiltins ¶
func (b *Base) WithBuiltins(schema frames.ToolsSchema) frames.ToolsSchema
WithBuiltins returns the toolset with the tools the service implements itself appended to the standard ones. An adapter renders the result, so a built-in tool reaches the model in the provider's own format rather than in one shape that has to suit every provider.
type Builtin ¶
Builtin is a tool the LLM service implements itself.
Whatever the model has to be told to use one is composed into the service's system instruction rather than carried here, so guidance shared by a family of built-in tools is stated once however many of them are offered.
type ConversionError ¶
type ConversionError struct {
// Cause is the error the conversion failed with.
Cause error
}
ConversionError reports that a conversation could not be mapped into a provider's message format. An adapter returns it from its conversion, and the LLM service reports it as a generation failure, so a context the provider cannot represent is told apart from the provider itself failing.
func (*ConversionError) Unwrap ¶
func (e *ConversionError) Unwrap() error
Unwrap returns the underlying conversion failure.
type Identifier ¶
type Identifier interface {
IDForLLMSpecificMessages() string
}
Identifier is the part of an adapter that names the provider it converts for. Every LLMAdapter satisfies it.
type LLMAdapter ¶
type LLMAdapter[P, T any] interface { // IDForLLMSpecificMessages is the identifier this provider's messages are // held under in a universal context, so a message written in one provider's // native format reaches that provider and no other. IDForLLMSpecificMessages() string // LLMInvocationParams converts the conversation into what this provider's API // takes. It returns a [ConversionError] for a conversation the provider has // no representation for. LLMInvocationParams(convo *frames.LLMContext, opts Options) (P, error) // ToProviderToolsFormat renders the advertised toolset in this provider's // format, including the custom tools written for it and leaving out those // written for another. ToProviderToolsFormat(schema frames.ToolsSchema) []T // MessagesForLogging renders the conversation as this provider will see it, // for a log or a trace. MessagesForLogging(convo *frames.LLMContext) []map[string]any }
LLMAdapter converts a universal conversation into one provider's invocation parameters. P is the parameter type that provider's API takes and T its tool type, both of which the adapter's own package defines.
type Options ¶
type Options struct {
// SystemInstruction is the instruction this call was given, which stands
// beside the conversation's own system prompt. It is what a one-shot
// inference runs under (see llm.InferenceOptions). Empty leaves the
// conversation's prompt to stand alone.
SystemInstruction string
// ConvertDeveloperToUser sends a developer message as a user message, for a
// provider with no developer role. What the role carries, the late results of
// an asynchronous tool, is worth more to the model than the role it arrives
// under.
ConvertDeveloperToUser bool
// EnablePromptCaching marks the conversation so the provider caches the
// prompt and reads the cache back on the next turn.
EnablePromptCaching bool
// EnsureLastMessageIsUser appends a minimal user message when the converted
// conversation ends on an assistant message. It is for a model without
// assistant-prefill support, which rejects a request ending that way.
EnsureLastMessageIsUser bool
}
Options tunes one conversion. The zero value converts the conversation as the provider's own API takes it.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package anthropic converts a universal conversation into the request Anthropic takes, and with it Bedrock, which serves the same models.
|
Package anthropic converts a universal conversation into the request Anthropic takes, and with it Bedrock, which serves the same models. |
|
Package gemini converts a universal conversation into the request Gemini takes, and with it Vertex AI, which serves the same models.
|
Package gemini converts a universal conversation into the request Gemini takes, and with it Vertex AI, which serves the same models. |
|
Package mistral converts a universal conversation into the request Mistral takes.
|
Package mistral converts a universal conversation into the request Mistral takes. |
|
Package openai converts a universal conversation into the chat-completions request OpenAI takes, and with it every endpoint that speaks OpenAI's API.
|
Package openai converts a universal conversation into the chat-completions request OpenAI takes, and with it every endpoint that speaks OpenAI's API. |
|
Package perplexity converts a universal conversation into the request Perplexity takes.
|
Package perplexity converts a universal conversation into the request Perplexity takes. |
|
Package realtime converts a universal conversation into what OpenAI's Realtime API takes on a session.
|
Package realtime converts a universal conversation into what OpenAI's Realtime API takes on a session. |
|
Package responses converts a universal conversation into the request OpenAI's Responses API takes, over HTTP and over the WebSocket alike: the two transports carry the same request.
|
Package responses converts a universal conversation into the request OpenAI's Responses API takes, over HTTP and over the WebSocket alike: the two transports carry the same request. |