Documentation
¶
Overview ¶
Package api provides higher level functions to wrap the OpenAI chat completions API.
Index ¶
- Variables
- func ChatCompletionToolParams(tools []ToolFunction) (params []openai.ChatCompletionToolUnionParam)
- func FromMessage(m Message, reasoningField string) openai.ChatCompletionMessageParamUnion
- func GetContent(raw string) (content, reasoning string)
- func MaxModelLength(server Server, baseURL string) (maxLen int, err error)
- func Pretty(v any) string
- func Tokenize(server Server, baseURL string, ...) (numTokens int, err error)
- type Accumulator
- type CallbackFunc
- type Client
- func (c *Client) ChatCompletion(ctx context.Context, request Conversation, callback CallbackFunc, ...) ([]Message, error)
- func (c *Client) ChatCompletionStream(ctx context.Context, request Conversation, callback CallbackFunc, ...) ([]Message, error)
- func (c *Client) CompactMessages(conv Conversation, limit int) error
- func (c *Client) NewRequest(modelName string, conv Conversation, tools ...ToolFunction) openai.ChatCompletionNewParams
- type Config
- type Conversation
- type Item
- type Message
- type Request
- type Response
- type Server
- type Stats
- type ToolConfig
- type ToolFunction
Constants ¶
This section is empty.
Variables ¶
var ( // Optional logging of raw JSON requests and responses TraceRequests = false TraceStream = false TraceTo = os.Stderr )
var ( // Used by DefaultConfig DefaultSystemMessage = "You are a helpful assistant. You should answer concisely unless more detail is requested. The current date is {{today}}." // Used by NewRequest ParallelToolCalls = true )
Functions ¶
func ChatCompletionToolParams ¶
func ChatCompletionToolParams(tools []ToolFunction) (params []openai.ChatCompletionToolUnionParam)
Tool parameters for given list of tools
func FromMessage ¶
func FromMessage(m Message, reasoningField string) openai.ChatCompletionMessageParamUnion
Convert our message format to openai chat completion struct
func GetContent ¶
Get content and reasoning content from raw JSON message
func MaxModelLength ¶
Get max content length for current model
Types ¶
type Accumulator ¶
type Accumulator struct {
openai.ChatCompletionAccumulator
Content string
Reasoning string
// contains filtered or unexported fields
}
type CallbackFunc ¶
Generated content where channel is either analysis (i.e. reasoning text), final (generated text) or tool (response from tool call) and index is the count of the message on the current stream, end is set on final message completion and full rather than delta content is sent
type Client ¶
type Client struct {
openai.Client
Server Server
BaseURL string
ModelName string
ReasoningField string
ContextLength int
}
Client and associated info
func NewClient ¶
Create new client with default settings if no options are given. If set then will use OPENAI_BASE_URL and OPENAI_API_KEY environment variables. The model name is optional for LlamaCPP and LLLM - they will use the currently loaded model.
func (*Client) ChatCompletion ¶
func (c *Client) ChatCompletion(ctx context.Context, request Conversation, callback CallbackFunc, statsCallback func(Stats), tools ...ToolFunction) ([]Message, error)
Chat completion without streaming with optional function call support. The list of new generated messages are returned. callback and statsCallback are called after each stage of generation - i.e. reasoning text, tool response and final response.
func (*Client) ChatCompletionStream ¶
func (c *Client) ChatCompletionStream(ctx context.Context, request Conversation, callback CallbackFunc, statsCallback func(Stats), tools ...ToolFunction) ([]Message, error)
As per ChatCompletion but will stream responses as they are generated.
func (*Client) CompactMessages ¶
func (c *Client) CompactMessages(conv Conversation, limit int) error
Estimate number of prompt tokens generated from the given list of messages. If this exceeds the given limit then the oldest messages in the conversation are marked as excluded.
func (*Client) NewRequest ¶
func (c *Client) NewRequest(modelName string, conv Conversation, tools ...ToolFunction) openai.ChatCompletionNewParams
Create a new chat completion request with given config settings. Messages with Excluded set are omitted from the request. Includes reasoning content starting from the beginning of the latest turn.
type Config ¶
type Config struct {
SystemPrompt string `json:"system_prompt"`
ReasoningEffort string `json:"reasoning_effort"` // low | medium | high | none
Tools []ToolConfig `json:"tools,omitzero"`
Temperature float64 `json:"temperature,omitzero"`
TopP float64 `json:"top_p,omitzero"`
TopK int `json:"top_k,omitzero"`
PresencePenalty float64 `json:"presence_penalty,omitzero"`
RepetitionPenalty float64 `json:"repetition_penalty,omitzero"`
CompactThreshold float64 `json:"compact_threshold,omitzero"` // if set then apply message compaction if hit this fraction of model context length
}
func DefaultConfig ¶
func DefaultConfig(tools ...ToolFunction) Config
Get default configuration with given tools enabled
type Conversation ¶
type Conversation struct {
ID string `json:"id"`
Config Config `json:"config"`
Messages []Message `json:"messages"`
NumTokens int `json:"num_tokens"`
ToolData map[string]json.RawMessage `json:"tool_data,omitzero"`
}
func NewConversation ¶
func NewConversation(cfg Config) Conversation
Create a new conversation and assign unique ID
func (Conversation) LastUserMessageNumber ¶
func (c Conversation) LastUserMessageNumber() int
Most recent non-excluded user message number or -1 if not found
type Message ¶
type Message struct {
Role string `json:"role"` // user | assistant | tool
Update bool `json:"update,omitzero"` // true if update to existing message
End bool `json:"end,omitzero"` // true if update and message is now complete
Content string `json:"content"`
Reasoning string `json:"reasoning,omitzero"`
ToolCall json.RawMessage `json:"tool_call,omitzero"`
ToolCallID string `json:"tool_call_id,omitzero"`
ContentTokens int `json:"content_tokens,omitzero"`
ReasoningTokens int `json:"reasoning_tokens,omitzero"`
Excluded bool `json:"excluded,omitzero"` // message is ignored by NewRequest if this is set
}
type Request ¶
type Request struct {
Action string `json:"action"` // add | list | load | delete | config
ID string `json:"id,omitzero"` // if action=load,delete uuid format
Message Message `json:"message,omitzero"` // if action=add
Config *Config `json:"config,omitzero"` // if action=config
}
Chat API request from frontend to webserver
type Response ¶
type Response struct {
Action string `json:"action"` // add | list | load | config | stats
Message Message `json:"message,omitzero"` // if action=add
Conversation Conversation `json:"conversation,omitzero"` // if action=load
List []Item `json:"list,omitzero"` // if action=list
Config Config `json:"config,omitzero"` // if action=config
Stats Stats `json:"stats,omitzero"` // if action=stats
}
Chat API response from webserver back to frontend
type Stats ¶
type Stats struct {
Model string `json:"model"` // model name
ApiCalls int `json:"api_calls"` // total number of API calls
ApiTime int `json:"api_time"` // total elapsed time in API calls in msec
CompletionTokens int `json:"completion_tokens"` // no. of completion tokens generated
PromptTokens int `json:"prompt_tokens"` // max prompt length
ToolCalls int `json:"tool_calls"` // total number of tool calls
Functions map[string]int `json:"functions"` // numer of tool calls by function name
ToolTime int `json:"tool_time"` // total elapsed time in tool calls in msec
}
func (*Stats) APICallInfo ¶
func (*Stats) CompletionTokensPerSec ¶
func (*Stats) ToolCallInfo ¶
type ToolConfig ¶
type ToolFunction ¶
type ToolFunction interface {
// function name and argument schema
Definition() shared.FunctionDefinitionParam
// call with args in JSON format - if err is set it is treated as fatal, else the model can retry with new args
Call(args string) (req, resp string, err error)
}
Interface implented by tools which can be called by the model.
Directories
¶
| Path | Synopsis |
|---|---|
|
browser
Package browser implements an OpenAI compatible web browser tool plugin which uses the brave search API and firecrawl scrape API.
|
Package browser implements an OpenAI compatible web browser tool plugin which uses the brave search API and firecrawl scrape API. |
|
python
Python tool to execute code in a docker container
|
Python tool to execute code in a docker container |
|
weather
Package weather implements tool functions to call the openweathermap API.
|
Package weather implements tool functions to call the openweathermap API. |