Documentation
¶
Overview ¶
Package client provides HTTP client functionality for communicating with the LLM Proxy API.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ChatClient ¶
ChatClient handles communication with the LLM Proxy chat API
func NewChatClient ¶
func NewChatClient(baseURL, token string) *ChatClient
NewChatClient creates a new chat client
func (*ChatClient) SendChatRequest ¶
func (c *ChatClient) SendChatRequest(messages []ChatMessage, options ChatOptions, readline *readline.Instance) (*ChatResponse, error)
SendChatRequest sends a chat request and returns the response
type ChatCompletionStreamResponse ¶
type ChatCompletionStreamResponse struct {
ID string `json:"id"`
Object string `json:"object"`
Created int `json:"created"`
Model string `json:"model"`
Choices []struct {
Index int `json:"index"`
Delta struct {
Role string `json:"role,omitempty"`
Content string `json:"content,omitempty"`
} `json:"delta"`
FinishReason string `json:"finish_reason"`
} `json:"choices"`
Usage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
} `json:"usage"`
}
ChatCompletionStreamResponse represents a chunked response in a stream
type ChatMessage ¶
ChatMessage represents a message in the chat
type ChatOptions ¶
type ChatOptions struct {
Model string
Temperature float64
MaxTokens int
UseStreaming bool
VerboseMode bool
}
ChatOptions configures chat request parameters
type ChatRequest ¶
type ChatRequest struct {
Model string `json:"model"`
Messages []ChatMessage `json:"messages"`
Temperature float64 `json:"temperature,omitempty"`
MaxTokens int `json:"max_tokens,omitempty"`
Stream bool `json:"stream"`
}
ChatRequest represents a request to the chat API
type ChatResponse ¶
type ChatResponse struct {
ID string `json:"id"`
Object string `json:"object"`
Created int `json:"created"`
Model string `json:"model"`
Choices []struct {
Index int `json:"index"`
Message ChatMessage `json:"message"`
FinishReason string `json:"finish_reason"`
} `json:"choices"`
Usage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
} `json:"usage"`
}
ChatResponse represents a response from the chat API
Click to show internal directories.
Click to hide internal directories.