protocol

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 3, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrInvalidRequest = errors.New("invalid chat request")

ErrInvalidRequest is returned when validation fails.

View Source
var ErrNoValue = io.EOF

StreamReaderWithConvert transforms stream elements. Return ErrNoValue from the convert function to skip an element.

Functions

func Pipe

func Pipe(bufferSize int) (StreamReader, StreamWriter)

Pipe creates a linked pair of StreamReader and StreamWriter. The buffer size controls how many chunks can be buffered.

func RegisterFactory

func RegisterFactory(provider ProviderType, factory ClientFactory)

RegisterFactory registers a factory function for a provider

Types

type ChatModel

type ChatModel interface {
	Name() string
	// Chat return common chat response
	Chat(ctx context.Context, req ChatRequest) (*ChatResponse, error)
	// StreamChat returns a streaming handle that yields incremental deltas.
	StreamChat(ctx context.Context, req ChatRequest) (ChatStream, error)
}

ChatModel defines the minimal capability surface for text-only chat models.

func NewClient

func NewClient(cfg ClientConfig) (ChatModel, error)

NewClient creates a ChatModel instance based on provider type. Providers must be registered via RegisterFactory or by importing their init packages.

Example usage:

import (
	"github.com/LingByte/lingllm/protocol"
	_ "github.com/LingByte/lingllm/protocol/openai"
)

cfg := protocol.ClientConfig{
	Provider: llm.ProviderOpenAI,
	APIKey:   "sk-...",
	BaseURL:  "https://api.openai.com/v1",
}
client, err := protocol.NewClient(cfg)

type ChatRequest

type ChatRequest struct {
	// Model is the target model identifier (e.g. "gpt-4o", "claude-3-opus").
	Model string `json:"model"`
	// Messages is the ordered conversation history.
	Messages []Message `json:"messages"`
	// MaxTokens limits generated tokens (provider specific defaults apply when zero).
	MaxTokens int `json:"max_tokens,omitempty"`
	// Temperature controls randomness (0–2 typical for OpenAI-compatible models).
	Temperature float32 `json:"temperature,omitempty"`
	// TopP enables nucleus sampling (provider specific).
	TopP float32 `json:"top_p,omitempty"`
	// Stop provides stop sequences.
	Stop []string `json:"stop,omitempty"`
	// Tools are functions the model can call.
	Tools []Tool `json:"tools,omitempty"`
	// ToolChoice controls tool calling behavior.
	ToolChoice ToolChoice `json:"tool_choice,omitempty"`
	// Metadata is an optional provider-specific bag for future extensions.
	Metadata map[string]string `json:"metadata,omitempty"`
}

ChatRequest captures a provider-agnostic chat generation request.

func NewChatRequest

func NewChatRequest(model string, messages ...Message) *ChatRequest

NewChatRequest creates a ChatRequest with the given model and messages.

func (*ChatRequest) Validate

func (r *ChatRequest) Validate() error

Validate ensures a ChatRequest contains minimally required fields.

func (*ChatRequest) WithMaxTokens

func (r *ChatRequest) WithMaxTokens(maxTokens int) *ChatRequest

WithMaxTokens sets the max tokens for the request.

func (*ChatRequest) WithMetadata

func (r *ChatRequest) WithMetadata(key, value string) *ChatRequest

WithMetadata sets metadata for the request.

func (*ChatRequest) WithStop

func (r *ChatRequest) WithStop(stop ...string) *ChatRequest

WithStop sets the stop sequences for the request.

func (*ChatRequest) WithTemperature

func (r *ChatRequest) WithTemperature(temp float32) *ChatRequest

WithTemperature sets the temperature for the request.

func (*ChatRequest) WithTopP

func (r *ChatRequest) WithTopP(topP float32) *ChatRequest

WithTopP sets the top_p for the request.

type ChatResponse

type ChatResponse struct {
	ID        string              `json:"id"`
	Model     string              `json:"model"`
	CreatedAt time.Time           `json:"created_at"`
	Choices   []Choice            `json:"choices"`
	Usage     TokenUsage          `json:"usage"`
	Metrics   metrics.CallMetrics `json:"metrics,omitempty"`
}

ChatResponse is a normalized chat completion result.

func CollectStream

func CollectStream(ctx context.Context, stream StreamReader) (*ChatResponse, error)

CollectStream reads all chunks from a stream and returns them as a single response.

func (*ChatResponse) FirstContent

func (r *ChatResponse) FirstContent() string

FirstContent returns the first choice's message content if available.

func (*ChatResponse) FirstMessage

func (r *ChatResponse) FirstMessage() *Message

FirstMessage returns the first choice's message if available.

type ChatStream

type ChatStream interface {
	Recv() (*ChatStreamChunk, error)
	Close() error
	Metrics() metrics.CallMetrics
}

ChatStream provides pull-based access to streaming deltas. Recv returns io.EOF when the stream ends.

type ChatStreamChunk

type ChatStreamChunk struct {
	Index        int         `json:"index"`
	Role         MessageRole `json:"role"`
	Delta        string      `json:"delta"`
	FinishReason string      `json:"finish_reason,omitempty"`
}

ChatStreamChunk represents an incremental delta from a streaming chat call.

type Choice

type Choice struct {
	Index        int     `json:"index"`
	Message      Message `json:"message"`
	FinishReason string  `json:"finish_reason"`
}

Choice represents a single candidate completion.

type ClientConfig

type ClientConfig struct {
	Provider     ProviderType
	APIKey       string
	BaseURL      string
	Organization string // OpenAI only
	Project      string // OpenAI only
}

ClientConfig holds configuration for creating LLM clients

type ClientFactory

type ClientFactory func(ClientConfig) (ChatModel, error)

ClientFactory is a function that creates ChatModel instances

type FunctionCall

type FunctionCall struct {
	Name      string          `json:"name"`
	Arguments json.RawMessage `json:"arguments"`
}

FunctionCall represents a function call with name and arguments.

type Message

type Message struct {
	Role       MessageRole `json:"role"`
	Content    string      `json:"content"`
	ToolCallID string      `json:"tool_call_id,omitempty"`
	ToolCalls  []ToolCall  `json:"tool_calls,omitempty"`
}

Message represents a single turn in a chat conversation.

func AssistantMessage

func AssistantMessage(content string) Message

AssistantMessage creates an assistant message.

func SystemMessage

func SystemMessage(content string) Message

SystemMessage creates a system message.

func ToolMessage

func ToolMessage(content string, toolCallID string) Message

ToolMessage creates a tool message.

func UserMessage

func UserMessage(content string) Message

UserMessage creates a user message.

type MessageRole

type MessageRole string

MessageRole defines the semantic role of a chat message.

const (
	RoleSystem    MessageRole = "system"
	RoleUser      MessageRole = "user"
	RoleAssistant MessageRole = "assistant"
	RoleTool      MessageRole = "tool"
)

type ProviderType

type ProviderType string

ProviderType defines supported LLM providers

const (
	ProviderOpenAI         ProviderType = "openai"
	ProviderAnthropic      ProviderType = "anthropic"
	ProviderOllama         ProviderType = "ollama"
	ProviderOpenAIResponse ProviderType = "openai-response"
)

type SourceEOF

type SourceEOF struct {
	Source string
}

SourceEOF indicates that a named source has ended.

type StreamReader

type StreamReader = ChatStream

StreamReader is an alias for ChatStream used by stream utilities and chains.

func MergeNamedStreamReaders

func MergeNamedStreamReaders(sources map[string]StreamReader) StreamReader

MergeNamedStreamReaders combines multiple named streams. Emits SourceEOF when each named source ends.

func MergeStreamReaders

func MergeStreamReaders(readers ...StreamReader) StreamReader

MergeStreamReaders combines multiple streams into one. Reads from each stream in order until all are exhausted.

func NewConvertedReader

func NewConvertedReader(ctx context.Context, upstream StreamReader, converter func(context.Context, *ChatStreamChunk) (*ChatStreamChunk, error)) StreamReader

NewConvertedReader creates a stream that transforms chunks.

func StreamReaderFromArray

func StreamReaderFromArray(chunks []*ChatStreamChunk) StreamReader

StreamReaderFromArray wraps a slice as a stream. Useful for testing or converting static data to streams.

func TransformStream

func TransformStream(ctx context.Context, reader StreamReader, transformer StreamTransformer) StreamReader

TransformStream applies a transformer to each chunk in a stream.

type StreamTransformer

type StreamTransformer interface {
	Transform(ctx context.Context, chunk *ChatStreamChunk) (*ChatStreamChunk, error)
}

StreamTransformer transforms stream chunks.

type StreamTransformerFunc

type StreamTransformerFunc func(ctx context.Context, chunk *ChatStreamChunk) (*ChatStreamChunk, error)

StreamTransformerFunc is a function that transforms stream chunks.

func (StreamTransformerFunc) Transform

Transform implements StreamTransformer.

type StreamWriter

type StreamWriter interface {
	// Send sends a chunk to the stream.
	Send(chunk *ChatStreamChunk, err error) error
	// Close closes the stream.
	Close() error
}

StreamWriter provides a write interface for streaming data.

type TokenUsage

type TokenUsage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

TokenUsage reports token accounting from the provider.

type Tool

type Tool struct {
	Name        string                 `json:"name"`
	Description string                 `json:"description"`
	Parameters  map[string]interface{} `json:"parameters,omitempty"`
}

Tool defines a tool/function that the model can call.

type ToolCall

type ToolCall struct {
	ID       string       `json:"id"`
	Type     string       `json:"type"`
	Function FunctionCall `json:"function"`
}

ToolCall represents a tool invocation request.

type ToolChoice

type ToolChoice string

ToolChoice controls whether the model must, may, or must not call tools.

const (
	ToolChoiceAuto     ToolChoice = "auto"
	ToolChoiceRequired ToolChoice = "required"
	ToolChoiceNone     ToolChoice = "none"
)

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL