llamaserver

package
v0.10.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 6, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package llamaserver wraps the OpenAI-compatible HTTP surface exposed by llama.cpp's built-in server.

Index

Constants

View Source 
const DefaultBaseURL = "http://localhost:8080/v1"

Variables

View Source 
var ProtocolCapabilities = openai.OpenAIProtocolCapabilities

ProtocolCapabilities mirrors the standard OpenAI-compatible surface.

Functions

func New 

func New(cfg Config) *openai.Provider

Types

type Config 

type Config struct {
	BaseURL      string
	APIKey       string
	Model        string
	ModelPattern string
	KnownModels  map[string]string
	Headers      map[string]string
	Reasoning    reasoning.Reasoning
}

type UtilizationProbe 

type UtilizationProbe struct {
	// contains filtered or unexported fields
}

UtilizationProbe queries llama-server observability endpoints and normalizes them into the shared endpoint utilization shape.

func NewUtilizationProbe 

func NewUtilizationProbe(baseURL string, client *http.Client) *UtilizationProbe

NewUtilizationProbe creates a probe for an OpenAI-compatible llama-server base URL.

func (*UtilizationProbe) Probe 

func (p *UtilizationProbe) Probe(ctx context.Context) utilization.EndpointUtilization

Probe first tries /metrics on the server root and falls back to /slots when metrics are unavailable.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.