llama

package
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 5, 2026 License: MIT Imports: 29 Imported by: 0

Documentation

Overview

Package llama ports Pi's built-in llama.cpp extension (packages/coding-agent/src/extensions/llama/): the router HTTP/SSE client, Hugging Face search and download, the dynamic provider, the manager UI, and the /llama command.

Index

Constants

View Source
const (
	CommandName        = "llama"
	CommandDescription = "Manage llama.cpp router models"
)

CommandName and CommandDescription mirror the pi.registerCommand call.

View Source
const DefaultLlamaServerURL = "http://127.0.0.1:8080"

DefaultLlamaServerURL mirrors DEFAULT_LLAMA_SERVER_URL.

View Source
const LlamaProviderID = "llama.cpp"

LlamaProviderID mirrors LLAMA_PROVIDER_ID.

Variables

This section is empty.

Functions

func FindHuggingFaceToken

func FindHuggingFaceToken(env func(string) string) string

FindHuggingFaceToken mirrors findHuggingFaceToken; env reads one variable, with "" meaning unset.

func FormatBytes

func FormatBytes(bytes float64) string

FormatBytes mirrors client.ts formatBytes.

func LlamaInferenceURL

func LlamaInferenceURL(serverURL string) (string, error)

LlamaInferenceURL mirrors client.ts llamaInferenceUrl.

func NormalizeLlamaServerURL

func NormalizeLlamaServerURL(value string) (string, error)

NormalizeLlamaServerURL mirrors client.ts normalizeLlamaServerUrl.

func RunWithProgress

func RunWithProgress[T any](ui LlamaUi, options RunWithProgressOptions[T]) (value T, cancelled bool, err error)

RunWithProgress mirrors runWithProgress: it shows progress while Run works and, when the user confirms a stop, calls Cancel and aborts Run with "Cancelled". Run is an owned goroutine joined before this returns. Progress updates and view mounting serialize so an initial state cannot overwrite a newer update.

func ShowLlamaUi

func ShowLlamaUi(ctx CommandContext, run func(ui LlamaUi) error)

ShowLlamaUi mirrors showLlamaUi: it shows a LlamaView through ctx.Custom and runs the flow on an owned goroutine that ends the view when it returns. A flow error is reported through Notify.

Types

type APIKeyAuth

type APIKeyAuth struct {
	Name    string
	Login   func(interaction AuthInteraction) (ai.Credential, error)
	Check   func(ctx context.Context, auth AuthContext, credential *ai.Credential) (*AuthCheck, error)
	Resolve func(ctx context.Context, auth AuthContext, credential *ai.Credential) (*AuthResult, error)
}

APIKeyAuth mirrors pi-ai ApiKeyAuth.

type AnyModel

type AnyModel interface {
	// contains filtered or unexported methods
}

AnyModel is a model the provider lists: Model (chat) or ClassifierModel.

type AuthCheck

type AuthCheck struct {
	Source string
	Type   string
}

AuthCheck mirrors pi-ai AuthCheck.

type AuthContext

type AuthContext struct {
	Env func(name string) (string, bool)
}

AuthContext mirrors pi-ai AuthContext. Env reports ok=false for an unset or blank variable, like defaultProviderAuthContext.

func DefaultAuthContext

func DefaultAuthContext() AuthContext

DefaultAuthContext mirrors defaultProviderAuthContext over the process environment.

type AuthInteraction

type AuthInteraction struct {
	Ctx    context.Context
	Prompt func(AuthPrompt) (string, error)
}

AuthInteraction mirrors ProviderAuthInteraction; Ctx carries its signal.

type AuthPrompt

type AuthPrompt struct {
	Type        string
	Message     string
	Placeholder string
}

AuthPrompt mirrors the text and secret members of pi-ai AuthPrompt, the only kinds this provider asks for.

type AuthResult

type AuthResult struct {
	Auth   ModelAuth
	Env    map[string]string
	Source string
}

AuthResult mirrors pi-ai AuthResult.

type ClassifierModel

type ClassifierModel struct {
	Type          string    `json:"type"`
	ID            string    `json:"id"`
	Name          string    `json:"name"`
	API           string    `json:"api"`
	Provider      string    `json:"provider"`
	BaseURL       string    `json:"baseUrl"`
	Input         []string  `json:"input"`
	Cost          ModelCost `json:"cost"`
	ContextWindow int       `json:"contextWindow"`
}

ClassifierModel mirrors the pi-ai ClassifierModel<"llama-cpp-classify"> the provider publishes and persists.

type CommandContext

type CommandContext struct {
	Ctx    context.Context
	Mode   string
	Notify func(message, notifyType string)
	// Custom shows the component the factory returns in the editor slot and
	// blocks until the factory's done callback runs, like ctx.ui.custom.
	Custom func(factory func(requestRender, done func()) CustomComponent)
}

CommandContext mirrors the ExtensionCommandContext members /llama uses. Notify may be called from the flow goroutine while Custom is showing.

type CustomComponent

type CustomComponent interface {
	tui.Component
	HandleInput(data string)
}

CustomComponent is the editor-slot component a command shows, mirroring the component ctx.ui.custom factories return.

type Host

type Host struct {
	// contains filtered or unexported fields
}

Host binds the llama.cpp provider to PiG's model registry, credential store, and models store. It ports the parts of Pi's model runtime (packages/ai/src/models.ts refresh, publishProviderModels, and getAuth) that a registered native provider relies on, for this one provider.

func NewHost

func NewHost(registry providerRegistry, credentials *ai.AuthStorage, store *ai.FileModelsStore) *Host

NewHost creates the llama.cpp provider and registers it, mirroring the built-in extension's pi.registerProvider call at startup.

func (*Host) CheckAuth

func (h *Host) CheckAuth(ctx context.Context) (*AuthCheck, error)

CheckAuth mirrors the side-effect-free availability check.

func (*Host) GetProviderAuth

func (h *Host) GetProviderAuth(ctx context.Context) (*AuthResult, error)

GetProviderAuth mirrors ModelRegistry.getProviderAuth for llama.cpp: the stored api-key credential resolved with the ambient environment.

func (*Host) HandleCommand

func (h *Host) HandleCommand(ctx CommandContext) error

HandleCommand mirrors the /llama command handler.

func (*Host) Provider

func (h *Host) Provider() *Provider

Provider returns the registered provider.

func (*Host) Refresh

func (h *Host) Refresh(ctx context.Context, allowNetwork bool) RefreshResult

Refresh mirrors Models.refresh for the llama.cpp provider: a cache-only phase restores the stored catalog, then, when allowNetwork is set, a network phase runs with the resolved credential. A newer refresh supersedes an older one, whose publications are then dropped.

func (*Host) SyncRegistration

func (h *Host) SyncRegistration(ctx context.Context)

SyncRegistration republishes the provider's catalog and resolved auth. An unresolved provider keeps no API key, so the registry reports it as not configured and hides its models.

type HuggingFaceClient

type HuggingFaceClient struct {
	// contains filtered or unexported fields
}

HuggingFaceClient mirrors huggingface.ts HuggingFaceClient.

func NewHuggingFaceClient

func NewHuggingFaceClient(token, baseURL string) *HuggingFaceClient

NewHuggingFaceClient mirrors the constructor; an empty baseURL selects https://huggingface.co.

func (*HuggingFaceClient) Details

Details mirrors HuggingFaceClient.details.

func (*HuggingFaceClient) Search

func (c *HuggingFaceClient) Search(ctx context.Context, query string) ([]HuggingFaceModel, error)

Search mirrors HuggingFaceClient.search.

type HuggingFaceGated

type HuggingFaceGated string

HuggingFaceGated mirrors `false | "auto" | "manual"`; the empty value is false.

type HuggingFaceModel

type HuggingFaceModel struct {
	ID        string
	Downloads float64
}

HuggingFaceModel mirrors one search result.

type HuggingFaceModelDetails

type HuggingFaceModelDetails struct {
	ID            string
	Gated         HuggingFaceGated
	Quantizations []HuggingFaceQuantization
}

HuggingFaceModelDetails mirrors the model details upstream reads.

type HuggingFaceQuantization

type HuggingFaceQuantization struct {
	Name string
	Size *float64
}

HuggingFaceQuantization mirrors one GGUF quantization. Size is nil when any file of the quantization has no reported size.

type HuggingFaceSearch

type HuggingFaceSearch struct {
	// contains filtered or unexported fields
}

HuggingFaceSearch mirrors ui.ts HuggingFaceSearch. Its state is guarded by the owning view's lock; the debounce timer and search goroutine take it.

func (*HuggingFaceSearch) HandleInput

func (s *HuggingFaceSearch) HandleInput(data string)

HandleInput mirrors HuggingFaceSearch.handleInput; the caller holds the view lock.

func (*HuggingFaceSearch) Invalidate

func (s *HuggingFaceSearch) Invalidate()

Invalidate invalidates the prompt, input, and results.

func (*HuggingFaceSearch) Render

func (s *HuggingFaceSearch) Render(width int) []string

Render draws the prompt, input, and results. The component reports no dirty state, so its container renders it every frame.

type HuggingFaceSearchFunc

type HuggingFaceSearchFunc func(ctx context.Context, query string) ([]HuggingFaceModel, error)

HuggingFaceSearchFunc performs one Hugging Face query.

type LlamaClient

type LlamaClient struct {
	ServerURL string
	// contains filtered or unexported fields
}

LlamaClient mirrors client.ts LlamaClient.

func NewLlamaClient

func NewLlamaClient(serverURL, apiKey string) (*LlamaClient, error)

NewLlamaClient mirrors the LlamaClient constructor, which normalizes the server URL and may throw.

func (*LlamaClient) Download

func (c *LlamaClient) Download(ctx context.Context, model string) error

Download mirrors LlamaClient.download.

func (*LlamaClient) DownloadAndWait

func (c *LlamaClient) DownloadAndWait(ctx context.Context, model string, onProgress func(LlamaProgress)) ([]LlamaModelInfo, error)

DownloadAndWait mirrors LlamaClient.downloadAndWait.

func (*LlamaClient) List

func (c *LlamaClient) List(ctx context.Context, reload bool) ([]LlamaModelInfo, error)

List mirrors LlamaClient.list.

func (*LlamaClient) Load

func (c *LlamaClient) Load(ctx context.Context, model string) error

Load mirrors LlamaClient.load.

func (*LlamaClient) LoadAndWait

func (c *LlamaClient) LoadAndWait(ctx context.Context, model string, onProgress func(LlamaProgress)) (LlamaModelInfo, error)

LoadAndWait mirrors LlamaClient.loadAndWait.

func (*LlamaClient) Props

func (c *LlamaClient) Props(ctx context.Context, model string) (LlamaServerProps, error)

Props mirrors LlamaClient.props.

func (*LlamaClient) Unload

func (c *LlamaClient) Unload(ctx context.Context, model string) error

Unload mirrors LlamaClient.unload.

func (*LlamaClient) UnloadAndWait

func (c *LlamaClient) UnloadAndWait(ctx context.Context, model string) error

UnloadAndWait mirrors LlamaClient.unloadAndWait.

func (*LlamaClient) Watch

func (c *LlamaClient) Watch(ctx context.Context, onEvent func(LlamaModelEvent)) error

Watch mirrors LlamaClient.watch: it reads /models/sse until the stream ends or ctx is cancelled.

type LlamaManagerAction

type LlamaManagerAction struct {
	Type  LlamaManagerActionType
	Model LlamaModelInfo
}

LlamaManagerAction mirrors LlamaManagerAction; Model is set for "model".

type LlamaManagerActionType

type LlamaManagerActionType string

LlamaManagerActionType mirrors the LlamaManagerAction union tags.

const (
	LlamaManagerActionModel    LlamaManagerActionType = "model"
	LlamaManagerActionDownload LlamaManagerActionType = "download"
	LlamaManagerActionClose    LlamaManagerActionType = "close"
)

type LlamaModelArchitecture

type LlamaModelArchitecture struct {
	InputModalities  []string
	OutputModalities []string
}

LlamaModelArchitecture mirrors LlamaModelInfo.architecture.

type LlamaModelEvent

type LlamaModelEvent struct {
	Model string
	Event string
	Data  any
}

LlamaModelEvent mirrors one /models/sse event.

type LlamaModelInfo

type LlamaModelInfo struct {
	ID           string
	Aliases      []string
	Status       LlamaModelInfoStatus
	Architecture *LlamaModelArchitecture
	Source       string
	Meta         *LlamaModelMeta
}

LlamaModelInfo mirrors one entry of the router's /models catalog.

type LlamaModelInfoStatus

type LlamaModelInfoStatus struct {
	Value    LlamaModelStatus
	Args     []string
	Failed   bool
	ExitCode *float64
	Progress any
}

LlamaModelInfoStatus mirrors LlamaModelInfo.status. Progress keeps the raw decoded JSON object because upstream only sums its entries.

type LlamaModelMeta

type LlamaModelMeta struct {
	NCtx      *float64
	NCtxTrain *float64
	Size      *float64
	Ftype     string
}

LlamaModelMeta mirrors LlamaModelInfo.meta. Pointers keep an absent value distinct from zero for the `??` reads upstream performs.

type LlamaModelStatus

type LlamaModelStatus string

LlamaModelStatus mirrors the llama.cpp router model status union.

const (
	LlamaModelStatusUnloaded    LlamaModelStatus = "unloaded"
	LlamaModelStatusLoading     LlamaModelStatus = "loading"
	LlamaModelStatusLoaded      LlamaModelStatus = "loaded"
	LlamaModelStatusDownloading LlamaModelStatus = "downloading"
	LlamaModelStatusSleeping    LlamaModelStatus = "sleeping"
)

type LlamaProgress

type LlamaProgress struct {
	Message string
	Ratio   *float64
	Detail  string
	// contains filtered or unexported fields
}

LlamaProgress mirrors a load or download progress update. Ratio is nil when upstream leaves it undefined.

type LlamaProviderController

type LlamaProviderController struct {
	Provider *Provider
	// contains filtered or unexported fields
}

LlamaProviderController mirrors LlamaProviderController.

func CreateLlamaProvider

func CreateLlamaProvider() *LlamaProviderController

CreateLlamaProvider mirrors createLlamaProvider.

func (*LlamaProviderController) SetCatalog

func (c *LlamaProviderController) SetCatalog(catalog []LlamaModelInfo, serverURL string, routerAutoload bool)

SetCatalog mirrors LlamaProviderController.setCatalog.

type LlamaServerProps

type LlamaServerProps struct {
	ModelsAutoload bool
	ChatTemplate   string
}

LlamaServerProps mirrors the /props fields upstream reads.

type LlamaUi

type LlamaUi interface {
	ShowModels(serverURL string, models []LlamaModelInfo) LlamaManagerAction
	Select(title string, options []string) (string, bool)
	Confirm(title, message string) bool
	ConnectionError(serverURL, message string) string
	SearchModels(search HuggingFaceSearchFunc) (string, bool)
	ShowStatus(title, message string)
	Progress(state ProgressState) <-chan struct{}
	UpdateProgress(state ProgressState)
}

LlamaUi mirrors ui.ts LlamaUi. Blocking methods return when the user answers; Progress returns a channel closed when the user asks to stop.

type LlamaView

type LlamaView struct {
	// contains filtered or unexported fields
}

LlamaView mirrors ui.ts LlamaView. The command's flow goroutine calls the LlamaUi methods while the interactive loop calls Render and HandleInput; one lock orders both, as the single JavaScript event loop does upstream.

func NewLlamaView

func NewLlamaView(requestRender func()) *LlamaView

NewLlamaView creates the manager view; requestRender asks the host to repaint after the flow changes the view.

func (*LlamaView) Confirm

func (v *LlamaView) Confirm(title, message string) bool

Confirm mirrors LlamaView.confirm.

func (*LlamaView) ConnectionError

func (v *LlamaView) ConnectionError(serverURL, message string) string

ConnectionError mirrors LlamaView.connectionError.

func (*LlamaView) HandleInput

func (v *LlamaView) HandleInput(data string)

HandleInput mirrors LlamaView.handleInput.

func (*LlamaView) Invalidate

func (v *LlamaView) Invalidate()

Invalidate mirrors LlamaView.invalidate.

func (*LlamaView) Progress

func (v *LlamaView) Progress(state ProgressState) <-chan struct{}

Progress mirrors LlamaView.progress: repeated calls share one pending stop signal until the user presses cancel or the content changes.

func (*LlamaView) Render

func (v *LlamaView) Render(width int) []string

Render mirrors LlamaView.render: content lines truncated to width.

func (*LlamaView) SearchModels

func (v *LlamaView) SearchModels(search HuggingFaceSearchFunc) (string, bool)

SearchModels mirrors LlamaView.searchModels. It returns after the search component closes and its pending search has stopped.

func (*LlamaView) Select

func (v *LlamaView) Select(title string, options []string) (string, bool)

Select mirrors LlamaView.select.

func (*LlamaView) ShowModels

func (v *LlamaView) ShowModels(serverURL string, models []LlamaModelInfo) LlamaManagerAction

ShowModels mirrors LlamaView.showModels.

func (*LlamaView) ShowStatus

func (v *LlamaView) ShowStatus(title, message string)

ShowStatus mirrors LlamaView.showStatus.

func (*LlamaView) UpdateProgress

func (v *LlamaView) UpdateProgress(state ProgressState)

UpdateProgress mirrors LlamaView.updateProgress.

type Model

type Model struct {
	ID               string            `json:"id"`
	Name             string            `json:"name"`
	API              string            `json:"api"`
	Provider         string            `json:"provider"`
	BaseURL          string            `json:"baseUrl"`
	Reasoning        bool              `json:"reasoning"`
	ThinkingLevelMap *ThinkingLevelMap `json:"thinkingLevelMap,omitempty"`
	Input            []string          `json:"input"`
	Cost             ModelCost         `json:"cost"`
	ContextWindow    int               `json:"contextWindow"`
	MaxTokens        int               `json:"maxTokens"`
	Compat           ModelCompat       `json:"compat"`
}

Model mirrors the pi-ai Model<"openai-completions"> the provider publishes and persists, in upstream field order.

type ModelAuth

type ModelAuth struct {
	APIKey  string
	BaseURL string
}

ModelAuth mirrors pi-ai ModelAuth.

type ModelCompat

type ModelCompat struct {
	SupportsStore            bool   `json:"supportsStore"`
	SupportsDeveloperRole    bool   `json:"supportsDeveloperRole"`
	SupportsReasoningEffort  bool   `json:"supportsReasoningEffort"`
	SupportsUsageInStreaming bool   `json:"supportsUsageInStreaming"`
	SupportsStrictMode       bool   `json:"supportsStrictMode"`
	MaxTokensField           string `json:"maxTokensField"`
	ThinkingFormat           string `json:"thinkingFormat,omitempty"`
	ThinkingTokenBudgetField string `json:"thinkingTokenBudgetField,omitempty"`
}

ModelCompat mirrors the openai-completions compat object toPiModel builds.

type ModelCost

type ModelCost struct {
	Input      float64 `json:"input"`
	Output     float64 `json:"output"`
	CacheRead  float64 `json:"cacheRead"`
	CacheWrite float64 `json:"cacheWrite"`
}

ModelCost mirrors Model.cost.

type ModelsPublication

type ModelsPublication struct {
	Persist *ai.ModelsStoreEntry
	Update  func()
}

ModelsPublication mirrors pi-ai ModelsPublication; a nil Persist leaves storage unchanged.

type ProgressState

type ProgressState struct {
	LlamaProgress
	Title string
	Model string
}

ProgressState mirrors ui.ts ProgressState.

type Provider

type Provider struct {
	ID           string
	Name         string
	BaseURL      string
	API          string
	APIKey       APIKeyAuth
	GetModels    func() []Model
	GetAllModels func() []AnyModel
	// Classify is the provider object's classifier operation. upstream: provider.ts:262 (classify)
	Classify      func(ctx context.Context, model ClassifierModel, request ai.ClassifierContext, options ai.ClassifierOptions) ai.ClassifierResult
	RefreshModels func(refresh RefreshModelsContext) error
}

Provider mirrors the pi-ai Provider object createLlamaProvider returns. Streaming uses the host's openai-completions implementation for API.

type RefreshModelsContext

type RefreshModelsContext struct {
	Ctx          context.Context
	Credential   *ai.Credential
	Stored       *ai.ModelsStoreEntry
	Publish      func(ModelsPublication) (bool, error)
	AllowNetwork bool
}

RefreshModelsContext mirrors pi-ai RefreshModelsContext.

type RefreshResult

type RefreshResult struct {
	Aborted bool
	Err     error
}

RefreshResult mirrors ModelsRefreshResult for this provider.

type RunWithProgressOptions

type RunWithProgressOptions[T any] struct {
	Title          string
	Model          string
	InitialMessage string
	CancelTitle    string
	CancelMessage  string
	Run            func(ctx context.Context, update func(LlamaProgress)) (T, error)
	Cancel         func() error
}

RunWithProgressOptions mirrors runWithProgress's options.

type ThinkingLevelMap

type ThinkingLevelMap struct {
	Off     *string `json:"off"`
	Minimal *string `json:"minimal"`
	Low     *string `json:"low"`
	Medium  *string `json:"medium"`
	High    *string `json:"high"`
	XHigh   *string `json:"xhigh"`
}

ThinkingLevelMap mirrors the thinkingLevelMap object toPiModel builds, in upstream key order; nil entries serialize as null.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL