llama

package

v0.2.0 Latest Latest Go to latest Published: Jan 3, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

Details

This section is empty.

This section is empty.

This section is empty.

type Model struct {
	// contains filtered or unexported fields
}

func LoadModel(modelPath string, nThreads int) (*Model, error)

LoadModel loads a GGUF model at modelPath and returns a Model. nThreads sets how many CPU threads to use (0 = default).

func (m *Model) Close()

func (m *Model) Predict(prompt string, opts PredictOptions) (string, error)

Predict runs the model and returns the text output

type PredictOptions struct {
	MaxTokens int
	Temp      float32
	TopK      int
	TopP      float32
}

PredictOptions controls generation