vlm

package
v0.8.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 14, 2026 License: Apache-2.0, MIT Imports: 24 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrTruncatedCompletion = errors.New("VLM completion truncated")

ErrTruncatedCompletion means the model exhausted its output budget.

Functions

This section is empty.

Types

type Config

type Config struct {
	Source        types.ModelSource
	BaseURL       string
	ModelName     string
	APIKey        string
	ModelID       string
	InterfaceType string // "ollama" or "openai" (default)
	Provider      string
	// MaxConcurrency caps concurrent background calls to this model; 0 falls
	// back to the process-wide default (see limiter.GateN).
	MaxConcurrency int
	// Spec carries per-row catalog overrides (protocol, compat, levels). VLM
	// calls now go through the chat factory, so the override has to travel
	// with them — otherwise /models reports capabilities computed WITH the
	// override while the actual request is built without it.
	Spec  *types.ModelSpecOverride
	Extra map[string]any
	// CustomHeaders 允许在调用远程 API 时附加自定义 HTTP 请求头(类似 OpenAI Python SDK 的 extra_headers)。
	CustomHeaders map[string]string
	AppID         string
	AppSecret     string
}

Config holds the configuration needed to create a VLM instance.

func ConfigFromModel

func ConfigFromModel(m *types.Model, appID, appSecret string) *Config

ConfigFromModel 根据 types.Model 构造 vlm.Config。 生产路径(从 DB 拉起)和测试连接路径(临时表单)共享这份映射。 appID / appSecret 是已解密的 WeKnoraCloud 凭证,调用方负责传入。 InterfaceType 会根据 source / 模型参数自动回退到合理默认值。

type OllamaVLM

type OllamaVLM struct {
	// contains filtered or unexported fields
}

OllamaVLM implements VLM via the local Ollama service.

func NewOllamaVLM

func NewOllamaVLM(config *Config, ollamaService *ollama.OllamaService) (*OllamaVLM, error)

NewOllamaVLM creates an Ollama-backed VLM instance.

func (*OllamaVLM) GetModelID

func (v *OllamaVLM) GetModelID() string

func (*OllamaVLM) GetModelName

func (v *OllamaVLM) GetModelName() string

func (*OllamaVLM) Predict

func (v *OllamaVLM) Predict(ctx context.Context, imgBytesList [][]byte, prompt string) (string, error)

Predict sends an image with a text prompt to the Ollama vision model.

type RemoteAPIVLM

type RemoteAPIVLM struct {
	// contains filtered or unexported fields
}

RemoteAPIVLM implements VLM on top of the catalog-driven chat client: a vision call is a chat call whose user turn carries images, so the vendor dialect (max_tokens field, sampling restrictions, protocol) is resolved exactly like it is for chat models.

func NewRemoteAPIVLM

func NewRemoteAPIVLM(config *Config) (*RemoteAPIVLM, error)

NewRemoteAPIVLM creates a remote-API backed VLM instance.

func (*RemoteAPIVLM) GetModelID

func (v *RemoteAPIVLM) GetModelID() string

func (*RemoteAPIVLM) GetModelName

func (v *RemoteAPIVLM) GetModelName() string

func (*RemoteAPIVLM) Predict

func (v *RemoteAPIVLM) Predict(ctx context.Context, imgBytesList [][]byte, prompt string) (string, error)

Predict sends images with a text prompt through the chat client.

type VLM

type VLM interface {
	// Predict sends one or more images with a text prompt to the VLM and returns the generated text.
	Predict(ctx context.Context, imgBytes [][]byte, prompt string) (string, error)

	GetModelName() string
	GetModelID() string
}

VLM defines the interface for Vision Language Model operations.

func NewVLM

func NewVLM(config *Config, ollamaService *ollama.OllamaService) (VLM, error)

NewVLM creates a VLM instance based on the provided configuration.

func NewVLMFromLegacyConfig

func NewVLMFromLegacyConfig(vlmCfg types.VLMConfig, ollamaService *ollama.OllamaService) (VLM, error)

NewVLMFromLegacyConfig creates a VLM from a legacy VLMConfig (inline BaseURL/APIKey/ModelName).

type WeKnoraCloudVLM

type WeKnoraCloudVLM struct {
	// contains filtered or unexported fields
}

WeKnoraCloudVLM implements VLM via the WeKnoraCloud API.

func NewWeKnoraCloudVLM

func NewWeKnoraCloudVLM(config *Config) (*WeKnoraCloudVLM, error)

NewWeKnoraCloudVLM creates a WeKnoraCloud-backed VLM instance.

func (*WeKnoraCloudVLM) GetModelID

func (v *WeKnoraCloudVLM) GetModelID() string

func (*WeKnoraCloudVLM) GetModelName

func (v *WeKnoraCloudVLM) GetModelName() string

func (*WeKnoraCloudVLM) Predict

func (v *WeKnoraCloudVLM) Predict(ctx context.Context, imgBytesList [][]byte, prompt string) (string, error)

Predict sends images with a text prompt to the WeKnoraCloud API.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL