vision

package
v0.11.0-beta.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: Apache-2.0, MIT Imports: 10 Imported by: 0

Documentation

Overview

Package vision turns image bytes into prose a text-only main agent can reason over (master plan §7.4). Used by remote gateways to inline-describe photos and by the vision_describe tool to caption browser screenshots.

We deliberately keep this separate from the providers package: the main agent's Message type is plain string today, and bolting image content blocks onto every provider for a single feature would be a large refactor for marginal gain. Instead a Describer takes bytes in, returns text out, and the caller injects the text wherever it needs it.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MIMEFromBytes

func MIMEFromBytes(b []byte) string

MIMEFromBytes sniffs PNG/JPEG/GIF/WebP magic so callers downloading arbitrary files don't have to guess. Falls back to image/png.

Types

type AnthropicDescriber

type AnthropicDescriber struct {
	APIKey    string
	Model     string        // e.g. "claude-sonnet-4-5-20250929"
	BaseURL   string        // override for tests; default https://api.anthropic.com
	HTTP      *http.Client  // overridable
	MaxTokens int           // default 1024
	Timeout   time.Duration // default 30s
}

AnthropicDescriber is the production Describer. Talks straight to the Messages API rather than going through the providers abstraction because we need image content blocks the providers.Message type doesn't carry yet.

func NewAnthropic

func NewAnthropic(apiKey, model string) *AnthropicDescriber

NewAnthropic returns a describer with sane defaults.

func (*AnthropicDescriber) Describe

func (d *AnthropicDescriber) Describe(ctx context.Context, images []Image, prompt string) (string, error)

Describe sends the images + prompt and returns the model's text. Any non-2xx surfaces with the response body for debuggability.

type Describer

type Describer interface {
	Describe(ctx context.Context, images []Image, prompt string) (string, error)
}

Describer turns one or more images plus an optional steering prompt into prose. Implementations MUST be safe to call concurrently.

type Image

type Image struct {
	Bytes []byte
	Mime  string
}

Image is one input frame to describe. Mime is the standard MIME type (image/png, image/jpeg, image/webp, image/gif). Empty Mime defaults to image/png — the dev browser screenshots are always PNG.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL