models

package

v0.153.1 Latest Latest Go to latest Published: Jul 26, 2026 License: Apache-2.0 Imports: 3 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/inference-gateway/cli

Links

Open Source Insights

Documentation ¶

Overview ¶

Package models provides utilities for working with LLM models.

Index ¶

func LookupContextWindow(model string) (int, bool)
func SetGatewayContextWindows(windows map[string]int)

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func LookupContextWindow ¶ added in v0.104.1

func LookupContextWindow(model string) (int, bool)

LookupContextWindow returns the matched context window size and whether a real match was found (from user config override or gateway data). Unknown models return (0, false) - there is no built-in fallback, so callers must gate window-dependent features on the second return.

func SetGatewayContextWindows ¶ added in v0.151.0

func SetGatewayContextWindows(windows map[string]int)

SetGatewayContextWindows replaces the gateway-reported context windows (from /v1/models?include=context_window). Keys are full "provider/model" ids; matching is exact on the lowercased id.

Types ¶

This section is empty.

Source Files ¶

View all Source files

context.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL