vllm

package

v1.8.0 Latest Latest Go to latest Published: Mar 10, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

Details

This section is empty.

This section is empty.

func Dimension(model string) (int, error)

Dimension returns the dimension of the model.

type Client struct {
	// contains filtered or unexported fields
}

Client wraps OpenAI client.

func NewClient(url string, log logr.Logger) *Client

NewClient creates a new OpenAI client.

func (c *Client) Embed(ctx context.Context, modelName, prompt string) ([]float32, error)

Embed creates embeddings.

func (c *Client) PullModel(ctx context.Context, modelName string) error

PullModel pulls a model.