llms

package module

v0.0.1 Latest Latest Go to latest Published: Feb 27, 2025 License: MIT Imports: 3 Imported by: 0

Details

View Source

const Version = "llms/v0.0.1"

This section is empty.

This section is empty.

type Model string

LLM pool specific parameter allowing to dynamically route request to models

func (Model) ChatterOpt()

type Pool struct {
	// contains filtered or unexported fields
}

LLM pool consists of default "route" and multiple named models.

func New(llms map[string]chatter.Chatter, fallback chatter.Chatter) *Pool

Creates LLMs pools instance

func (p *Pool) Prompt(ctx context.Context, prompt []fmt.Stringer, opts ...chatter.Opt) (chatter.Reply, error)

func (p *Pool) UsedInputTokens() int

func (p *Pool) UsedReplyTokens() int