common

func CalculateLogprobs ¶ added in v0.12.11

func CalculateLogprobs(logits []float32, selectedToken int, topK int, decoder TokenDecoderFunc) []llm.Logprob

CalculateLogprobs converts raw logits to log probabilities and finds top K tokens. It uses numerically stable softmax to compute log probabilities.

func ContainsStopSuffix ¶

func ContainsStopSuffix(sequence string, stops []string) bool

func FindStop ¶

func FindStop(sequence string, stops []string) (bool, string)

func IncompleteUnicode ¶

func IncompleteUnicode(token string) bool

func TruncateStop ¶

func TruncateStop(pieces []string, stop string) ([]string, bool)

TruncateStop removes the provided stop string from pieces, returning the partial pieces with stop removed, including truncating the last piece if required (and signalling if this was the case)