Documentation
¶
Index ¶
- Constants
- func DetectArgFrom(playground *inferenceapi.Playground) string
- func FetchModelsByPlayground(ctx context.Context, k8sClient client.Client, ...) (models []*coreapi.OpenModel, err error)
- func FetchModelsByService(ctx context.Context, k8sClient client.Client, service *inferenceapi.Service) (models []*coreapi.OpenModel, err error)
- func FirstAssignedFlavor(model *coreapi.OpenModel, playground *inferenceapi.Playground) []coreapi.Flavor
- func RecommendedConfigName(playground *inferenceapi.Playground) string
Constants ¶
View Source
const ( DefaultArg string = "default" SpeculativeDecodingArg string = "speculative-decoding" )
These two modes are preset.
Variables ¶
This section is empty.
Functions ¶
func DetectArgFrom ¶ added in v0.0.9
func DetectArgFrom(playground *inferenceapi.Playground) string
DetectArgFrom wil auto detect the arg from model roles if not set explicitly.
func FetchModelsByPlayground ¶
func FetchModelsByPlayground(ctx context.Context, k8sClient client.Client, playground *inferenceapi.Playground) (models []*coreapi.OpenModel, err error)
func FetchModelsByService ¶
func FirstAssignedFlavor ¶ added in v0.1.0
func FirstAssignedFlavor(model *coreapi.OpenModel, playground *inferenceapi.Playground) []coreapi.Flavor
FirstAssignedFlavor will return the first assigned flavor of the model.
func RecommendedConfigName ¶ added in v0.1.1
func RecommendedConfigName(playground *inferenceapi.Playground) string
Types ¶
This section is empty.
Source Files
Directories
Click to show internal directories.
Click to hide internal directories.