Documentation
¶
Index ¶
- Constants
- func DetectArgFrom(playground *inferenceapi.Playground) string
- func FetchModelsByPlayground(ctx context.Context, k8sClient client.Client, ...) (models []*coreapi.OpenModel, err error)
- func FetchModelsByService(ctx context.Context, k8sClient client.Client, service *inferenceapi.Service) (models []*coreapi.OpenModel, err error)
- func FirstAssignedFlavor(model *coreapi.OpenModel, playground *inferenceapi.Playground) []coreapi.Flavor
- func RecommendedConfigName(playground *inferenceapi.Playground) string
- func SkipModelLoader(obj metav1.Object) bool
- type GlobalConfigs
Constants ¶
View Source
const ( DefaultArg string = "default" SpeculativeDecodingArg string = "speculative-decoding" )
These two modes are preset.
Variables ¶
This section is empty.
Functions ¶
func DetectArgFrom ¶ added in v0.0.9
func DetectArgFrom(playground *inferenceapi.Playground) string
DetectArgFrom wil auto detect the arg from model roles if not set explicitly.
func FetchModelsByPlayground ¶
func FetchModelsByPlayground(ctx context.Context, k8sClient client.Client, playground *inferenceapi.Playground) (models []*coreapi.OpenModel, err error)
func FetchModelsByService ¶
func FirstAssignedFlavor ¶ added in v0.1.0
func FirstAssignedFlavor(model *coreapi.OpenModel, playground *inferenceapi.Playground) []coreapi.Flavor
FirstAssignedFlavor will return the first assigned flavor of the model.
func RecommendedConfigName ¶ added in v0.1.1
func RecommendedConfigName(playground *inferenceapi.Playground) string
func SkipModelLoader ¶ added in v0.1.4
Types ¶
type GlobalConfigs ¶ added in v0.1.4
type GlobalConfigs struct {
SchedulerName string `yaml:"scheduler-name"`
InitContainerImage string `yaml:"init-container-image"`
}
func ParseGlobalConfigmap ¶ added in v0.1.4
func ParseGlobalConfigmap(cm *corev1.ConfigMap) (*GlobalConfigs, error)
Source Files
Directories
Click to show internal directories.
Click to hide internal directories.