Affected by 
Documentation
¶
Index ¶
- func CreateImageGenModel(modelName, modelDir, quantize string, createLayer LayerCreator, ...) error
- func CreateSafetensorsModel(modelName, modelDir, quantize string, createLayer LayerCreator, ...) error
- func GetModelArchitecture(modelName string) (string, error)
- func IsImageGenModel(modelName string) bool
- func IsSafetensorsLLMModel(modelName string) bool
- func IsSafetensorsModel(modelName string) bool
- func IsSafetensorsModelDir(dir string) bool
- func IsTensorModelDir(dir string) bool
- func ShouldQuantize(name, component string) bool
- func ShouldQuantizeTensor(name string, shape []int32) bool
- type LayerCreator
- type LayerInfo
- type Manifest
- type ManifestLayer
- type ManifestWriter
- type ModelConfig
- type QuantizingTensorLayerCreator
- type TensorLayerCreator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CreateImageGenModel ¶
func CreateImageGenModel(modelName, modelDir, quantize string, createLayer LayerCreator, createTensorLayer QuantizingTensorLayerCreator, writeManifest ManifestWriter, fn func(status string)) error
CreateImageGenModel imports an image generation model from a directory. Stores each tensor as a separate blob for fine-grained deduplication. If quantize is specified, linear weights in transformer/text_encoder are quantized. Supported quantization types: fp8 (or empty for no quantization). Layer creation and manifest writing are done via callbacks to avoid import cycles.
func CreateSafetensorsModel ¶
func CreateSafetensorsModel(modelName, modelDir, quantize string, createLayer LayerCreator, createTensorLayer QuantizingTensorLayerCreator, writeManifest ManifestWriter, fn func(status string)) error
CreateSafetensorsModel imports a standard safetensors model from a directory. This handles Hugging Face style models with config.json and *.safetensors files. Stores each tensor as a separate blob for fine-grained deduplication. If quantize is non-empty (e.g., "fp8"), eligible tensors will be quantized.
func GetModelArchitecture ¶
GetModelArchitecture returns the architecture from the model's config.json layer.
func IsImageGenModel ¶
IsImageGenModel checks if a model is an image generation model (has image capability).
func IsSafetensorsLLMModel ¶
IsSafetensorsLLMModel checks if a model is a safetensors LLM model (has completion capability, not image generation).
func IsSafetensorsModel ¶
IsSafetensorsModel checks if a model was created with the experimental safetensors builder by checking the model format in the config.
func IsSafetensorsModelDir ¶
IsSafetensorsModelDir checks if the directory contains a standard safetensors model by looking for config.json and at least one .safetensors file.
func IsTensorModelDir ¶
IsTensorModelDir checks if the directory contains a diffusers-style tensor model by looking for model_index.json, which is the standard diffusers pipeline config.
func ShouldQuantize ¶
ShouldQuantize returns true if a tensor should be quantized. For image gen models (component non-empty): quantizes linear weights, skipping VAE, embeddings, norms. For LLM models (component empty): quantizes linear weights, skipping embeddings, norms, and small tensors.
func ShouldQuantizeTensor ¶
ShouldQuantizeTensor returns true if a tensor should be quantized based on name and shape. This is a more detailed check that also considers tensor dimensions.
Types ¶
type LayerCreator ¶
LayerCreator is called to create a blob layer. name is the path-style name (e.g., "tokenizer/tokenizer.json")
type LayerInfo ¶
type LayerInfo struct {
Digest string
Size int64
MediaType string
Name string // Path-style name: "component/tensor" or "path/to/config.json"
}
LayerInfo holds metadata for a created layer.
type Manifest ¶
type Manifest struct {
SchemaVersion int `json:"schemaVersion"`
MediaType string `json:"mediaType"`
Config ManifestLayer `json:"config"`
Layers []ManifestLayer `json:"layers"`
}
Manifest represents the manifest JSON structure.
type ManifestLayer ¶
type ManifestLayer struct {
MediaType string `json:"mediaType"`
Digest string `json:"digest"`
Size int64 `json:"size"`
Name string `json:"name,omitempty"`
}
ManifestLayer represents a layer in the manifest.
type ManifestWriter ¶
ManifestWriter writes the manifest file.
type ModelConfig ¶
type ModelConfig struct {
ModelFormat string `json:"model_format"`
Capabilities []string `json:"capabilities"`
}
ModelConfig represents the config blob stored with a model.
type QuantizingTensorLayerCreator ¶
type QuantizingTensorLayerCreator func(r io.Reader, name, dtype string, shape []int32, quantize string) ([]LayerInfo, error)
QuantizingTensorLayerCreator creates tensor layers with optional quantization. When quantize is non-empty (e.g., "fp8"), returns multiple layers (weight + scales + biases).
Source Files
Directories