Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var ( // Config file patterns - supported configuration file extensions. ConfigFilePatterns = []string{ "*.json", "*.jsonl", "*.yaml", "*.yml", "*.toml", "*.ini", "*.config", "*.modelcard", "*.meta", "*tokenizer.model*", "config.json.*", } // Model file patterns - supported model file extensions. ModelFilePatterns = []string{ "*.safetensors", "*.bin", "*.pt", "*.pth", "*.tflite", "*.h5", "*.hdf", "*.hdf5", "*.ot", "*.engine", "*.trt", "*.onnx", "*.gguf", "*.msgpack", "*.model", } // Code file patterns - supported script and notebook files. CodeFilePatterns = []string{ "*.py", "*.sh", "*.ipynb", "*.patch", } // Doc file patterns - supported documentation files DocFilePatterns = []string{ "*.txt", "*.md", "*.pdf", "LICENSE*", "README*", "SETUP*", "*requirements*", "*.jpg", "*.jpeg", "*.png", "*.gif", "*.bmp", "*.tiff", "*.ico", } )
Functions ¶
func IsFileType ¶
IsFileType checks if the filename matches any of the given patterns
Types ¶
type Modelfile ¶
type Modelfile interface {
// GetConfigs returns the args of the config command in the modelfile,
// and deduplicates the args. The order of the args is the same as the
// order in the modelfile.
GetConfigs() []string
// GetModels returns the args of the model command in the modelfile,
// and deduplicates the args. The order of the args is the same as the
// order in the modelfile.
GetModels() []string
// GetCode returns the args of the code command in the modelfile,
// and deduplicates the args. The order of the args is the same as the
// order in the modelfile.
GetCodes() []string
// GetDatasets returns the args of the dataset command in the modelfile,
// and deduplicates the args. The order of the args is the same as the
// order in the modelfile.
GetDatasets() []string
// GetDocs returns the args of the doc command in the modelfile,
// and deduplicates the args. The order of the args is the same as the
// order in the modelfile.
GetDocs() []string
// GetName returns the value of the name command in the modelfile.
GetName() string
// GetArch returns the value of the arch command in the modelfile.
GetArch() string
// GetFamily returns the value of the family command in the modelfile.
GetFamily() string
// GetFormat returns the value of the format command in the modelfile.
GetFormat() string
// GetParamsize returns the value of the paramsize command in the modelfile.
GetParamsize() string
// GetPrecision returns the value of the precision command in the modelfile.
GetPrecision() string
// GetQuantization returns the value of the quantization command in the modelfile.
GetQuantization() string
// Content returns the content of the modelfile.
Content() []byte
}
Modelfile is the interface for the modelfile. It is used to parse the modelfile by the path and get the information of the modelfile.
func NewModelfile ¶
NewModelfile creates a new modelfile by the path of the modelfile. It parses the modelfile and returns the modelfile interface.
func NewModelfileByWorkspace ¶
func NewModelfileByWorkspace(workspace string, config *configmodelfile.GenerateConfig) (Modelfile, error)
NewModelfileByWorkspace creates a new modelfile by the workspace.
It generates the modelfile by the following steps:
- It walks the workspace and gets the files, and generates the modelfile by the files.
- It generates the modelfile by the model config, such as config.json and generation_config.json.
- It generates the modelfile by the generate config, such as name, arch, family, format, paramsize, precision, and quantization.
Source Files
Directories
Click to show internal directories.
Click to hide internal directories.