loader

package
v0.7.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 3, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package loader provides model weight loading functionality for Born ML framework.

This package implements readers for popular model weight formats:

  • SafeTensors: Zero-copy loading with memory mapping (Hugging Face standard)
  • GGUF: Efficient format for quantized models (llama.cpp ecosystem)

Supported architectures:

  • LLaMA (including LLaMA 2, LLaMA 3)
  • Mistral (7B, 8x7B MoE)
  • DeepSeek (DeepSeek-V2, DeepSeek-Coder)

The loader package focuses on inference use cases for v0.5.0 LLM support. Training is not supported in this version.

Example: 

// Auto-detect format and load model
model, err := loader.OpenModel("path/to/model.safetensors")
if err != nil {
    log.Fatal(err)
}
defer model.Close()

// Load specific tensor
tensor, err := model.LoadTensor("model.layers.0.attn.q_proj.weight")
if err != nil {
    log.Fatal(err)
}

Design principles:

  • Pure Go: No CGO dependencies
  • Zero-copy: Memory mapping when possible (Unix/Darwin)
  • Lazy loading: Load tensors on-demand
  • Type safety: Proper dtype handling (F16, BF16, F32, Q4_0, Q8_0)

Index

Constants

View Source 
const (
	ArchitectureLLaMA    = "llama"
	ArchitectureMistral  = "mistral"
	ArchitectureDeepSeek = "deepseek"
)

Architecture names.

Variables

This section is empty.

Functions

func DetectArchitecture 

func DetectArchitecture(names []string) string

DetectArchitecture attempts to detect model architecture from weight names.

Types

type DeepSeekMapper 

type DeepSeekMapper struct{}

DeepSeekMapper maps DeepSeek weight names to Born standard names. Supports DeepSeek-V2 and DeepSeek-Coder.

func NewDeepSeekMapper 

func NewDeepSeekMapper() *DeepSeekMapper

NewDeepSeekMapper creates a new DeepSeek weight mapper.

func (*DeepSeekMapper) Architecture 

func (m *DeepSeekMapper) Architecture() string

Architecture returns "deepseek".

func (*DeepSeekMapper) MapName 

func (m *DeepSeekMapper) MapName(name string) (string, error)

MapName converts DeepSeek weight names to Born standard names. DeepSeek uses similar naming to LLaMA but may have architecture-specific differences.

type GGUFDType 

type GGUFDType uint32

GGUFDType represents GGUF tensor data types. 

const (
	GGUFDTypeF32  GGUFDType = 0
	GGUFDTypeF16  GGUFDType = 1
	GGUFDTypeQ4_0 GGUFDType = 2
	GGUFDTypeQ4_1 GGUFDType = 3
	GGUFDTypeQ8_0 GGUFDType = 8
)

GGUF tensor dtypes.

type GGUFMetadata 

type GGUFMetadata map[string]interface{}

GGUFMetadata stores GGUF metadata key-value pairs.

type GGUFReader 

type GGUFReader struct {
	// contains filtered or unexported fields
}

GGUFReader reads GGUF format files.

func NewGGUFReader 

func NewGGUFReader(path string) (*GGUFReader, error)

NewGGUFReader creates a new GGUF reader.

func (*GGUFReader) Close 

func (r *GGUFReader) Close() error

Close closes the GGUF file.

func (*GGUFReader) Metadata 

func (r *GGUFReader) Metadata() GGUFMetadata

Metadata returns the metadata map.

func (*GGUFReader) ReadTensorData 

func (r *GGUFReader) ReadTensorData(name string) ([]byte, error)

ReadTensorData reads raw tensor data for a given tensor.

func (*GGUFReader) TensorInfo 

func (r *GGUFReader) TensorInfo(name string) (*GGUFTensorInfo, error)

TensorInfo returns information about a specific tensor.

func (*GGUFReader) TensorNames 

func (r *GGUFReader) TensorNames() []string

TensorNames returns a list of all tensor names.

type GGUFTensorInfo 

type GGUFTensorInfo struct {
	Name   string
	Dims   []uint64 // Dimensions (reversed compared to normal order)
	DType  GGUFDType
	Offset uint64 // Offset in data section
}

GGUFTensorInfo describes a tensor in GGUF format.

type GGUFType 

type GGUFType uint32

GGUFType represents GGUF value types. 

const (
	GGUFTypeUint8   GGUFType = 0
	GGUFTypeInt8    GGUFType = 1
	GGUFTypeUint16  GGUFType = 2
	GGUFTypeInt16   GGUFType = 3
	GGUFTypeUint32  GGUFType = 4
	GGUFTypeInt32   GGUFType = 5
	GGUFTypeFloat32 GGUFType = 6
	GGUFTypeBool    GGUFType = 7
	GGUFTypeString  GGUFType = 8
	GGUFTypeArray   GGUFType = 9
	GGUFTypeUint64  GGUFType = 10
	GGUFTypeInt64   GGUFType = 11
	GGUFTypeFloat64 GGUFType = 12
)

GGUF value types.

type LLaMAMapper 

type LLaMAMapper struct{}

LLaMAMapper maps LLaMA weight names to Born standard names. Supports LLaMA, LLaMA 2, and LLaMA 3.

func NewLLaMAMapper 

func NewLLaMAMapper() *LLaMAMapper

NewLLaMAMapper creates a new LLaMA weight mapper.

func (*LLaMAMapper) Architecture 

func (m *LLaMAMapper) Architecture() string

Architecture returns "llama".

func (*LLaMAMapper) MapName 

func (m *LLaMAMapper) MapName(name string) (string, error)

MapName converts LLaMA weight names to Born standard names. LLaMA format:

  • model.embed_tokens.weight -> embedding.weight
  • model.layers.{i}.self_attn.q_proj.weight -> layers.{i}.attn.q.weight
  • model.layers.{i}.mlp.gate_proj.weight -> layers.{i}.ffn.gate.weight
  • model.norm.weight -> norm.weight

type MistralMapper 

type MistralMapper struct{}

MistralMapper maps Mistral weight names to Born standard names. Supports Mistral 7B and Mixtral 8x7B.

func NewMistralMapper 

func NewMistralMapper() *MistralMapper

NewMistralMapper creates a new Mistral weight mapper.

func (*MistralMapper) Architecture 

func (m *MistralMapper) Architecture() string

Architecture returns "mistral".

func (*MistralMapper) MapName 

func (m *MistralMapper) MapName(name string) (string, error)

MapName converts Mistral weight names to Born standard names. Mistral uses similar naming to LLaMA but with some differences.

type ModelFormat 

type ModelFormat int

ModelFormat represents the model weight format. 

const (
	FormatUnknown ModelFormat = iota
	FormatSafeTensors
	FormatGGUF
)

Supported model formats.

func (ModelFormat) String 

func (f ModelFormat) String() string

String returns the format name.

type ModelReader 

type ModelReader interface {
	// Close closes the underlying file.
	Close() error

	// Format returns the model format.
	Format() ModelFormat

	// Architecture returns the detected architecture (llama, mistral, deepseek).
	Architecture() string

	// Metadata returns model metadata.
	Metadata() map[string]interface{}

	// TensorNames returns all tensor names in the model.
	TensorNames() []string

	// LoadTensor loads a tensor by name with optional name mapping.
	LoadTensor(name string, backend tensor.Backend) (*tensor.RawTensor, error)

	// ReadTensorData reads raw tensor bytes (for custom conversion).
	ReadTensorData(name string) ([]byte, error)
}

ModelReader provides a unified interface for loading model weights.

func OpenModel 

func OpenModel(path string) (ModelReader, error)

OpenModel opens a model file and auto-detects the format. Supports .safetensors and .gguf files.

Example: 

model, err := loader.OpenModel("path/to/model.safetensors")
if err != nil {
    log.Fatal(err)
}
defer model.Close()

fmt.Printf("Format: %s\n", model.Format())
fmt.Printf("Architecture: %s\n", model.Architecture())

type SafeTensorInfo 

type SafeTensorInfo struct {
	DType       SafeTensorsDType `json:"dtype"`
	Shape       []int            `json:"shape"`
	DataOffsets [2]int64         `json:"data_offsets"` // [start, end]
}

SafeTensorInfo describes a tensor in SafeTensors format.

type SafeTensorsDType 

type SafeTensorsDType string

SafeTensorsDType represents supported SafeTensors data types. 

const (
	SafeTensorsF16  SafeTensorsDType = "F16"
	SafeTensorsF32  SafeTensorsDType = "F32"
	SafeTensorsF64  SafeTensorsDType = "F64"
	SafeTensorsBF16 SafeTensorsDType = "BF16"
	SafeTensorsI32  SafeTensorsDType = "I32"
	SafeTensorsI64  SafeTensorsDType = "I64"
	SafeTensorsU8   SafeTensorsDType = "U8"
	SafeTensorsBool SafeTensorsDType = "BOOL"
)

Supported SafeTensors dtypes.

type SafeTensorsHeader 

type SafeTensorsHeader struct {
	Metadata map[string]string          `json:"__metadata__"`
	Tensors  map[string]SafeTensorInfo  `json:"-"`
	RawMap   map[string]json.RawMessage `json:"-"`
}

SafeTensorsHeader is the JSON header in SafeTensors format.

func (*SafeTensorsHeader) UnmarshalJSON 

func (h *SafeTensorsHeader) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for SafeTensorsHeader.

type SafeTensorsReader 

type SafeTensorsReader struct {
	// contains filtered or unexported fields
}

SafeTensorsReader reads SafeTensors format files.

func NewSafeTensorsReader 

func NewSafeTensorsReader(path string) (*SafeTensorsReader, error)

NewSafeTensorsReader creates a new SafeTensors reader.

func (*SafeTensorsReader) Close 

func (r *SafeTensorsReader) Close() error

Close closes the SafeTensors file.

func (*SafeTensorsReader) LoadTensor 

func (r *SafeTensorsReader) LoadTensor(name string, backend tensor.Backend) (*tensor.RawTensor, error)

LoadTensor loads a tensor from SafeTensors file into Born tensor. For F16/BF16, this function returns an error - caller must use ReadTensorData and convert manually.

func (*SafeTensorsReader) Metadata 

func (r *SafeTensorsReader) Metadata() map[string]string

Metadata returns the metadata map from the header.

func (*SafeTensorsReader) ReadTensorData 

func (r *SafeTensorsReader) ReadTensorData(name string) ([]byte, error)

ReadTensorData reads raw tensor data for a given tensor name.

func (*SafeTensorsReader) TensorInfo 

func (r *SafeTensorsReader) TensorInfo(name string) (*SafeTensorInfo, error)

TensorInfo returns information about a specific tensor.

func (*SafeTensorsReader) TensorNames 

func (r *SafeTensorsReader) TensorNames() []string

TensorNames returns a list of all tensor names in the file.

type WeightMapper 

type WeightMapper interface {
	// MapName converts a model-specific weight name to Born standard name.
	MapName(name string) (string, error)

	// Architecture returns the architecture name (e.g., "llama", "mistral").
	Architecture() string
}

WeightMapper maps model-specific weight names to standard Born names.

func GetMapper 

func GetMapper(architecture string) WeightMapper

GetMapper returns the appropriate weight mapper for an architecture.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.