mlx

package
v0.40.13 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 28, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package mlx wraps the MLX C API.

MLX keeps stream and backend state in thread-locals, so all calls into this package must come from a single goroutine locked to its OS thread (see mlx/mlxthread).

Index

Constants

View Source
const End = math.MaxInt32

End is a sentinel value meaning "to the end of the dimension", equivalent to an omitted stop in Python (e.g. a[i:]).

View Source
const FastSDPANoWindow = -1

FastSDPANoWindow disables sliding-window restriction for SDPA - each query attends to the full key context.

View Source
const Nvfp4MaxProduct = 448 * 6

Nvfp4MaxProduct is the product of the maximum E4M3 and E2M1 values. Every globalScale below is in MLX's representation: a checkpoint's multiplier m (ModelOpt's weight_scale_2) is held as m*Nvfp4MaxProduct, so Nvfp4MaxProduct itself is the identity. Wrappers that hand the scale to MLX pass it through; wrappers that apply it themselves divide it back out.

Variables

View Source
var GELU = Compile1("GELU", gelu, Shapeless())

GELU returns the exact erf formulation used by torch.nn.functional.gelu.

View Source
var GELUApprox = Compile1(
	"GELUApprox",
	func(x *Array) *Array {

		dt := x.DType()
		half := FromValue[float32](0.5).AsType(dt)
		coeff := FromValue(geluCoeff).AsType(dt)
		c := FromValue[float32](0.044715).AsType(dt)
		one := FromValue[float32](1.0).AsType(dt)

		x3 := x.Multiply(x).Multiply(x)
		inner := x.Add(c.Multiply(x3))
		tanh := coeff.Multiply(inner).Tanh()
		return half.Multiply(x).Multiply(one.Add(tanh))
	},
	Shapeless(),
)

GELUApprox returns 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3))) as a fused kernel.

View Source
var GeGLU = Compile2(
	"GeGLU",
	func(gate, up *Array) *Array {
		return GELUApprox(gate).Multiply(up)
	},
	Shapeless(),
)

GeGLU returns gelu_approx(gate) * up as a fused kernel. Matches mlx_lm's geglu, used by Gemma-family MLP and MoE paths.

View Source
var LogitSoftcap = Compile2(
	"LogitSoftcap",
	func(x, cap *Array) *Array {
		return x.Divide(cap).Tanh().Multiply(cap)
	},
	Shapeless(),
)

LogitSoftcap returns tanh(x / cap) * cap as a fused kernel. Matches mlx_lm's logit_softcap. cap must have the same dtype as x.

View Source
var ReLUSquared = Compile1(
	"ReLUSquared",
	func(x *Array) *Array {
		zero := FromValue[float32](0).AsType(x.DType())
		x = Maximum(x, zero)
		return x.Multiply(x)
	},
	Shapeless(),
)

ReLUSquared returns relu(x)^2 as a fused kernel.

View Source
var SiLU = Compile1(
	"SiLU",
	func(a *Array) *Array {
		return a.Multiply(a.Sigmoid())
	},
	Shapeless(),
)

SiLU returns a * sigmoid(a) as a fused kernel.

View Source
var SoftplusF32 = Compile1(
	"SoftplusF32",
	func(x *Array) *Array {
		dt := x.DType()
		zero := FromValue[float32](0)
		return Logaddexp(x.AsType(DTypeFloat32), zero).AsType(dt)
	},
	Shapeless(),
)

SoftplusF32 returns softplus(x) computed in float32 precision and cast back to x's original dtype, as a fused kernel. Matches the laguna attention output-gate formula: softplus(cast_f32(x)).cast(orig_dtype).

View Source
var SwiGLU = Compile2(
	"SwiGLU",
	func(gate, up *Array) *Array {
		return SiLU(gate).Multiply(up)
	},
	Shapeless(),
)

SwiGLU returns silu(gate) * up as a fused kernel.

Functions

func ActiveMemory

func ActiveMemory() int

func AsyncEval

func AsyncEval(outputs ...*Array)

func CUDAIsAvailable

func CUDAIsAvailable() bool

CUDAIsAvailable returns true if a CUDA GPU is available.

func CUDAToolkitDirForLoadedLibrary added in v0.40.7

func CUDAToolkitDirForLoadedLibrary(libraryPath string) (string, bool)

CUDAToolkitDirForLoadedLibrary returns the selected MLX CUDA variant directory when it carries bundled CUDA headers for MLX JIT compilation.

func CacheMemory

func CacheMemory() int

func CheckInit

func CheckInit() error

CheckInit returns any error that occurred during MLX dynamic library initialization.

func ClearCache

func ClearCache()

func Compile1

func Compile1(name string, fn func(*Array) *Array, opts ...CompileOption) func(*Array) *Array

Compile1 compiles a unary function. See Compile.

func Compile2

func Compile2(name string, fn func(*Array, *Array) *Array, opts ...CompileOption) func(*Array, *Array) *Array

Compile2 compiles a binary function. See Compile.

func Compile3

func Compile3(name string, fn func(*Array, *Array, *Array) *Array, opts ...CompileOption) func(*Array, *Array, *Array) *Array

Compile3 compiles a ternary function. See Compile.

func DisableCompile

func DisableCompile()

func EnableCompile

func EnableCompile()

func Eval

func Eval(outputs ...*Array)

func GPUIsAvailable

func GPUIsAvailable() bool

GPUIsAvailable returns true if a GPU device is available.

func GatedDelta

func GatedDelta(packed, ba, dtBias, aExp, state, mask *Array, captureAll bool) (y, nextState *Array, interior []*Array)

GatedDelta runs the whole gated-delta step — q/k norms, decay gate, and the scan — in one launch over the activated causal-conv output. packed is [B, T, 2*Hk*Dk + Hv*Dv] with rows packed [q | k | v], ba is [B, T, 2*Hv] packed [beta | alpha] rows, and state is [B, Hv, Dv, Dk]. captureAll additionally emits every interior per-token state. Inputs that fit the one-launch kernels' contract run there; anything else runs the same step as graph ops.

When mask is non-nil, it must be a [B, T] bool tensor identifying real (true) vs. padded (false) positions. Padded rows are neutralized before the kernels' own preprocessing: zeroed conv rows make the q/k/v norms emit zeros, and -inf beta/alpha rows yield beta 0 and decay 1, so each padded position is an identity step with zero output, exactly as on the recurrence path.

func GatedDeltaPrecise added in v0.40.4

func GatedDeltaPrecise(packed, ba, dtBias, aExp, state, mask *Array, captureAll bool) (y, nextState *Array, interior []*Array)

GatedDeltaPrecise keeps the decay gate in FP32 for models trained with that recurrence. Other models continue through GatedDelta's original path.

func LibraryPathsForLoadedLibrary added in v0.40.7

func LibraryPathsForLoadedLibrary(libraryPath string) []string

LibraryPathsForLoadedLibrary returns the runtime library search paths needed for the selected MLX dynamic library.

func Load

func Load(path string) iter.Seq2[string, *Array]

func LoadedLibraryPath

func LoadedLibraryPath() (string, error)

LoadedLibraryPath returns the MLX dynamic library path selected by this package.

func Mamba2Scan

func Mamba2Scan(hidden, bState, cState, dt, state, a, d, dtBias, mask *Array, captureAll bool) (y, nextState *Array, interior []*Array)

Mamba2Scan runs the Mamba2 recurrent scan. Inputs must be float32 with shapes hidden [B, T, H, D], bState/cState [B, T, G, S] where H%G == 0, dt [B, T, H], state [B, H, D, S], and a/d/dtBias [H]. captureAll also returns the state after every token but the last.

mask, when non-nil, is a [B, T] bool marking real (true) vs padded positions; padded positions are identity steps.

func MaxRecommendedWorkingSetSize

func MaxRecommendedWorkingSetSize() (int, error)

MaxRecommendedWorkingSetSize returns the device's recommended upper bound for resident Metal allocations.

func MetalIsAvailable

func MetalIsAvailable() bool

MetalIsAvailable returns true if a Metal GPU is available.

func PeakMemory

func PeakMemory() int

func PrettyBytes

func PrettyBytes(n int) fmt.Stringer

func ResetPeakMemory

func ResetPeakMemory()

func SaveSafetensors

func SaveSafetensors(path string, arrays map[string]*Array) error

SaveSafetensors saves arrays to a safetensors file without metadata.

func SaveSafetensorsWithMetadata

func SaveSafetensorsWithMetadata(path string, arrays map[string]*Array, metadata map[string]string) error

SaveSafetensorsWithMetadata saves arrays to a safetensors file with metadata.

func Scoped

func Scoped(fn func())

Scoped runs fn in a function scope. Arrays created or released inside it are freed when fn returns.

func SetDefaultDeviceGPU

func SetDefaultDeviceGPU()

SetDefaultDeviceGPU sets the default MLX device to GPU.

func SetWiredLimit

func SetWiredLimit(limit int) (int, error)

SetWiredLimit sets the maximum amount of Metal memory MLX keeps resident and returns the previous limit.

func Slice

func Slice(args ...int) slice

func Version

func Version() string

Version returns the MLX core library version string.

Types

type Array

type Array struct {
	// contains filtered or unexported fields
}

An Array's lifetime is governed by the scope it belongs to; see scope.go.

func Add

func Add(a, b *Array) *Array

func AddMM

func AddMM(c, a, b *Array, alpha, beta float32) *Array

func AddScalar

func AddScalar(a *Array, s float32) *Array

func Arange

func Arange(start, stop, step float64, dtype DType) *Array

func Argpartition

func Argpartition(a *Array, kth int, axis int) *Array

func Argsort

func Argsort(a *Array, axis int) *Array

func Bernoulli

func Bernoulli(p *Array) *Array

func BernoulliWithKey

func BernoulliWithKey(p *Array, key *Array) *Array

func BroadcastTo

func BroadcastTo(a *Array, shape ...int32) *Array

BroadcastTo broadcasts an array to the given shape.

func Clamp

func Clamp(a *Array, minVal, maxVal float32) *Array

Clamp clamps array values to [min, max].

func Clip

func Clip(a, aMin, aMax *Array) *Array

func Collect

func Collect(v any) []*Array

func Concatenate

func Concatenate(arrays []*Array, axis int) *Array

func Contiguous

func Contiguous(a *Array, allowColMajor bool) *Array

func Conv1d

func Conv1d(x, weight *Array, bias *Array, stride, padding, dilation, groups int32) *Array

func Conv2d

func Conv2d(x, weight *Array, strideH, strideW, padH, padW, dilationH, dilationW, groups int32) *Array

Conv2d performs 2D convolution: x [N,H,W,C_in], weight [C_out,kH,kW,C_in]. MLX uses NHWC layout.

func Cos

func Cos(a *Array) *Array

func DepthwiseConv1d

func DepthwiseConv1d(x, weight *Array, bias *Array) *Array

func DepthwiseConvSiLU

func DepthwiseConvSiLU(x, w, bias *Array, outLen int) *Array

DepthwiseConvSiLU computes SiLU of a valid depthwise conv: x [B, T+K-1, C] and w [C, K] give [B, T, C], each output reading the K trailing input rows starting at its own index. bias, when non-nil, is [C]. Inputs that fit the fused kernel's contract run there; anything else runs the same computation as graph ops, bit for bit.

func Dequantize

func Dequantize(w, scales, biases *Array, groupSize, bits int, mode string, globalScale *Array) *Array

Dequantize applies globalScale itself rather than forwarding it: MLX's own argument accepts only a scalar, and callers pass per-expert banks.

func Div

func Div(a, b *Array) *Array

func DivScalar

func DivScalar(a *Array, s float32) *Array

func Erf

func Erf(a *Array) *Array

func Exp

func Exp(a *Array) *Array

func ExpandDims

func ExpandDims(a *Array, axis int) *Array

func FastScaledDotProductAttention

func FastScaledDotProductAttention(q, k, v *Array, scale float32, windowSize int, mode string, mask *Array) *Array

func FastScaledDotProductAttentionWithSinks added in v0.40.8

func FastScaledDotProductAttentionWithSinks(q, k, v *Array, scale float32, windowSize int, mode string, mask, sinks *Array) *Array

FastScaledDotProductAttentionWithSinks adds one learned softmax denominator term per query head. A sink has no corresponding value vector.

windowSize restricts each query to the most recent windowSize keys (sliding window attention); pass FastSDPANoWindow for full context.

func Flatten

func Flatten(a *Array) *Array

func FloorDivideScalar

func FloorDivideScalar(a *Array, s int32) *Array

func FromFP8

func FromFP8(x *Array, dtype DType) *Array

func FromValue

func FromValue[T scalarTypes](t T) *Array

func FromValues

func FromValues[S ~[]E, E arrayTypes](s S, shape ...int) *Array

func GLU

func GLU(a *Array) *Array

GLU applies Gated Linear Unit: splits x along last dim into two halves, returns first * sigmoid(second).

func GatherMM

func GatherMM(a, b *Array, lhsIndices, rhsIndices *Array, sortedIndices bool) *Array

func GatherQMM

func GatherQMM(x, w, scales *Array, biases, lhsIndices, rhsIndices *Array, transpose bool, groupSize, bits int, mode string, globalScale *Array, sortedIndices bool) *Array

GatherQMM multiplies x against the quantized weight banks selected by lhsIndices and rhsIndices, with one globalScale entry per expert. Metal applies it inside the gather kernel, other backends scale the gathered output rows in the wrapper.

func GatherWeightedSum added in v0.40.5

func GatherWeightedSum(values, scores, indices *Array) *Array

GatherWeightedSum reduces K expert rows per token in FP32, then rounds to values' dtype. values is [N*K, D], scores is [N, K], and optional indices maps each token/expert slot to a row in values. Fusing the gather and reduction avoids materializing reordered expert outputs and their FP32 weighted product.

func LayerNormFn

func LayerNormFn(x, weight, bias *Array, eps float32) *Array

func Log

func Log(a *Array) *Array

func Logaddexp

func Logaddexp(a, b *Array) *Array

func Matmul

func Matmul(a, b *Array) *Array

func Maximum

func Maximum(a, b *Array) *Array

Maximum returns element-wise maximum of two arrays.

func Mean

func Mean(a *Array, axis int, keepDims bool) *Array

func Minimum

func Minimum(a, b *Array) *Array

Minimum returns element-wise minimum of two arrays.

func Mul

func Mul(a, b *Array) *Array

func MulScalar

func MulScalar(a *Array, s float32) *Array

func Neg

func Neg(a *Array) *Array

func New

func New(name string) *Array

func NewArrayInt32

func NewArrayInt32(data []int32, shape []int32) *Array

func NewScalarArray

func NewScalarArray(value float32) *Array

func Pad

func Pad(a *Array, axes []int, lowPad, highPad []int, padValue *Array, mode string) *Array

Pad pads array a along the given axes with specified low/high pad sizes. mode should be "constant", "edge", or "reflect".

func PadConstant

func PadConstant(a *Array, axes []int, lowPad, highPad []int) *Array

PadConstant pads with zeros along the given axes.

func Quantize

func Quantize(w *Array, groupSize, bits int, mode string) (weights, scales, biases *Array)

func QuantizedMatmul

func QuantizedMatmul(x, w, scales, biases *Array, transpose bool, groupSize, bits int, mode string, globalScale *Array) *Array

QuantizedMatmul applies globalScale to the output; MLX's quantized_matmul has none of its own.

func RMSNormFn

func RMSNormFn(x, weight *Array, eps float32) *Array

func RSqrt

func RSqrt(a *Array) *Array

func RandomKey

func RandomKey(seed uint64) *Array

func ReLU

func ReLU(a *Array) *Array

ReLU computes max(0, x).

func Reshape

func Reshape(a *Array, shape ...int32) *Array

func RoPEWithBase

func RoPEWithBase(x *Array, dims int, traditional bool, base, scale float32, offsets *Array) *Array

RoPEWithBase applies rotary position embeddings to x. offsets is an int32 array of shape [B] giving each batch row's starting position; the kernel applies positions offsets[b] + 0..T-1 per row.

func RoPEWithFreqs

func RoPEWithFreqs(x *Array, dims int, traditional bool, base, scale float32, offsets *Array, freqs *Array) *Array

RoPEWithFreqs applies RoPE with optional custom frequencies. When freqs is non-nil, it is used instead of computing from base. Note: MLX takes reciprocal(freqs) internally to get inv_freq, so pass the actual frequencies (base^(2i/dim)), not the inverse frequencies.

func ScopedArrays

func ScopedArrays(fn func() []*Array) []*Array

ScopedArrays runs fn in a function scope and moves the arrays it returns to the caller's scope.

func ScopedAsyncEval

func ScopedAsyncEval(fn func() []*Array) []*Array

ScopedAsyncEval is ScopedEval with an asynchronous evaluation.

func ScopedEval

func ScopedEval(fn func() []*Array) []*Array

ScopedEval runs fn in a function scope, moves the arrays it returns to the caller's scope, ends the scope, and then evaluates them.

func Sigmoid

func Sigmoid(a *Array) *Array

func SigmoidRouter

func SigmoidRouter(gates, bias *Array) (origScores, negScores *Array)

SigmoidRouter returns (sigmoid(gates), -(sigmoid(gates)+bias)) as a fused kernel — the DeepSeek-V2 / GLM-MoE aux-loss-free router head.

func Sin

func Sin(a *Array) *Array

func SliceStartStop

func SliceStartStop(a *Array, start, stop []int32) *Array

func SoftmaxAxis

func SoftmaxAxis(a *Array, axis int, precise bool) *Array

func Softplus

func Softplus(a *Array) *Array

Softplus computes log(1 + exp(x)) using logaddexp for numerical stability.

func Squeeze

func Squeeze(a *Array, axis int) *Array

func Stack

func Stack(arrays []*Array, axis int) *Array

func Sub

func Sub(a, b *Array) *Array

func Sum

func Sum(a *Array, axis int, keepDims bool) *Array

func SwiGLUScaled added in v0.34.4

func SwiGLUScaled(gate, gateScale, up, upScale *Array) *Array

SwiGLUScaled applies each non-nil quantization global scale, rounds the projection back to its input dtype, then evaluates SwiGLU. A nil scale means that projection is already complete.

func Take

func Take(a *Array, indices *Array, axis int) *Array

func TakeAlongAxis

func TakeAlongAxis(a, indices *Array, axis int) *Array

func Tile

func Tile(a *Array, reps []int32) *Array

func ToFP8

func ToFP8(x *Array) *Array

func Transpose

func Transpose(a *Array, axes ...int) *Array

func Tri

func Tri(n, m int32, k int) *Array

func Where

func Where(condition, a, b *Array) *Array

func Zeros

func Zeros(dtype DType, shape ...int) *Array

func ZerosF32

func ZerosF32(shape []int32) *Array

func (*Array) Abs

func (t *Array) Abs() *Array

func (*Array) Add

func (t *Array) Add(other *Array) *Array

func (*Array) Addmm

func (t *Array) Addmm(a, b *Array, alpha, beta float32) *Array

func (*Array) Argmax

func (t *Array) Argmax(axis int, keepDims bool) *Array

func (*Array) ArgpartitionAxis

func (t *Array) ArgpartitionAxis(kth int, axis int) *Array

func (*Array) ArgsortAxis

func (t *Array) ArgsortAxis(axis int) *Array

func (*Array) AsStrided

func (t *Array) AsStrided(shape []int, strides []int, offset int) *Array

func (*Array) AsType

func (t *Array) AsType(dtype DType) *Array

func (*Array) BitwiseAnd

func (t *Array) BitwiseAnd(other *Array) *Array

func (*Array) BitwiseXor

func (t *Array) BitwiseXor(other *Array) *Array

func (*Array) Categorical

func (t *Array) Categorical(axis int) *Array

func (*Array) CategoricalWithKey

func (t *Array) CategoricalWithKey(axis int, key *Array) *Array

func (*Array) Clone

func (t *Array) Clone() *Array

func (*Array) Concatenate

func (t *Array) Concatenate(axis int, others ...*Array) *Array

func (*Array) Cumsum

func (t *Array) Cumsum(axis int, reverse, inclusive bool) *Array

func (*Array) DType

func (t *Array) DType() DType

func (*Array) Dim

func (t *Array) Dim(dim int) int

func (*Array) Dims

func (t *Array) Dims() []int

func (*Array) Divide

func (t *Array) Divide(other *Array) *Array

func (*Array) Equal

func (t *Array) Equal(other *Array) *Array

func (*Array) ExpandDims

func (t *Array) ExpandDims(axis int) *Array

func (*Array) Flatten

func (t *Array) Flatten(startAxis, endAxis int) *Array

func (*Array) Float

func (t *Array) Float() float32

func (*Array) Floats

func (t *Array) Floats() []float32

func (*Array) FloorDivide

func (t *Array) FloorDivide(other *Array) *Array

func (*Array) GatherMM

func (t *Array) GatherMM(other, lhs, rhs *Array, sorted bool) *Array

func (*Array) Greater

func (t *Array) Greater(other *Array) *Array

func (*Array) Int

func (t *Array) Int() int32

func (*Array) Ints

func (t *Array) Ints() []int32

func (*Array) Less

func (t *Array) Less(other *Array) *Array

func (*Array) LessEqual

func (t *Array) LessEqual(other *Array) *Array

func (*Array) LogValue

func (t *Array) LogValue() slog.Value

func (*Array) LogsumexpAxis

func (t *Array) LogsumexpAxis(axis int, keepDims bool) *Array

func (*Array) Matmul

func (t *Array) Matmul(other *Array) *Array

func (*Array) MaxAxis

func (t *Array) MaxAxis(axis int, keepDims bool) *Array

func (*Array) Multiply

func (t *Array) Multiply(other *Array) *Array

func (*Array) Negative

func (t *Array) Negative() *Array

func (*Array) NumBytes

func (t *Array) NumBytes() int

func (*Array) NumDims

func (t *Array) NumDims() int

func (*Array) Power

func (t *Array) Power(exponent *Array) *Array

func (*Array) PutAlongAxis

func (t *Array) PutAlongAxis(indices, values *Array, axis int) *Array

func (*Array) Remainder

func (t *Array) Remainder(other *Array) *Array

func (*Array) Reshape

func (t *Array) Reshape(axes ...int) *Array

func (*Array) RightShift

func (t *Array) RightShift(other *Array) *Array

func (*Array) Save

func (t *Array) Save(name string) error

func (*Array) ScatterAddAxis

func (t *Array) ScatterAddAxis(indices, values *Array, axis int) *Array

func (*Array) Set

func (t *Array) Set(other *Array)

func (*Array) Sigmoid

func (t *Array) Sigmoid() *Array

func (*Array) Sign

func (t *Array) Sign() *Array

func (*Array) Size

func (t *Array) Size() int

func (*Array) Slice

func (t *Array) Slice(slices ...slice) *Array

func (*Array) SliceUpdate

func (t *Array) SliceUpdate(other *Array, slices ...slice) *Array

func (*Array) Sqrt

func (t *Array) Sqrt() *Array

func (*Array) Squeeze

func (t *Array) Squeeze(axis int) *Array

func (*Array) StackAxis

func (t *Array) StackAxis(axis int, others ...*Array) *Array

func (*Array) String

func (t *Array) String() string

func (*Array) Subtract

func (t *Array) Subtract(other *Array) *Array

func (*Array) SumAxis

func (t *Array) SumAxis(axis int, keepDims bool) *Array

func (*Array) TakeAlongAxis

func (t *Array) TakeAlongAxis(indices *Array, axis int) *Array

func (*Array) TakeAxis

func (t *Array) TakeAxis(indices *Array, axis int) *Array

func (*Array) Tanh

func (t *Array) Tanh() *Array

func (*Array) Transpose

func (t *Array) Transpose(axes ...int) *Array

type Byte

type Byte int

func (Byte) String

func (b Byte) String() string

type CompileFunc

type CompileFunc func(inputs ...*Array) []*Array

CompileFunc is the signature of a function that can be compiled.

func Compile

func Compile(name string, fn CompileFunc, opts ...CompileOption) CompileFunc

Compile returns a compiled version of fn. When called during another compile's trace, fn is inlined directly so outer compiles can fuse through inner ones.

Compiled functions must not have side effects outside of the function. Do not access data other than the arguments passed in (either Go data or MLX arrays) unless it is a constant.

type CompileOption

type CompileOption func(*compileConfig)

CompileOption configures Compile behavior.

func Shapeless

func Shapeless() CompileOption

Shapeless traces the function once against symbolic shapes so the compiled graph accepts any input shape afterwards. Without this option, MLX re-traces on each new (shape, dtype) combination and caches each specialization.

type DType

type DType int
const (
	DTypeBool      DType = C.MLX_BOOL
	DTypeUint8     DType = C.MLX_UINT8
	DTypeUint16    DType = C.MLX_UINT16
	DTypeUint32    DType = C.MLX_UINT32
	DTypeUint64    DType = C.MLX_UINT64
	DTypeInt8      DType = C.MLX_INT8
	DTypeInt16     DType = C.MLX_INT16
	DTypeInt32     DType = C.MLX_INT32
	DTypeInt64     DType = C.MLX_INT64
	DTypeFloat16   DType = C.MLX_FLOAT16
	DTypeFloat32   DType = C.MLX_FLOAT32
	DTypeFloat64   DType = C.MLX_FLOAT64
	DTypeBFloat16  DType = C.MLX_BFLOAT16
	DTypeComplex64 DType = C.MLX_COMPLEX64
)

func (DType) String

func (t DType) String() string

func (*DType) UnmarshalJSON

func (t *DType) UnmarshalJSON(b []byte) error

type Device

type Device struct {
	// contains filtered or unexported fields
}

func DefaultDevice

func DefaultDevice() Device

func (Device) LogValue

func (d Device) LogValue() slog.Value

type Embedding

type Embedding struct {
	Weight *Array `weight:"weight"`
}

func (*Embedding) AsLinear

func (e *Embedding) AsLinear() Linear

func (*Embedding) Forward

func (e *Embedding) Forward(indices *Array) *Array

type GibiByte

type GibiByte int

func (GibiByte) String

func (b GibiByte) String() string

type KibiByte

type KibiByte int

func (KibiByte) String

func (b KibiByte) String() string

type LayerNorm

type LayerNorm struct {
	Weight *Array `weight:"weight"`
	Bias   *Array `weight:"bias"`
}

func (*LayerNorm) Forward

func (r *LayerNorm) Forward(x *Array, eps float32) *Array

type Linear

type Linear struct {
	Weight *Array `weight:"weight"`
	Bias   *Array `weight:"bias"`
}

func (*Linear) Forward

func (m *Linear) Forward(x *Array) *Array

Forward computes the linear transformation: x @ Weight.T + Bias

func (*Linear) Gather

func (m *Linear) Gather(x, lhs, rhs *Array, sorted bool) *Array

type MebiByte

type MebiByte int

func (MebiByte) String

func (b MebiByte) String() string

type Memory

type Memory struct{}

func (Memory) LogValue

func (Memory) LogValue() slog.Value

type RMSNorm

type RMSNorm struct {
	Weight *Array `weight:"weight"`
}

func (*RMSNorm) Forward

func (r *RMSNorm) Forward(x *Array, eps float32) *Array

type SafetensorsFile

type SafetensorsFile struct {
	// contains filtered or unexported fields
}

SafetensorsFile represents a loaded safetensors file.

func LoadSafetensorsNative

func LoadSafetensorsNative(path string) (*SafetensorsFile, error)

LoadSafetensorsNative loads a safetensors file using MLX's native loader.

func (*SafetensorsFile) Free

func (s *SafetensorsFile) Free()

Free releases the loaded safetensors maps.

func (*SafetensorsFile) Get

func (s *SafetensorsFile) Get(name string) *Array

Get retrieves a tensor by name.

func (*SafetensorsFile) GetMetadata

func (s *SafetensorsFile) GetMetadata(key string) string

GetMetadata retrieves a metadata value by key.

type Scope

type Scope struct {
	// contains filtered or unexported fields
}

func NewScope

func NewScope() *Scope

func (*Scope) Attach

func (s *Scope) Attach(arrays ...*Array)

Attach takes arrays from the function scope that built them or from the root. An array some held scope already holds, this one included, is that holder's to discard or detach first: a second Attach means two owners.

func (*Scope) Close

func (s *Scope) Close()

Close frees whatever the scope still holds. A nil *Scope holds nothing.

func (*Scope) Detach

func (s *Scope) Detach(arrays ...*Array)

Detach moves arrays back to the current scope, for a caller that still reads them.

func (*Scope) Discard

func (s *Scope) Discard(arrays ...*Array)

type Stream

type Stream struct {
	// contains filtered or unexported fields
}

func DefaultStream

func DefaultStream() Stream

func (Stream) LogValue

func (s Stream) LogValue() slog.Value

func (Stream) Synchronize added in v0.40.0

func (s Stream) Synchronize()

Synchronize waits for submitted work and its completion handlers on the stream.

type TebiByte

type TebiByte int

func (TebiByte) String

func (b TebiByte) String() string

Directories

Path Synopsis
Package mlxtest provides shared scaffolding for tests that exercise MLX through the cgo wrapper in mlx.
Package mlxtest provides shared scaffolding for tests that exercise MLX through the cgo wrapper in mlx.
mlxthreadtest
Package mlxthreadtest runs tests on a persistent MLX worker thread.
Package mlxthreadtest runs tests on a persistent MLX worker thread.
Package quant holds the quantization format facts shared by the model importer, the runtime loader, and `ollama show`.
Package quant holds the quantization format facts shared by the model importer, the runtime loader, and `ollama show`.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL