Documentation
¶
Overview ¶
Package linear provides reusable ANE-backed linear forward execution.
It compiles and caches in-memory kernels keyed by shape and weights, then executes row-major x*w^T using ANE runtime plumbing from this repository.
DynamicExecutor provides a compile-once variant for workloads whose weights change across evaluations.
Index ¶
- type CallStats
- type DynamicExecutor
- func (e *DynamicExecutor) Close()
- func (e *DynamicExecutor) Linear(ctx context.Context, x, w []float32, batch, inDim, outDim int) ([]float32, error)
- func (e *DynamicExecutor) LinearIOIntoWithStats(ctx context.Context, dst, x, wIO []float32, batch, inDim, outDim int) (CallStats, error)
- func (e *DynamicExecutor) LinearIntoWithStats(ctx context.Context, dst, x, w []float32, batch, inDim, outDim int) (CallStats, error)
- func (e *DynamicExecutor) LinearOneHotIOIntoWithStats(ctx context.Context, dst []float32, xs []int, batch, inDim, outDim int) (CallStats, error)
- func (e *DynamicExecutor) LinearWithStats(ctx context.Context, x, w []float32, batch, inDim, outDim int) ([]float32, CallStats, error)
- func (e *DynamicExecutor) Prepare(batch, inDim, outDim int) error
- func (e *DynamicExecutor) PrimeWeightsIO(batch, inDim, outDim int, wIO []float32) error
- func (e *DynamicExecutor) Stats() Stats
- func (e *DynamicExecutor) UpdateWeightsIORows(batch, inDim, outDim int, wIO []float32, rows []int) error
- type Executor
- func (e *Executor) Close()
- func (e *Executor) Linear(ctx context.Context, x, w []float32, batch, inDim, outDim int) ([]float32, error)
- func (e *Executor) LinearIntoWithStats(ctx context.Context, dst, x, w []float32, batch, inDim, outDim int) (CallStats, error)
- func (e *Executor) LinearWithStats(ctx context.Context, x, w []float32, batch, inDim, outDim int) ([]float32, CallStats, error)
- func (e *Executor) Prepare(w []float32, batch, inDim, outDim int) error
- func (e *Executor) Stats() Stats
- type Options
- type Stats
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DynamicExecutor ¶
type DynamicExecutor struct {
// contains filtered or unexported fields
}
DynamicExecutor caches compile-once ANE kernels keyed by shape and accepts runtime-provided weights on each call.
It preserves the row-major linear API used by Executor:
- x is [batch, inDim]
- w is [outDim, inDim]
- y is [batch, outDim]
func NewDynamic ¶
func NewDynamic(opts Options) *DynamicExecutor
NewDynamic creates a dynamic linear executor.
func (*DynamicExecutor) Close ¶
func (e *DynamicExecutor) Close()
Close releases all cached kernels.
func (*DynamicExecutor) Linear ¶
func (e *DynamicExecutor) Linear(ctx context.Context, x, w []float32, batch, inDim, outDim int) ([]float32, error)
Linear computes x*w^T where x is [batch,inDim] and w is [outDim,inDim].
func (*DynamicExecutor) LinearIOIntoWithStats ¶
func (e *DynamicExecutor) LinearIOIntoWithStats(ctx context.Context, dst, x, wIO []float32, batch, inDim, outDim int) (CallStats, error)
LinearIOIntoWithStats computes x*w where w is already laid out as [inDim,outDim].
This is a fast path for callers that maintain the dynamic kernel's native weight layout and want to avoid transposing [outDim,inDim] on every call.
func (*DynamicExecutor) LinearIntoWithStats ¶
func (e *DynamicExecutor) LinearIntoWithStats(ctx context.Context, dst, x, w []float32, batch, inDim, outDim int) (CallStats, error)
LinearIntoWithStats computes x*w^T into dst and returns per-call execution stats.
func (*DynamicExecutor) LinearOneHotIOIntoWithStats ¶
func (e *DynamicExecutor) LinearOneHotIOIntoWithStats(ctx context.Context, dst []float32, xs []int, batch, inDim, outDim int) (CallStats, error)
LinearOneHotIOIntoWithStats computes y = x*w for one-hot activations encoded by xs and a previously primed IO-layout weight matrix.
func (*DynamicExecutor) LinearWithStats ¶
func (e *DynamicExecutor) LinearWithStats(ctx context.Context, x, w []float32, batch, inDim, outDim int) ([]float32, CallStats, error)
LinearWithStats computes x*w^T and returns per-call execution stats.
func (*DynamicExecutor) Prepare ¶
func (e *DynamicExecutor) Prepare(batch, inDim, outDim int) error
Prepare compiles and caches a dynamic kernel for the provided shape.
func (*DynamicExecutor) PrimeWeightsIO ¶
func (e *DynamicExecutor) PrimeWeightsIO(batch, inDim, outDim int, wIO []float32) error
PrimeWeightsIO stores an IO-layout weight matrix in the cached dynamic kernel.
func (*DynamicExecutor) Stats ¶
func (e *DynamicExecutor) Stats() Stats
Stats returns a snapshot of executor counters.
func (*DynamicExecutor) UpdateWeightsIORows ¶
func (e *DynamicExecutor) UpdateWeightsIORows(batch, inDim, outDim int, wIO []float32, rows []int) error
UpdateWeightsIORows patches a subset of IO-layout weight rows in the cached kernel.
type Executor ¶
type Executor struct {
// contains filtered or unexported fields
}
Executor caches compiled kernels and runs linear forwards.
func (*Executor) Linear ¶
func (e *Executor) Linear(ctx context.Context, x, w []float32, batch, inDim, outDim int) ([]float32, error)
Linear computes x*w^T where x is [batch,inDim] and w is [outDim,inDim].
x and w use row-major layout.
func (*Executor) LinearIntoWithStats ¶
func (e *Executor) LinearIntoWithStats(ctx context.Context, dst, x, w []float32, batch, inDim, outDim int) (CallStats, error)
LinearIntoWithStats computes x*w^T into dst and returns per-call execution stats.
dst must have length batch*outDim and uses row-major layout.
func (*Executor) LinearWithStats ¶
func (e *Executor) LinearWithStats(ctx context.Context, x, w []float32, batch, inDim, outDim int) ([]float32, CallStats, error)
LinearWithStats computes x*w^T and returns per-call execution stats.
Source Files