espresso

package
v0.4.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 21, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package espresso provides high-level access to Apple's Espresso neural network execution framework with multi-backend support (CPU, GPU, ANE).

Unlike github.com/tmc/apple/x/ane, which targets only the Apple Neural Engine, Espresso can dispatch individual layers to the best available backend. This is the same execution path that CoreML uses internally. 

ctx, err := espresso.Open()
if err != nil {
	log.Fatal(err)
}
defer ctx.Close()

net, err := ctx.LoadNetwork(irJSON)
if err != nil {
	log.Fatal(err)
}
defer net.Close()

f := espresso.NewFrame()
f.SetInput("input", data, espresso.Shape{Batch: 1, Channels: 3, Height: 224, Width: 224})
if err := net.Eval(f); err != nil {
	log.Fatal(err)
}
output, _ := f.Output("output")

Architecture

Espresso is Apple's private neural network IR and execution framework. It sits between CoreML/MIL and hardware backends (ANE, GPU via Metal Performance Shaders, CPU). Where x/ane provides direct ANE-only access, Espresso is the multi-backend orchestrator that CoreML uses internally.

The key advantage over x/ane is per-layer backend selection: Espresso can run some layers on ANE, fall back to GPU for unsupported ops, and use CPU as a last resort. x/ane requires the entire model to be ANE-compatible. 

CoreML (.mlmodel) → Espresso IR (JSON + weights)
                         │
              ┌──────────┼──────────┐
              ANE        GPU        CPU

Execution

A Context manages execution state and selects the platform. Context.LoadNetwork loads a network from Espresso IR (JSON + binary weights). Frame binds named input and output tensors using string keys matching the model's tensor names. Network.Eval executes the network.

Profiling

ProfileNetwork collects per-layer execution data including which engine ran each layer, runtime samples, and platform support. This is useful for understanding how Espresso partitions a model across backends.

Optimization Passes

The Pass interface wraps Espresso's built-in optimization passes (fusion, strength reduction, quantization, normalization merging). Use Optimize to apply a standard set, or OptimizeWith for a custom pipeline.

Metal Interop

For networks using the ANE backend, [Network.MetalBuffer] provides zero-copy GPU access to ANE IOSurfaces via Metal buffers.

Example (Argmax) 
package main

import (
	"fmt"

	"github.com/tmc/apple/x/espresso"
)

func main() {
	data := []float32{0.1, 0.7, 0.2}
	fmt.Println(espresso.Argmax(data))
}

Output:
1
Example (Softmax) 
package main

import (
	"fmt"

	"github.com/tmc/apple/x/espresso"
)

func main() {
	data := []float32{1.0, 2.0, 3.0}
	result := espresso.Softmax(data)
	fmt.Printf("%.4f %.4f %.4f\n", result[0], result[1], result[2])
}

Output:
0.0900 0.2447 0.6652
Example (TopK) 
package main

import (
	"fmt"

	"github.com/tmc/apple/x/espresso"
)

func main() {
	data := []float32{0.1, 0.9, 0.3, 0.7, 0.5}
	top3 := espresso.TopK(data, 3)
	fmt.Println(top3)
}

Output:
[1 3 4]
Example (Types) 
package main

import (
	"fmt"

	"github.com/tmc/apple/x/espresso"
)

func main() {
	fmt.Println(espresso.PlatformAuto)
	fmt.Println(espresso.PlatformCPU)
	fmt.Println(espresso.PlatformGPU)
	fmt.Println(espresso.PlatformANE)

	s := espresso.Shape{Batch: 1, Channels: 3, Height: 224, Width: 224}
	fmt.Println(s.Elements())

	fmt.Println(espresso.EngineCPU)
	fmt.Println(espresso.EngineGPU)
	fmt.Println(espresso.EngineANE)

}

Output:
auto
cpu
gpu
ane
150528
cpu
gpu
ane

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	ErrFramework    = errors.New("espresso: framework not available")
	ErrLoad         = errors.New("espresso: network load failed")
	ErrNilNetwork   = errors.New("espresso: nil network returned")
	ErrEval         = errors.New("espresso: evaluation failed")
	ErrBindInput    = errors.New("espresso: input binding failed")
	ErrBindOutput   = errors.New("espresso: output binding failed")
	ErrNoSuchOutput = errors.New("espresso: output not found")
	ErrClosed       = errors.New("espresso: context closed")
)
View Source 
var ErrUnsupported = errors.New("espresso: unsupported platform")

ErrUnsupported is returned on platforms that do not support Espresso.

Functions

func Argmax 

func Argmax(data []float32) int

Argmax returns the index of the maximum value in the slice.

func IsMemoryTight 

func IsMemoryTight() bool

IsMemoryTight returns whether Espresso's shared Metal singleton reports memory pressure.

func Optimize 

func Optimize(_ *Network) error

Optimize applies a standard set of optimization passes.

func OptimizeWith 

func OptimizeWith(_ *Network, _ ...Pass) error

OptimizeWith applies the given passes in order.

func Softmax 

func Softmax(data []float32) []float32

Softmax applies softmax normalization to a float32 slice in-place and returns it.

func TopK 

func TopK(data []float32, k int) []int

TopK returns the indices of the K largest values in descending order.

Types

type ANEProfile 

type ANEProfile struct {
	ANETimePerEvalNS  uint64
	TotalANETimeNS    uint64
	CompilerFileNames []string
}

ANEProfile contains ANE-specific profiling data.

type ANESurface 

type ANESurface struct{}

ANESurface provides access to Espresso's ANE IOSurface for zero-copy Metal buffer creation and multi-buffer async execution.

func NewANESurface 

func NewANESurface() *ANESurface

NewANESurface creates a new ANE IOSurface wrapper.

func NewANESurfaceFromLayout 

func NewANESurfaceFromLayout(_ SurfaceLayout) *ANESurface

NewANESurfaceFromLayout creates an ANE IOSurface with explicit model-driven dimensions.

func NewANESurfaceWithProperties 

func NewANESurfaceWithProperties(_, _ any) *ANESurface

NewANESurfaceWithProperties creates an ANE IOSurface with specific properties and pixel formats.

func (*ANESurface) AliasingMem 

func (a *ANESurface) AliasingMem() any

AliasingMem returns the external storage blob used for aliasing memory.

func (*ANESurface) Cleanup 

func (a *ANESurface) Cleanup()

Cleanup releases surface resources.

func (*ANESurface) CreateIOSurface 

func (a *ANESurface) CreateIOSurface(_ any) uintptr

CreateIOSurface creates a new IOSurface with optional extra properties.

func (*ANESurface) ForceAlloc 

func (a *ANESurface) ForceAlloc(_ any)

ForceAlloc forces non-lazy allocation.

func (*ANESurface) IOSurfaceForFrame 

func (a *ANESurface) IOSurfaceForFrame(_ uint64) (uintptr, error)

IOSurfaceForFrame returns the IOSurface backing the given multi-buffer frame.

func (*ANESurface) LazilyAllocFrame 

func (a *ANESurface) LazilyAllocFrame(_ uint64)

LazilyAllocFrame lazily allocates an IOSurface for the given frame.

func (*ANESurface) MatchesCVImageBuffer 

func (a *ANESurface) MatchesCVImageBuffer(_ uintptr) bool

MatchesCVImageBuffer returns whether the surface matches a CVImageBuffer.

func (*ANESurface) MatchesIOSurface 

func (a *ANESurface) MatchesIOSurface(_ uintptr) bool

MatchesIOSurface returns whether the surface matches an IOSurfaceRef.

func (*ANESurface) NFrames 

func (a *ANESurface) NFrames() uint64

NFrames returns the number of allocated multi-buffer frames.

func (*ANESurface) PixelFormat 

func (a *ANESurface) PixelFormat() uint32

PixelFormat returns the surface pixel format.

func (*ANESurface) ReadFrame 

func (a *ANESurface) ReadFrame(_ uint64, _ []byte) error

ReadFrame reads raw bytes from the IOSurface backing the given frame.

func (*ANESurface) ReadFrameF32 

func (a *ANESurface) ReadFrameF32(_ uint64, _ []float32) error

ReadFrameF32 reads float32 data from the IOSurface backing the given frame.

func (*ANESurface) ResizeForAsync 

func (a *ANESurface) ResizeForAsync(_ uint64)

ResizeForAsync resizes the surface for multiple async buffer frames.

func (*ANESurface) RestoreAllInternalStorage 

func (a *ANESurface) RestoreAllInternalStorage()

RestoreAllInternalStorage restores internal storage for all multi-buffer frames.

func (*ANESurface) RestoreInternalStorage 

func (a *ANESurface) RestoreInternalStorage(_ uint64)

RestoreInternalStorage restores internal storage for a single slot.

func (*ANESurface) SetAliasingMem 

func (a *ANESurface) SetAliasingMem(_ any)

SetAliasingMem sets the external storage blob for aliasing memory.

func (*ANESurface) SetExternalStorage 

func (a *ANESurface) SetExternalStorage(_ uint64, _ uintptr)

SetExternalStorage replaces the IOSurface backing a storage slot.

func (*ANESurface) WriteFrame 

func (a *ANESurface) WriteFrame(_ uint64, _ []byte) error

WriteFrame writes raw bytes to the IOSurface backing the given frame.

func (*ANESurface) WriteFrameF32 

func (a *ANESurface) WriteFrameF32(_ uint64, _ []float32) error

WriteFrameF32 writes float32 data to the IOSurface backing the given frame.

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder constructs Espresso IR JSON (model.espresso.net + model.espresso.shape) and optional weight binaries for inference on Apple's Espresso framework.

func NewBuilder 

func NewBuilder() *Builder

NewBuilder creates a new Espresso IR builder with format version 200.

func (*Builder) Activation 

func (b *Builder) Activation(name, mode string, opts ...LayerOption) *Builder

Activation adds an activation layer.

func (*Builder) BatchNorm 

func (b *Builder) BatchNorm(name string, channels int, opts ...LayerOption) *Builder

BatchNorm adds a batch normalization layer.

func (*Builder) Build 

func (b *Builder) Build() (netJSON []byte, shapeJSON []byte, err error)

Build produces the Espresso IR JSON files (net and shape).

func (*Builder) BuildWithWeights 

func (b *Builder) BuildWithWeights() (netJSON, shapeJSON, weightsBin []byte, err error)

BuildWithWeights produces IR JSON and a weight binary.

func (*Builder) Concat 

func (b *Builder) Concat(name string, opts ...LayerOption) *Builder

Concat adds a concatenation layer.

func (*Builder) Convolution 

func (b *Builder) Convolution(name string, nOutput, kernelH, kernelW int, opts ...LayerOption) *Builder

Convolution adds a convolution layer.

func (*Builder) Elementwise 

func (b *Builder) Elementwise(name, operation string, opts ...LayerOption) *Builder

Elementwise adds an elementwise operation layer with a string operation name.

func (*Builder) ElementwiseOped 

func (b *Builder) ElementwiseOped(name string, op ElementwiseOp, opts ...LayerOption) *Builder

ElementwiseOped adds an elementwise operation layer with a typed operation code.

func (*Builder) InnerProduct 

func (b *Builder) InnerProduct(name string, nOutput int, opts ...LayerOption) *Builder

InnerProduct adds a fully connected layer.

func (*Builder) Pooling 

func (b *Builder) Pooling(name, mode string, kernelH, kernelW int, opts ...LayerOption) *Builder

Pooling adds a pooling layer.

func (*Builder) Reshape 

func (b *Builder) Reshape(name string, shape [4]int, opts ...LayerOption) *Builder

Reshape adds a reshape layer.

func (*Builder) Softmax 

func (b *Builder) Softmax(name string, opts ...LayerOption) *Builder

Softmax adds a softmax layer.

type Context 

type Context struct{}

Context manages execution state for Espresso networks.

func Open 

func Open(_ ...ContextOption) (*Context, error)

Open creates a new Espresso execution context.

func (*Context) Close 

func (c *Context) Close() error

Close releases context resources.

func (*Context) LoadNetwork 

func (c *Context) LoadNetwork(_ []byte, _ ...NetworkOption) (*Network, error)

LoadNetwork loads a network from Espresso IR.

func (*Context) LoadNetworkFromFile 

func (c *Context) LoadNetworkFromFile(_ string, _ ...NetworkOption) (*Network, error)

LoadNetworkFromFile loads a network from an Espresso net.json file.

func (*Context) Platform 

func (c *Context) Platform() int

Platform returns the platform this context was created with.

type ContextOption 

type ContextOption func(*contextConfig)

ContextOption configures a Context.

func WithGPUPriority 

func WithGPUPriority(_ uint32) ContextOption

WithGPUPriority sets the GPU command buffer priority.

func WithLowPriority 

func WithLowPriority(_ float32) ContextOption

WithLowPriority sets the max ms per command buffer for low priority.

func WithPlatform 

func WithPlatform(_ Platform) ContextOption

WithPlatform selects the execution backend.

type ElementwiseOp 

type ElementwiseOp int

ElementwiseOp identifies an Espresso elementwise operation by its numeric code. 

const (
	ElementwiseAdd   ElementwiseOp = 0
	ElementwiseMul   ElementwiseOp = 1
	ElementwiseMax   ElementwiseOp = 2
	ElementwiseMin   ElementwiseOp = 3
	ElementwiseSub   ElementwiseOp = 5
	ElementwiseDiv   ElementwiseOp = 6
	ElementwiseSqrt  ElementwiseOp = 10
	ElementwiseExp   ElementwiseOp = 11
	ElementwiseRsqrt ElementwiseOp = 12
	ElementwiseLog   ElementwiseOp = 13
	ElementwisePow   ElementwiseOp = 14
	ElementwiseAbs   ElementwiseOp = 15
)

Espresso elementwise operation codes.

type Engine 

type Engine int

Engine identifies which hardware executed a layer. 

const (
	EngineCPU Engine = iota
	EngineGPU
	EngineANE
)

func (Engine) String 

func (e Engine) String() string

type Error 

type Error struct {
	Op  string
	Err error
}

Error wraps an error from the Espresso subsystem.

func (*Error) Error 

func (e *Error) Error() string

func (*Error) Unwrap 

func (e *Error) Unwrap() error

type EvalOptions 

type EvalOptions struct {
	UseCVPixelBuffers bool
	Timeout           time.Duration
}

EvalOptions configures execution behavior.

type Frame 

type Frame struct{}

Frame holds named input and output tensors for network execution.

func NewFrame 

func NewFrame() *Frame

NewFrame creates an empty data frame.

func (*Frame) FunctionName 

func (f *Frame) FunctionName() string

FunctionName returns the function name set on this frame.

func (*Frame) InputNames 

func (f *Frame) InputNames() []string

InputNames returns the names of all input attachments.

func (*Frame) Output 

func (f *Frame) Output(_ string) ([]float32, error)

Output returns the named output tensor data.

func (*Frame) OutputFloat64 

func (f *Frame) OutputFloat64(_ string) ([]float64, error)

OutputFloat64 returns the named output as float64 values.

func (*Frame) OutputInto 

func (f *Frame) OutputInto(_ string, _ []float32) error

OutputInto copies the named output tensor data into the caller's buffer.

func (*Frame) OutputNames 

func (f *Frame) OutputNames() []string

OutputNames returns the names of all output attachments.

func (*Frame) OutputRaw 

func (f *Frame) OutputRaw(_ string) ([]byte, error)

OutputRaw returns the raw bytes of a named output.

func (*Frame) SetFunctionName 

func (f *Frame) SetFunctionName(_ string)

SetFunctionName sets the function name.

func (*Frame) SetGroundTruth 

func (f *Frame) SetGroundTruth(_ string, _ []float32, _ Shape) error

SetGroundTruth attaches a named ground truth tensor.

func (*Frame) SetInput 

func (f *Frame) SetInput(_ string, _ []float32, _ Shape) error

SetInput attaches a named float32 input tensor.

func (*Frame) SetInputBytes 

func (f *Frame) SetInputBytes(_ string, _ []byte) error

SetInputBytes attaches a named raw byte input.

func (*Frame) SetInputCVPixelBuffer 

func (f *Frame) SetInputCVPixelBuffer(_ string, _ uintptr) error

SetInputCVPixelBuffer attaches a named input from a CVPixelBuffer.

func (*Frame) SetInputIOSurface 

func (f *Frame) SetInputIOSurface(_ string, _ uintptr) error

SetInputIOSurface attaches a named input backed by an IOSurface.

func (*Frame) SetInputImage 

func (f *Frame) SetInputImage(_ string, _ []byte, _ int) error

SetInputImage attaches a named image input with raw pixel data and channel count.

type FrameStorage 

type FrameStorage struct{}

FrameStorage holds reusable multi-frame storage.

func NewFrameStorage 

func NewFrameStorage(_ ...*Frame) *FrameStorage

NewFrameStorage creates reusable storage from the given frames.

type LayerOption 

type LayerOption func(*layerDef)

LayerOption configures a layer definition.

func WithBias 

func WithBias(data []float32) LayerOption

WithBias assigns float32 bias data to a layer.

func WithBottom 

func WithBottom(names ...string) LayerOption

WithBottom sets the input blob names for a layer.

func WithGroups 

func WithGroups(g int) LayerOption

WithGroups sets the convolution groups.

func WithHasReLU 

func WithHasReLU(v bool) LayerOption

WithHasReLU enables fused ReLU on the layer.

func WithPad 

func WithPad(t, b, l, r int) LayerOption

WithPad sets padding [top, bottom, left, right].

func WithStride 

func WithStride(h, w int) LayerOption

WithStride sets stride [height, width].

func WithTop 

func WithTop(names ...string) LayerOption

WithTop sets the output blob names for a layer.

func WithWeights 

func WithWeights(data []float32) LayerOption

WithWeights assigns float32 weight data to a layer.

type LayerProfile 

type LayerProfile struct {
	Name           string
	DebugName      string
	AvgRuntimeMS   float64
	RuntimeSamples []float64
	Engine         Engine
	Supported      bool
	HasPerfWarning bool
	LayerType      string
}

LayerProfile contains profiling data for a single layer.

func (*LayerProfile) IsANEResident 

func (l *LayerProfile) IsANEResident() bool

IsANEResident reports whether this layer runs on the ANE.

type LayerShape 

type LayerShape struct {
	Name string // blob name
	N    int    // batch
	C    int    // channels
	H    int    // height
	W    int    // width
}

LayerShape describes the tensor shape for a layer blob as parsed from model.espresso.shape JSON.

func (LayerShape) Elements 

func (s LayerShape) Elements() int

Elements returns the total number of elements in this shape.

type MetalDevice 

type MetalDevice struct{}

MetalDevice wraps a Metal GPU device for zero-copy interop with Espresso.

func OpenMetal 

func OpenMetal() (*MetalDevice, error)

OpenMetal opens the system default Metal device.

func (*MetalDevice) Close 

func (d *MetalDevice) Close() error

Close releases the Metal device.

type Network 

type Network struct {
	// contains filtered or unexported fields
}

Network is a loaded Espresso neural network ready for execution.

func (*Network) Close 

func (n *Network) Close() error

Close releases network resources.

func (*Network) Eval 

func (n *Network) Eval(_ *Frame) error

Eval executes the network on the given frame.

func (*Network) EvalAsync 

func (n *Network) EvalAsync(_ *Frame) <-chan error

EvalAsync executes the network asynchronously.

func (*Network) EvalAsyncWithCallback 

func (n *Network) EvalAsyncWithCallback(_ *Frame, fn func(error))

EvalAsyncWithCallback executes the network asynchronously with a callback.

func (*Network) EvalWithOptions 

func (n *Network) EvalWithOptions(_ *Frame, _ EvalOptions) error

EvalWithOptions executes with custom options.

func (*Network) EvalWithStorage 

func (n *Network) EvalWithStorage(_ *FrameStorage, _ EvalOptions) error

EvalWithStorage executes the network using pre-allocated storage.

func (*Network) HasLayer 

func (n *Network) HasLayer(_ string) bool

HasLayer reports whether the network contains a layer with the given name.

func (*Network) LayerCount 

func (n *Network) LayerCount() int

LayerCount returns the number of layers.

func (*Network) LayerNames 

func (n *Network) LayerNames() []string

LayerNames returns the names of all layers.

type NetworkOption 

type NetworkOption func(*networkConfig)

NetworkOption configures network loading.

func WithBinSerializerID 

func WithBinSerializerID(_ []byte) NetworkOption

WithBinSerializerID sets the binary serializer ID.

func WithComputePath 

func WithComputePath(_ int) NetworkOption

WithComputePath sets the compute path.

type Pass 

type Pass interface {
	// Name returns the pass identifier.
	Name() string
	// Run applies the optimization to the network. Returns true if modified.
	Run(n *Network) bool
}

Pass is a network optimization pass that transforms an Espresso network.

type Platform 

type Platform int

Platform selects the execution backend. 

const (
	PlatformAuto Platform = iota // Espresso selects per-layer
	PlatformCPU
	PlatformGPU
	PlatformANE
)

func (Platform) String 

func (p Platform) String() string

type Profile 

type Profile struct {
	Path   string
	Layers []LayerProfile
	ANE    *ANEProfile
}

Profile contains execution profiling data for a network.

func ProfileNetwork 

func ProfileNetwork(_ *Network) (*Profile, error)

ProfileNetwork extracts profiling data from a loaded network.

func (*Profile) ANEResidentLayers 

func (p *Profile) ANEResidentLayers() []LayerProfile

ANEResidentLayers returns layers that are resident on the ANE.

func (*Profile) CPUFallbackLayers 

func (p *Profile) CPUFallbackLayers() []LayerProfile

CPUFallbackLayers returns supported layers that fell back to CPU instead of ANE.

func (*Profile) IsFullyANEResident 

func (p *Profile) IsFullyANEResident() bool

IsFullyANEResident reports whether all supported layers run on the ANE.

func (*Profile) LayersByEngine 

func (p *Profile) LayersByEngine() map[Engine][]LayerProfile

LayersByEngine returns layers grouped by their execution engine.

func (*Profile) TotalRuntimeMS 

func (p *Profile) TotalRuntimeMS() float64

TotalRuntimeMS returns the sum of average runtimes across all layers.

func (*Profile) UnsupportedLayers 

func (p *Profile) UnsupportedLayers() []LayerProfile

UnsupportedLayers returns layers that are not supported by the main engine.

type Shape 

type Shape struct {
	Batch    int
	Channels int
	Height   int
	Width    int
	Sequence int // for RNN/transformer models
}

Shape describes tensor dimensions.

func (Shape) Elements 

func (s Shape) Elements() int

Elements returns the total number of elements.

type ShapeMap 

type ShapeMap struct {
	Shapes map[string]LayerShape
}

ShapeMap holds parsed shape information for all blobs in a model.

func ReadEspressoShape 

func ReadEspressoShape(data []byte) (*ShapeMap, error)

ReadEspressoShape parses a model.espresso.shape JSON file into a ShapeMap.

The shape JSON has the structure: 

{
  "layer_shapes": {
    "blob_name": { "n": 1, "k": 64, "h": 1, "w": 128 },
    ...
  }
}

where k=channels, h=height, w=width, n=batch.

func ReadEspressoShapeFile 

func ReadEspressoShapeFile(path string) (*ShapeMap, error)

ReadEspressoShapeFile reads and parses a model.espresso.shape JSON file.

func (*ShapeMap) Get 

func (m *ShapeMap) Get(name string) (LayerShape, bool)

Get returns the shape for a named blob.

func (*ShapeMap) Names 

func (m *ShapeMap) Names() []string

Names returns sorted blob names.

type SurfaceLayout 

type SurfaceLayout struct {
	Width       int
	Height      int
	BytesPerRow int
	AllocSize   int
	ElemSize    int
}

SurfaceLayout describes the IOSurface dimensions required by a compiled model.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.