options

package
v0.6.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 17, 2026 License: Apache-2.0 Imports: 3 Imported by: 3

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type ExtraExecutionProvider added in v0.6.3 

type ExtraExecutionProvider struct {
	Name    string
	Options map[string]string
}

type GoMLXOptions added in v0.4.0 

type GoMLXOptions struct {
	// BatchBuckets defines the bucket sizes for batch dimension padding.
	// Coarse bucketing reduces JIT cache pressure by limiting unique shapes.
	// Default: []int{1, 8, 32}
	BatchBuckets []int
	// SequenceBuckets defines the bucket sizes for sequence length padding.
	// Coarse bucketing reduces JIT cache pressure by limiting unique shapes.
	// Default: []int{32, 128, 512}
	SequenceBuckets []int
	Cuda            bool
	XLA             bool
	TPU             bool
}

type GraphOptimizationLevel added in v0.6.3 

type GraphOptimizationLevel int
const (
	GraphOptimizationLevelDisableAll     GraphOptimizationLevel = 0
	GraphOptimizationLevelEnableBasic    GraphOptimizationLevel = 1
	GraphOptimizationLevelEnableExtended GraphOptimizationLevel = 2
	GraphOptimizationLevelEnableAll      GraphOptimizationLevel = 99
)

type LoggingLevel added in v0.6.3 

type LoggingLevel int
const (
	LoggingLevelVerbose LoggingLevel = 0
	LoggingLevelInfo    LoggingLevel = 1
	LoggingLevelWarning LoggingLevel = 2
	LoggingLevelError   LoggingLevel = 3
	LoggingLevelFatal   LoggingLevel = 4
)

type Options 

type Options struct {
	BackendOptions any
	ORTOptions     *OrtOptions
	GoMLXOptions   *GoMLXOptions
	Destroy        func() error
	Backend        string
}

func Defaults 

func Defaults() *Options

type OrtOptions 

type OrtOptions struct {
	LibraryPath             *string
	LibraryDir              *string
	Telemetry               *bool
	IntraOpNumThreads       *int
	InterOpNumThreads       *int
	CPUMemArena             *bool
	MemPattern              *bool
	ParallelExecutionMode   *bool
	IntraOpSpinning         *bool
	InterOpSpinning         *bool
	LogSeverityLevel        *LoggingLevel
	EnvLoggingLevel         *LoggingLevel
	GraphOptimizationLevel  *GraphOptimizationLevel
	CudaOptions             map[string]string
	CoreMLOptions           map[string]string
	DirectMLOptions         *int
	OpenVINOOptions         map[string]string
	TensorRTOptions         map[string]string
	NvTensorRTRTXOptions    map[string]string
	ExtraExecutionProviders []ExtraExecutionProvider
	OptimizedModelFilePath  *string
	ProfilingEnabled        *bool
	ProfilingFilePrefix     *string
}

type WithOption 

type WithOption func(o *Options) error

WithOption is the interface for all option functions.

func WithCPUMemArena added in v0.5.9 

func WithCPUMemArena(enable bool) WithOption

WithCPUMemArena (ORT only) Enable/Disable the usage of the memory arena on CPU. Arena may pre-allocate memory for future usage. Default is true.

func WithCoreML 

func WithCoreML(flags map[string]string) WithOption

WithCoreML (ORT only) Use this function to set the CoreML options flags for the ONNX backend configuration. The `flags` parameter represents the CoreML options flags. The `o.CoreMLOptions` field in `OrtOptions` struct will be set to the provided flags parameter.

func WithCuda 

func WithCuda(options map[string]string) WithOption

WithCuda Use this function to set the options for CUDA provider. It takes a map of CUDA parameters as input. The options will be applied to the OrtOptions or GoMLXOptions struct, depending on your current backend.

func WithDirectML 

func WithDirectML(deviceID int) WithOption

WithDirectML (ORT only) Use this function to set the DirectML device ID for the onnxbackend. By default, this option is not set.

func WithEnvLoggingLevel added in v0.6.3 

func WithEnvLoggingLevel(level LoggingLevel) WithOption

WithEnvLoggingLevel (ORT only) Sets the log severity level for the environment.

func WithExecutionMode added in v0.3.1 

func WithExecutionMode(parallel bool) WithOption

WithExecutionMode sets the parallel execution mode for the ORT backend. Returns an error if the backend is not ORT.

func WithExtraExecutionProvider added in v0.6.3 

func WithExtraExecutionProvider(name string, options map[string]string) WithOption

WithExtraExecutionProvider (ORT only) Adds an extra execution provider to the session.

func WithGoMLXBatchBuckets added in v0.6.1 

func WithGoMLXBatchBuckets(buckets []int) WithOption

WithGoMLXBatchBuckets (XLA and GO only) sets the bucket sizes for batch dimension padding. Inputs are padded to the smallest bucket >= their batch size. Fewer/coarser buckets reduce JIT cache pressure but increase padding overhead. Default is []int{1, 8, 32}. IMPORTANT: Ensure MaxCache >= len(BatchBuckets) * len(SequenceBuckets).

func WithGoMLXSequenceBuckets added in v0.6.1 

func WithGoMLXSequenceBuckets(buckets []int) WithOption

WithGoMLXSequenceBuckets (XLA and GO only) sets the bucket sizes for sequence length padding. Inputs are padded to the smallest bucket >= their sequence length. Fewer/coarser buckets reduce JIT cache pressure but increase padding overhead. Default is []int{32, 128, 512}. IMPORTANT: Ensure MaxCache >= len(BatchBuckets) * len(SequenceBuckets).

func WithGraphOptimizationLevel added in v0.6.3 

func WithGraphOptimizationLevel(level GraphOptimizationLevel) WithOption

WithGraphOptimizationLevel (ORT only) Sets the graph optimization level for the session.

func WithInterOpNumThreads 

func WithInterOpNumThreads(numThreads int) WithOption

WithInterOpNumThreads (ORT only) Sets the number of threads used to parallelize execution across separate onnxbackend graph nodes. If unspecified, onnxbackend uses the number of physical CPU cores.

func WithInterOpSpinning added in v0.3.1 

func WithInterOpSpinning(spinning bool) WithOption

WithInterOpSpinning sets the spinning behavior for inter-op threads when the backend is ORT. It returns an error if used with a backend other than ORT.

func WithIntraOpNumThreads 

func WithIntraOpNumThreads(numThreads int) WithOption

WithIntraOpNumThreads (ORT only) Sets the number of threads used to parallelize execution within onnxbackend graph nodes. If unspecified, onnxbackend uses the number of physical CPU cores.

func WithIntraOpSpinning added in v0.3.1 

func WithIntraOpSpinning(spinning bool) WithOption

WithIntraOpSpinning configures whether intra-op spinning is enabled for the ORT backend. It returns an error if used with a backend other than ORT.

func WithLogSeverityLevel added in v0.6.3 

func WithLogSeverityLevel(level LoggingLevel) WithOption

WithLogSeverityLevel (ORT only) Sets the log severity level for the session.

func WithMemPattern 

func WithMemPattern(enable bool) WithOption

WithMemPattern (ORT only) Enable/Disable the memory pattern optimization. If this is enabled memory is preallocated if all shapes are known. Default is true.

func WithNvTensorRTRTX added in v0.6.3 

func WithNvTensorRTRTX(options map[string]string) WithOption

WithNvTensorRTRTX (ORT only) Use this function to set the options for the NvTensorRTRTX provider. The options parameter should be a pointer to an instance of NvTensorRTRTXOptions. By default, the options will be nil and the NvTensorRTRTX provider will not be used. Example usage: 

WithNvTensorRTRTX(map[string]string{"device_id": "0"})

Note: For the TensorRT provider to work, the onnxbackend library must be built with TensorRT support.

func WithOnnxLibraryPath 

func WithOnnxLibraryPath(ortLibraryPath string) WithOption

WithOnnxLibraryPath (ORT only) Use this function to set the path to the "libonnxuntime.so", "libonnxuntime.dylib" or "onnxruntime.dll" files.

func WithOpenVINO 

func WithOpenVINO(options map[string]string) WithOption

WithOpenVINO (ORT only) Use this function to set the OpenVINO options for the OpenVINO execution provider. The options parameter should be a map of string keys and string values, representing the configuration options. For each key-value pair in the map, the specified option will be set in the OpenVINO execution provider. Example usage: WithOpenVINO(map[string]string{"device_type": "CPU", "num_threads": "4"}) This will configure the OpenVINO execution provider to use CPU as the device type and set the number of threads to 4.

func WithOptimizedModelFilePath added in v0.6.5 

func WithOptimizedModelFilePath(path string) WithOption

WithOptimizedModelFilePath (ORT only) Sets the path to the optimized model file.

func WithProfiling added in v0.6.5 

func WithProfiling(enabled bool, filePrefix string) WithOption

WithProfiling (ORT only) Enables or disables profiling.

func WithTPU added in v0.6.0 

func WithTPU() WithOption

WithTPU (XLA only) Use this function to enable TPU acceleration for the XLA backend. Requires libtpu.so to be available (pre-installed on GKE TPU nodes). Set PJRT_PLUGIN_LIBRARY_PATH to the directory containing pjrt_plugin_tpu.so or libtpu.so.

func WithTelemetry 

func WithTelemetry() WithOption

WithTelemetry (ORT only) Enables telemetry events for the onnxbackend environment. Default is off.

func WithTensorRT 

func WithTensorRT(options map[string]string) WithOption

WithTensorRT (ORT only) Use this function to set the options for the TensorRT provider. The options parameter should be a pointer to an instance of TensorRTProviderOptions. By default, the options will be nil and the TensorRT provider will not be used. Example usage: 

WithTensorRT(map[string]string{"device_id": "0"})

Note: For the TensorRT provider to work, the onnxbackend library must be built with TensorRT support.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.