README
¶
go-pytorch
Go binding for Pytorch C++ API. This is used by the Pytorch agent in MLModelScope to perform model inference in Go.
Installation
The binding requires Pytorch C++ (libtorch) and other Go packages.
Pytorch C++ (libtorch) Library
The Pytorch C++ library is expected to be under /opt/libtorch.
The binding is built using libtorch 1.8.1.
To install Pytorch C++ on your system, you can
-
download pre-built binary from Pytorch website: Choose
Pytorch Build = Stable,Your OS = <fill>,Package = LibTorch,Language = C++andCUDA = <fill>. Then downloadcxx11 ABIversion. Unzip the packaged directory and copy to/opt/libtorch(or modify the correspondingCFLAGSandLDFLAGSpaths if using a custom location). -
build it from source: Refer to our scripts or the
LIBRARY INSTALLATIONsection in the dockefiles.
-
The default blas is OpenBLAS. The default OpenBLAS path for macOS is
/usr/local/opt/openblasif installed throught homebrew (openblas is keg-only, which means it was not symlinked into /usr/local, because macOS provides BLAS and LAPACK in the Accelerate framework). -
The default pytorch C++ installation path is
/opt/libtorchfor linux, darwin and ppc64le without powerai -
The default CUDA path is
/usr/local/cuda
See lib.go for details.
If you get an error about not being able to write to /opt then perform the following
sudo mkdir -p /opt/libtorch
sudo chown -R `whoami` /opt/libtorch
If you are using Pytorch docker images or other libary paths, change CGO_CFLAGS, CGO_CXXFLAGS and CGO_LDFLAGS enviroment variables. Refer to Using cgo with the go command.
For example,
export CGO_CFLAGS="${CGO_CFLAGS} -I/tmp/libtorch/include"
export CGO_CXXFLAGS="${CGO_CXXFLAGS} -I/tmp/libtorch/include"
export CGO_LDFLAGS="${CGO_LDFLAGS} -L/tmp/libtorch/lib"
There is an issue when using libtorch with version < 1.6.0, the work around here is to set LRU_CACHE_CAPACITY=1 in the environmental variable.
Configure Environmental Variables
Configure the linker environmental variables since the Pytorch C++ library is under a non-system directory. Place the following in either your ~/.bashrc or ~/.zshrc file
Linux
export LIBRARY_PATH=$LIBRARY_PATH:/opt/libtorch/lib
export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/opt/libtorch/lib
macOS
export LIBRARY_PATH=$LIBRARY_PATH:/opt/libtorch/lib
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/opt/libtorch/lib
Check the Build
Run go build to check the dependences installation and library paths set-up.
On linux, the default is to use GPU, if you don't have a GPU, do go build -tags=nogpu instead of go build.
Note : The CGO interface passes go pointers to the C API. This is an error by the CGO runtime. Disable the error by placing
export GODEBUG=cgocheck=0
in your ~/.bashrc or ~/.zshrc file and then run either source ~/.bashrc or source ~/.zshrc
Credits
Parts of the implementation have been borrowed from orktes/go-torch
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( Version = "1.3.1" BuildDate = "undefined" GitCommit = "undefined" )
Functions ¶
func GetErrorString ¶ added in v0.1.0
func GetErrorString() string
func PanicOnError ¶ added in v0.1.0
func PanicOnError()
func PrintTensors ¶ added in v0.1.0
func PrintTensors(inputs ...*Tensor)
PrintTensors prints tensors contents
func ResetError ¶ added in v0.1.0
func ResetError()
Types ¶
type DeviceKind ¶ added in v0.1.0
type DeviceKind C.Torch_DeviceKind
const ( UnknownDeviceKind DeviceKind = C.UNKNOWN_DEVICE_KIND CPUDeviceKind DeviceKind = C.CPU_DEVICE_KIND CUDADeviceKind DeviceKind = C.CUDA_DEVICE_KIND )
type Error ¶ added in v0.1.0
type Error struct {
// contains filtered or unexported fields
}
Error errors returned by torch functions
type Predictor ¶
type Predictor struct {
// contains filtered or unexported fields
}
func (*Predictor) ReadPredictionOutput ¶
func (*Predictor) ReadProfile ¶ added in v0.1.0
type Tensor ¶ added in v0.1.0
type Tensor struct {
// contains filtered or unexported fields
}
Tensor holds a multi-dimensional array of elements of a single data type.
func NewTensor ¶ added in v0.1.0
func NewTensor(value interface{}, device DeviceKind, deviceId int) (*Tensor, error)
NewTensor converts from a Go value to a Tensor. Valid values are scalars, slices, and arrays. Every element of a slice must have the same length so that the resulting Tensor has a valid shape.
func NewTensorWithShape ¶ added in v0.1.0
func NewTensorWithShape(value interface{}, shape []int64, dt DType, device DeviceKind, deviceId int) (*Tensor, error)
NewTensorWithShape converts a single dimensional Go array or slice into a Tensor with given shape
type Trace ¶ added in v0.1.0
type Trace struct {
StartTime time.Time
TraceEvents TraceEvents
}
func (*Trace) Publish ¶ added in v0.1.0
func (t *Trace) Publish(ctx context.Context, lvl tracer.Level, opts ...opentracing.StartSpanOption) error
type TraceEvent ¶ added in v0.1.0
type TraceEvent struct {
Name string `json:"name,omitempty"`
Phase string `json:"ph,omitempty"`
Timestamp float64 `json:"ts,omitempty"`
Duration float64 `json:"dur,omitempty"`
ProcessID string `json:"pid,omitempty"`
ThreadID int64 `json:"tid,omitempty"`
Shape string `json:"shape,omitempty"`
AllocatedMemory int64 `json:"allocated_memory,omitempty"`
PeakMemory int64 `json:"peak_memory,omitempty"`
Index int64 `json:"layer_sequence_index,omitempty"`
Start int64 `json:"-"`
End int64 `json:"-"`
StartTime time.Time `json:"-"`
EndTime time.Time `json:"-"`
Seq int64 `json:"-"`
}
func (TraceEvent) ID ¶ added in v0.1.0
func (t TraceEvent) ID() string
func (*TraceEvent) Publish ¶ added in v0.1.0
func (event *TraceEvent) Publish(ctx context.Context, lvl tracer.Level, opts ...opentracing.StartSpanOption) error
type TraceEvents ¶ added in v0.1.0
type TraceEvents []TraceEvent
func (TraceEvents) Len ¶ added in v0.1.0
func (t TraceEvents) Len() int
func (TraceEvents) Less ¶ added in v0.1.0
func (t TraceEvents) Less(i, j int) bool
func (TraceEvents) Swap ¶ added in v0.1.0
func (t TraceEvents) Swap(i, j int)
Source Files