Documentation
¶
Index ¶
- Variables
- func AddBiasLastDim(outDev, biasDev DevicePtr, totalElems, lastDim int) error
- func AppMemFromBaseline(current MemStats) (appUsed, appFree uint64, ok bool)
- func ClearScratch()
- func DeviceDownloadF32(p DevicePtr, data []float32, elemOff, nElems int) error
- func DeviceDownloadSlice(dev DevicePtr, data []float32, hostOff, n int) error
- func DeviceFree(p DevicePtr)
- func DeviceMemLine() string
- func DeviceMemset(p DevicePtr, nElems int)
- func DeviceMemsetAt(p DevicePtr, elemOff, nElems int)
- func DeviceUploadF32(p DevicePtr, data []float32, elemOff, nElems int) error
- func DeviceUploadSlice(dev DevicePtr, data []float32, hostOff, n int) error
- func GemmDevice(aDev, bDev, cDev DevicePtr, M, N, K int) error
- func GemmDeviceAt(aDev, bDev, cDev DevicePtr, aOff, bOff, cOff, M, N, K int) error
- func GemmF32(A, B, C []float32, M, N, K int) error
- func GemmStridedBatchedAt(aDev, bDev, cDev DevicePtr, aOff, bOff, cOff, M, N, K, batchCount int) error
- func Init() error
- func RecordMemBaseline()
- func Shutdown()
- type DevicePtr
- type MemStats
- type TimingStats
Constants ¶
This section is empty.
Variables ¶
var ( // UseGemmScratch enables reusable GEMM operand scratch buffers (op0/op1). UseGemmScratch = true )
Functions ¶
func AddBiasLastDim ¶
AddBiasLastDim adds bias[lastDim] to each row-major element out[i].
func AppMemFromBaseline ¶
AppMemFromBaseline returns GPU bytes used by this app and free left from startup budget.
func DeviceDownloadF32 ¶
DeviceDownloadF32 copies device FP16 into host float32 at elemOff.
func DeviceDownloadSlice ¶
DeviceDownloadSlice copies device FP16 at dev base into host float32[hostOff:hostOff+n].
func DeviceFree ¶
func DeviceFree(p DevicePtr)
func DeviceMemLine ¶
func DeviceMemLine() string
DeviceMemLine formats total/available GPU memory for startup logging.
func DeviceMemset ¶
func DeviceMemsetAt ¶
DeviceMemsetAt zeros nElems at elemOff in a device buffer.
func DeviceUploadF32 ¶
DeviceUploadF32 copies host float32 into device FP16 at elemOff.
func DeviceUploadSlice ¶
DeviceUploadSlice copies host float32[hostOff:hostOff+n] into device FP16 at dev base.
func GemmDevice ¶
GemmDevice computes C[M,N] = A[M,K] @ B[K,N] on device (all offsets zero).
func GemmDeviceAt ¶
GemmDeviceAt computes one row-major GEMM slice on device (FP16).
func GemmStridedBatchedAt ¶
func GemmStridedBatchedAt(aDev, bDev, cDev DevicePtr, aOff, bOff, cOff, M, N, K, batchCount int) error
GemmStridedBatchedAt runs batchCount row-major GEMMs on device (FP16).
func RecordMemBaseline ¶
func RecordMemBaseline()
RecordMemBaseline snapshots device used/free before this process allocates weights.
Types ¶
type DevicePtr ¶
type DevicePtr uintptr
DevicePtr is a GPU allocation (FP16 elements).
func DeviceAlloc ¶
DeviceAlloc allocates nbytes on the GPU.
func ScratchGemmOp ¶
ScratchGemmOp returns a reusable operand buffer; which must be 0 or 1.
func ScratchGemmOut ¶
ScratchGemmOut returns a reusable GEMM output buffer.
type MemStats ¶
MemStats is device memory usage from cudaMemGetInfo.
func DeviceMemStats ¶
DeviceMemStats returns used and total GPU memory in bytes.
type TimingStats ¶
type TimingStats struct {
H2D time.Duration
Gemm time.Duration
D2H time.Duration
AllocWait time.Duration
OOM uint64 // bytes that failed to allocate this interval
}
TimingStats is cumulative CUDA timing buckets.
func SnapshotTimings ¶
func SnapshotTimings() TimingStats
SnapshotTimings returns current cumulative CUDA timing counters.
func (TimingStats) Delta ¶
func (s TimingStats) Delta(base TimingStats) TimingStats
Delta returns cumulative counter difference s-base.
Source Files