Documentation
¶
Overview ¶
Package normalization provides various normalization layers for neural networks.
Index ¶
- type LayerNormalization
- func (ln *LayerNormalization[T]) Backward(ctx context.Context, dOut *tensor.Tensor[T], inputs ...*tensor.Tensor[T]) ([]*tensor.Tensor[T], error)
- func (ln *LayerNormalization[T]) Forward(ctx context.Context, inputs ...*tensor.Tensor[T]) (*tensor.Tensor[T], error)
- func (ln *LayerNormalization[T]) OutputShape(inputShapes ...[]int) ([]int, error)
- func (ln *LayerNormalization[T]) Parameters() []graph.Parameter[T]
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type LayerNormalization ¶
LayerNormalization implements the Layer Normalization operation.
func NewLayerNormalization ¶
func NewLayerNormalization[T tensor.Numeric](engine compute.Engine[T], featureDim int, epsilon T) (*LayerNormalization[T], error)
NewLayerNormalization creates a new LayerNormalization layer. featureDim: The dimension over which to normalize (typically the last dimension). epsilon: A small constant to avoid division by zero.
func (*LayerNormalization[T]) Backward ¶
func (ln *LayerNormalization[T]) Backward(ctx context.Context, dOut *tensor.Tensor[T], inputs ...*tensor.Tensor[T]) ([]*tensor.Tensor[T], error)
Backward computes the gradients for LayerNormalization.
func (*LayerNormalization[T]) Forward ¶
func (ln *LayerNormalization[T]) Forward(ctx context.Context, inputs ...*tensor.Tensor[T]) (*tensor.Tensor[T], error)
Forward computes the Layer Normalization.
func (*LayerNormalization[T]) OutputShape ¶
func (ln *LayerNormalization[T]) OutputShape(inputShapes ...[]int) ([]int, error)
OutputShape returns the output shape, which is the same as the input shape.
func (*LayerNormalization[T]) Parameters ¶
func (ln *LayerNormalization[T]) Parameters() []graph.Parameter[T]
Parameters returns the trainable gamma and beta parameters.
Source Files
Click to show internal directories.
Click to hide internal directories.