Documentation
¶
Overview ¶
Package transformer provides transformer building blocks such as the Transformer `Block` used in encoder/decoder stacks.
Index ¶
- type Block
- func (b *Block[T]) Attention() graph.Node[T]
- func (b *Block[T]) Attributes() map[string]any
- func (b *Block[T]) Backward(_ context.Context, _ types.BackwardMode, dOut *tensor.TensorNumeric[T], ...) ([]*tensor.TensorNumeric[T], error)
- func (b *Block[T]) Engine() compute.Engine[T]
- func (b *Block[T]) Forward(ctx context.Context, inputs ...*tensor.TensorNumeric[T]) (*tensor.TensorNumeric[T], error)
- func (b *Block[T]) OpType() string
- func (b *Block[T]) OutputShape() []int
- func (b *Block[T]) Parameters() []*graph.Parameter[T]
- type BlockOption
- type BlockOptions
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Block ¶
Block represents a single Transformer block.
func NewTransformerBlock ¶
func NewTransformerBlock[T tensor.Numeric]( engine compute.Engine[T], ops numeric.Arithmetic[T], modelDim, ffnDim int, attention graph.Node[T], opts ...BlockOption[T], ) (*Block[T], error)
NewTransformerBlock creates a new Transformer block.
func (*Block[T]) Attention ¶ added in v0.3.0
Attention returns the attention layer of the Transformer block.
func (*Block[T]) Attributes ¶ added in v0.3.0
Attributes returns the attributes of the Transformer block.
func (*Block[T]) Backward ¶
func (b *Block[T]) Backward(_ context.Context, _ types.BackwardMode, dOut *tensor.TensorNumeric[T], _ ...*tensor.TensorNumeric[T]) ([]*tensor.TensorNumeric[T], error)
Backward computes the backward pass of the Transformer block.
func (*Block[T]) Engine ¶ added in v0.3.0
Engine returns the compute engine used by the Transformer block.
func (*Block[T]) Forward ¶
func (b *Block[T]) Forward(ctx context.Context, inputs ...*tensor.TensorNumeric[T]) (*tensor.TensorNumeric[T], error)
Forward computes the forward pass of the Transformer block.
func (*Block[T]) OpType ¶ added in v0.3.0
OpType returns the operator type of the Transformer block.
func (*Block[T]) OutputShape ¶
OutputShape returns the output shape of the Transformer block.
func (*Block[T]) Parameters ¶
Parameters returns the parameters of the Transformer block.
type BlockOption ¶ added in v0.3.0
type BlockOption[T tensor.Numeric] func(*BlockOptions[T])
BlockOption is a function that applies an option to BlockOptions.
func WithEpsilon ¶ added in v0.3.0
func WithEpsilon[T tensor.Numeric](epsilon T) BlockOption[T]
WithEpsilon sets the epsilon value for the RMS normalization layers.
type BlockOptions ¶ added in v0.3.0
BlockOptions holds configuration options for the Transformer block.
Source Files