transformers

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 10, 2026 License: Apache-2.0 Imports: 29 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	ErrMultiDimensionalArray = errors.New("multi-dimensional arrays are not supported")
	ErrInvalidArrayOptions   = errors.New("invalid array options")
)
View Source
var (
	ErrUnsupportedValueType     = errors.New("unsupported value type for transformer")
	ErrUnsupportedGenerator     = errors.New("transformer doesn't support the configured generator")
	ErrUnsupportedTransformer   = errors.New("unsupported transformer config")
	ErrInvalidParameters        = errors.New("invalid transformer parameters")
	ErrInvalidDynamicParameters = errors.New("invalid transformer dynamic parameters")
	ErrUnknownParameter         = errors.New("unknown parameter provided to transformer")
)

Functions

func FindDynamicValue added in v0.5.0

func FindDynamicValue[T any](param *DynamicParameter, dynamicValues map[string]any, defaultValue T) (T, error)

func FindParameter

func FindParameter[T any](params ParameterValues, name string) (T, bool, error)

func FindParameterArray added in v0.4.1

func FindParameterArray[T any](params ParameterValues, name string) ([]T, bool, error)

func FindParameterWithDefault added in v0.5.0

func FindParameterWithDefault[T any](params ParameterValues, name string, defaultValue T) (T, error)

func ParseDynamicParameters added in v0.5.0

func ParseDynamicParameters(params ParameterValues) (map[string]*DynamicParameter, error)

func ValidateParameters added in v0.5.0

func ValidateParameters(provided map[string]any, expected []string) error

ValidateParameters checks if all provided parameters are in the expected set

Types

type ArrayConfig added in v1.5.0

type ArrayConfig struct {
	ElementTransformer Transformer
	ArrayOID           uint32
	ElementOID         uint32
	ElementTypeName    string
	Column             string
	Generator          ArrayGenerator
	MinCount           int
	MaxCount           int
}

ArrayConfig describes an ArrayTransformer.

type ArrayGenerator added in v1.5.0

type ArrayGenerator string

ArrayGenerator selects how the elements of the transformed array are drawn from the elements of the source array.

const (
	// ArrayGeneratorMap transforms every source element in order, preserving
	// the length of the array.
	ArrayGeneratorMap ArrayGenerator = "map"
	// ArrayGeneratorRandom emits between MinCount and MaxCount elements, each
	// the transform of a source element chosen at random with replacement.
	ArrayGeneratorRandom ArrayGenerator = "random"
)

type ArrayTransformer added in v1.5.0

type ArrayTransformer struct {
	// contains filtered or unexported fields
}

ArrayTransformer applies an element transformer to every element of a one-dimensional postgres array.

func NewArrayTransformer added in v1.5.0

func NewArrayTransformer(cfg ArrayConfig) (*ArrayTransformer, error)

func (*ArrayTransformer) Close added in v1.5.0

func (t *ArrayTransformer) Close() error

func (*ArrayTransformer) CompatibleTypes added in v1.5.0

func (t *ArrayTransformer) CompatibleTypes() []SupportedDataType

func (*ArrayTransformer) IsDynamic added in v1.5.0

func (t *ArrayTransformer) IsDynamic() bool

func (*ArrayTransformer) Transform added in v1.5.0

func (t *ArrayTransformer) Transform(ctx context.Context, value Value) (any, error)

func (*ArrayTransformer) Type added in v1.5.0

func (*ArrayTransformer) Uniqueness added in v1.5.0

func (t *ArrayTransformer) Uniqueness() Uniqueness

Uniqueness classifies the whole array transform, which is what validation against a unique index needs. Mapping a length preserving, injective element transform over an array is injective, so map inherits the element transformer's classification verbatim.

type Config

type Config struct {
	Name              TransformerType
	Parameters        ParameterValues
	DynamicParameters ParameterValues
}

type Definition added in v0.6.0

type Definition struct {
	SupportedTypes []SupportedDataType
	Parameters     []Parameter
	Uniqueness     Uniqueness
}

func EmailTransformerDefinition added in v0.8.2

func EmailTransformerDefinition() *Definition

func EncryptedAESSIVTransformerDefinition added in v1.2.0

func EncryptedAESSIVTransformerDefinition() *Definition

func FPEFF1TransformerDefinition added in v1.5.0

func FPEFF1TransformerDefinition() *Definition

func HstoreTransformerDefinition added in v0.8.1

func HstoreTransformerDefinition() *Definition

func JSONTransformerDefinition added in v0.6.5

func JSONTransformerDefinition() *Definition

func LiteralStringTransformerDefinition added in v0.6.0

func LiteralStringTransformerDefinition() *Definition

func LookupChoiceTransformerDefinition added in v1.5.0

func LookupChoiceTransformerDefinition() *Definition

func MaskingTransformerDefinition added in v0.6.0

func MaskingTransformerDefinition() *Definition

func PGAnonymizerTransformerDefinition added in v0.8.4

func PGAnonymizerTransformerDefinition() *Definition

func PhoneNumberTransformerDefinition added in v0.6.0

func PhoneNumberTransformerDefinition() *Definition

func StringTransformerDefinition added in v0.6.0

func StringTransformerDefinition() *Definition

func TemplateTransformerDefinition added in v0.6.0

func TemplateTransformerDefinition() *Definition

type DynamicParameter added in v0.5.0

type DynamicParameter struct {
	Column string
}

type EmailTransformer added in v0.8.2

type EmailTransformer struct {
	// contains filtered or unexported fields
}

func NewEmailTransformer added in v0.8.2

func NewEmailTransformer(params ParameterValues) (*EmailTransformer, error)

func (*EmailTransformer) Close added in v0.8.4

func (et *EmailTransformer) Close() error

func (*EmailTransformer) CompatibleTypes added in v0.8.2

func (et *EmailTransformer) CompatibleTypes() []SupportedDataType

func (*EmailTransformer) IsDynamic added in v0.8.9

func (et *EmailTransformer) IsDynamic() bool

func (*EmailTransformer) Transform added in v0.8.2

func (et *EmailTransformer) Transform(_ context.Context, v Value) (any, error)

func (*EmailTransformer) Type added in v0.8.2

func (et *EmailTransformer) Type() TransformerType

func (*EmailTransformer) Uniqueness added in v1.4.0

func (et *EmailTransformer) Uniqueness() Uniqueness

type EncryptedAESSIVTransformer added in v1.2.0

type EncryptedAESSIVTransformer struct {
	// contains filtered or unexported fields
}

EncryptedAESSIVTransformer encrypts values with AES-SIV (RFC 5297), a deterministic authenticated encryption scheme: the same input, key and associated data always produce the same token, so equality relationships between column values are preserved across rows, tables and runs. Unlike hashing, the transformation is reversible by holders of the key, and tampered or forged tokens fail authenticated decryption. Tokens are base64url-encoded (no padding), safe for URLs and file names. For bytea columns the raw bytes are encrypted regardless of how the value arrives (raw during snapshots, hex-text during replication) and the token is returned as bytes.

func NewEncryptedAESSIVTransformer added in v1.2.0

func NewEncryptedAESSIVTransformer(params ParameterValues) (*EncryptedAESSIVTransformer, error)

func (*EncryptedAESSIVTransformer) Close added in v1.2.0

func (est *EncryptedAESSIVTransformer) Close() error

func (*EncryptedAESSIVTransformer) CompatibleTypes added in v1.2.0

func (est *EncryptedAESSIVTransformer) CompatibleTypes() []SupportedDataType

func (*EncryptedAESSIVTransformer) IsDynamic added in v1.2.0

func (est *EncryptedAESSIVTransformer) IsDynamic() bool

func (*EncryptedAESSIVTransformer) Transform added in v1.2.0

func (est *EncryptedAESSIVTransformer) Transform(_ context.Context, v Value) (any, error)

func (*EncryptedAESSIVTransformer) Type added in v1.2.0

func (*EncryptedAESSIVTransformer) Uniqueness added in v1.4.0

func (est *EncryptedAESSIVTransformer) Uniqueness() Uniqueness

type FPEFF1Transformer added in v1.5.0

type FPEFF1Transformer struct {
	// contains filtered or unexported fields
}

FPEFF1Transformer encrypts values with FF1 (NIST SP 800-38G). The output contains the same characters as the input, and it has the same length. Two different inputs always give two different outputs, thus the transformer is safe on a column with a unique index. The transformer copies the characters that are not in the alphabet to the same positions in the output. A person who has the key can decrypt the values.

func NewFPEFF1Transformer added in v1.5.0

func NewFPEFF1Transformer(params ParameterValues) (*FPEFF1Transformer, error)

func (*FPEFF1Transformer) Close added in v1.5.0

func (ft *FPEFF1Transformer) Close() error

func (*FPEFF1Transformer) CompatibleTypes added in v1.5.0

func (ft *FPEFF1Transformer) CompatibleTypes() []SupportedDataType

func (*FPEFF1Transformer) IsDynamic added in v1.5.0

func (ft *FPEFF1Transformer) IsDynamic() bool

func (*FPEFF1Transformer) Transform added in v1.5.0

func (ft *FPEFF1Transformer) Transform(_ context.Context, v Value) (any, error)

func (*FPEFF1Transformer) Type added in v1.5.0

func (*FPEFF1Transformer) Uniqueness added in v1.5.0

func (ft *FPEFF1Transformer) Uniqueness() Uniqueness

type HstoreTransformer added in v0.8.1

type HstoreTransformer struct {
	// contains filtered or unexported fields
}

func NewHstoreTransformer added in v0.8.1

func NewHstoreTransformer(params ParameterValues) (*HstoreTransformer, error)

func (*HstoreTransformer) Close added in v0.8.4

func (t *HstoreTransformer) Close() error

func (*HstoreTransformer) CompatibleTypes added in v0.8.1

func (t *HstoreTransformer) CompatibleTypes() []SupportedDataType

func (*HstoreTransformer) IsDynamic added in v0.8.9

func (t *HstoreTransformer) IsDynamic() bool

func (*HstoreTransformer) Transform added in v0.8.1

func (t *HstoreTransformer) Transform(_ context.Context, value Value) (any, error)

func (*HstoreTransformer) Type added in v0.8.1

func (*HstoreTransformer) Uniqueness added in v1.4.0

func (t *HstoreTransformer) Uniqueness() Uniqueness

type JSONTransformer added in v0.6.5

type JSONTransformer struct {
	// contains filtered or unexported fields
}

func NewJSONTransformer added in v0.6.5

func NewJSONTransformer(params ParameterValues) (*JSONTransformer, error)

func (*JSONTransformer) Close added in v0.8.4

func (jt *JSONTransformer) Close() error

func (*JSONTransformer) CompatibleTypes added in v0.6.5

func (jt *JSONTransformer) CompatibleTypes() []SupportedDataType

func (*JSONTransformer) IsDynamic added in v0.8.9

func (jt *JSONTransformer) IsDynamic() bool

func (*JSONTransformer) Transform added in v0.6.5

func (jt *JSONTransformer) Transform(_ context.Context, value Value) (any, error)

func (*JSONTransformer) Type added in v0.6.5

func (jt *JSONTransformer) Type() TransformerType

func (*JSONTransformer) Uniqueness added in v1.4.0

func (jt *JSONTransformer) Uniqueness() Uniqueness

type LiteralStringTransformer added in v0.5.0

type LiteralStringTransformer struct {
	// contains filtered or unexported fields
}

func NewLiteralStringTransformer added in v0.5.0

func NewLiteralStringTransformer(params ParameterValues) (*LiteralStringTransformer, error)

func (*LiteralStringTransformer) Close added in v0.8.4

func (lst *LiteralStringTransformer) Close() error

func (*LiteralStringTransformer) CompatibleTypes added in v0.5.0

func (lst *LiteralStringTransformer) CompatibleTypes() []SupportedDataType

func (*LiteralStringTransformer) IsDynamic added in v0.8.9

func (lst *LiteralStringTransformer) IsDynamic() bool

func (*LiteralStringTransformer) Transform added in v0.5.0

func (lst *LiteralStringTransformer) Transform(_ context.Context, value Value) (any, error)

func (*LiteralStringTransformer) Type added in v0.6.0

func (*LiteralStringTransformer) Uniqueness added in v1.4.0

func (lst *LiteralStringTransformer) Uniqueness() Uniqueness

type LookupChoiceTransformer added in v1.5.0

type LookupChoiceTransformer struct {
	// contains filtered or unexported fields
}

LookupChoiceTransformer replaces a value with one taken from a column of another table. The values are read once, when the transformer is built, so the configuration does not have to be regenerated when the lookup table changes.

func NewLookupChoiceTransformer added in v1.5.0

func NewLookupChoiceTransformer(params ParameterValues) (*LookupChoiceTransformer, error)

NewLookupChoiceTransformer reads the values of the configured lookup column and returns a transformer that chooses from them. The connection is only held for the duration of the read.

func (*LookupChoiceTransformer) Close added in v1.5.0

func (t *LookupChoiceTransformer) Close() error

Close is a no-op: the connection used to read the values is released as soon as the read completes.

func (*LookupChoiceTransformer) CompatibleTypes added in v1.5.0

func (t *LookupChoiceTransformer) CompatibleTypes() []SupportedDataType

CompatibleTypes reports the pg types the lookup column's values can be written to, so that a rule pointing a column at a lookup column of an incompatible type fails on startup rather than mid load.

func (*LookupChoiceTransformer) IsDynamic added in v1.5.0

func (t *LookupChoiceTransformer) IsDynamic() bool

func (*LookupChoiceTransformer) Transform added in v1.5.0

func (t *LookupChoiceTransformer) Transform(_ context.Context, value Value) (any, error)

func (*LookupChoiceTransformer) Type added in v1.5.0

func (*LookupChoiceTransformer) Uniqueness added in v1.5.0

func (t *LookupChoiceTransformer) Uniqueness() Uniqueness

Uniqueness is lossy in both generator modes: the values come from a set that is normally much smaller than the number of rows being transformed.

type MaskingTransformer added in v0.5.0

type MaskingTransformer struct {
	// contains filtered or unexported fields
}

MaskingTransformer is a transformer that masks sensitive data using the Masker library.

func NewMaskingTransformer added in v0.5.0

func NewMaskingTransformer(params ParameterValues) (*MaskingTransformer, error)

NewMaskingTransformer creates a new MaskingTransformer with the given masking function.

func (*MaskingTransformer) Close added in v0.8.4

func (t *MaskingTransformer) Close() error

func (*MaskingTransformer) CompatibleTypes added in v0.5.0

func (t *MaskingTransformer) CompatibleTypes() []SupportedDataType

func (*MaskingTransformer) IsDynamic added in v0.8.9

func (t *MaskingTransformer) IsDynamic() bool

func (*MaskingTransformer) Transform added in v0.5.0

func (t *MaskingTransformer) Transform(_ context.Context, value Value) (any, error)

Transform applies the masking function to the input value and returns the masked value.

func (*MaskingTransformer) Type added in v0.6.0

func (*MaskingTransformer) Uniqueness added in v1.4.0

func (t *MaskingTransformer) Uniqueness() Uniqueness

type PGAnonymizerTransformer added in v0.8.4

type PGAnonymizerTransformer struct {
	// contains filtered or unexported fields
}

func NewPGAnonymizerTransformer added in v0.8.4

func NewPGAnonymizerTransformer(params ParameterValues) (*PGAnonymizerTransformer, error)

NewPGAnonymizerTransformer creates a new transformer that supports pg_anonymizer functions. Unsupported functions: - destruction ('MASKED WITH VALUE ”CONFIDENTIAL” ) - anon.ternary (conditional masking) - anon.generalize... (data generalization - only makes sense with views)

func (*PGAnonymizerTransformer) Close added in v0.8.4

func (t *PGAnonymizerTransformer) Close() error

func (*PGAnonymizerTransformer) CompatibleTypes added in v0.8.4

func (t *PGAnonymizerTransformer) CompatibleTypes() []SupportedDataType

func (*PGAnonymizerTransformer) IsDynamic added in v0.8.9

func (t *PGAnonymizerTransformer) IsDynamic() bool

func (*PGAnonymizerTransformer) Transform added in v0.8.4

func (t *PGAnonymizerTransformer) Transform(ctx context.Context, value Value) (any, error)

func (*PGAnonymizerTransformer) Type added in v0.8.4

func (*PGAnonymizerTransformer) Uniqueness added in v1.4.0

func (t *PGAnonymizerTransformer) Uniqueness() Uniqueness

type Parameter added in v0.6.0

type Parameter struct {
	Name          string
	SupportedType string
	Default       any
	Dynamic       bool
	Required      bool
	Values        []any
}

type ParameterValues added in v0.6.0

type ParameterValues map[string]any

type PhoneNumberTransformer added in v0.5.0

type PhoneNumberTransformer struct {
	// contains filtered or unexported fields
}

func NewPhoneNumberTransformer added in v0.5.0

func NewPhoneNumberTransformer(params, dynamicParams ParameterValues) (*PhoneNumberTransformer, error)

func (*PhoneNumberTransformer) Close added in v0.8.4

func (t *PhoneNumberTransformer) Close() error

func (*PhoneNumberTransformer) CompatibleTypes added in v0.5.0

func (t *PhoneNumberTransformer) CompatibleTypes() []SupportedDataType

func (*PhoneNumberTransformer) IsDynamic added in v0.8.9

func (t *PhoneNumberTransformer) IsDynamic() bool

func (*PhoneNumberTransformer) Transform added in v0.5.0

func (t *PhoneNumberTransformer) Transform(_ context.Context, value Value) (any, error)

func (*PhoneNumberTransformer) Type added in v0.6.0

func (*PhoneNumberTransformer) Uniqueness added in v1.4.0

func (t *PhoneNumberTransformer) Uniqueness() Uniqueness

type StringTransformer

type StringTransformer struct {
}

func NewStringTransformer

func NewStringTransformer(params ParameterValues) (*StringTransformer, error)

func (*StringTransformer) Close added in v0.8.4

func (st *StringTransformer) Close() error

func (*StringTransformer) CompatibleTypes added in v0.5.0

func (st *StringTransformer) CompatibleTypes() []SupportedDataType

func (*StringTransformer) IsDynamic added in v0.8.9

func (st *StringTransformer) IsDynamic() bool

func (*StringTransformer) Transform

func (st *StringTransformer) Transform(_ context.Context, v Value) (any, error)

func (*StringTransformer) Type added in v0.6.0

func (*StringTransformer) Uniqueness added in v1.4.0

func (st *StringTransformer) Uniqueness() Uniqueness

type SupportedDataType added in v0.5.0

type SupportedDataType string
const (
	ByteArrayDataType      SupportedDataType = "byte_array"
	StringDataType         SupportedDataType = "string"
	BooleanDataType        SupportedDataType = "boolean"
	Integer8DataType       SupportedDataType = "integer8"
	UInteger8DataType      SupportedDataType = "uinteger8"
	Integer16DataType      SupportedDataType = "integer16"
	UInteger16DataType     SupportedDataType = "uinteger16"
	Integer32DataType      SupportedDataType = "integer32"
	UInteger32DataType     SupportedDataType = "uinteger32"
	Integer64DataType      SupportedDataType = "integer64"
	UInteger64DataType     SupportedDataType = "uinteger64"
	Float32DataType        SupportedDataType = "float32"
	Float64DataType        SupportedDataType = "float64"
	UUIDDataType           SupportedDataType = "uuid"
	UInt8ArrayOf16DataType SupportedDataType = "uint8_array_of_16"
	DateDataType           SupportedDataType = "date"
	DatetimeDataType       SupportedDataType = "datetime"
	JSONDataType           SupportedDataType = "json"
	AllDataTypes           SupportedDataType = "all"
	CitextDataType         SupportedDataType = "citext"
	HstoreDataType         SupportedDataType = "hstore"
	EnumDataType           SupportedDataType = "enum"
)

type TemplateTransformer added in v0.6.0

type TemplateTransformer struct {
	// contains filtered or unexported fields
}

func NewTemplateTransformer added in v0.6.0

func NewTemplateTransformer(params ParameterValues) (*TemplateTransformer, error)

func (*TemplateTransformer) Close added in v0.8.4

func (t *TemplateTransformer) Close() error

func (*TemplateTransformer) CompatibleTypes added in v0.6.0

func (t *TemplateTransformer) CompatibleTypes() []SupportedDataType

func (*TemplateTransformer) IsDynamic added in v0.8.9

func (t *TemplateTransformer) IsDynamic() bool

func (*TemplateTransformer) Transform added in v0.6.0

func (t *TemplateTransformer) Transform(_ context.Context, value Value) (any, error)

func (*TemplateTransformer) Type added in v0.6.0

func (*TemplateTransformer) Uniqueness added in v1.4.0

func (t *TemplateTransformer) Uniqueness() Uniqueness

type Transformer

type Transformer interface {
	Transform(context.Context, Value) (any, error)
	IsDynamic() bool
	CompatibleTypes() []SupportedDataType
	Type() TransformerType
	Close() error
}

type TransformerType

type TransformerType string
const (
	Email                  TransformerType = "email"
	String                 TransformerType = "string"
	LiteralString          TransformerType = "literal_string"
	PhoneNumber            TransformerType = "phone_number"
	NeosyncString          TransformerType = "neosync_string"
	GreenmaskString        TransformerType = "greenmask_string"
	NeosyncFirstName       TransformerType = "neosync_firstname"
	GreenmaskFirstName     TransformerType = "greenmask_firstname"
	NeosyncLastName        TransformerType = "neosync_lastname"
	NeosyncFullName        TransformerType = "neosync_fullname"
	NeosyncEmail           TransformerType = "neosync_email"
	GreenmaskInteger       TransformerType = "greenmask_integer"
	GreenmaskFloat         TransformerType = "greenmask_float"
	GreenmaskUUID          TransformerType = "greenmask_uuid"
	GreenmaskBoolean       TransformerType = "greenmask_boolean"
	GreenmaskChoice        TransformerType = "greenmask_choice"
	GreenmaskUnixTimestamp TransformerType = "greenmask_unix_timestamp"
	GreenmaskDate          TransformerType = "greenmask_date"
	GreenmaskUTCTimestamp  TransformerType = "greenmask_utc_timestamp"
	Masking                TransformerType = "masking"
	Template               TransformerType = "template"
	JSON                   TransformerType = "json"
	Hstore                 TransformerType = "hstore"
	PGAnonymizer           TransformerType = "pg_anonymizer"
	EncryptedAESSIV        TransformerType = "encrypted_aes_siv"
	LookupChoice           TransformerType = "lookup_choice"
	FPEFF1                 TransformerType = "fpe_ff1"
)

type Uniqueness added in v1.4.0

type Uniqueness int

validated against unique indexes

const (
	// zero value, so a transformer that never classified itself is
	// distinguishable from one deliberately classified lossy
	UniquenessUnspecified Uniqueness = iota
	UniquenessLossy
	UniquenessNotGuaranteed
	UniquenessPreserved
)

func UniquenessOf added in v1.4.0

func UniquenessOf(t Transformer) Uniqueness

UniquenessOf reports what t guarantees about the uniqueness of its output. A transformer that does not classify itself is assumed to be capable of producing duplicates, but not known to, so that both ways of leaving it unclassified answer the same thing.

func (Uniqueness) String added in v1.4.0

func (u Uniqueness) String() string

type UniquenessReporter added in v1.4.0

type UniquenessReporter interface {
	Uniqueness() Uniqueness
}

optional so that Transformer implementations outside this module keep compiling; UniquenessOf treats a non-implementer as unknown

type Value added in v0.5.0

type Value struct {
	TransformValue any
	TransformType  string
	DynamicValues  map[string]any
}

func NewValue added in v0.5.0

func NewValue(transformValue any, transformType string, dynamicValues map[string]any) Value

func (*Value) GetDynamicValue added in v0.6.0

func (v *Value) GetDynamicValue(name string) (any, error)

func (*Value) GetValue added in v0.6.0

func (v *Value) GetValue() any

Directories

Path Synopsis
internal
lookup
Package lookup holds the connection seam used by transformers that read their values from the database when they are built.
Package lookup holds the connection seam used by transformers that read their values from the database when they are built.
pool
Package pool provides a concurrency-safe pool of instances for libraries whose types are not safe for concurrent use.
Package pool provides a concurrency-safe pool of instances for libraries whose types are not safe for concurrent use.
template
Package template builds the text/template instances used by the transformers that accept user provided templates.
Package template builds the text/template instances used by the transformers that accept user provided templates.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL