filter

package
v0.25.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 25, 2025 License: MIT Imports: 10 Imported by: 0

README

Memo Filter Engine

This package houses the memo-only filter engine that turns CEL expressions into SQL fragments. The engine follows a three phase pipeline inspired by systems such as Calcite or Prisma:

  1. Parsing – CEL expressions are parsed with cel-go and validated against the memo-specific environment declared in schema.go. Only fields that exist in the schema can surface in the filter.
  2. Normalization – the raw CEL AST is converted into an intermediate representation (IR) defined in ir.go. The IR is a dialect-agnostic tree of conditions (logical operators, comparisons, list membership, etc.). This step enforces schema rules (e.g. operator compatibility, type checks).
  3. Rendering – the renderer in render.go walks the IR and produces a SQL fragment plus placeholder arguments tailored to a target dialect (sqlite, mysql, or postgres). Dialect differences such as JSON access, boolean semantics, placeholders, and LIKE vs ILIKE are encapsulated in renderer helpers.

The entry point is filter.DefaultEngine() from engine.go. It lazily constructs an Engine configured with the memo schema and exposes:

engine, _ := filter.DefaultEngine()
stmt, _ := engine.CompileToStatement(ctx, `has_task_list && visibility == "PUBLIC"`, filter.RenderOptions{
	Dialect: filter.DialectPostgres,
})
// stmt.SQL  -> "((memo.payload->'property'->>'hasTaskList')::boolean IS TRUE AND memo.visibility = $1)"
// stmt.Args -> ["PUBLIC"]

Core Files

File Responsibility
schema.go Declares memo fields, their types, backing columns, CEL environment options
ir.go IR node definitions used across the pipeline
parser.go Converts CEL Expr into IR while applying schema validation
render.go Translates IR into SQL, handling dialect-specific behavior
engine.go Glue between the phases; exposes Compile, CompileToStatement, and DefaultEngine
helpers.go Convenience helpers for store integration (appending conditions)

SQL Generation Notes

  • Placeholders — ? is used for SQLite/MySQL, $n for Postgres. The renderer tracks offsets to compose queries with pre-existing arguments.
  • JSON Fields — Memo metadata lives in memo.payload. The renderer handles JSON_EXTRACT/json_extract/->/->> variations and boolean coercion.
  • Tag Operations — tag in [...] and "tag" in tags become JSON array predicates. SQLite uses LIKE patterns, MySQL uses JSON_CONTAINS, and Postgres uses @>.
  • Boolean Flags — Fields such as has_task_list render as IS TRUE equality checks, or comparisons against CAST('true' AS JSON) depending on the dialect.

Typical Integration

  1. Fetch the engine with filter.DefaultEngine().
  2. Call CompileToStatement using the appropriate dialect enum.
  3. Append the emitted SQL fragment/args to the existing WHERE clause.
  4. Execute the resulting query through the store driver.

The helpers.AppendConditions helper encapsulates steps 2–3 when a driver needs to process an array of filters.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AppendConditions added in v0.25.2 

func AppendConditions(ctx context.Context, engine *Engine, filters []string, dialect DialectName, where *[]string, args *[]any) error

AppendConditions compiles the provided filters and appends the resulting SQL fragments and args.

Types

type Column added in v0.25.2 

type Column struct {
	Table string
	Name  string
}

Column identifies the backing table column.

type ComparisonCondition added in v0.25.2 

type ComparisonCondition struct {
	Left     ValueExpr
	Operator ComparisonOperator
	Right    ValueExpr
}

ComparisonCondition represents a binary comparison.

type ComparisonOperator added in v0.25.2 

type ComparisonOperator string

ComparisonOperator lists supported comparison operators. 

const (
	CompareEq  ComparisonOperator = "="
	CompareNeq ComparisonOperator = "!="
	CompareLt  ComparisonOperator = "<"
	CompareLte ComparisonOperator = "<="
	CompareGt  ComparisonOperator = ">"
	CompareGte ComparisonOperator = ">="
)

type Condition added in v0.25.2 

type Condition interface {
	// contains filtered or unexported methods
}

Condition represents a boolean expression derived from the CEL filter.

type ConstantCondition added in v0.25.2 

type ConstantCondition struct {
	Value bool
}

ConstantCondition captures a literal boolean outcome.

type ContainsCondition added in v0.25.2 

type ContainsCondition struct {
	Field string
	Value string
}

ContainsCondition models the <field>.contains(<value>) call.

type DialectName added in v0.25.2 

type DialectName string

DialectName enumerates supported SQL dialects. 

const (
	DialectSQLite   DialectName = "sqlite"
	DialectMySQL    DialectName = "mysql"
	DialectPostgres DialectName = "postgres"
)

type ElementInCondition added in v0.25.2 

type ElementInCondition struct {
	Element ValueExpr
	Field   string
}

ElementInCondition represents the CEL syntax `"value" in field`.

type Engine added in v0.25.2 

type Engine struct {
	// contains filtered or unexported fields
}

Engine parses CEL filters into a dialect-agnostic condition tree.

func DefaultEngine added in v0.25.2 

func DefaultEngine() (*Engine, error)

DefaultEngine returns the process-wide memo filter engine.

func NewEngine added in v0.25.2 

func NewEngine(schema Schema) (*Engine, error)

NewEngine builds a new Engine for the provided schema.

func (*Engine) Compile added in v0.25.2 

func (e *Engine) Compile(_ context.Context, filter string) (*Program, error)

Compile parses the filter string into an executable program.

func (*Engine) CompileToStatement added in v0.25.2 

func (e *Engine) CompileToStatement(ctx context.Context, filter string, opts RenderOptions) (Statement, error)

CompileToStatement compiles and renders the filter in a single step.

type Field added in v0.25.2 

type Field struct {
	Name                 string
	Kind                 FieldKind
	Type                 FieldType
	Column               Column
	JSONPath             []string
	AliasFor             string
	SupportsContains     bool
	Expressions          map[DialectName]string
	AllowedComparisonOps map[ComparisonOperator]bool
}

Field captures the schema metadata for an exposed CEL identifier.

type FieldKind added in v0.25.2 

type FieldKind string

FieldKind describes how a field is stored. 

const (
	FieldKindScalar       FieldKind = "scalar"
	FieldKindBoolColumn   FieldKind = "bool_column"
	FieldKindJSONBool     FieldKind = "json_bool"
	FieldKindJSONList     FieldKind = "json_list"
	FieldKindVirtualAlias FieldKind = "virtual_alias"
)

type FieldPredicateCondition added in v0.25.2 

type FieldPredicateCondition struct {
	Field string
}

FieldPredicateCondition asserts that a field evaluates to true.

type FieldRef added in v0.25.2 

type FieldRef struct {
	Name string
}

FieldRef references a named schema field.

type FieldType added in v0.25.2 

type FieldType string

FieldType represents the logical type of a field. 

const (
	FieldTypeString    FieldType = "string"
	FieldTypeInt       FieldType = "int"
	FieldTypeBool      FieldType = "bool"
	FieldTypeTimestamp FieldType = "timestamp"
)

type FunctionValue added in v0.25.2 

type FunctionValue struct {
	Name string
	Args []ValueExpr
}

FunctionValue captures simple function calls like size(tags).

type InCondition added in v0.25.2 

type InCondition struct {
	Left   ValueExpr
	Values []ValueExpr
}

InCondition represents an IN predicate with literal list values.

type LiteralValue added in v0.25.2 

type LiteralValue struct {
	Value interface{}
}

LiteralValue holds a literal scalar.

type LogicalCondition added in v0.25.2 

type LogicalCondition struct {
	Operator LogicalOperator
	Left     Condition
	Right    Condition
}

LogicalCondition composes two conditions with a logical operator.

type LogicalOperator added in v0.25.2 

type LogicalOperator string

LogicalOperator enumerates the supported logical operators. 

const (
	LogicalAnd LogicalOperator = "AND"
	LogicalOr  LogicalOperator = "OR"
)

type NotCondition added in v0.25.2 

type NotCondition struct {
	Expr Condition
}

NotCondition negates a child condition.

type Program added in v0.25.2 

type Program struct {
	// contains filtered or unexported fields
}

Program stores a compiled filter condition.

func (*Program) ConditionTree added in v0.25.2 

func (p *Program) ConditionTree() Condition

ConditionTree exposes the underlying condition tree.

func (*Program) Render added in v0.25.2 

func (p *Program) Render(opts RenderOptions) (Statement, error)

Render converts the program into a dialect-specific SQL fragment.

type RenderOptions added in v0.25.2 

type RenderOptions struct {
	Dialect           DialectName
	PlaceholderOffset int
	DisableNullChecks bool
}

RenderOptions configure SQL rendering.

type Schema added in v0.25.2 

type Schema struct {
	Name       string
	Fields     map[string]Field
	EnvOptions []cel.EnvOption
}

Schema collects CEL environment options and field metadata.

func NewSchema added in v0.25.2 

func NewSchema() Schema

NewSchema constructs the memo filter schema and CEL environment.

func (Schema) Field added in v0.25.2 

func (s Schema) Field(name string) (Field, bool)

Field returns the field metadata if present.

func (Schema) ResolveAlias added in v0.25.2 

func (s Schema) ResolveAlias(name string) (Field, bool)

ResolveAlias resolves a virtual alias to its target field.

type Statement added in v0.25.2 

type Statement struct {
	SQL  string
	Args []any
}

Statement contains the rendered SQL fragment and its args.

type ValueExpr added in v0.25.2 

type ValueExpr interface {
	// contains filtered or unexported methods
}

ValueExpr models arithmetic or scalar expressions whose result feeds a comparison.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.