filter

package
v0.101.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 20, 2026 License: MIT Imports: 10 Imported by: 0

README

Memo Filter Engine

This package houses the memo-only filter engine that turns CEL expressions into SQL fragments. The engine follows a three phase pipeline inspired by systems such as Calcite or Prisma:

  1. Parsing – CEL expressions are parsed with cel-go and validated against the memo-specific environment declared in schema.go. Only fields that exist in the schema can surface in the filter.
  2. Normalization – the raw CEL AST is converted into an intermediate representation (IR) defined in ir.go. The IR is a dialect-agnostic tree of conditions (logical operators, comparisons, list membership, etc.). This step enforces schema rules (e.g. operator compatibility, type checks).
  3. Rendering – the renderer in render.go walks the IR and produces a SQL fragment plus placeholder arguments tailored to a target dialect (sqlite, mysql, or postgres). Dialect differences such as JSON access, boolean semantics, placeholders, and LIKE vs ILIKE are encapsulated in renderer helpers.

The entry point is filter.DefaultEngine() from engine.go. It lazily constructs an Engine configured with the memo schema and exposes:

engine, _ := filter.DefaultEngine()
stmt, _ := engine.CompileToStatement(ctx, `has_task_list && visibility == "PUBLIC"`, filter.RenderOptions{
	Dialect: filter.DialectPostgres,
})
// stmt.SQL  -> "((memo.payload->'property'->>'hasTaskList')::boolean IS TRUE AND memo.visibility = $1)"
// stmt.Args -> ["PUBLIC"]

Core Files

File Responsibility
schema.go Declares memo fields, their types, backing columns, CEL environment options
ir.go IR node definitions used across the pipeline
parser.go Converts CEL Expr into IR while applying schema validation
render.go Translates IR into SQL, handling dialect-specific behavior
engine.go Glue between the phases; exposes Compile, CompileToStatement, and DefaultEngine
helpers.go Convenience helpers for store integration (appending conditions)

SQL Generation Notes

  • Placeholders — ? is used for SQLite/MySQL, $n for Postgres. The renderer tracks offsets to compose queries with pre-existing arguments.
  • JSON Fields — Memo metadata lives in memo.payload. The renderer handles JSON_EXTRACT/json_extract/->/->> variations and boolean coercion.
  • Tag Operations — tag in [...] and "tag" in tags become JSON array predicates. SQLite uses LIKE patterns, MySQL uses JSON_CONTAINS, and Postgres uses @>.
  • Boolean Flags — Fields such as has_task_list render as IS TRUE equality checks, or comparisons against CAST('true' AS JSON) depending on the dialect.

Typical Integration

  1. Fetch the engine with filter.DefaultEngine().
  2. Call CompileToStatement using the appropriate dialect enum.
  3. Append the emitted SQL fragment/args to the existing WHERE clause.
  4. Execute the resulting query through the store driver.

The helpers.AppendConditions helper encapsulates steps 2–3 when a driver needs to process an array of filters.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AppendConditions 

func AppendConditions(ctx context.Context, engine *Engine, filters []string, dialect DialectName, where *[]string, args *[]any) error

AppendConditions compiles the provided filters and appends the resulting SQL fragments and args.

Types

type Column 

type Column struct {
	Table string
	Name  string
}

Column identifies the backing table column.

type ComparisonCondition 

type ComparisonCondition struct {
	Left     ValueExpr
	Right    ValueExpr
	Operator ComparisonOperator
}

ComparisonCondition represents a binary comparison.

type ComparisonOperator 

type ComparisonOperator string

ComparisonOperator lists supported comparison operators. 

const (
	CompareEq  ComparisonOperator = "="
	CompareNeq ComparisonOperator = "!="
	CompareLt  ComparisonOperator = "<"
	CompareLte ComparisonOperator = "<="
	CompareGt  ComparisonOperator = ">"
	CompareGte ComparisonOperator = ">="
)

type ComprehensionKind 

type ComprehensionKind string

ComprehensionKind enumerates the types of list comprehensions. 

const (
	ComprehensionExists ComprehensionKind = "exists"
)

type Condition 

type Condition interface {
	// contains filtered or unexported methods
}

Condition represents a boolean expression derived from the CEL filter.

type ConstantCondition 

type ConstantCondition struct {
	Value bool
}

ConstantCondition captures a literal boolean outcome.

type ContainsCondition 

type ContainsCondition struct {
	Field string
	Value string
}

ContainsCondition models the <field>.contains(<value>) call.

type ContainsPredicate 

type ContainsPredicate struct {
	Substring string
}

ContainsPredicate represents t.contains("substring").

type DialectName 

type DialectName string

DialectName enumerates supported SQL dialects. 

const (
	DialectSQLite   DialectName = "sqlite"
	DialectPostgres DialectName = "postgres"
)

type ElementInCondition 

type ElementInCondition struct {
	Element ValueExpr
	Field   string
}

ElementInCondition represents the CEL syntax `"value" in field`.

type EndsWithPredicate 

type EndsWithPredicate struct {
	Suffix string
}

EndsWithPredicate represents t.endsWith("suffix").

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine parses CEL filters into a dialect-agnostic condition tree.

func DefaultAttachmentEngine 

func DefaultAttachmentEngine() (*Engine, error)

DefaultAttachmentEngine returns the process-wide attachment filter engine.

func DefaultEngine 

func DefaultEngine() (*Engine, error)

DefaultEngine returns the process-wide memo filter engine.

func NewEngine 

func NewEngine(schema Schema) (*Engine, error)

NewEngine builds a new Engine for the provided schema.

func (*Engine) Compile 

func (e *Engine) Compile(_ context.Context, filter string) (*Program, error)

Compile parses the filter string into an executable program.

func (*Engine) CompileToStatement 

func (e *Engine) CompileToStatement(ctx context.Context, filter string, opts RenderOptions) (Statement, error)

CompileToStatement compiles and renders the filter in a single step.

type Field 

type Field struct {
	Expressions          map[DialectName]string
	AllowedComparisonOps map[ComparisonOperator]bool
	Column               Column
	Name                 string
	Kind                 FieldKind
	Type                 FieldType
	AliasFor             string
	JSONPath             []string
	SupportsContains     bool
}

Field captures the schema metadata for an exposed CEL identifier.

type FieldKind 

type FieldKind string

FieldKind describes how a field is stored. 

const (
	FieldKindScalar       FieldKind = "scalar"
	FieldKindBoolColumn   FieldKind = "bool_column"
	FieldKindJSONBool     FieldKind = "json_bool"
	FieldKindJSONList     FieldKind = "json_list"
	FieldKindVirtualAlias FieldKind = "virtual_alias"
)

type FieldPredicateCondition 

type FieldPredicateCondition struct {
	Field string
}

FieldPredicateCondition asserts that a field evaluates to true.

type FieldRef 

type FieldRef struct {
	Name string
}

FieldRef references a named schema field.

type FieldType 

type FieldType string

FieldType represents the logical type of a field. 

const (
	FieldTypeString    FieldType = "string"
	FieldTypeInt       FieldType = "int"
	FieldTypeBool      FieldType = "bool"
	FieldTypeTimestamp FieldType = "timestamp"
)

type FunctionValue 

type FunctionValue struct {
	Name string
	Args []ValueExpr
}

FunctionValue captures simple function calls like size(tags).

type InCondition 

type InCondition struct {
	Left   ValueExpr
	Values []ValueExpr
}

InCondition represents an IN predicate with literal list values.

type ListComprehensionCondition 

type ListComprehensionCondition struct {
	Predicate PredicateExpr
	Kind      ComprehensionKind
	Field     string
	IterVar   string
}

ListComprehensionCondition represents CEL macros like exists(), all(), filter().

type LiteralValue 

type LiteralValue struct {
	Value interface{}
}

LiteralValue holds a literal scalar.

type LogicalCondition 

type LogicalCondition struct {
	Left     Condition
	Right    Condition
	Operator LogicalOperator
}

LogicalCondition composes two conditions with a logical operator.

type LogicalOperator 

type LogicalOperator string

LogicalOperator enumerates the supported logical operators. 

const (
	LogicalAnd LogicalOperator = "AND"
	LogicalOr  LogicalOperator = "OR"
)

type NotCondition 

type NotCondition struct {
	Expr Condition
}

NotCondition negates a child condition.

type PredicateExpr 

type PredicateExpr interface {
	// contains filtered or unexported methods
}

PredicateExpr represents predicates used in comprehensions.

type Program 

type Program struct {
	// contains filtered or unexported fields
}

Program stores a compiled filter condition.

func (*Program) ConditionTree 

func (p *Program) ConditionTree() Condition

ConditionTree exposes the underlying condition tree.

func (*Program) Render 

func (p *Program) Render(opts RenderOptions) (Statement, error)

Render converts the program into a dialect-specific SQL fragment.

type RenderOptions 

type RenderOptions struct {
	Dialect           DialectName
	PlaceholderOffset int
	DisableNullChecks bool
}

RenderOptions configure SQL rendering.

type Schema 

type Schema struct {
	Name       string
	Fields     map[string]Field
	EnvOptions []cel.EnvOption
}

Schema collects CEL environment options and field metadata.

func NewAttachmentSchema 

func NewAttachmentSchema() Schema

NewAttachmentSchema constructs the attachment filter schema and CEL environment.

func NewSchema 

func NewSchema() Schema

NewSchema constructs the memo filter schema and CEL environment.

func (Schema) Field 

func (s Schema) Field(name string) (Field, bool)

Field returns the field metadata if present.

func (Schema) ResolveAlias 

func (s Schema) ResolveAlias(name string) (Field, bool)

ResolveAlias resolves a virtual alias to its target field.

type StartsWithPredicate 

type StartsWithPredicate struct {
	Prefix string
}

StartsWithPredicate represents t.startsWith("prefix").

type Statement 

type Statement struct {
	SQL  string
	Args []any
}

Statement contains the rendered SQL fragment and its args.

type ValueExpr 

type ValueExpr interface {
	// contains filtered or unexported methods
}

ValueExpr models arithmetic or scalar expressions whose result feeds a comparison.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.