Documentation
¶
Overview ¶
Package filter — CLI mini-syntax parsers. See filter.go for the Directives type and the package overview.
Package filter implements subsetting axes for `datatug db copy`: table include/exclude, structured row predicates (--where), and per-table row limits. Column subsetting is deferred to a follow-up Feature once the upstream DALgo QueryBuilder gains a field-projection API; the Directives struct carries PerTableColumns and GlobalExcludeColumns as no-op placeholders to preserve type stability for that follow-up.
Spec: spec/features/cli/db/copy/filtering/README.md
Index ¶
- func ApplyTableFilter(refs []dal.CollectionRef, d *Directives) ([]dal.CollectionRef, error)
- func CoerceValue(raw string, colType dbschema.Type) (any, error)
- func CompileWhereForTable(table string, group *PredicateGroup, def *dbschema.CollectionDef) ([]dal.Condition, error)
- func ParseLimitFlag(s string) (table string, n int, err error)
- func ParseOperator(token string) (dal.Operator, error)
- func ParseTableList(s string) []string
- func ValidateWhereAgainstSchema(table string, p Predicate, def *dbschema.CollectionDef) error
- type ColumnRule
- type Directives
- type GroupOperator
- type OperatorToken
- type Predicate
- type PredicateGroup
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ApplyTableFilter ¶
func ApplyTableFilter(refs []dal.CollectionRef, d *Directives) ([]dal.CollectionRef, error)
ApplyTableFilter narrows the source collection list per --include / --exclude. Returns the filtered list, or an error if a named table does not exist on source (REQ:table-not-found).
Input refs are in source-introspection order; output preserves source order (NOT the order in IncludeTables).
func CoerceValue ¶
CoerceValue converts the raw string value (as received from CLI or YAML) into the typed value expected by the column's introspected dbschema.Type. Returns the coerced value or an error naming the expected type and offending input (REQ:where-type-coercion).
Type-mapping rules:
- Int → int64 via strconv.ParseInt
- Float → float64 via strconv.ParseFloat
- Decimal → float64 (Decimal is carried as float per dalgo2ingitdb's type-mapping table; lossy carrier documented in db copy spec)
- Bool → bool via strconv.ParseBool ("true"/"false"/"1"/"0", case-insensitive)
- Time → time.Time via time.Parse(time.DateOnly) for ISO date, fallback time.RFC3339 for full datetime
- String → passthrough
Unknown column types (e.g. dbschema.Null, dbschema.Bytes) are passed through as strings — the source query will surface a type error if the operator can't handle them.
func CompileWhereForTable ¶
func CompileWhereForTable(table string, group *PredicateGroup, def *dbschema.CollectionDef) ([]dal.Condition, error)
CompileWhereForTable validates each predicate in group against def and returns the dal.Condition slice ready to pass to QueryBuilder.Where.
MVP only compiles flat AND-composed predicates; OR-groups (group.Subgroups) are deferred per REQ:config-file-schema's reserved-`or:`-key rule, gated on the upstream `dalgo-group-condition-ctor` Idea. The Subgroups field stays on PredicateGroup as a no-op carrier for the follow-up Feature.
REQ:where-and-semantics, REQ:push-down-only.
func ParseLimitFlag ¶
ParseLimitFlag parses one --limit <table>:<N> token. N must be a positive integer. REQ:limit-flag.
func ParseOperator ¶
ParseOperator returns the dal.Operator for token, or an error listing every supported operator if token is not in the fixed vocabulary (REQ:operator-vocabulary).
func ParseTableList ¶
ParseTableList parses a comma-separated list of table names, trimming whitespace and dropping empty segments. Returns nil for the empty string. Used by --include and --exclude (REQ:include-flag, REQ:exclude-flag).
func ValidateWhereAgainstSchema ¶
func ValidateWhereAgainstSchema(table string, p Predicate, def *dbschema.CollectionDef) error
ValidateWhereAgainstSchema verifies that the predicate's field exists on the table's introspected schema and that the value can be coerced to the field's type. On unknown field, returns an error suggesting the Levenshtein-closest source field (REQ:where-unknown-field). If no field is within the Levenshtein threshold, the error lists every known field on the collection as the suggestion set.
Every MVP operator takes a value (REQ:operator-vocabulary defers the null-test operators), so coercion is unconditional.
Types ¶
type ColumnRule ¶
ColumnRule is the per-table column selection. Reserved for the deferred column-subsetting follow-up Feature.
type Directives ¶
type Directives struct {
// IncludeTables lists source tables to copy. Mutually exclusive with
// ExcludeTables. Empty means "copy all source tables".
IncludeTables []string
// ExcludeTables lists source tables to skip. Mutually exclusive with
// IncludeTables. Empty means "no skip rule".
ExcludeTables []string
// Where holds row-level predicates keyed by table name. Each entry's
// value is a per-table predicate group (AND-composed across CLI
// flags; may contain config-file OR-groups). Nil means "no row filter".
Where map[string]*PredicateGroup
// LimitsByTable maps source-table name → positive row limit. Nil or
// missing-key means "no limit for that table".
LimitsByTable map[string]int
// PerTableColumns holds per-table column rules. Reserved for the
// deferred column-subsetting follow-up Feature; not populated in MVP.
PerTableColumns map[string]*ColumnRule
// GlobalExcludeColumns lists columns to drop from every copied table
// that contains them. Reserved for the deferred column-subsetting
// follow-up Feature; not populated in MVP.
GlobalExcludeColumns []string
}
Directives is the resolved, pre-validation, pre-compilation set of filter axes. Constructed by either ParseCLI (cmd_db_copy.go flags) or ParseConfig (YAML --filter-config). Always non-nil; empty fields mean "no constraint".
func ParseConfigFile ¶
func ParseConfigFile(path string) (*Directives, error)
ParseConfigFile reads and decodes the YAML at path into a Directives. Rejects unrecognized top-level keys (REQ:config-file-schema). The `columns:` (top-level) and nested `or:` (inside where:<table>:) keys are recognized but reserved for deferred follow-up Features and MUST produce errors naming the deferral.
func (*Directives) IsEmpty ¶
func (d *Directives) IsEmpty() bool
IsEmpty reports whether the Directives has no filter rules at all. Used by engine.go to fast-path the no-filter case.
func (*Directives) PreValidate ¶
func (d *Directives) PreValidate() error
PreValidate runs validation rules that do NOT require source-schema introspection: mutex checks, structural sanity. Called immediately after parsing (CLI or YAML), before opening source/target.
REQ:include-exclude-mutex. Column-rule mutex (REQ:columns-mutex-per-table) is checked here too even though column subsetting is deferred, so the type stays useful for the follow-up Feature without breaking changes.
type GroupOperator ¶
type GroupOperator int
GroupOperator is And or Or.
const ( And GroupOperator = iota Or )
type OperatorToken ¶
type OperatorToken string
OperatorToken is the string form of a row-filter operator. The fixed vocabulary and the token → dal.Operator mapping are defined in operator.go (added in Task 4). Declared here as a typed alias so the Predicate struct compiles cleanly from Task 1 onward.
const ( OpEqual OperatorToken = "=" OpLessThan OperatorToken = "<" OpLessOrEqual OperatorToken = "<=" OpGreaterThan OperatorToken = ">" OpGreaterOrEqual OperatorToken = ">=" OpIn OperatorToken = "in" )
The fixed MVP operator vocabulary. Six tokens covering the operators that DALgo today exposes via dal.Operator constants. Four originally planned operators (!=, not_in, is_null, is_not_null) are deferred — dal-go/dalgo does not currently provide NotEqual, NotIn, IsNull, or IsNotNull. See REQ:operator-vocabulary and the sibling `dalgo-extended-operators` Idea (to be filed in dal-go/dalgo).
Note: DALgo's constants spell "Then" instead of "Than" (dal.LessThen, dal.GreaterThen). Preserved as-is — fixing the typo is part of the upstream follow-up work.
type Predicate ¶
type Predicate struct {
Field string
Operator OperatorToken
Value string // pre-coercion; coerced against introspected dbschema.Type at validate time
}
Predicate is a single field-operator-value triple.
func ParseWhereFlag ¶
ParseWhereFlag parses one --where <table>:<field>:<op>:<value> token. Returns the table name and resolved Predicate. The colon delimiter can be escaped with a leading backslash (\:) inside the value slot.
REQ:where-cli-syntax, REQ:operator-vocabulary.
type PredicateGroup ¶
type PredicateGroup struct {
Operator GroupOperator
Conditions []Predicate
Subgroups []*PredicateGroup
}
PredicateGroup is an AND/OR group of conditions for a single table. Conditions and Subgroups compose at the GroupOperator level (default And).
Source Files