semantic

semantic

Pure logic — no I/O, no mutation of its inputs — that turns a project's model (Entity, EntityField.Mappings/NamePatterns, QueryDef.Parameters[].Meta) plus a scanned schema into the three things the core-investigation-loop feature's investigation loop needs: which column means which entity field (Resolve), what else is reachable from a selected value (RelatedLookups), and which library queries can already be run with the values on hand (Applicable). Safe to call from datatug-cli's HTTP resolver, the TUI, or a future datatug context verb — this package never talks to a database or a project store itself.

Resolve

Maps physical columns (source + collection + scanned column list) to the entity fields a project's model says they mean, per REQ:field-mapping-model: declared EntityField.Mappings win; absent a declared mapping, EntityField.NamePatterns are tried and the result is labelled inferred; a column matching neither is simply absent from the result.

entities := []*datatug.Entity{
	{
		ProjectItem: datatug.ProjectItem{ProjItemBrief: datatug.ProjItemBrief{ID: "customer"}},
		Fields: datatug.EntityFields{
			{
				ID: "id", Type: "string",
				Mappings: datatug.PhysicalRefs{
					{Source: "chinook", Collection: "Customer", Column: "CustomerId"},
				},
			},
			{
				ID: "email", Type: "string",
				NamePatterns: datatug.StringPatterns{{Type: "exact", Value: "Email"}},
			},
		},
	},
}

results := semantic.Resolve(entities, "chinook", "Customer", []semantic.Column{
	{Name: "CustomerId", Type: "int"},
	{Name: "Email", Type: "string"},
	{Name: "FirstName", Type: "string"},
})
// results == []semantic.Resolution{
//   {Column: "CustomerId", Entity: "customer", Field: "id", Provenance: semantic.Declared},
//   {Column: "Email", Entity: "customer", Field: "email", Provenance: semantic.Inferred, Rule: "namePattern:exact:Email"},
// }
// "FirstName" matches neither a declared mapping nor a name pattern, so it
// has no entry at all.

RelatedLookups

Given a selected SemanticValue (a resolved cell — build one from a Resolution plus the value the cell held), returns every other collection reachable from it: through declared foreign keys in both directions within its own source, and through mappings of the same entity field in other sources/collections, per REQ:related-lookup-model. It only says where to look — the caller (datatug-cli, per REQ:related-lookup-execution) builds and runs the actual filtered query through the access-policy path.

For an outgoing foreign key, the join column is the referenced column the key names (datatug.ForeignKey.RefColumns, paired by position with the selected column), for a single-column and a composite key alike. Only a key without referenced columns (read from a source that does not report them) keeps the older guess: the referenced table's single-column primary key.

schema := map[semantic.SchemaKey]semantic.TableSchema{
	{Source: "chinook", Collection: "Customer"}: {
		PrimaryKey: &datatug.UniqueKey{Name: "PK_Customer", Columns: []string{"CustomerId"}},
		ReferencedBy: datatug.ReferencedBys{
			{
				DBCollectionKey: datatug.NewTableKey("Invoice", "", "", nil),
				ForeignKeys:     []*datatug.RefByForeignKey{{Name: "FK_Invoice_Customer", Columns: []string{"CustomerId"}}},
			},
		},
	},
}

selected := semantic.SemanticValue{
	Entity: "customer", Field: "id", Value: 5,
	Source: "chinook", Collection: "Customer", Column: "CustomerId",
	Provenance: semantic.Declared,
}

lookups := semantic.RelatedLookups(entities, schema, selected)
// lookups == []semantic.Lookup{
//   {Source: "chinook", Collection: "Invoice", Column: "CustomerId", Kind: semantic.LookupReferencedBy, Via: "FK_Invoice_Customer"},
//   {Source: "support-notes", Collection: "Customer", Column: "CustomerId", Kind: semantic.LookupSameField, Via: "field:customer.id"},
// }
// (the second entry assumes entities also declares a mapping of customer.id
// to source "support-notes", collection "Customer")

Applicable

Given the library's queries and the semantic values on hand (the current selection plus the Investigation Context), splits them into queries every required, Meta-tagged parameter of which can be bound, and queries still missing at least one — per REQ:applicable-queries and REQ:parameter-auto-binding. A parameter with no Meta, or an unsatisfied optional one, never blocks applicability.

query := &datatug.QueryDef{
	ProjectItem: datatug.ProjectItem{ProjItemBrief: datatug.ProjItemBrief{ID: "customer-invoices", Title: "Customer invoices"}},
	Type: datatug.QueryTypeSQL,
	Parameters: datatug.Parameters{
		{ID: "customerId", Type: "integer", IsRequired: true, Meta: &datatug.EntityFieldRef{Entity: "Customer", Field: "ID"}},
	},
}

available := []semantic.SemanticValue{
	{Entity: "Customer", Field: "ID", Value: 5, Source: "chinook", Collection: "Customer", Column: "CustomerId", Provenance: semantic.Declared},
}

applicable, notYet := semantic.Applicable([]*datatug.QueryDef{query}, available)
// applicable[0].Bindings  == []semantic.Binding{{Parameter: "customerId", Value: 5, From: available[0]}}
// applicable[0].Chain     == []string{"CustomerId → Customer.ID (declared) → parameter customerId"}
// notYet                  == nil (this query had everything it needed)

LookupSources

Given a parameter that means an entity field, returns the collections its value can be picked from, per the parameter-lookup feature (datatug/backstage spec/features/parameter-lookup, REQ:lookup-source-resolution): every declared mapping of that field (never a NamePatterns inference) whose collection's primary key is exactly that one column, with a key kind (ColumnKeyKind: integer or text affinity) the parameter's type can hold (ParameterAcceptsKey). Each source carries its display columns (DisplayColumns): the entity's display field list when it yields any column in that collection, otherwise up to three inferred ones (other mapped fields, then the schema's first text columns), labelled Inferred.

customer := &datatug.Entity{
	ProjectItem: datatug.ProjectItem{ProjItemBrief: datatug.ProjItemBrief{ID: "Customer"}},
	Display:     []string{"CompanyName"},
	Fields: datatug.EntityFields{
		{ID: "ID", Type: "string", Mappings: datatug.PhysicalRefs{{Source: "northwind", Collection: "Customers", Column: "CustomerID"}}},
		{ID: "CompanyName", Type: "string", Mappings: datatug.PhysicalRefs{{Source: "northwind", Collection: "Customers", Column: "CompanyName"}}},
	},
}
schema := map[semantic.SchemaKey]semantic.TableSchema{
	{Source: "northwind", Collection: "Customers"}: {
		PrimaryKey: &datatug.UniqueKey{Name: "PK_Customers", Columns: []string{"CustomerID"}},
		Columns:    []semantic.Column{{Name: "CustomerID", Type: "nchar(5)"}, {Name: "CompanyName", Type: "nvarchar(40)"}},
	},
}
param := datatug.ParameterDef{ID: "CustomerID", Type: "string", Meta: &datatug.EntityFieldRef{Entity: "Customer", Field: "ID"}}

sources := semantic.LookupSources([]*datatug.Entity{customer}, schema, param, "")
// sources == []semantic.LookupSource{{
//   Source: "northwind", Collection: "Customers", KeyColumn: "CustomerID", KeyKind: semantic.KeyText,
//   Display: []semantic.DisplayColumn{{Column: "CompanyName", Field: "CompanyName", Label: "CompanyName", Provenance: semantic.Declared}},
//   DisplayProvenance: semantic.Declared,
//   Meaning: semantic.ParameterMeaning{Ref: datatug.EntityFieldRef{Entity: "Customer", Field: "ID"}, Provenance: semantic.Declared},
// }}

ResolveParameterMeaning

The field a parameter means (REQ:inferred-parameter-meaning): its declared Meta when set; otherwise the one key field (isKeyField: true) whose NamePatterns match the parameter's name without @, labelled Inferred with the matching pattern in Rule. Patterns match as in Resolve. When key fields of more than one entity field match, the name is ambiguous and there is no meaning (so no lookup), never a guess. LookupSources calls it, and each source's Meaning says which way the meaning was found.

customer.Fields[0].IsKeyField = true
customer.Fields[0].NamePatterns = datatug.StringPatterns{{Type: "regexp", Value: `\w+CustomerID`, CaseSensitive: true}}
meaning, ok := semantic.ResolveParameterMeaning([]*datatug.Entity{customer}, datatug.ParameterDef{ID: "@ShipCustomerID", Type: "string"})
// ok == true
// meaning == semantic.ParameterMeaning{Ref: {Entity: "Customer", Field: "ID"}, Provenance: semantic.Inferred, Rule: `namePattern:regexp:\w+CustomerID`}

It only says where to pick from; building and running the bounded filter query is the runtime's job (the browser SQLite worker for a public fixture, or POST /datatug/semantic/lookup on a server, under its access policy).