secureread

package
v0.137.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 3, 2026 License: Apache-2.0 Imports: 43 Imported by: 0

README

pkg/secureread

Policy-enforced read executor for datatug serve (Phase 1 plan task 6, executor half). Runs structured, TugQL and native-SQL queries against any pkg/dbcopy-supported source through one fixed Session's DALgo access policies, so every read the web UI can trigger goes through the same access-control path regardless of query shape (spec/features/core-investigation-loop/README.md, REQ:server-acl-all-reads).

This package does not touch HTTP, pkg/server, pkg/storage or cmd_serve.go — those are stream S1's lane. This README is the wiring contract between the two: what S1 calls, with what, and what comes back.

Quick shape

session, err := secureread.NewSession(secureread.SessionOptions{
    As: asFlag, Roles: roleFlags, Groups: groupFlags, // same shape as `query run`
    PoliciesDir: policiesDirFlag, PolicyFiles: policyFlags, NoPolicies: noPoliciesFlag,
})
executor := secureread.NewExecutor(session) // build ONCE, for the server's whole life

result, err := executor.RunStructured(ctx, sourceURL, query, variables)
result, err := executor.RunTugQL(ctx, sourceURL, tugqlYAMLBytes, variables)
result, err := executor.RunNativeSQL(ctx, sourceURL, sqlText)

sourceURL is a pkg/dbcopy URL (sqlite:///abs/path.db, ingitdb://./dir) — datatug serve already resolves a project's configured database connection to a filesystem path; turning that into a dbcopy URL is S1's job, not something this package can do without knowing the server's connection model.

Result{Columns []string, Rows []Row{Key string, Data map[string]any}, Limitations []Limitation} — see result.go for the full doc comments. Limitations is what REQ:limitation-visible requires the web UI to show; never drop it when shaping the HTTP response.

Errors: errors.Is(err, secureread.ErrAccessDenied) for a refusal (map to HTTP 403, matching cmd_query.go's exitCodeAccessDenied convention); errors.Is(err, secureread.ErrNativeSQLUnsupported) for a source with no SQL-text surface; errors.Is(err, secureread.ErrNoPrincipal) only ever surfaces at NewSession time (server startup), never per-request.

The three call sites

1. GET /datatug/exec/select (existing — pkg/server/endpoints/execute_endpoints.go: executeSelectHandler)

Today this handler builds an unsecured api.SelectRequest{Project, Environment, Database, From, SQL, Where, Limit, Parameters} straight from query-string parameters and never checks policy. It already has the right shape to route to either executor method:

  • request.SQL != "" → opaque SQL text → Executor.RunNativeSQL(ctx, sourceURL, request.SQL).
  • otherwise → structured, built from request.From/request.Where (a dal.NewQueryBuilder(dal.From(dal.NewRootCollectionRef(request.From, ""))).Where(...).SelectColumns(), mirroring apps/datatugapp/commands/cmd_query.go's buildQuery) → Executor.RunStructured(ctx, sourceURL, query, variablesFromParameters).

request.Parameters []datatug.Parameter becomes the variables map[string]any argument (by .ID/.Value); request.Limit becomes dal.Query.Limit() on the built query, same as --from already does in cmd_query.go.

2. Saved-query run (by QueryDef.ID)

Saved TugQL queries have one body: <id>.query.tugql for text or <id>.query.tugql.yaml for the structured document. Core selects the representation from the filename and rejects multiple bodies. A write request sets QueryDef.StructuredBody for the structured form; the flag is not stored in the JSON sidecar. Executor.RunTugQL accepts the structured document, with the caller's bound parameter values as variables. Saved text is parsed through DALgo before entering that same policy-bound executor. SQL queries use Executor.RunNativeSQL.

Text authoring definitions, imports, CTEs, and inline parameter declarations require an authorized resolver context and currently fail closed. Separately declared project-query parameters remain available. Core's temporary reader can read and execute pre-cutover bodies, but this CLI's Git-backed writer refuses to edit them until they use a canonical modern filename; it never guesses which old tracked file to remove.

3. POST /datatug/exec/run_query (new — named by AC

restricted-rows-and-columns-server and tugql-query-runs)

The HTTP surface for #2: request body names a queryID (or project-relative query path) plus bound parameter values; the handler resolves the QueryDef exactly as #2 describes and shapes the JSON response from the returned secureread.Result — columns, rows, and limitations (with rowsFiltered/hiddenColumns/nativeSql derived by filtering Result.Limitations by Kind, per REQ:limitation-visible and AC restricted-rows-and-columns-server's exact field names limitations.rowsFiltered / limitations.hiddenColumns).

What this package deliberately does not decide

  • Source URL resolution. How a project's configured database connection (environment + database name) becomes a pkg/dbcopy URL is S1's concern; this package only consumes the resulting URL string.
  • HTTP status/JSON shape. ErrAccessDenied → 403, a dbcopy.Parse error → 400, everything else → 500 is a reasonable mapping (mirrors cmd_query.go's exit codes) but is S1's call, not enforced here.
  • Related-record lookups (REQ:related-lookup-execution). Those are server-built structured queries too and should go through Executor.RunStructured the same way, but building the FK-based lookup query itself is the semantic-resolution work of a different plan task — out of this stream's scope.
  • Non-sqlite native SQL. RunNativeSQL supports sqlite:// sources only today (see its doc comment for why); a future SQL-capable adapter for another scheme needs a new branch there, not a change at the call site.

Documentation

Overview

Package secureread is the policy-enforced read executor for the DataTug agent server (`datatug serve`). It runs structured, TugQL and native-SQL queries against any pkg/dbcopy-supported source through one fixed Session's DALgo access policies, so every read the web UI can trigger goes through the same access-control path regardless of the query shape (Feature core-investigation-loop, REQ:server-acl-all-reads).

Design

A Session is built once, for the whole `datatug serve` process lifetime, from the same --as/--role/--group/policy flags `datatug query run` already accepts (REQ:principal-selection): the server never re-binds its principal per request. An Executor is bound to that Session and reused across requests; each call names only the source URL and the query.

  • RunStructured executes a dal.Query (typically built with dal.NewQueryBuilder) through pkg/accesspolicies.Run: row conditions are AND-ed into the query, field allow-lists redact each row, and a query that explicitly references a field no policy allows is refused with ErrAccessDenied rather than silently emptied (AC hidden-column-refused).
  • RunTugQL deserializes a TugQL-YAML document (dal-go/dalgo/tugql) and runs it exactly like RunStructured (REQ:tugql-query-type).
  • RunNativeSQL executes raw SQL text read-only. DALgo cannot rewrite text it does not parse, so per REQ:opaque-sql-limitation (assumption A2 in the hub Feature) only the source-level allow/deny — an access.OpaqueQueryScope rule — is enforced, never row conditions or field allow-lists; the session is pinned read-only at the SQLite engine level (PRAGMA query_only) and the Result is always stamped with a LimitationNativeSQL entry. Only sqlite:// sources support native SQL today — see RunNativeSQL's doc comment.

Every Result carries the Limitations a caller must show, never apply silently (REQ:limitation-visible): which policy narrowed the rows, whether rows were filtered, which columns an implicit select hid, and whether row/ column policy was skipped for opaque SQL.

See README.md for the exact HTTP call sites this package is meant to back.

Index

Constants

View Source 
const (
	MaxStatisticDistinctValues = 256
	// 2,048 covers ordinary Chat result limits (including a full Chinook
	// invoice history) while still preventing an unbounded time-series map.
	MaxStatisticDateBuckets      = 2048
	MaxStatisticDateNumericPairs = 64
)

The caps keep result analysis bounded. An incomplete flag always accompanies a retained lower bound, so callers never mistake it for an exact result.

View Source 
const ExactTableReadProfile = "exact-table-readonly-v1"

Variables

View Source 
var ErrAccessDenied = access.ErrAccessDenied

ErrAccessDenied is access.ErrAccessDenied, re-exported so callers can check a secureread error with errors.Is(err, secureread.ErrAccessDenied) without importing the access package themselves. It is the exact same sentinel value dal-go/dalgo/access uses, not a wrapped copy.

View Source 
var ErrInvalidSession = errors.New("secureread: invalid session configuration")

ErrInvalidSession rejects contradictory or unknown session configuration.

View Source 
var ErrNativeSQLUnsupported = errors.New("secureread: native SQL is not supported for this source")

ErrNativeSQLUnsupported is returned by RunNativeSQL for a source scheme with no native SQL-text execution surface in this repository yet. See RunNativeSQL's doc comment for the exact schemes this covers today.

View Source 
var ErrNoPrincipal = errors.New("secureread: no principal named; pass --as, --role or --group, or run Unrestricted")

ErrNoPrincipal is returned by NewSession when the session is secured (NoPolicies is unset) but no principal was named via --as/--role/--group. REQ:principal-selection requires `datatug serve` to fix a principal for the whole session; constructing a secured Session with nobody named would let every request quietly fall through to each policy's own default (usually deny) with no caller-visible identity to blame or audit against. An Unrestricted session (--no-policies) may omit a principal since nothing is enforced.

View Source 
var ErrOpaqueSQLNotGranted = errors.New("secureread: native SQL execution requires an explicit opaque-query grant (--allow-opaque-sql) or an unrestricted session")

ErrOpaqueSQLNotGranted is returned by RunNativeSQL before any dispatch when the session is neither Unrestricted nor carries an explicit AllowOpaqueSQL grant (REQ:opaque-sql-limitation: "The protected profile MUST refuse opaque native SQL before execution with UNSUPPORTED_PROTECTED_EXECUTION"). Every caller of RunNativeSQL — exec/run_query and the legacy exec/select / exec/execute_commands routes alike, since they share one Executor/Session — is refused the same way.

View Source 
var ErrSnapshotPolicyUnexpressible = errors.New("snapshot policy evaluation is not safely expressible")

ErrSnapshotPolicyUnexpressible means SQLite cannot represent a recorded value without changing policy-comparison semantics. Callers must deny the read without disclosing the value.

View Source 
var ErrSourceBinding = fmt.Errorf("%w: configured source binding is unavailable; validate a new session", access.ErrAccessDenied)

Functions

func ToContractLimitations added in v0.20.6 

func ToContractLimitations(in []Limitation) []apicontract.Limitation

ToContractLimitations folds this package's own internally-split Limitation entries (a standalone "rowsFiltered" marker, a standalone "hiddenColumns" marker, one "policy"/"nativeSql" entry per applied policy — see result.go) into api-contract.md's single-shape Limitation{policy, rowsFiltered, hiddenColumns} list — the SAME shape every policy-enforced read endpoint reports (S101: exec/select gets this treatment too, not just exec/run_query, which is where this folding logic originally lived before moving here so both callers share one implementation).

Phase 1's demo scenarios apply exactly one named policy per request (customers-support, security-matrix, etc.), so folding every applied policy name into ONE combined entry alongside the overall rowsFiltered/ hiddenColumns flags is accurate for them. A project with more than one DISTINCT policy applying different row/column restrictions to the SAME request would see them combined into that one entry too (its rowsFiltered/hiddenColumns already reflect the union). Attributing rowsFiltered/hiddenColumns to the SPECIFIC policy that caused each would need this package's own internal Limitation shape to carry that association, which it does not yet — flagged as a Task 13 ("converge protected execution") follow-up, not fixed here.

func ValidateRecursiveQuery added in v0.73.0 

func ValidateRecursiveQuery(query dal.StructuredQuery) error

ValidateRecursiveQuery refuses shapes whose generic evaluator cannot preserve exact arithmetic. This runs before opening a source and is also used by previews.

func WithFederatedProgress added in v0.46.0 

func WithFederatedProgress(ctx context.Context, observer func(dal.FederatedProgress)) context.Context

WithFederatedProgress installs a request-scoped observer without changing the saved-query API or routing progress through stdout result data.

Types

type BoundSessionOptions added in v0.136.0 

type BoundSessionOptions struct {
	Required         bool
	As               string
	Roles, Groups    []string
	ExtraPolicyFiles []string
	ExtraPoliciesDir string
}

BoundSessionOptions contains trusted startup options, never request fields. Required profile policies are always loaded; optional files add intersections.

type ColumnStatistics added in v0.41.0 

type ColumnStatistics struct {
	Name                  string           `json:"name"`
	NullCount             int              `json:"nullCount"`
	NonNullCount          int              `json:"nonNullCount"`
	Cardinality           int              `json:"cardinality"`
	CardinalityIncomplete bool             `json:"cardinalityIncomplete,omitempty"`
	Frequencies           []ValueFrequency `json:"frequencies,omitempty"`
	FrequenciesIncomplete bool             `json:"frequenciesIncomplete,omitempty"`
	Types                 TypeObservations `json:"types"`
	DateBuckets           []DateBucket     `json:"dateBuckets,omitempty"`
	DateBucketsIncomplete bool             `json:"dateBucketsIncomplete,omitempty"`
}

type DatabaseProfile added in v0.132.0 

type DatabaseProfile uint8

DatabaseProfile classifies the existing database-read policy mode. It never conveys project ownership or permission to perform a database mutation. 

const (
	// DatabaseProfileUnspecified accepts legacy literals at construction only.
	DatabaseProfileUnspecified DatabaseProfile = iota
	// DatabasePolicyGoverned requires the existing policy evaluator to grant reads.
	DatabasePolicyGoverned
	// DatabasePolicyFreeReads permits today's no-policy reads. Loaded policies
	// and the existing native safeguards still apply.
	DatabasePolicyFreeReads
)

func (DatabaseProfile) PolicyFreeReads added in v0.132.0 

func (p DatabaseProfile) PolicyFreeReads() bool

PolicyFreeReads reports the configured read profile independently of project ownership. Loaded policies still govern; this is never a policy bypass.

type DateBucket added in v0.41.0 

type DateBucket struct {
	Bucket string `json:"bucket"`
	Count  int    `json:"count"`
}

type DateNumericBucket added in v0.41.0 

type DateNumericBucket struct {
	Bucket string  `json:"bucket"`
	Sum    float64 `json:"sum"`
	Count  int     `json:"count"`
}

type DateNumericSum added in v0.41.0 

type DateNumericSum struct {
	DateColumn    string              `json:"dateColumn"`
	NumericColumn string              `json:"numericColumn"`
	Buckets       []DateNumericBucket `json:"buckets,omitempty"`
	Incomplete    bool                `json:"incomplete,omitempty"`
}

type ExactAggregateResult added in v0.73.0 

type ExactAggregateResult struct {
	Result
	ColumnTypes map[string]string
}

ExactAggregateResult carries authoritative types for derived aliases. The caller must persist these with Result; catalog field-name inference is not sufficient for aggregate outputs.

type Executor 

type Executor struct {
	// contains filtered or unexported fields
}

Executor runs queries against any pkg/dbcopy-supported source URL through one fixed Session's access policies. `datatug serve` constructs one Executor for its whole life (REQ:principal-selection) and calls it once per request, naming only the source the request targets — see README.md for the three HTTP call sites this backs.

func NewExecutor 

func NewExecutor(session Session) *Executor

NewExecutor returns an Executor bound to session.

func (*Executor) Authorize added in v0.86.0 

func (e *Executor) Authorize(ctx context.Context, query dal.Query, variables map[string]any) error

Authorize applies the same shared policies as RunStructured without dispatching a data query or reading data rows. Legacy profiles use a data-less session and perform no source I/O. The configured SQLite profile opens its fixed read-only source, then verifies native identity metadata after policy admission. Supported bound tables must exist with exact ordinary main names. A policy denial itself never depends on whether the collection exists.

func (*Executor) CanReadWholeCollection added in v0.40.0 

func (e *Executor) CanReadWholeCollection(ctx context.Context, collection string) error

CanReadWholeCollection is a no-row-read policy preflight for a prospective JOIN target. A JOIN must not turn a target's row predicate or field mask into an unprotected read through its parent relation. The actual JOIN still goes through RunTugQL and its normal secure-read authorization.

func (*Executor) RunExactAggregateTugQL added in v0.108.0 

func (e *Executor) RunExactAggregateTugQL(ctx context.Context, sourceURL string, originalDocument, document []byte, variables map[string]any, columns []resultops.Column) (ExactAggregateResult, error)

RunExactAggregateTugQL executes a narrow canonical Group preview over a single source. OriginalDocument supplies projection/type provenance; document alone determines the input scope. Every source row is acquired through the current policy path. No native opaque SQL or generic float aggregation is used. Input budgets are hard errors, never partial aggregates. Integer/decimal results use exact JSON numeric lexemes; FLOAT follows resultops IEEE addition.

func (*Executor) RunFederatedTugQL added in v0.108.0 

func (e *Executor) RunFederatedTugQL(ctx context.Context, document []byte, sourceURLs map[string]string, variables map[string]any) (Result, error)

RunFederatedTugQL runs a named-database query through secured, independent leaf reads. The caller supplies source URLs from its project environment. No joined query is delegated to a database or to an OVDB server.

func (*Executor) RunNativeSQL 

func (e *Executor) RunNativeSQL(ctx context.Context, sourceURL, sqlText string, args ...dal.QueryArg) (Result, error)

RunNativeSQL executes raw SQL text read-only against sourceURL and stamps the Result with LimitationNativeSQL. DALgo cannot rewrite text it does not parse, so per REQ:opaque-sql-limitation (assumption A2) only the source-level allow/deny is enforced: the query becomes a dal.TextQuery, whose access.Resource is access.OpaqueQuery(sqlText) — a policy needs an access.OpaqueQueryScope rule to allow it at all, and no policy's row condition or field allow-list is applied to the returned rows regardless of what accesspolicies.Explain reports for the collection-scoped rules.

Before any of that, this process's own operator-level grant is checked first (Task 12, api-contract.md REQ:opaque-sql-limitation: "the support demo has no such grant"): unless the session is Unrestricted or was explicitly started with AllowOpaqueSQL (`datatug serve --allow-opaque-sql`), RunNativeSQL refuses with ErrOpaqueSQLNotGranted before doing anything else — this is what makes every native-SQL caller (exec/run_query and the legacy exec/select/exec/execute_commands routes, which all share one Executor/Session) obey the same boundary, rather than each endpoint needing its own check.

args are the query's own bind values (dal.QueryArg{Name, Value}) — a named arg (Name != "") binds an "@name"/":name"/"$name" placeholder in sqlText; a positional arg (Name == "") binds an ordinary "?" placeholder, in order. They reach the database/sql driver exactly as given: dal-go/dalgo2sql v0.11.7+ converts a named dal.QueryArg to sql.Named(Name, Value) and a positional one to its bare Value (see dal-go/dalgo2sql#177 — a real bind, not string substitution into sqlText, so a value can never be mistaken for SQL syntax).

SQLite opens its main file with mode=ro, retains PRAGMA query_only, and installs a connection authorizer that rejects writes, attachments and disabling query_only. Multi-statement SQL cannot weaken this boundary.

PostgreSQL, MySQL, Oracle and SQL Server sources run native SQL through runServerNativeSQL. DuckDB accepts a single SELECT, opens file databases read-only and initializes every physical connection with a read-only transaction. Because DuckDB SELECTs can read external files and URLs, native DuckDB queries require an unrestricted session. SQLite uses the connection authorizer described above. Other sources without SQL text execution fail with ErrNativeSQLUnsupported.

func (*Executor) RunRecursiveTugQL added in v0.108.0 

func (e *Executor) RunRecursiveTugQL(ctx context.Context, sourceURL string, document []byte, variables map[string]any) (Result, error)

RunRecursiveTugQL evaluates a derived query over one explicitly selected source. Every nested leaf passes through the same policy session and bound variables. Named databases are refused: they require a routing executor.

func (*Executor) RunSnapshot added in v0.27.0 

func (e *Executor) RunSnapshot(ctx context.Context, collection string, recordset apicontract.Recordset) (Result, error)

RunSnapshot replays recorded typed rows through the session's current access policies. A temporary private SQLite source lets the same accesspolicies.Run path enforce both column projections and expressible row conditions; an unsupported condition fails closed in RunStructured.

func (*Executor) RunStructured 

func (e *Executor) RunStructured(ctx context.Context, sourceURL string, query dal.Query, variables map[string]any) (Result, error)

RunStructured executes a structured query (typically built with dal.NewQueryBuilder) against sourceURL through the session's access policies (accesspolicies.Run): row conditions are AND-ed in, field allow-lists redact each row, and a query that explicitly references a field no policy allows is refused with ErrAccessDenied rather than silently emptied. variables resolves the query's own `param` nodes (including $currentUser, bound automatically from the session's principal); pass nil when the query has none.

func (*Executor) RunStructuredInsecureForTest added in v0.20.4 

func (e *Executor) RunStructuredInsecureForTest(ctx context.Context, sourceURL string, query dal.Query, variables map[string]any) (Result, error)

RunStructuredInsecureForTest is RunStructured, except the underlying HTTP(S) source is opened via dbcopy.BackendRef.OpenForTest instead of Open: every dalgo2http.Collection it builds gets Collection.InsecureAllowLoopback set (dal-go/dalgo2http v0.2.0's TEST-ONLY escape hatch — see httpsource.AllowInsecureLoopback's doc comment). This lets a test point a QueryDef's .query.http file at a loopback httptest.Server, or a deliberately-unreachable loopback address (e.g. 127.0.0.1:1, for a fast deterministic live-failure), while still exercising RunStructured's exact real wiring — accesspolicies.Run, the dalgo2http.Recorder context, Result.Provenance — end to end, instead of weakening the test by bypassing RunStructured altogether.

NEVER call this from production code. It exists for this package's own tests (executor_provenance_test.go) and for other packages' tests that drive the same sourceURL -> pkg/dbcopy -> pkg/httpsource pipeline in-process (e.g. apps/datatugapp/commands). A production descriptor file can never request this itself: dalgo2http excludes InsecureAllowLoopback from its YAML/JSON schema, and this method is only reachable by Go code that calls it explicitly.

func (*Executor) RunTugQL added in v0.108.0 

func (e *Executor) RunTugQL(ctx context.Context, sourceURL string, tugqlDoc []byte, variables map[string]any) (Result, error)

RunTugQL deserializes a TugQL-YAML document (dal-go/dalgo/tugql.Deserialize) and executes it exactly like RunStructured (REQ:tugql-query-type).

func (*Executor) StreamFederatedTugQL added in v0.108.0 

func (e *Executor) StreamFederatedTugQL(ctx context.Context, document []byte, sourceURLs map[string]string, variables map[string]any) (*FederatedStream, error)

StreamFederatedTugQL exposes the secured DALgo reader without collecting rows.

type FederatedStream added in v0.46.0 

type FederatedStream struct {
	Reader dal.RecordsReader
	Query  dal.StructuredQuery
	// contains filtered or unexported fields
}

FederatedStream owns every opened source until Close. Reader is already policy filtered; callers must close the stream even after a read error.

func (*FederatedStream) Close added in v0.46.0 

func (s *FederatedStream) Close() error

func (*FederatedStream) Limitations added in v0.46.0 

func (s *FederatedStream) Limitations() []Limitation

type Limitation 

type Limitation struct {
	Kind LimitationKind
	// Policy names the policy document responsible (its access.Policy.Name());
	// "native-sql" for LimitationNativeSQL.
	Policy string
	// Note is a human-readable explanation, e.g. accesspolicies.Line.String()
	// or the opaque-SQL note text.
	Note string
	// Count is reserved for a future exact filtered-row count (e.g. "3 of 12
	// rows hidden"); nil until a caller computes it — Executor does not run
	// the extra unrestricted count query needed to fill it in.
	Count *int
	// Columns holds the hidden field names for LimitationHiddenColumns.
	Columns []string
}

Limitation is one policy effect a Result's caller MUST be told about (REQ:limitation-visible) rather than have applied silently. Not every field is set for every Kind: LimitationRowsFiltered carries no Policy; LimitationHiddenColumns carries Columns; LimitationPolicy and LimitationNativeSQL carry Policy and Note.

type LimitationKind 

type LimitationKind string

LimitationKind names the shape of one applied limitation, per REQ:limitation-visible ("policy name, rows filtered yes/no, hidden columns") plus REQ:opaque-sql-limitation's native-SQL note. 

const (
	// LimitationPolicy attributes a row condition or field allow-list to
	// the policy that applied it, with a human-readable explanation in Note
	// (accesspolicies.Line.String()).
	LimitationPolicy LimitationKind = "policy"
	// LimitationRowsFiltered marks that at least one policy's row condition
	// narrowed the query (the result may legitimately be empty).
	LimitationRowsFiltered LimitationKind = "rowsFiltered"
	// LimitationHiddenColumns lists the queried collection's fields some
	// policy's field allow-list hid from an implicit (wildcard) select.
	// Columns holds the hidden field names.
	LimitationHiddenColumns LimitationKind = "hiddenColumns"
	// LimitationNativeSQL marks that row and column policies were not
	// applied because the query was opaque SQL text
	// (REQ:opaque-sql-limitation) — only source-level allow/deny ran.
	LimitationNativeSQL LimitationKind = "nativeSql"
)

type OrdinalColumn added in v0.132.0 

type OrdinalColumn struct {
	NativeType string `json:"nativeType"`
	Encoding   string `json:"encoding,omitempty"`
}

OrdinalColumn describes a column by position, never by its possibly duplicate label. Compound cells use JSON text of recursively typed cells (maps use typed key/value pairs); this preserves nested UInt64 and Decimal without floats.

type RecordSetStatistics added in v0.41.0 

type RecordSetStatistics struct {
	RowCount                  int                `json:"rowCount"`
	Columns                   []ColumnStatistics `json:"columns"`
	DateNumericSums           []DateNumericSum   `json:"dateNumericSums,omitempty"`
	DateNumericSumsIncomplete bool               `json:"dateNumericSumsIncomplete,omitempty"`
}

RecordSetStatistics is the renderer-independent, immutable analysis of a Result. Slices are sorted deterministically before being exposed or stored.

func StatisticsForRows added in v0.41.0 

func StatisticsForRows(columns []string, rows []Row) RecordSetStatistics

StatisticsForRows derives statistics from already materialized rows. It is used only for legacy result payloads; live reads update the same accumulator as they load records.

func StatisticsFromRecordset added in v0.41.0 

func StatisticsFromRecordset(recordset apicontract.Recordset) RecordSetStatistics

StatisticsFromRecordset preserves apicontract's tagged source types. It is deliberately used after RunSnapshot's SQLite policy replay instead of inferring date, decimal, or boolean semantics from SQLite driver values.

type Result 

type Result struct {
	// AnonymousRecordset is an ordered, keyless result. Rows must never be
	// reconstructed by column name: duplicate labels and rows are meaningful.
	AnonymousRecordset *apicontract.Recordset
	OrdinalColumns     []OrdinalColumn
	Columns            []string
	// ColumnTypes is optional executor-owned output metadata. Derived values
	// have types without asserting physical entity or key provenance.
	ColumnTypes map[string]string
	Rows        []Row
	// ColumnOrder is the columns of a native SQL result in the order of its
	// SELECT list, where the executor could read it; nil otherwise. Columns
	// keeps its own order (the sorted union), which stdout documents; only
	// the stored result files follow ColumnOrder.
	ColumnOrder []string
	Statistics  RecordSetStatistics
	Limitations []Limitation
	// Collection is the base collection a structured query actually read.
	// Callers use the executor-derived value for evidence provenance and
	// current-policy replay; it is empty only for opaque native queries.
	Collection string
	// Provenance is set only when the source observed one while producing
	// this Result — today, only a pkg/httpsource-backed source (opened via
	// pkg/dbcopy's http(s):// scheme) ever calls the dalgo2http.Observer
	// RunStructured wires into the query's context; every other backend
	// (sqlite, ingitdb) leaves it nil. nil means "not observed", never
	// "this was live" — mirrors pkg/httpsource.Result's own doc comment on
	// this exact ambiguity, and matches the ad-hoc `datatug query run --db
	// http://...` path's own ($provenance / "source: ..." line, PR #204)
	// treatment of the same nil-vs-live distinction.
	Provenance *dalgo2http.Provenance
	// SnapshotRecordset is populated only by RunSnapshot. It restores the
	// original contract value types after SQLite has applied current policy;
	// callers must not infer evidence types from database driver values.
	SnapshotRecordset *apicontract.Recordset
}

Result is what every Executor.Run* method returns: the columns and rows a principal is allowed to see, and the limitations that applied to produce them.

type Row 

type Row struct {
	Key  string
	Data map[string]any
}

Row is one returned record: its key (native SQL rows have none, so Key is "") and its JSON-shaped data, already policy-redacted by the time it reaches here.

type Session 

type Session struct {
	Principal         *access.Principal
	Policies          []accesspolicies.Loaded
	LocalProjectOwner bool
	DatabaseProfile   DatabaseProfile
	// AllowAnonymousLocalIdentity preserves the local read authentication fallback.
	// It grants neither project ownership nor database access.
	AllowAnonymousLocalIdentity bool
	// Unrestricted is translated only when DatabaseProfile is unspecified. Its
	// output mirrors the read profile; execution never uses this field.
	// Deprecated: use the separate ownership, database and identity fields.
	Unrestricted bool
	// AllowOpaqueSQL is the existing native-query capability, never a data-write grant.
	AllowOpaqueSQL bool
	// contains filtered or unexported fields
}

Session fixes identity, project ownership and the database read profile. Database requests use DatabaseProfile and the existing policy evaluator; project-file and asset operations use LocalProjectOwner.

func LoadBoundSession added in v0.136.0 

func LoadBoundSession(ctx context.Context, projectDir string, options BoundSessionOptions) (Session, bool, error)

LoadBoundSession loads the committed local source profile, or reports no declared profile for unchanged legacy startup. Required=true never falls back. ctx must be nonnil, as required by the Go context contract. The concrete pinned filestore loader filters nil catalogs before returning its catalog list.

func NewSession 

func NewSession(o SessionOptions) (Session, error)

NewSession loads the session's policies (accesspolicies.Load) and resolves its principal. A secured session (NoPolicies unset) MUST name a principal via As, Roles or Groups — REQ:principal-selection fixes who `datatug serve` runs as for the whole session, never anonymously — so NewSession returns ErrNoPrincipal rather than silently starting an unidentified secured session. An Unrestricted session may omit a principal entirely.

func NormalizeSession added in v0.132.0 

func NormalizeSession(s Session) (Session, error)

NormalizeSession translates legacy construction input and validates profiles. On error the returned session retains no authority. Explicit profiles never rederive ownership or identity from the legacy compatibility echo.

func RefusedSession added in v0.136.0 

func RefusedSession(err error) Session

RefusedSession returns a session without authority that retains a failed trusted configuration decision. No caller-supplied policy or principal can restore authority to it; a fresh successful configuration is required.

func (Session) BoundBackend added in v0.136.0 

func (s Session) BoundBackend(ctx context.Context, url string) (dbcopy.BackendRef, bool, error)

func (Session) BoundScope added in v0.136.0 

func (s Session) BoundScope() (project, environment, catalog string, bound bool)

func (Session) BoundSource added in v0.136.0 

func (s Session) BoundSource(ctx context.Context, project, root, environment, catalog string) (string, bool, error)

func (Session) CheckSourceBinding added in v0.136.0 

func (s Session) CheckSourceBinding(ctx context.Context) error

CheckSourceBinding revalidates the fixed configuration. ctx must be nonnil.

func (Session) SourceBindingFingerprint added in v0.136.0 

func (s Session) SourceBindingFingerprint() string

func (Session) SourceBound added in v0.136.0 

func (s Session) SourceBound() bool

func (Session) ValidateBoundProjects added in v0.136.0 

func (s Session) ValidateBoundProjects(paths map[string]string) error

func (Session) ValidateBoundServer added in v0.136.0 

func (s Session) ValidateBoundServer(paths map[string]string, writes, opaque, live bool) error

type SessionOptions 

type SessionOptions struct {
	// As is the --as principal ID; empty means no ID (roles/groups only, or
	// no principal at all).
	As string
	// Roles are the --role values (repeatable).
	Roles []string
	// Groups are the --group values (repeatable).
	Groups []string
	// PoliciesDir is the --policies-dir value; empty means
	// $DATATUG_POLICIES_DIR or ~/.datatug/policies (accesspolicies.ResolveDir).
	PoliciesDir string
	// PolicyFiles are additional --policy documents, applied after the
	// directory.
	PolicyFiles []string
	// NoPolicies runs the session Unrestricted (--no-policies): no policies
	// are loaded and no principal is required.
	NoPolicies bool
}

SessionOptions mirrors the serve-style flags `datatug query run` already accepts (see apps/datatugapp/commands/cmd_query.go), so `datatug serve` can build a Session from the same --as/--role/--group/--policies-dir/ --policy/--no-policies flags without re-deriving the plumbing.

type StatisticsCollector added in v0.41.0 

type StatisticsCollector struct {
	// contains filtered or unexported fields
}

StatisticsCollector lets a codec or reader update analysis as each row is already materialized, avoiding a separate chart-only row pass.

func NewStatisticsCollector added in v0.41.0 

func NewStatisticsCollector() *StatisticsCollector

func (*StatisticsCollector) AddRow added in v0.41.0 

func (c *StatisticsCollector) AddRow(row Row)

func (*StatisticsCollector) Finalize added in v0.41.0 

func (c *StatisticsCollector) Finalize(columns []string) RecordSetStatistics

type TypeObservations added in v0.41.0 

type TypeObservations struct {
	Null     int `json:"null"`
	String   int `json:"string"`
	Boolean  int `json:"boolean"`
	Number   int `json:"number"`
	Decimal  int `json:"decimal"`
	Date     int `json:"date"`
	Datetime int `json:"datetime"`
	Other    int `json:"other"`
}

type ValueFrequency added in v0.41.0 

type ValueFrequency struct {
	Label string    `json:"label"`
	Type  ValueKind `json:"type"`
	Count int       `json:"count"`
}

type ValueKind added in v0.41.0 

type ValueKind string

ValueKind records the source value family observed by the analysis layer. It intentionally does not depend on a renderer's data model. 

const (
	ValueKindNull     ValueKind = "null"
	ValueKindString   ValueKind = "string"
	ValueKindBoolean  ValueKind = "boolean"
	ValueKindNumber   ValueKind = "number"
	ValueKindDecimal  ValueKind = "decimal"
	ValueKindDate     ValueKind = "date"
	ValueKindDatetime ValueKind = "datetime"
	ValueKindOther    ValueKind = "other"
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.