Documentation
¶
Overview ¶
Package sqlite holds the dialect-generic SQL query provider and registers it under the "sqlite" key. A host that only needs SQLite-backed record stores enables it without the cloud providers:
import _ "github.com/flanksource/commons-db/query/providers/sqlite"
query/providers registers the remaining SQL keys (sql, postgres, mysql, sqlserver, clickhouse) through NewSQLProvider.
Index ¶
- func CompareSQLite(a, b any) (int, error)
- func MergeSQLPages(ctx context.Context, request SQLMergeRequest) iter.Seq2[query.Page, error]
- func NewSQLProvider(key, connType string) query.Provider
- type Partition
- type PartitionSet
- type SQLMember
- type SQLMergeRequest
- type SQLOptions
- type SQLPageRequest
- type SQLPageResult
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CompareSQLite ¶ added in v0.1.69
CompareSQLite orders two scanned values the way SQLite's BINARY collation does, so rows read from several files merge in the order one file would have returned them. Integers compare exactly against each other and against reals; a float64 detour would tie distinct int64 keys above 2^53. A time.Time is TEXT in storage and compares chronologically against another time, and as its RFC3339Nano text against a string.
func MergeSQLPages ¶ added in v0.1.69
MergeSQLPages is a k-way merge of the same paged read over several databases.
Every member reads with ReadSQLPage under the same filters and order, each resuming from its own last row, and the merge emits the smallest head row across members until the page is full. Rows whose keys tie across members — the order's unique column is only unique within one database — are broken by member name, and because each member resumes from its own position a resumed walk neither repeats nor skips them.
Only cursor paging is supported: an offset into a merge would have to read and discard every row before it from every member.
func NewSQLProvider ¶
NewSQLProvider returns the SQL provider registered under key. connType presets the connection driver type; empty takes it from the request's options or connection.
Types ¶
type Partition ¶ added in v0.1.69
type Partition struct {
// Name is the file's path, which keeps it stable across requests and
// unique within the set.
Name string
DSN string
Labels map[string]string
}
Partition is one resolved file of a PartitionSet.
type PartitionSet ¶ added in v0.1.69
type PartitionSet struct {
// Files lists the partitions explicitly. Exclusive with Glob.
Files []string `json:"files,omitempty"`
// Glob matches the partitions, with ** for any depth.
Glob string `json:"glob,omitempty"`
// Labels is a regular expression over each partition's path whose named
// groups become constant columns of that partition's rows, such as
// `dt=(?P<dt>[0-9-]+)`. Every partition must match it. An exact-value filter
// on a label column skips the partitions it excludes without opening them.
Labels string `json:"labels,omitempty"`
// Immutable opens every partition with immutable=1: SQLite then skips all
// locking, which is only correct for sealed files nothing writes again.
Immutable bool `json:"immutable,omitempty"`
// Concurrency bounds the partitions read at once; zero means 8.
Concurrency int `json:"concurrency,omitempty"`
}
PartitionSet names SQLite files sharing one schema that a sqlite profile reads as one result, such as one file per day or per tenant.
func (PartitionSet) Resolve ¶ added in v0.1.69
func (s PartitionSet) Resolve(filters []query.ColumnFilterValue) ([]Partition, error)
Resolve lists the partitions, extracts their labels and drops those an exact-value filter on a label excludes. An empty result is an error: a read over no files names a layout that does not exist, which an empty table would hide.
type SQLMember ¶ added in v0.1.69
type SQLMember struct {
// Name identifies the member in the cursor and breaks ties between members
// holding rows with equal keys, so it must be stable across requests.
Name string
Client *sql.DB
// Labels become constant columns of every row this member returns, so
// filters, order and projections can address them like stored columns.
Labels map[string]string
}
SQLMember is one partition of a merged read: a client on one of several databases sharing a schema.
type SQLMergeRequest ¶ added in v0.1.69
type SQLMergeRequest struct {
Driver string
Members []SQLMember
// Read is the request every member runs. Its Position is the merged
// cursor: the per-member positions travel in Position.State under
// query.ProviderStateKey.
Read SQLPageRequest
// Concurrency bounds the members read at once; zero means 8.
Concurrency int
}
SQLMergeRequest reads Read from every member and merges the rows into one ordered result.
type SQLOptions ¶ added in v0.1.68
type SQLOptions struct {
// Type overrides the connection driver type (postgres, mysql, sql_server, clickhouse, sqlite).
Type string `json:"type,omitempty"`
// URL is an inline DSN used when no stored connection is referenced.
URL string `json:"url,omitempty"`
// Database overrides the database from the hydrated connection URL.
Database string `json:"database,omitempty"`
// Partitions reads the query over several SQLite files sharing a schema
// and merges their rows. Exclusive with a connection and URL.
Partitions *PartitionSet `json:"partitions,omitempty"`
}
SQLOptions are the provider-specific knobs decoded from ProviderRequest.Options.
func (SQLOptions) Validate ¶ added in v0.1.68
func (o SQLOptions) Validate() error
Validate refuses a Type that names no SQL engine the provider builds for.
type SQLPageRequest ¶
type SQLPageRequest struct {
// Connection names the connection client is, for the SQL statement tap.
Connection string
Query string
QueryArgs []any
QueryIdentifiers []string
Filters []query.ColumnFilterValue
Order query.Order
Position query.CursorPosition
Page query.PageRequest
Diagnostics *query.ProviderDiagnostics
}
type SQLPageResult ¶
type SQLPageResult struct {
Rows []query.Row
ColumnTypes []*sql.ColumnType
Total *query.Total
HasMore bool
}
func ReadSQLPage ¶
func ReadSQLPage(ctx context.Context, client *sql.DB, driver string, request SQLPageRequest) (result SQLPageResult, err error)