Documentation
¶
Overview ¶
Package toolkitcfg resolves typed per-toolkit connection configuration out of the platform's raw toolkits config (map[string]any decoded from YAML/JSON).
The platform stores toolkit config as toolkits.<kind>.instances.<name>.<key>; these helpers walk that structure, pick the default instance when none is named, and extract the typed DataHub / Trino / S3 settings the providers need. Split out of pkg/platform to keep that package under its size budget (#756); the primitive typed-map accessors live in the sibling cfgmap package.
Index ¶
- Constants
- func AutoEnableKind(toolkits map[string]any, kind string)
- func InstanceConfig(toolkits map[string]any, kind, instance string) map[string]any
- func KindEnabled(kindMap map[string]any) bool
- func MergeInstance(toolkits map[string]any, kind, name string, cfg map[string]any)
- func MissingDefaults(toolkits map[string]any) []string
- func PinDeclaredDefaults(toolkits map[string]any)
- func ResolveDefaultInstance(kindCfg, instances map[string]any) string
- type DataHub
- type DeclaredConnections
- type S3
- type Trino
Constants ¶
const ( DefaultTrinoPort = 8080 DefaultTrinoQueryLimit = 1000 DefaultTrinoMaxLimit = 10000 )
Trino connection defaults, applied when an instance omits the setting.
Variables ¶
This section is empty.
Functions ¶
func AutoEnableKind ¶ added in v1.123.0
AutoEnableKind ensures toolkits[kind] exists with enabled=true so the toolkit loader will instantiate it. Idempotent and non-overriding: if the operator has already declared the kind block (enabled OR disabled), their explicit choice is respected.
Logs at Debug, not Info: this is the platform's documented default behavior, not an exceptional condition that requires operator attention. Operators who want to silence the path entirely can set the kind explicitly in YAML (with either enabled state).
func InstanceConfig ¶
InstanceConfig retrieves one instance's config map for a toolkit kind. When instance is "" it resolves the default (or first) instance. Returns nil if the kind, its instances map, or the named instance is absent or malformed.
func KindEnabled ¶ added in v1.123.0
KindEnabled reports whether a toolkit kind map has enabled=true. It handles both bool and string values, because environment-variable expansion produces strings.
func MergeInstance ¶ added in v1.123.0
MergeInstance merges one stored connection into the toolkit config map under its kind's instances. It is a no-op when the kind is absent or disabled, and when the kind already carries an instance of that name: file config takes precedence over a connection held in the database.
Instances merged here arrive after Config.Validate has run, so they are not covered by MissingDefaults: a kind can hold several of them with no "default", which is why ResolveDefaultInstance resolves deterministically rather than relying on that refusal.
func MissingDefaults ¶ added in v1.123.0
MissingDefaults returns one message per toolkit kind that configures more than one instance without a "default" key naming which of them a lookup that omits the instance means. Kinds and the instance names within a message are sorted, so the same config produces the same messages on every run. Callers treat a non-empty result as a config error; Config.Validate does.
Two DataHub catalogs with no default is not a deployment that chose either one: whichever the platform picks binds the semantic provider, the query provider and the managed-resource blob store to a connection the operator never named. Naming the candidates lets them make that choice.
A kind is checked whether or not it is enabled, because the providers read an instance's config through InstanceConfig without consulting the enable flag: a catalog used only for enrichment registers no tools and still has to say which of its instances the enrichment reads.
This covers only the instances a config declares. Connections held in the database merge into the toolkits config after validation; PinDeclaredDefaults keeps them from taking over the lookup a declared instance answers today.
func PinDeclaredDefaults ¶ added in v1.123.0
PinDeclaredDefaults records, for every kind that declares instances without a "default", the instance its config resolves to today. Call it once before merging connections held in the database.
Without it, a kind that declares a single instance and needs no "default" changes meaning when an admin-UI connection whose name sorts earlier joins the same map: the next restart resolves the unqualified lookup to the new connection and moves a provider, or managed-resource blob storage, off the connection the file pointed at. Pinning first is the same rule MergeInstance already follows, that file config outranks a stored connection.
func ResolveDefaultInstance ¶
ResolveDefaultInstance determines which instance a lookup that names none means: the one named by the kind's "default" key, else the lexicographically first instance, else "".
The fallback compares names rather than taking whatever a map range hands back first. Go randomizes map iteration order, so ranging resolved a different instance on every process start: two replicas built from one config disagreed about which connection an unqualified lookup meant, and a restart could point managed-resource blob storage at a different S3 connection than the one existing resources were written through. The multi-connection Trino toolkit picks its own default the same way (pkg/toolkits/trino/toolkit.go).
An empty "default" is treated as absent, so it agrees with MissingDefaults about which configs have named an instance.
Types ¶
type DeclaredConnections ¶ added in v1.123.0
DeclaredConnections records, per toolkit kind, the connection instances the config file declared. It is the only record of which connections the file owns: MergeInstance puts stored connections into the same instances map the file produced, and connbackfill seeds a connection_instances row for every file-configured connection, so neither the merged config nor the store can answer "did the file declare this one" afterwards.
The keys are the instance names, which is the same namespace a connection_instances row's name and a MergeInstance call use.
func Declared ¶ added in v1.123.0
func Declared(toolkits map[string]any) DeclaredConnections
Declared snapshots the instances each kind declares. Call it before MergeInstance merges a stored connection into the same map, and before PinDeclaredDefaults, which reads instances but adds none.
Every kind is captured, including the ones the admin connection API does not manage: a file-declared datahub instance is as much the file's as a trino one.
func (DeclaredConnections) Has ¶ added in v1.123.0
func (d DeclaredConnections) Has(kind, name string) bool
Has reports whether the config file declared name as an instance of kind. The zero value declares nothing, so a caller holding no snapshot treats every connection as database-owned.
type S3 ¶
type S3 struct {
Region string
Endpoint string
AccessKeyID string
SecretKey string
BucketPrefix string
ConnectionName string
UsePathStyle bool
}
S3 holds extracted S3 configuration.
type Trino ¶
type Trino struct {
Host string
Port int
User string
Password string // #nosec G117 -- Trino connection credential from admin config
Catalog string
Schema string
SSL bool
SSLVerify bool
Timeout time.Duration
DefaultLimit int
MaxLimit int
ReadOnly bool
// ConnectionName is the name a call binds this connection by: the resolved
// `instances:` key, which is what the Trino toolkit routes on. The query
// provider stamps it onto every availability answer, so it is the name an
// agent is told to pass as `connection` — a label the router does not know
// would send the agent to a connection that refuses it (#1396).
ConnectionName string
}
Trino holds extracted Trino configuration.
func TrinoConfig ¶
TrinoConfig extracts Trino configuration for the named instance (or the default instance when instance is ""). Returns nil if not configured.
ConnectionName is the resolved instance name rather than the instance's `connection_name`: Trino routes by instance, so that key is the name a `connection` argument carries and a persona rule matches, and it is what the toolkit reports as its connection. Reading `connection_name` here published a name no call could bind, and reading nothing published an empty one (#1396).
Source Files