Documentation
¶
Overview ¶
Package connsource maps connections to their DataHub URN components (platform name and catalog mapping), with forward and reverse lookups. It is extracted from the platform facade so that package stays within its size budget; the platform re-exports Source and Map as type aliases for its existing callers.
Index ¶
- func ConnectionNamesForURN(sources *Map, toolkits []registry.Toolkit, urn string) []string
- func DefaultSourceName(kind string) string
- func PlatformFromURN(urn string) string
- type Map
- func (m *Map) Add(src Source)
- func (m *Map) ConnectionsForSource(datahubSourceName string) []*Source
- func (m *Map) ConnectionsForURN(urn string) []*Source
- func (m *Map) DataHubSourceName(kind, name string) string
- func (m *Map) ForConnection(kind, name string) *Source
- func (m *Map) Overlay(src Source)
- func (m *Map) Remove(kind, name string)
- func (m *Map) Seed(entries []Source)
- type Source
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ConnectionNamesForURN ¶ added in v1.123.0
ConnectionNamesForURN returns the names of the connections that could serve a catalog URN, empty when none can be determined. It is the one resolution shared by the enrichment middleware (which tells an agent which connections can query a dataset) and the discovery connection boundary (which decides whether the caller's persona reaches any of them).
It walks the LIVE connections rather than the source map's entries, because the two are not the same set: the map is built once at startup, so it does not hold a connection added to a toolkit after it. Each connection's platform comes from its own map entry when it has one and otherwise from its toolkit's instance entry; a connection with neither is not a candidate, which leaves the URN unattributable rather than attributed by guess.
func DefaultSourceName ¶ added in v1.123.0
DefaultSourceName returns the DataHub platform a kind's datasets carry when nothing overrides it. An unmapped kind returns "", which leaves its connections out of every URN's candidate set rather than in the wrong one.
func PlatformFromURN ¶ added in v1.117.0
PlatformFromURN extracts the platform name from a DataHub URN. Example: "urn:li:dataset:(urn:li:dataPlatform:trino,...)" returns "trino". Exported so callers that resolve a URN against the live connection set (rather than this map's entries) parse the platform the same way.
Types ¶
type Map ¶
type Map struct {
// contains filtered or unexported fields
}
Map provides forward and reverse lookups between connections and DataHub URN components.
One map is shared for the life of the process: tool-call goroutines read it on every enrichment and URN build, while the admin connection routes add, overlay and remove entries from HTTP goroutines. Go maps are not safe for concurrent read and write — that pair is a fatal runtime error, not a recoverable one — so every accessor takes the lock, and the readers that answer with a slice answer with their own copy of it.
func (*Map) Add ¶
Add registers a connection's DataHub source mapping. If the same connection (kind+name) already exists, the old entry is replaced so that bySourceName never contains duplicates.
func (*Map) ConnectionsForSource ¶
ConnectionsForSource returns all connections that map to the given DataHub source name (e.g. "trino" returns all Trino connections).
func (*Map) ConnectionsForURN ¶
ConnectionsForURN parses a DataHub URN and returns all connections whose source name matches the URN's platform. Returns nil if the URN can't be parsed.
func (*Map) DataHubSourceName ¶
DataHubSourceName returns the DataHub source name mapped to a connection, or "" when none; it adapts the map to the connview.SourceResolver capability.
func (*Map) ForConnection ¶
ForConnection returns the DataHub source info for a connection. Returns nil if the connection has no mapping.
Kind and name together identify a connection, and both are required because a deployment may legitimately carry one name across several kinds. A lookup by name alone used to answer with whichever entry the map iteration reached first, which put an s3 platform on a Trino warehouse table's URN and returned a different platform on the next call (#1384). Every caller reads the kind off the audit event or PlatformContext it reads the name from, so this map is not the place to guess one.
func (*Map) Overlay ¶ added in v1.123.0
Overlay files a stored connection's mapping over the entry the map holds for the same connection: a field the stored config leaves unset keeps the value already there, and the kind's default answers only when nothing else did.
The entry it overlays MUST be the one the running configuration derives — call Seed first, or overlay onto a map the registry arm has just populated. Two behaviors depend on that. Replacing the entry outright would lose the deployment's urn_mapping on every boot after the first, because the connection backfill seeds a config-less row for each file-configured connection and a row with nothing to say must not out-rank the file. And overlaying onto a previous overlay would make a field unclearable: an operator who removes datahub_source_name from a stored connection would keep reading the value they deleted until the next restart (#1396).
func (*Map) Seed ¶ added in v1.123.0
Seed files the entry the running configuration derives for one connection, replacing whatever the map held. It is what an Overlay must land on and what a deleted stored override falls back to, so a connection is never left carrying a value no configuration states.
type Source ¶
type Source struct {
// Kind is the toolkit kind (trino, s3).
Kind string `json:"kind"`
// Name is the connection name.
Name string `json:"name"`
// DataHubSourceName is the platform identifier in DataHub URNs
// (e.g. "trino", "postgres", "s3"). Multiple connections can share the same
// source name.
DataHubSourceName string `json:"datahub_source_name"`
// CatalogMapping maps connection catalog names to DataHub catalog names.
// For example: {"rdbms": "postgres"} means the connection's "rdbms" catalog
// corresponds to "postgres" in DataHub URNs.
CatalogMapping map[string]string `json:"catalog_mapping,omitempty"`
// Description is the human-readable connection description.
Description string `json:"description,omitempty"`
}
Source holds the DataHub mapping for a single connection.
func RegistryEntries ¶ added in v1.123.0
func RegistryEntries(kind string, names []string, urnPlatform string, catalogMapping map[string]string) []Source
RegistryEntries returns the source entries a live toolkit of the given kind contributes: one per connection name it serves, all sharing the kind's mapping. urnPlatform and catalogMapping are the deployment's semantic urn_mapping, which only the query engine's kind carries. A kind that names no DataHub platform contributes nothing.
It is the one derivation of "what does the running configuration say about this connection", so the startup build and the admin path that has to restore it after a stored override is deleted cannot answer differently (#1396).
Source Files