knowledge

package
v1.100.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 8, 2026 License: Apache-2.0 Imports: 31 Imported by: 0

Documentation

Overview

Package knowledge provides a knowledge capture toolkit for the MCP data platform.

Index

Constants

View Source 
const (
	MinInsightTextLen   = 10
	MaxInsightTextLen   = 4000
	MaxEntityURNs       = 10
	MaxRelatedColumns   = 20
	MaxSuggestedActions = 5
	MaxApplyChanges     = 20
	MaxInsightIDs       = 50
)

Insight validation constraints.

View Source 
const (
	StatusPending    = "pending"
	StatusApproved   = "approved"
	StatusRejected   = "rejected"
	StatusApplied    = "applied"
	StatusSuperseded = "superseded"
	StatusRolledBack = "rolled_back"
)

Insight status constants.

View Source 
const DefaultLimit = 20

DefaultLimit is the default page size for list queries.

View Source 
const MaxLimit = 100

MaxLimit is the maximum page size for list queries.

View Source 
const PendingStalenessThresholdDays = 30

PendingStalenessThresholdDays is the age, in days, at or past which a pending insight is counted as stale review debt (reported as pending_over_30d). An unreviewed queue erodes the knowledge flywheel, so this threshold makes the aging visible to reviewers and to agents nudging them (#764). "At or past" (age >= threshold) matches the portal's age badge, which flags a row stale at the same 30-day mark, so the count and the badge never disagree.

Variables

View Source 
var ErrChangesetAlreadyRolledBack = errors.New("changeset already rolled back")

ErrChangesetAlreadyRolledBack is returned when a rollback targets a changeset that has already been rolled back.

Functions

func AgeDays added in v1.100.0 

func AgeDays(t, now time.Time) int

AgeDays returns the whole-day age of t relative to now, floored at 0 so a clock skew that puts t slightly in the future never reports negative. It is the single definition of whole-day insight age shared by bulk_review and platform_info so the two never drift (#764); the portal mirrors it in TS.

func NormalizeConfidence 

func NormalizeConfidence(c string) string

NormalizeConfidence returns the confidence value, defaulting to "medium" if empty.

func NormalizeSource added in v0.21.1 

func NormalizeSource(s string) string

NormalizeSource returns the source value, defaulting to "user" if empty.

func PageTargetURN added in v1.89.0 

func PageTargetURN(slug string) string

Page changeset markers. The changeset target_urn is free-form text (migration 000008), so a page promotion records "kp:<slug>" and shares the same changeset store / list / rollback surface as DataHub changesets. PageTargetURN is the changeset target_urn for a knowledge page, keyed by slug. Exported so other packages (the portal lineage endpoint) reference the same format instead of duplicating the "kp:" prefix.

func ValidateAction 

func ValidateAction(action string) error

ValidateAction checks whether an action value is valid.

func ValidateApplyChanges 

func ValidateApplyChanges(changes []ApplyChange) error

ValidateApplyChanges validates the changes slice for the apply action.

func ValidateCategory 

func ValidateCategory(c string) error

ValidateCategory checks whether a category value is valid.

func ValidateConfidence 

func ValidateConfidence(c string) error

ValidateConfidence checks whether a confidence value is valid. An empty string is valid and defaults to "medium".

func ValidateEntityURNs 

func ValidateEntityURNs(urns []string) error

ValidateEntityURNs validates the entity URN slice.

func ValidateInsightText 

func ValidateInsightText(text string) error

ValidateInsightText checks whether the insight text meets length requirements.

func ValidateRelatedColumns 

func ValidateRelatedColumns(cols []RelatedColumn) error

ValidateRelatedColumns validates the related columns slice.

func ValidateSource added in v0.21.1 

func ValidateSource(s string) error

ValidateSource checks whether a source value is valid. An empty string is valid and defaults to "user".

func ValidateStatusTransition 

func ValidateStatusTransition(from, to string) error

ValidateStatusTransition checks whether a status transition is allowed.

func ValidateSuggestedActions 

func ValidateSuggestedActions(actions []SuggestedAction) error

ValidateSuggestedActions validates a slice of suggested actions.

Types

type ApplyChange 

type ApplyChange struct {
	ChangeType       string `json:"change_type"`
	Target           string `json:"target"`
	Detail           string `json:"detail"`
	QuerySQL         string `json:"query_sql,omitempty"`
	QueryDescription string `json:"query_description,omitempty"`
}

ApplyChange represents a single change to apply to DataHub.

type ApplyConfig 

type ApplyConfig struct {
	Enabled             bool   `yaml:"enabled"`
	DataHubConnection   string `yaml:"datahub_connection"`
	RequireConfirmation bool   `yaml:"require_confirmation"`
}

ApplyConfig configures the apply_knowledge tool.

type BackfillStats added in v1.88.0 

type BackfillStats struct {
	PagesScanned int `json:"pages_scanned"`
	InlineRefs   int `json:"inline_refs"`
	PromotedRefs int `json:"promoted_refs"`
}

BackfillStats reports what a reference backfill touched.

type Changeset 

type Changeset struct {
	ID               string         `json:"id" example:"cs_x1y2z3a4b5c6d7e8"`
	CreatedAt        time.Time      `json:"created_at" example:"2026-01-15T16:00:00Z"`
	TargetURN        string         `json:"target_urn" example:"urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)"`
	ChangeType       string         `json:"change_type" example:"update_description"`
	PreviousValue    map[string]any `json:"previous_value"`
	NewValue         map[string]any `json:"new_value"`
	SourceInsightIDs []string       `json:"source_insight_ids"`
	ApprovedBy       string         `json:"approved_by" example:"admin@example.com"`
	AppliedBy        string         `json:"applied_by" example:"admin@example.com"`
	RolledBack       bool           `json:"rolled_back" example:"false"`
	RolledBackBy     string         `json:"rolled_back_by,omitempty"`
	RolledBackAt     *time.Time     `json:"rolled_back_at,omitempty"`
}

Changeset records a set of changes applied to DataHub from insights.

type ChangesetFilter 

type ChangesetFilter struct {
	EntityURN string
	AppliedBy string
	// SourceInsightID filters to changesets whose source_insight_ids array
	// contains this insight id (the thread -> insight -> changeset bridge).
	SourceInsightID string
	Since           *time.Time
	Until           *time.Time
	RolledBack      *bool
	Limit           int
	Offset          int
}

ChangesetFilter defines filtering criteria for listing changesets.

func (*ChangesetFilter) EffectiveLimit 

func (f *ChangesetFilter) EffectiveLimit() int

EffectiveLimit returns the limit to use, applying defaults and caps.

type ChangesetStore 

type ChangesetStore interface {
	InsertChangeset(ctx context.Context, cs Changeset) error
	GetChangeset(ctx context.Context, id string) (*Changeset, error)
	ListChangesets(ctx context.Context, filter ChangesetFilter) ([]Changeset, int, error)
	RollbackChangeset(ctx context.Context, id, rolledBackBy string) error
}

ChangesetStore persists and queries knowledge changesets.

func NewNoopChangesetStore 

func NewNoopChangesetStore() ChangesetStore

NewNoopChangesetStore creates a no-op ChangesetStore.

func NewPostgresChangesetStore 

func NewPostgresChangesetStore(db *sql.DB) ChangesetStore

NewPostgresChangesetStore creates a new PostgreSQL changeset store.

type DataHubClientWriter 

type DataHubClientWriter struct {
	// contains filtered or unexported fields
}

DataHubClientWriter is a real DataHubWriter implementation that delegates to the mcp-datahub client for read and write operations against DataHub.

func NewDataHubClientWriter 

func NewDataHubClientWriter(c *dhclient.Client) *DataHubClientWriter

NewDataHubClientWriter creates a DataHubClientWriter from an existing client. 

func (w *DataHubClientWriter) AddDocumentationLink(ctx context.Context, urn, linkURL, description string) error

AddDocumentationLink adds a documentation link to an entity.

func (*DataHubClientWriter) ApplyGlossaryTermChanges added in v1.94.1 

func (w *DataHubClientWriter) ApplyGlossaryTermChanges(ctx context.Context, urn string, add, remove []string) error

ApplyGlossaryTermChanges adds and removes glossary terms on an entity in a single read-modify-write of the glossaryTerms aspect. Delegating to the upstream client's per-term AddGlossaryTerm/RemoveGlossaryTerm issues a read-modify-write per term; because DataHub aspect writes are eventually consistent, back-to-back calls read stale state and the final write overwrites the whole aspect with a single term, silently dropping the rest (#729). Reading once, merging every add/remove, and writing once is lossless. A term present in both add and remove is removed.

func (*DataHubClientWriter) ApplyOwnerChanges added in v1.95.0 

func (w *DataHubClientWriter) ApplyOwnerChanges(ctx context.Context, urn string, add []OwnerChange, remove []string) error

ApplyOwnerChanges removes then adds owners on an entity. Unlike tags and glossary terms, ownership is written through the server-side additive GraphQL mutations addOwner/removeOwner (upstream write_owners.go), which modify the ownership aspect per association and are lossless, so this needs no read-modify-write batching of the whole aspect. An owner URN present in both add and remove is left removed, matching the ApplyTagChanges/ApplyGlossaryTermChanges contracts; removes are applied before adds for the same reason.

func (*DataHubClientWriter) ApplyTagChanges added in v1.94.1 

func (w *DataHubClientWriter) ApplyTagChanges(ctx context.Context, urn string, add, remove []string) error

ApplyTagChanges adds and removes tags on an entity in a single read-modify-write of the globalTags aspect. Delegating to the upstream client's per-tag AddTag/ RemoveTag issues a read-modify-write per tag; because DataHub aspect writes are eventually consistent, back-to-back calls read stale tag state and the final write overwrites the whole aspect with a single tag, silently dropping the rest (#721). Reading once, merging every add/remove, and writing once is lossless. A tag present in both add and remove is removed.

func (*DataHubClientWriter) CreateCuratedQuery added in v0.25.0 

func (w *DataHubClientWriter) CreateCuratedQuery(ctx context.Context, entityURN, name, sqlText, description string) (string, error)

CreateCuratedQuery creates a Query entity in DataHub associated with the given dataset.

func (*DataHubClientWriter) DeleteContextDocument added in v1.46.0 

func (w *DataHubClientWriter) DeleteContextDocument(ctx context.Context, documentID string) error

DeleteContextDocument removes a context document by its ID.

func (*DataHubClientWriter) DeleteTag added in v1.95.0 

func (w *DataHubClientWriter) DeleteTag(ctx context.Context, tagURN string) error

DeleteTag removes a tag definition entirely (#726).

func (*DataHubClientWriter) GetCurrentMetadata 

func (w *DataHubClientWriter) GetCurrentMetadata(ctx context.Context, urn string) (*EntityMetadata, error)

GetCurrentMetadata retrieves current metadata for an entity from DataHub.

The generic entity query (client.GetEntity) populates description and owners only for dataset and dashboard entities, so glossaryTerm and dataProduct read those fields through dedicated getters. Tags and glossary terms come from the authoritative REST aspects for the entity types that expose them over REST, and from the entity query for the GraphQL-only types (domain, glossaryTerm, glossaryNode), which mcp-datahub >= v1.10.2 surfaces on GetEntity via the experimental aspects API. This keeps both resulting_state and the rollback before-image complete for non-dataset types, where they were previously empty (#723): an empty before-image otherwise let a rollback strip tags or glossary terms the entity already had.

Description and owners are still empty for types that have neither a generic-query fragment nor a dedicated getter (e.g. container, chart, dataFlow, dataJob).

func (*DataHubClientWriter) GetIncidents added in v1.94.1 

func (w *DataHubClientWriter) GetIncidents(ctx context.Context, entityURN string) ([]types.Incident, error)

GetIncidents returns the incidents currently on an entity.

func (*DataHubClientWriter) RaiseIncident added in v1.44.0 

func (w *DataHubClientWriter) RaiseIncident(ctx context.Context, entityURN, title, description string) (string, error)

RaiseIncident creates a new incident on an entity.

func (*DataHubClientWriter) RemoveCustomProperties added in v1.95.0 

func (w *DataHubClientWriter) RemoveCustomProperties(ctx context.Context, urn string, keys []string) error

RemoveCustomProperties removes the given customProperties keys from an entity (#726). 

func (w *DataHubClientWriter) RemoveDocumentationLink(ctx context.Context, urn, linkURL string) error

RemoveDocumentationLink removes a documentation link from an entity by URL.

func (*DataHubClientWriter) RemoveStructuredProperty added in v1.44.0 

func (w *DataHubClientWriter) RemoveStructuredProperty(ctx context.Context, urn, propertyURN string) error

RemoveStructuredProperty removes a structured property from an entity.

func (*DataHubClientWriter) ResolveIncident added in v1.44.0 

func (w *DataHubClientWriter) ResolveIncident(ctx context.Context, incidentURN, message string) error

ResolveIncident marks an incident as resolved.

func (*DataHubClientWriter) SetCustomProperties added in v1.95.0 

func (w *DataHubClientWriter) SetCustomProperties(ctx context.Context, urn string, properties map[string]string) error

SetCustomProperties sets the given customProperties key/values on an entity (#726).

func (*DataHubClientWriter) SetDomain added in v1.95.0 

func (w *DataHubClientWriter) SetDomain(ctx context.Context, entityURN, domainURN string) error

SetDomain assigns a domain to an entity, replacing any existing domain.

func (*DataHubClientWriter) UnsetDomain added in v1.95.0 

func (w *DataHubClientWriter) UnsetDomain(ctx context.Context, entityURN string) error

UnsetDomain removes the domain from an entity.

func (*DataHubClientWriter) UpdateColumnDescription added in v0.22.1 

func (w *DataHubClientWriter) UpdateColumnDescription(ctx context.Context, urn, fieldPath, description string) error

UpdateColumnDescription sets the editable description for a specific column.

func (*DataHubClientWriter) UpdateColumnDescriptionBatch added in v1.53.0 

func (w *DataHubClientWriter) UpdateColumnDescriptionBatch(ctx context.Context, urn string, columns map[string]string) error

UpdateColumnDescriptionBatch sets descriptions for multiple columns in a single read-modify-write cycle. This avoids the stale-read bug where back-to-back single UpdateColumnDescription calls lose all but the last column due to DataHub's eventual consistency.

func (*DataHubClientWriter) UpdateDescription 

func (w *DataHubClientWriter) UpdateDescription(ctx context.Context, urn, description string) error

UpdateDescription sets the editable description for an entity.

func (*DataHubClientWriter) UpsertContextDocument added in v1.46.0 

func (w *DataHubClientWriter) UpsertContextDocument(ctx context.Context, entityURN string, doc types.ContextDocumentInput) (*types.ContextDocument, error)

UpsertContextDocument creates or updates a context document on an entity.

func (*DataHubClientWriter) UpsertStructuredProperties added in v1.44.0 

func (w *DataHubClientWriter) UpsertStructuredProperties(ctx context.Context, urn, propertyURN string, values []any) error

UpsertStructuredProperties sets a structured property on an entity.

type DataHubWriter 

type DataHubWriter interface {
	GetCurrentMetadata(ctx context.Context, urn string) (*EntityMetadata, error)
	UpdateDescription(ctx context.Context, urn string, description string) error
	UpdateColumnDescription(ctx context.Context, urn string, fieldPath string, description string) error
	// UpdateColumnDescriptionBatch sets descriptions for multiple columns in a single
	// read-modify-write cycle, avoiding the stale-read bug where back-to-back single
	// calls lose all but the last column.
	UpdateColumnDescriptionBatch(ctx context.Context, urn string, columns map[string]string) error
	// ApplyTagChanges adds and removes the given tags in a single read-modify-write
	// of the globalTags aspect. Per-tag writes are read-modify-write against
	// DataHub's eventually consistent store, so back-to-back single calls read stale
	// tag state and the last write clobbers the rest, silently dropping tags (#721).
	// Batching every add/remove for an entity into one read-modify-write is lossless.
	// add and remove hold full TagUrns; a tag in both add and remove is removed.
	ApplyTagChanges(ctx context.Context, urn string, add, remove []string) error
	// ApplyGlossaryTermChanges adds and removes the given glossary terms in a single
	// read-modify-write of the glossaryTerms aspect. Per-term writes are
	// read-modify-write against DataHub's eventually consistent store, so back-to-back
	// single calls read stale state and the last write clobbers the rest, silently
	// dropping terms (#729, same class of bug as #721). add and remove hold full
	// glossaryTerm URNs; a term in both add and remove is removed.
	ApplyGlossaryTermChanges(ctx context.Context, urn string, add, remove []string) error
	AddDocumentationLink(ctx context.Context, urn string, url string, description string) error
	// RemoveDocumentationLink removes a documentation link by URL. Used to revert add_documentation.
	RemoveDocumentationLink(ctx context.Context, urn string, url string) error
	CreateCuratedQuery(ctx context.Context, entityURN, name, sql, description string) (string, error)

	// Structured properties (DataHub 1.4.x)
	UpsertStructuredProperties(ctx context.Context, urn string, propertyURN string, values []any) error
	RemoveStructuredProperty(ctx context.Context, urn string, propertyURN string) error

	// Curation (#726, mcp-datahub v1.11.0). DeleteTag removes a tag definition
	// entirely; SetCustomProperties/RemoveCustomProperties edit an entity's legacy
	// customProperties map.
	DeleteTag(ctx context.Context, tagURN string) error
	SetCustomProperties(ctx context.Context, urn string, properties map[string]string) error
	RemoveCustomProperties(ctx context.Context, urn string, keys []string) error

	// Incidents (DataHub 1.4.x)
	RaiseIncident(ctx context.Context, entityURN, title, description string) (string, error)
	ResolveIncident(ctx context.Context, incidentURN, message string) error
	// GetIncidents returns the incidents currently on an entity, used to avoid raising
	// a duplicate quality-issue incident.
	GetIncidents(ctx context.Context, entityURN string) ([]types.Incident, error)

	// Context documents (DataHub 1.4.x with document support)
	UpsertContextDocument(ctx context.Context, entityURN string, doc types.ContextDocumentInput) (*types.ContextDocument, error)
	DeleteContextDocument(ctx context.Context, documentID string) error
}

DataHubWriter provides write-back operations to DataHub.

type EntityInsightSummary 

type EntityInsightSummary struct {
	EntityURN  string   `json:"entity_urn"`
	Count      int      `json:"count"`
	Categories []string `json:"categories"`
	LatestAt   string   `json:"latest_at"`
}

EntityInsightSummary summarizes insights for a single entity.

type EntityMetadata 

type EntityMetadata struct {
	Description   string   `json:"description"`
	Tags          []string `json:"tags"`
	GlossaryTerms []string `json:"glossary_terms"`
	Owners        []string `json:"owners"`
}

EntityMetadata holds current metadata for an entity from DataHub.

type Insight 

type Insight struct {
	ID               string            `json:"id" example:"a1b2c3d4e5f67890a1b2c3d4e5f67890"`
	CreatedAt        time.Time         `json:"created_at" example:"2026-01-15T14:30:00Z"`
	SessionID        string            `json:"session_id" example:"sess_abc123"`
	CapturedBy       string            `json:"captured_by" example:"analyst@example.com"`
	Persona          string            `json:"persona" example:"analyst"`
	Source           string            `json:"source" example:"user"`
	Category         string            `json:"category" example:"correction"`
	InsightText      string            `json:"insight_text" example:"The amount column represents gross margin before returns, not revenue."`
	Confidence       string            `json:"confidence" example:"high"`
	EntityURNs       []string          `json:"entity_urns"`
	RelatedColumns   []RelatedColumn   `json:"related_columns"`
	SuggestedActions []SuggestedAction `json:"suggested_actions"`
	Status           string            `json:"status" example:"pending"`
	// SinkClass is the #633 organizing axis carried onto the backing memory
	// record so the unified write path and apply_knowledge sink router can
	// route by it. Empty for insights captured before #633.
	SinkClass string `json:"sink_class,omitempty" example:"schema_entity"`

	// Lifecycle fields (populated by migrations 000007 and 000008)
	ReviewedBy   string     `json:"reviewed_by,omitempty" example:"admin@example.com"`
	ReviewedAt   *time.Time `json:"reviewed_at,omitempty"`
	ReviewNotes  string     `json:"review_notes,omitempty" example:"Verified with data engineering team"`
	AppliedBy    string     `json:"applied_by,omitempty"`
	AppliedAt    *time.Time `json:"applied_at,omitempty"`
	ChangesetRef string     `json:"changeset_ref,omitempty"`
}

Insight represents a captured domain knowledge insight.

type InsightFilter 

type InsightFilter struct {
	Status     string
	Category   string
	EntityURN  string
	CapturedBy string
	Confidence string
	Source     string
	Since      *time.Time
	Until      *time.Time
	Limit      int
	Offset     int
	// OrderCreatedAsc lists insights oldest-first (created_at ASC) instead of the
	// default newest-first (created_at DESC). The review queue uses it to work the
	// oldest, stalest review debt first (#764).
	OrderCreatedAsc bool
}

InsightFilter defines filtering criteria for listing insights.

func (*InsightFilter) EffectiveLimit 

func (f *InsightFilter) EffectiveLimit() int

EffectiveLimit returns the limit to use, applying defaults and caps.

type InsightSearchQuery added in v1.80.0 

type InsightSearchQuery struct {
	QueryText  string
	Embedding  []float32
	CapturedBy string
	Status     string
	Limit      int
}

InsightSearchQuery parameterizes a relevance-ranked insight search. It is owner-scoped (CapturedBy) and, like List, restricted to the knowledge dimension by the adapter. Embedding drives semantic (hybrid) ranking when non-empty; a nil/empty Embedding selects the lexical-only path used when no embedding provider is configured. The caller (the portal search handler) owns the embedder and precomputes Embedding so the adapter does not depend on an embedding provider.

type InsightSearcher added in v1.81.0 

type InsightSearcher interface {
	Search(ctx context.Context, q InsightSearchQuery) ([]ScoredInsight, error)
}

InsightSearcher is the optional relevance-search capability of an InsightStore. Only the memory-backed adapter implements it; the legacy SQL store and the noop store do not. Both the recall_insight tool and the portal insight-search route type-assert the wired store against this to gate registration, so the capability is declared once here next to the query and result types it uses.

type InsightStats 

type InsightStats struct {
	TotalPending int                    `json:"total_pending"`
	ByEntity     []EntityInsightSummary `json:"by_entity"`
	ByCategory   map[string]int         `json:"by_category"`
	ByConfidence map[string]int         `json:"by_confidence"`
	ByStatus     map[string]int         `json:"by_status"`
	// OldestPendingAt is the created_at of the oldest pending insight, or nil
	// when the pending queue is empty. It surfaces review-queue staleness so
	// pending insights do not silently age (#764).
	OldestPendingAt *time.Time `json:"oldest_pending_at,omitempty"`
	// PendingOver30d counts pending insights older than
	// PendingStalenessThresholdDays, the accumulating review debt (#764).
	PendingOver30d int `json:"pending_over_30d"`
}

InsightStats holds aggregated insight statistics.

type InsightStore 

type InsightStore interface {
	Insert(ctx context.Context, insight Insight) error
	Get(ctx context.Context, id string) (*Insight, error)
	List(ctx context.Context, filter InsightFilter) ([]Insight, int, error)
	UpdateStatus(ctx context.Context, id, status, reviewedBy, reviewNotes string) error
	Update(ctx context.Context, id string, updates InsightUpdate) error
	Stats(ctx context.Context, filter InsightFilter) (*InsightStats, error)
	MarkApplied(ctx context.Context, id, appliedBy, changesetRef string) error
	// MarkRolledBack transitions an applied insight to rolled_back. It is a no-op
	// for insights not currently in the applied state, so re-running a rollback
	// does not error or double-transition.
	MarkRolledBack(ctx context.Context, id, rolledBackBy string) error
	Supersede(ctx context.Context, entityURN string, excludeID string) (int, error)
}

InsightStore persists and queries captured insights.

func NewMemoryInsightAdapter added in v1.52.0 

func NewMemoryInsightAdapter(store memory.Store) InsightStore

NewMemoryInsightAdapter creates an InsightStore backed by a memory.Store.

func NewNoopStore 

func NewNoopStore() InsightStore

NewNoopStore creates a no-op InsightStore for use when no database is available.

func NewPostgresStore 

func NewPostgresStore(db *sql.DB) InsightStore

NewPostgresStore creates a new PostgreSQL insight store.

func ParseConfig 

func ParseConfig(cfg map[string]any) InsightStore

ParseConfig extracts an InsightStore from a configuration map. The store must be provided programmatically (not from YAML).

type InsightUpdate 

type InsightUpdate struct {
	InsightText string `json:"insight_text,omitempty"`
	Category    string `json:"category,omitempty"`
	Confidence  string `json:"confidence,omitempty"`
}

InsightUpdate holds fields that can be edited on a non-applied insight.

type NoopDataHubWriter 

type NoopDataHubWriter struct{}

NoopDataHubWriter is a no-op implementation for when DataHub write-back is not configured. 

func (*NoopDataHubWriter) AddDocumentationLink(_ context.Context, _, _, _ string) error

AddDocumentationLink is a no-op.

func (*NoopDataHubWriter) ApplyGlossaryTermChanges added in v1.94.1 

func (*NoopDataHubWriter) ApplyGlossaryTermChanges(_ context.Context, _ string, _, _ []string) error

ApplyGlossaryTermChanges is a no-op.

func (*NoopDataHubWriter) ApplyTagChanges added in v1.94.1 

func (*NoopDataHubWriter) ApplyTagChanges(_ context.Context, _ string, _, _ []string) error

ApplyTagChanges is a no-op.

func (*NoopDataHubWriter) CreateCuratedQuery added in v0.25.0 

func (*NoopDataHubWriter) CreateCuratedQuery(_ context.Context, _, _, _, _ string) (string, error)

CreateCuratedQuery is a no-op.

func (*NoopDataHubWriter) DeleteContextDocument added in v1.46.0 

func (*NoopDataHubWriter) DeleteContextDocument(_ context.Context, _ string) error

DeleteContextDocument is a no-op.

func (*NoopDataHubWriter) DeleteTag added in v1.95.0 

func (*NoopDataHubWriter) DeleteTag(_ context.Context, _ string) error

DeleteTag is a no-op.

func (*NoopDataHubWriter) GetCurrentMetadata 

func (*NoopDataHubWriter) GetCurrentMetadata(_ context.Context, _ string) (*EntityMetadata, error)

GetCurrentMetadata returns empty metadata.

func (*NoopDataHubWriter) GetIncidents added in v1.94.1 

func (*NoopDataHubWriter) GetIncidents(_ context.Context, _ string) ([]types.Incident, error)

GetIncidents returns no incidents.

func (*NoopDataHubWriter) RaiseIncident added in v1.44.0 

func (*NoopDataHubWriter) RaiseIncident(_ context.Context, _, _, _ string) (string, error)

RaiseIncident is a no-op.

func (*NoopDataHubWriter) RemoveCustomProperties added in v1.95.0 

func (*NoopDataHubWriter) RemoveCustomProperties(_ context.Context, _ string, _ []string) error

RemoveCustomProperties is a no-op. 

func (*NoopDataHubWriter) RemoveDocumentationLink(_ context.Context, _, _ string) error

RemoveDocumentationLink is a no-op.

func (*NoopDataHubWriter) RemoveStructuredProperty added in v1.44.0 

func (*NoopDataHubWriter) RemoveStructuredProperty(_ context.Context, _, _ string) error

RemoveStructuredProperty is a no-op.

func (*NoopDataHubWriter) ResolveIncident added in v1.44.0 

func (*NoopDataHubWriter) ResolveIncident(_ context.Context, _, _ string) error

ResolveIncident is a no-op.

func (*NoopDataHubWriter) SetCustomProperties added in v1.95.0 

func (*NoopDataHubWriter) SetCustomProperties(_ context.Context, _ string, _ map[string]string) error

SetCustomProperties is a no-op.

func (*NoopDataHubWriter) UpdateColumnDescription added in v0.22.1 

func (*NoopDataHubWriter) UpdateColumnDescription(_ context.Context, _, _, _ string) error

UpdateColumnDescription is a no-op.

func (*NoopDataHubWriter) UpdateColumnDescriptionBatch added in v1.53.0 

func (*NoopDataHubWriter) UpdateColumnDescriptionBatch(_ context.Context, _ string, _ map[string]string) error

UpdateColumnDescriptionBatch is a no-op.

func (*NoopDataHubWriter) UpdateDescription 

func (*NoopDataHubWriter) UpdateDescription(_ context.Context, _, _ string) error

UpdateDescription is a no-op.

func (*NoopDataHubWriter) UpsertContextDocument added in v1.46.0 

func (*NoopDataHubWriter) UpsertContextDocument(_ context.Context, _ string, _ types.ContextDocumentInput) (*types.ContextDocument, error)

UpsertContextDocument is a no-op.

func (*NoopDataHubWriter) UpsertStructuredProperties added in v1.44.0 

func (*NoopDataHubWriter) UpsertStructuredProperties(_ context.Context, _, _ string, _ []any) error

UpsertStructuredProperties is a no-op.

type OwnerChange added in v1.95.0 

type OwnerChange struct {
	// OwnerURN is the corpUser or corpGroup URN being added as an owner.
	OwnerURN string
	// OwnershipType is the ownership type (e.g. "TECHNICAL_OWNER"); when empty
	// the upstream client defaults it.
	OwnershipType string
}

OwnerChange is an owner to add along with its ownership type.

type PageEditedError added in v1.88.0 

type PageEditedError struct {
	Slug             string
	CurrentVersion   int
	ChangesetVersion int
}

PageEditedError is returned when a knowledge-page promotion cannot be rolled back because the page was edited after the promotion (its current version advanced past the version the changeset produced), so reverting would clobber the later edit. The page sink's counterpart of RollbackConflictError.

func (*PageEditedError) Error added in v1.88.0 

func (e *PageEditedError) Error() string

Error implements the error interface.

type PageReverter added in v1.88.0 

type PageReverter interface {
	GetBySlug(ctx context.Context, slug string) (*knowledgepage.Page, error)
	Update(ctx context.Context, id string, updates knowledgepage.Update) error
	SoftDelete(ctx context.Context, id string) error
	// Entity references (#664): restore the page's promoted references on rollback,
	// scoped to source=promoted so manual and inline references are not clobbered.
	ReplaceEntityRefsBySource(ctx context.Context, pageID, source string, refs []knowledgepage.EntityRef) error
}

PageReverter is the slice of the knowledge-page store a rollback needs: look up by slug, restore a prior version, or soft-delete a newly created page. It is the read-and-revert subset of pageWriter, satisfied by portal/knowledgepage. Store, and is exported so the admin REST rollback (pkg/admin) can supply it via RollbackDeps.Pages.

type PendingReview added in v1.100.0 

type PendingReview struct {
	TotalPending    int        `json:"total_pending"`
	OldestPendingAt *time.Time `json:"oldest_pending_at,omitempty"`
	PendingOver30d  int        `json:"pending_over_30d"`
}

PendingReview is the lightweight review-queue summary: the pending count and its staleness rollup, without the by-category/by-confidence group-bys that the full InsightStats carries. It lets platform_info surface the review-debt nudge per session without the heavier Stats fan-out (#764).

func PendingReviewOf added in v1.100.0 

func PendingReviewOf(ctx context.Context, store InsightStore) (*PendingReview, error)

PendingReviewOf returns the lightweight review-queue summary, using the store's PendingReviewStats fast path when it implements PendingReviewStater and falling back to the full Stats otherwise (so every InsightStore, including test doubles and the noop, is supported without change).

type PendingReviewStater added in v1.100.0 

type PendingReviewStater interface {
	PendingReviewStats(ctx context.Context) (*PendingReview, error)
}

PendingReviewStater is an optional InsightStore capability: a cheap pending count plus staleness rollup, without the by-category/by-confidence group-bys the full Stats issues. platform_info needs only the review-debt nudge, so it prefers this fast path (via PendingReviewOf) to avoid a four-query fan-out on every per-session orientation call (#764).

type PromptCreator added in v1.51.0 

type PromptCreator interface {
	Create(ctx context.Context, p *prompt.Prompt) error
	RegisterRuntimePrompt(p *prompt.Prompt)
}

PromptCreator creates and registers prompts at runtime.

type ProposedChange 

type ProposedChange struct {
	ChangeType       string   `json:"change_type"`
	Target           string   `json:"target"`
	CurrentValue     string   `json:"current_value"`
	SuggestedValue   string   `json:"suggested_value"`
	SourceInsightIDs []string `json:"source_insight_ids"`
}

ProposedChange represents a deterministic change proposal from synthesis.

type RelatedColumn 

type RelatedColumn struct {
	URN       string `json:"urn" example:"urn:li:dataset:(urn:li:dataPlatform:trino,hive.sales.orders,PROD)"`
	Column    string `json:"column" example:"amount"`
	Relevance string `json:"relevance" example:"direct"`
}

RelatedColumn represents a column related to an insight.

type RequiredFieldError added in v1.51.0 

type RequiredFieldError string

RequiredFieldError is a sentinel-like error for missing required fields.

func (RequiredFieldError) Error added in v1.51.0 

func (e RequiredFieldError) Error() string

Error implements the error interface.

type RollbackConflictError added in v1.70.0 

type RollbackConflictError struct {
	ConflictingIDs []string
	Aspects        []string
}

RollbackConflictError is returned when a newer, not-yet-rolled-back changeset has since mutated the same metadata aspect. Reverting would silently clobber that newer change, so the rollback is refused.

func (*RollbackConflictError) Error added in v1.70.0 

func (e *RollbackConflictError) Error() string

Error implements the error interface.

type RollbackDeps added in v1.70.0 

type RollbackDeps struct {
	Writer     DataHubWriter
	Changesets ChangesetStore
	Insights   InsightStore
	// Pages reverts knowledge-page promotions (target "kp:<slug>"). Optional; a
	// nil Pages makes a page-changeset rollback return a clear "not configured"
	// error rather than mis-routing through the DataHub inverse-op path.
	// PageReverter and PageEditedError live in page_sink.go with the rest of the
	// page-sink machinery.
	Pages PageReverter
}

RollbackDeps bundles the stores and writer a rollback operates on. It is the shared dependency set for both the apply_knowledge MCP tool and the admin REST endpoint.

type RollbackResult added in v1.70.0 

type RollbackResult struct {
	ChangesetID        string   `json:"changeset_id"`
	TargetURN          string   `json:"target_urn"`
	RevertedChanges    []string `json:"reverted_changes"`
	SkippedChanges     []string `json:"skipped_changes,omitempty"`
	InsightsRolledBack []string `json:"insights_rolled_back"`
	RolledBackBy       string   `json:"rolled_back_by,omitempty"`
}

RollbackResult summarizes the outcome of a successful changeset rollback.

func RevertChangeset added in v1.70.0 

func RevertChangeset(ctx context.Context, deps RollbackDeps, cs *Changeset, rolledBackBy string) (*RollbackResult, error)

RevertChangeset reverts the DataHub aspects mutated by a changeset back to their pre-change state, transitions the source insights to rolled_back, and marks the changeset as rolled back. It is the single rollback implementation shared by the apply_knowledge MCP tool and the admin REST endpoint.

It refuses (rather than silently no-ops) when the changeset is already rolled back, when it contains change types whose prior state was not captured, or when a newer changeset has since touched the same aspect.

type ScoredInsight added in v1.80.0 

type ScoredInsight struct {
	Insight Insight
	Score   float64
}

ScoredInsight pairs an insight with its search relevance score.

type SearchableInsightStore added in v1.88.0 

type SearchableInsightStore interface {
	InsightStore
	InsightSearcher
}

SearchableInsightStore is an InsightStore that also supports relevance search. Only the memory-backed adapter satisfies it; the legacy SQL store and the noop store implement InsightStore but not InsightSearcher. The unified search wiring asserts the wired store against this so the insights search provider gets both the entity-keyed lookup (List, filtered by EntityURN) and the relevance search (Search) from one value.

type SuggestedAction 

type SuggestedAction struct {
	ActionType       string `json:"action_type" example:"update_description"`
	Target           string `json:"target" example:"amount"`
	Detail           string `json:"detail" example:"Gross margin before returns"`
	QuerySQL         string `json:"query_sql,omitempty"`
	QueryDescription string `json:"query_description,omitempty"`
}

SuggestedAction represents a proposed catalog change.

type Toolkit 

type Toolkit struct {
	// contains filtered or unexported fields
}

Toolkit implements the knowledge capture toolkit.

func New 

func New(name string, store InsightStore) (*Toolkit, error)

New creates a new knowledge toolkit. If store is nil, a no-op store is used.

func (*Toolkit) BackfillPageRefs added in v1.88.0 

func (t *Toolkit) BackfillPageRefs(ctx context.Context, pages knowledgepage.Store) (BackfillStats, error)

BackfillPageRefs re-derives entity references for existing knowledge pages, so pages that predate the reference feature get their references without a manual re-save (#664 Phase 5). For each live page it:

  1. re-scans the body for inline references and reconciles them as source=inline (the same reconcile a page save performs); and
  2. re-derives the references the page's source insights carried, via the page's changesets, and adds them as source=promoted.

It is idempotent (the inline reconcile is a source-scoped replace, the promoted add unions) and best-effort per page: a page whose reference cannot be written (for example an inline mention of a since-deleted entity, the same case the live save path swallows) is logged and skipped, so one bad page never aborts the pass. Only a failure to list pages is returned. Pages are processed in batches because the store caps a single list at backfillPageBatch.

func (*Toolkit) Close 

func (*Toolkit) Close() error

Close releases resources.

func (*Toolkit) Connection 

func (*Toolkit) Connection() string

Connection returns the connection name for audit logging.

func (*Toolkit) Kind 

func (*Toolkit) Kind() string

Kind returns the toolkit kind.

func (*Toolkit) Name 

func (t *Toolkit) Name() string

Name returns the toolkit instance name.

func (*Toolkit) PromptInfos added in v1.38.0 

func (*Toolkit) PromptInfos() []registry.PromptInfo

PromptInfos returns metadata for prompts registered by the knowledge toolkit.

func (*Toolkit) RegisterTools 

func (t *Toolkit) RegisterTools(s *mcp.Server)

RegisterTools registers the knowledge toolkit's tools. Capture moved to the memory toolkit's memory_capture verb (#633) and reading is search; this toolkit owns admin promotion (apply_knowledge) and the capture-guidance prompts.

func (*Toolkit) RunGuardedBackfill added in v1.88.0 

func (t *Toolkit) RunGuardedBackfill(ctx context.Context, db *sql.DB, pages knowledgepage.Store)

RunGuardedBackfill runs BackfillPageRefs once, guarded by the platform_backfills sentinel: if the sentinel is absent it runs the backfill and records it. It is idempotent and leaves the sentinel unset on failure so it retries on the next start. Failures are logged, never fatal. Intended to be called in a goroutine at startup.

func (*Toolkit) SetApplyConfig 

func (t *Toolkit) SetApplyConfig(cfg ApplyConfig, csStore ChangesetStore, writer DataHubWriter)

SetApplyConfig enables the apply_knowledge tool with its dependencies.

func (*Toolkit) SetPageGuards added in v1.94.0 

func (t *Toolkit) SetPageGuards(guards knowledgepage.PageGuards, emb embedding.Provider)

SetPageGuards wires the resolved knowledge-page write-guard thresholds and the embedding provider used by the create-time duplicate gate (#705). A nil provider (or the noop placeholder) leaves the gate inactive: without a real embedding the cosine similarity is not defined, so the create proceeds unguarded.

func (*Toolkit) SetPageWriter added in v1.88.0 

func (t *Toolkit) SetPageWriter(pw pageWriter)

SetPageWriter wires the knowledge-page store so apply can promote captures to canonical pages (the destination is the sink chosen at apply, not the capture-time class). A nil store leaves the page sink unavailable (apply with sink=knowledge_page then errors rather than silently no-oping).

func (*Toolkit) SetPromptCreator added in v1.51.0 

func (t *Toolkit) SetPromptCreator(pc PromptCreator)

SetPromptCreator sets the prompt creator for add_prompt change type support.

func (*Toolkit) SetQueryProvider 

func (t *Toolkit) SetQueryProvider(provider query.Provider)

SetQueryProvider sets the query execution provider.

func (*Toolkit) SetSemanticProvider 

func (t *Toolkit) SetSemanticProvider(provider semantic.Provider)

SetSemanticProvider sets the semantic metadata provider.

func (*Toolkit) Tools 

func (t *Toolkit) Tools() []string

Tools returns the list of tool names provided by this toolkit.

type UnrevertibleError added in v1.70.0 

type UnrevertibleError struct {
	ChangeTypes []string
}

UnrevertibleError is returned when a changeset contains change types whose pre-change state is not captured in the changeset's before-image, so they cannot be reverted automatically. The recovery path for these is a manual re-apply of the desired state.

func (*UnrevertibleError) Error added in v1.70.0 

func (e *UnrevertibleError) Error() string

Error implements the error interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.