Help us shape the future of Go! Take the 2026 Go Developer Survey

discovery

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 18, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ManualDispositionReason = "当前版本无可用脱敏算法，请人工处置"
	GenericReviewReason     = "可能含敏感自由文本，请人工判断后选择 block/hash"
)
View Source 
const (
	MinTables                = 1
	MaxTables                = 20
	MaxMetadataColumns       = 500
	MaxCandidateColumns      = 50
	MaxColumnsPerSampleQuery = 10
	DefaultSampleRows        = 10
	MaxSampleRows            = 20
	DefaultSampleValues      = 500
	MaxSampleValues          = 1000
)

Fixed first-release limits. Callers may expose these values but must not raise them at runtime.

Variables

View Source 
var (
	ErrInvalidRequest         = errors.New("discovery request is invalid")
	ErrScopeLimitExceeded     = errors.New("discovery scope limit exceeded")
	ErrCandidateLimitExceeded = errors.New("discovery candidate limit exceeded")
	ErrSampleLimitExceeded    = errors.New("discovery sample limit exceeded")
	ErrScopeNotVisible        = errors.New("discovery scope is not visible")
	ErrNotApplicable          = errors.New("discovery finding is not applicable")
	ErrUnknownCategory        = errors.New("unknown discovery category")
	ErrPermissionDenied       = errors.New("discovery permission denied")
	ErrRateLimited            = errors.New("discovery rate limited")
	ErrTimeout                = errors.New("discovery timeout")
	ErrInternal               = errors.New("discovery internal error")
)
View Source 
var (
	ErrRangeExceeded  = ErrScopeLimitExceeded
	ErrCandidateLimit = ErrCandidateLimitExceeded
)

Compatibility aliases use the shorter terminology found in the design.

Functions

func ClassifyColumnName 

func ClassifyColumnName(
	name string,
	categories []Category,
) (Category, []Signal, bool, error)

ClassifyColumnName performs only token/phrase matching. It never uses substring matching. The returned signal name belongs to a fixed vocabulary.

func IsKnownCategory 

func IsKnownCategory(category Category) bool

IsKnownCategory reports whether category is currently exposed by the discovery rollout gate. Callers must use this instead of maintaining a second category allowlist outside this package.

func NewPortError 

func NewPortError(code ErrorCode) error

NewPortError creates a redacted error that a trusted adapter may return through SchemaLister or LimitedQuerier. Only stable discovery sentinels are accepted; adapter and driver error text must never be attached.

func ValidateApplicable 

func ValidateApplicable(category Category, rule RecommendedRule) error

ValidateApplicable checks the automatic-advice contract. For number, a range rule is applicable only when its explicit width survives canonicalization.

func ValidateSample 

func ValidateSample(category Category, value string, referenceDate time.Time) bool

ValidateSample validates one complete non-NULL sample using the category's fixed first-release rule. referenceDate makes birth/age checks deterministic.

Types

type Category 

type Category string

Category identifies a first-release discovery category. 

const (
	CategoryPhone     Category = "phone"
	CategoryEmail     Category = "email"
	CategoryIDCard    Category = "idcard"
	CategoryBankCard  Category = "bankcard"
	CategoryIP        Category = "ip"
	CategoryBirthdate Category = "birthdate"
	CategoryNumber    Category = "number"
	CategoryDate      Category = "date"
	CategoryGeneric   Category = "generic"
)

type ClassifiedError 

type ClassifiedError struct {
	Code ErrorCode
	// contains filtered or unexported fields
}

ClassifiedError deliberately carries no adapter error text or sample data.

func (*ClassifiedError) Error 

func (err *ClassifiedError) Error() string

func (*ClassifiedError) Unwrap 

func (err *ClassifiedError) Unwrap() error

type ColumnMeta 

type ColumnMeta struct {
	Schema   string `json:"schema"`
	Table    string `json:"table"`
	Column   string `json:"column"`
	DataType string `json:"data_type"`
	Ordinal  int    `json:"ordinal"`
}

ColumnMeta is the minimum metadata needed by discovery.

type ColumnRef 

type ColumnRef struct {
	Schema string `json:"schema"`
	Table  string `json:"table"`
	Column string `json:"column"`
}

ColumnRef is a metadata-confirmed column identity. It is deliberately not a SQL fragment.

type Confidence 

type Confidence string

Confidence is the confidence assigned by the fixed state machine. 

const (
	ConfidenceHigh   Confidence = "high"
	ConfidenceMedium Confidence = "medium"
	ConfidenceLow    Confidence = "low"
)

type ErrorCode 

type ErrorCode string

ErrorCode is a stable error classification for later HTTP mapping. 

const (
	CodeInvalidRequest   ErrorCode = "DISCOVERY_INVALID_REQUEST"
	CodeScopeLimit       ErrorCode = "DISCOVERY_SCOPE_LIMIT"
	CodeCandidateLimit   ErrorCode = "DISCOVERY_CANDIDATE_LIMIT"
	CodeSampleLimit      ErrorCode = "DISCOVERY_SAMPLE_LIMIT"
	CodeScopeNotVisible  ErrorCode = "DISCOVERY_SCOPE_NOT_VISIBLE"
	CodeNotApplicable    ErrorCode = "DISCOVERY_NOT_APPLICABLE"
	CodeUnknownCategory  ErrorCode = "DISCOVERY_UNKNOWN_CATEGORY"
	CodePermissionDenied ErrorCode = "DISCOVERY_PERMISSION_DENIED"
	CodeRateLimited      ErrorCode = "DISCOVERY_RATE_LIMITED"
	CodeTimeout          ErrorCode = "DISCOVERY_TIMEOUT"
	CodeInternal         ErrorCode = "DISCOVERY_INTERNAL"
)

type Finding 

type Finding struct {
	Schema          string           `json:"schema"`
	Table           string           `json:"table"`
	Column          string           `json:"column"`
	DataType        string           `json:"data_type"`
	Category        Category         `json:"category"`
	Signals         []Signal         `json:"signals"`
	Confidence      Confidence       `json:"confidence"`
	Sampled         bool             `json:"sampled"`
	MatchedSamples  int              `json:"matched_samples"`
	EligibleSamples int              `json:"eligible_samples"`
	RecommendedRule *RecommendedRule `json:"recommended_rule"`
	Applicable      bool             `json:"applicable"`
	ExistingRule    bool             `json:"existing_rule"`
	Reason          string           `json:"reason,omitempty"`
}

Finding is safe to return from the core: it contains counts, never raw sample values.

type LimitedQuerier 

type LimitedQuerier interface {
	QueryColumns(
		ctx context.Context,
		datasourceID string,
		table TableRef,
		columns []ColumnRef,
		limit int,
	) (SampleBatch, error)
}

LimitedQuerier samples metadata-confirmed typed refs with an enforced row limit. It never accepts SQL text.

type Limits 

type Limits struct {
	MaxTables                int `json:"max_tables"`
	MaxMetadataColumns       int `json:"max_metadata_columns"`
	MaxCandidateColumns      int `json:"max_candidate_columns"`
	MaxColumnsPerSampleQuery int `json:"max_columns_per_sample_query"`
	DefaultSampleRows        int `json:"default_sample_rows"`
	MaxSampleRows            int `json:"max_sample_rows"`
	DefaultSampleValues      int `json:"default_sample_values"`
	MaxSampleValues          int `json:"max_sample_values"`
}

Limits reports the immutable core limits.

type RangeHint 

type RangeHint struct {
	BucketWidth  *int64 `json:"bucket_width,omitempty"`
	BucketOffset *int64 `json:"bucket_offset,omitempty"`
	Granularity  string `json:"granularity,omitempty"`
}

RangeHint is the narrow discovery representation of range parameters. It is intentionally independent of mask.RangeParams and persistence models.

type RecommendedRule 

type RecommendedRule struct {
	SensitiveType mask.SensitiveType `json:"sensitive_type"`
	Algo          mask.Algorithm     `json:"algo"`
	Range         *RangeHint         `json:"range,omitempty"`
}

RecommendedRule contains only combinations accepted by mask.NewRedactor.

func Advise 

func Advise(category Category, rangeHints ...*RangeHint) (*RecommendedRule, bool, string, error)

Advise returns the automatic recommendation for a category. A number range is returned only when the caller explicitly supplies one valid width hint; otherwise number defaults to block. Automatic advice never returns hash.

func CanonicalizeRule 

func CanonicalizeRule(category Category, rule RecommendedRule) (RecommendedRule, error)

CanonicalizeRule is the sole discovery range canonicalizer. It validates a category-aligned rule, normalizes semantically equivalent empty/zero forms, and returns fresh pointer fields that callers may safely retain.

type Result 

type Result = ScanResult

Result is kept as a concise alias for callers that prefer it.

type SampleBatch 

type SampleBatch struct {
	// contains filtered or unexported fields
}

SampleBatch is an in-process transfer object between the controlled reader and Scanner. Its raw values are intentionally inaccessible outside this package and are never copied into a Finding.

func NewSampleBatch 

func NewSampleBatch(columns []ColumnRef, rows [][]*string) SampleBatch

NewSampleBatch copies a controlled-reader result into a short-lived batch. Nil cells represent SQL NULL. Structural validation is repeated by Scanner.

type ScanRequest 

type ScanRequest struct {
	Tables     []TableRef `json:"tables"`
	Sampling   *bool      `json:"sampling,omitempty"`
	SampleRows int        `json:"sample_rows,omitempty"`
	Categories []Category `json:"categories,omitempty"`
}

ScanRequest always names its table scope explicitly. A nil Sampling value means true. SampleRows zero means the default; negative values and values above MaxSampleRows are rejected.

type ScanResult 

type ScanResult struct {
	Scope    Scope     `json:"scope"`
	Stats    Stats     `json:"stats"`
	Limits   Limits    `json:"limits"`
	Findings []Finding `json:"findings"`
}

ScanResult is the complete pure-core result.

type Scanner 

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner coordinates metadata classification and bounded candidate-only sampling. It has no database, HTTP, MCP, or raw-SQL dependency.

func NewScanner 

func NewScanner(lister SchemaLister, querier LimitedQuerier) *Scanner

func (*Scanner) Scan 

func (scanner *Scanner) Scan(
	ctx context.Context,
	datasourceID string,
	request ScanRequest,
) (ScanResult, error)

Scan fails closed on invalid scope, invisible tables, malformed adapter output, or any hard limit. Adapter error strings are never propagated.

type SchemaLister 

type SchemaLister interface {
	ListColumns(ctx context.Context, datasourceID string, tables []TableRef) ([]ColumnMeta, error)
}

SchemaLister returns metadata only for the explicitly requested typed refs.

type Scope 

type Scope struct {
	DatasourceID string     `json:"datasource_id"`
	Tables       []TableRef `json:"tables"`
	Sampling     bool       `json:"sampling"`
	SampleRows   int        `json:"sample_rows"`
	Categories   []Category `json:"categories"`
}

Scope is the normalized effective scope used for a scan.

type Signal 

type Signal struct {
	Name  string `json:"name"`
	Count int    `json:"count"`
}

Signal contains only a controlled signal name and an aggregate count. It never contains a sample or captured text.

type Stats 

type Stats struct {
	TablesRequested    int `json:"tables_requested"`
	TablesScanned      int `json:"tables_scanned"`
	ColumnsSeen        int `json:"columns_seen"`
	CandidateColumns   int `json:"candidate_columns"`
	SampledColumns     int `json:"sampled_columns"`
	SampledValuesCount int `json:"sampled_values_count"`
	FindingsCount      int `json:"findings_count"`
}

Stats contains only aggregate counters.

type TableRef 

type TableRef struct {
	Schema string `json:"schema"`
	Table  string `json:"table"`
}

TableRef is a metadata-confirmed table identity. It is deliberately not a SQL fragment.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.