discovery

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 17, 2026 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ManualDispositionReason = "当前版本无可用脱敏算法，请人工处置"
	GenericReviewReason     = "可能含敏感自由文本，请人工判断后选择 block/hash"
)
View Source 
const (
	MinTables                = 1
	MaxTables                = 20
	MaxMetadataColumns       = 500
	MaxCandidateColumns      = 50
	MaxColumnsPerSampleQuery = 10
	DefaultSampleRows        = 10
	MaxSampleRows            = 20
	DefaultSampleValues      = 500
	MaxSampleValues          = 1000
)

Fixed first-release limits. Callers may expose these values but must not raise them at runtime.

Variables

View Source 
var (
	ErrInvalidRequest         = errors.New("discovery request is invalid")
	ErrScopeLimitExceeded     = errors.New("discovery scope limit exceeded")
	ErrCandidateLimitExceeded = errors.New("discovery candidate limit exceeded")
	ErrSampleLimitExceeded    = errors.New("discovery sample limit exceeded")
	ErrScopeNotVisible        = errors.New("discovery scope is not visible")
	ErrNotApplicable          = errors.New("discovery finding is not applicable")
	ErrUnknownCategory        = errors.New("unknown discovery category")
	ErrPermissionDenied       = errors.New("discovery permission denied")
	ErrRateLimited            = errors.New("discovery rate limited")
	ErrTimeout                = errors.New("discovery timeout")
	ErrInternal               = errors.New("discovery internal error")
)
View Source 
var (
	ErrRangeExceeded  = ErrScopeLimitExceeded
	ErrCandidateLimit = ErrCandidateLimitExceeded
)

Compatibility aliases use the shorter terminology found in the design.

Functions

func ClassifyColumnName 

func ClassifyColumnName(
	name string,
	categories []Category,
) (Category, []Signal, bool, error)

ClassifyColumnName performs only token/phrase matching. It never uses substring matching. The returned signal name belongs to a fixed vocabulary.

func IsKnownCategory 

func IsKnownCategory(category Category) bool

IsKnownCategory reports whether category is currently exposed by the discovery rollout gate. Callers must use this instead of maintaining a second category allowlist outside this package.

func NewPortError 

func NewPortError(code ErrorCode) error

NewPortError creates a redacted error that a trusted adapter may return through SchemaLister or LimitedQuerier. Only stable discovery sentinels are accepted; adapter and driver error text must never be attached.

func ValidateApplicable 

func ValidateApplicable(category Category, rule RecommendedRule) error

ValidateApplicable checks the automatic-advice contract. For number, a range rule is applicable only when its explicit width survives canonicalization.

func ValidateSample 

func ValidateSample(category Category, value string, referenceDate time.Time) bool

ValidateSample validates one complete non-NULL sample using the category's fixed first-release rule. referenceDate makes birth/age checks deterministic.

Types

type Category 

type Category string

Category identifies a first-release discovery category. 

const (
	CategoryPhone     Category = "phone"
	CategoryEmail     Category = "email"
	CategoryIDCard    Category = "idcard"
	CategoryBankCard  Category = "bankcard"
	CategoryIP        Category = "ip"
	CategoryBirthdate Category = "birthdate"
	CategoryNumber    Category = "number"
	CategoryDate      Category = "date"
	CategoryGeneric   Category = "generic"
)

type ClassifiedError 

type ClassifiedError struct {
	Code ErrorCode
	// contains filtered or unexported fields
}

ClassifiedError deliberately carries no adapter error text or sample data.

func (*ClassifiedError) Error 

func (err *ClassifiedError) Error() string

func (*ClassifiedError) Unwrap 

func (err *ClassifiedError) Unwrap() error

type ColumnMeta 

type ColumnMeta struct {
	Schema   string `json:"schema"`
	Table    string `json:"table"`
	Column   string `json:"column"`
	DataType string `json:"data_type"`
	Ordinal  int    `json:"ordinal"`
}

ColumnMeta is the minimum metadata needed by discovery.

type ColumnRef 

type ColumnRef struct {
	Schema string `json:"schema"`
	Table  string `json:"table"`
	Column string `json:"column"`
}

ColumnRef is a metadata-confirmed column identity. It is deliberately not a SQL fragment.

type Confidence 

type Confidence string

Confidence is the confidence assigned by the fixed state machine. 

const (
	ConfidenceHigh   Confidence = "high"
	ConfidenceMedium Confidence = "medium"
	ConfidenceLow    Confidence = "low"
)

type ErrorCode 

type ErrorCode string

ErrorCode is a stable error classification for later HTTP mapping. 

const (
	CodeInvalidRequest   ErrorCode = "DISCOVERY_INVALID_REQUEST"
	CodeScopeLimit       ErrorCode = "DISCOVERY_SCOPE_LIMIT"
	CodeCandidateLimit   ErrorCode = "DISCOVERY_CANDIDATE_LIMIT"
	CodeSampleLimit      ErrorCode = "DISCOVERY_SAMPLE_LIMIT"
	CodeScopeNotVisible  ErrorCode = "DISCOVERY_SCOPE_NOT_VISIBLE"
	CodeNotApplicable    ErrorCode = "DISCOVERY_NOT_APPLICABLE"
	CodeUnknownCategory  ErrorCode = "DISCOVERY_UNKNOWN_CATEGORY"
	CodePermissionDenied ErrorCode = "DISCOVERY_PERMISSION_DENIED"
	CodeRateLimited      ErrorCode = "DISCOVERY_RATE_LIMITED"
	CodeTimeout          ErrorCode = "DISCOVERY_TIMEOUT"
	CodeInternal         ErrorCode = "DISCOVERY_INTERNAL"
)

type Finding 

type Finding struct {
	Schema          string           `json:"schema"`
	Table           string           `json:"table"`
	Column          string           `json:"column"`
	DataType        string           `json:"data_type"`
	Category        Category         `json:"category"`
	Signals         []Signal         `json:"signals"`
	Confidence      Confidence       `json:"confidence"`
	Sampled         bool             `json:"sampled"`
	MatchedSamples  int              `json:"matched_samples"`
	EligibleSamples int              `json:"eligible_samples"`
	RecommendedRule *RecommendedRule `json:"recommended_rule"`
	Applicable      bool             `json:"applicable"`
	ExistingRule    bool             `json:"existing_rule"`
	Reason          string           `json:"reason,omitempty"`
}

Finding is safe to return from the core: it contains counts, never raw sample values.

type LimitedQuerier 

type LimitedQuerier interface {
	QueryColumns(
		ctx context.Context,
		datasourceID string,
		table TableRef,
		columns []ColumnRef,
		limit int,
	) (SampleBatch, error)
}

LimitedQuerier samples metadata-confirmed typed refs with an enforced row limit. It never accepts SQL text.

type Limits 

type Limits struct {
	MaxTables                int `json:"max_tables"`
	MaxMetadataColumns       int `json:"max_metadata_columns"`
	MaxCandidateColumns      int `json:"max_candidate_columns"`
	MaxColumnsPerSampleQuery int `json:"max_columns_per_sample_query"`
	DefaultSampleRows        int `json:"default_sample_rows"`
	MaxSampleRows            int `json:"max_sample_rows"`
	DefaultSampleValues      int `json:"default_sample_values"`
	MaxSampleValues          int `json:"max_sample_values"`
}

Limits reports the immutable core limits.

type RangeHint 

type RangeHint struct {
	BucketWidth  *int64 `json:"bucket_width,omitempty"`
	BucketOffset *int64 `json:"bucket_offset,omitempty"`
	Granularity  string `json:"granularity,omitempty"`
}

RangeHint is the narrow discovery representation of range parameters. It is intentionally independent of mask.RangeParams and persistence models.

type RecommendedRule 

type RecommendedRule struct {
	SensitiveType mask.SensitiveType `json:"sensitive_type"`
	Algo          mask.Algorithm     `json:"algo"`
	Range         *RangeHint         `json:"range,omitempty"`
}

RecommendedRule contains only combinations accepted by mask.NewRedactor.

func Advise 

func Advise(category Category, rangeHints ...*RangeHint) (*RecommendedRule, bool, string, error)

Advise returns the automatic recommendation for a category. A number range is returned only when the caller explicitly supplies one valid width hint; otherwise number defaults to block. Automatic advice never returns hash.

func CanonicalizeRule 

func CanonicalizeRule(category Category, rule RecommendedRule) (RecommendedRule, error)

CanonicalizeRule is the sole discovery range canonicalizer. It validates a category-aligned rule, normalizes semantically equivalent empty/zero forms, and returns fresh pointer fields that callers may safely retain.

type Result 

type Result = ScanResult

Result is kept as a concise alias for callers that prefer it.

type SampleBatch 

type SampleBatch struct {
	// contains filtered or unexported fields
}

SampleBatch is an in-process transfer object between the controlled reader and Scanner. Its raw values are intentionally inaccessible outside this package and are never copied into a Finding.

func NewSampleBatch 

func NewSampleBatch(columns []ColumnRef, rows [][]*string) SampleBatch

NewSampleBatch copies a controlled-reader result into a short-lived batch. Nil cells represent SQL NULL. Structural validation is repeated by Scanner.

type ScanRequest 

type ScanRequest struct {
	Tables     []TableRef `json:"tables"`
	Sampling   *bool      `json:"sampling,omitempty"`
	SampleRows int        `json:"sample_rows,omitempty"`
	Categories []Category `json:"categories,omitempty"`
}

ScanRequest always names its table scope explicitly. A nil Sampling value means true. SampleRows zero means the default; negative values and values above MaxSampleRows are rejected.

type ScanResult 

type ScanResult struct {
	Scope    Scope     `json:"scope"`
	Stats    Stats     `json:"stats"`
	Limits   Limits    `json:"limits"`
	Findings []Finding `json:"findings"`
}

ScanResult is the complete pure-core result.

type Scanner 

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner coordinates metadata classification and bounded candidate-only sampling. It has no database, HTTP, MCP, or raw-SQL dependency.

func NewScanner 

func NewScanner(lister SchemaLister, querier LimitedQuerier) *Scanner

func (*Scanner) Scan 

func (scanner *Scanner) Scan(
	ctx context.Context,
	datasourceID string,
	request ScanRequest,
) (ScanResult, error)

Scan fails closed on invalid scope, invisible tables, malformed adapter output, or any hard limit. Adapter error strings are never propagated.

type SchemaLister 

type SchemaLister interface {
	ListColumns(ctx context.Context, datasourceID string, tables []TableRef) ([]ColumnMeta, error)
}

SchemaLister returns metadata only for the explicitly requested typed refs.

type Scope 

type Scope struct {
	DatasourceID string     `json:"datasource_id"`
	Tables       []TableRef `json:"tables"`
	Sampling     bool       `json:"sampling"`
	SampleRows   int        `json:"sample_rows"`
	Categories   []Category `json:"categories"`
}

Scope is the normalized effective scope used for a scan.

type Signal 

type Signal struct {
	Name  string `json:"name"`
	Count int    `json:"count"`
}

Signal contains only a controlled signal name and an aggregate count. It never contains a sample or captured text.

type Stats 

type Stats struct {
	TablesRequested    int `json:"tables_requested"`
	TablesScanned      int `json:"tables_scanned"`
	ColumnsSeen        int `json:"columns_seen"`
	CandidateColumns   int `json:"candidate_columns"`
	SampledColumns     int `json:"sampled_columns"`
	SampledValuesCount int `json:"sampled_values_count"`
	FindingsCount      int `json:"findings_count"`
}

Stats contains only aggregate counters.

type TableRef 

type TableRef struct {
	Schema string `json:"schema"`
	Table  string `json:"table"`
}

TableRef is a metadata-confirmed table identity. It is deliberately not a SQL fragment.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.