testspecs

package
v0.0.0-...-c2011ee Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 30, 2026 License: Apache-2.0 Imports: 33 Imported by: 0

Documentation

Overview

Package testspecs provides shared testing infrastructure for CrossCodex BDD tests.

This package contains reusable behavior specifications, common test fixtures, standardized setup helpers, and custom Gomega matchers that eliminate duplication across package test suites.

Key components: - shared_behaviors.go: Reusable BDD specs for common patterns - interfaces.go: Contracts for behavior composition - fixtures.go: Standard test data and contexts - helpers.go: Setup/teardown utilities - matchers.go: Custom Gomega matchers for CrossCodex-specific patterns - graphdb_contract.go: GraphDBContractBehavior, the contract specs every graphdb.GraphDB driver runs

Architecture:

The testspecs package enables a three-layer BDD testing approach:

  1. Behavioral specs - High-level behavior specifications that can be shared across multiple components (e.g., "behaves like a tenant-isolated component")

  2. Technical tests - Package-specific implementation tests that verify concrete behavior of individual components

  3. Integration specs - End-to-end tests that verify component interactions and system behavior under realistic conditions

Interface Contracts:

Components that implement the provided interfaces can automatically inherit shared behavior specifications, ensuring consistent behavior patterns across the codebase.

For example, any component implementing TenantIsolatedComponent can use the shared "behaves like a tenant-isolated component" spec, which tests tenant context propagation, access validation, and isolation boundaries.

Usage: 

import . "github.com/complytime-labs/crosscodex/internal/testspecs"

var _ = Describe("MyComponent", func() {
  var component TenantIsolatedComponent

  BeforeEach(func() {
    component = NewMyComponent()
  })

  BehavesLikeATenantIsolatedComponent(&component)

  Describe("component-specific behavior", func() {
    It("should validate tenant IDs", func() {
      Expect("acme-corp").To(BeValidTenantID())
    })

    It("should have secure error handling", func() {
      err := component.DoSomething()
      Expect(err).To(BeSecureError())
      Expect(err).To(BeActionableError())
    })
  })
})

Custom Matchers:

The package provides custom Gomega matchers for common CrossCodex patterns: - BeValidTenantID(): Validates tenant ID format (3-52 chars, alphanumeric + hyphens) - HaveValidTenantPrefix(): Validates strings with tenant prefixes (tenant-id/path) - BeSecureError(): Ensures errors don't leak sensitive information - BeActionableError(): Ensures errors provide actionable guidance to users - HaveValidConfigStructure(): Validates required configuration sections - MatchConfigPrecedence(): Validates XDG-compliant configuration precedence - HaveIsolatedTenantData(): Validates proper tenant data isolation - BeSanitizedOutput(): Validates output is sanitized of sensitive information - ContainValidTimestamp(field): Validates RFC3339 timestamps in data structures - BeValidDatabaseConnection(): Validates active database connections - BeWithinTolerance(expected, tolerance): Validates numeric values within tolerance

Index

Constants

This section is empty.

Variables

View Source 
var StandardConfigPaths = map[string]ConfigPathFixture{
	"xdg-precedence": {
		Paths: []string{
			"$XDG_CONFIG_HOME/crosscodex/config.yml",
			"$HOME/.config/crosscodex/config.yml",
			"$HOME/.crosscodex.yml",
		},
		Priority:    "user-config",
		Description: "XDG Base Directory specification paths with user precedence",
	},
	"system-paths": {
		Paths: []string{
			"/etc/crosscodex/config.yml",
			"/usr/local/etc/crosscodex/config.yml",
		},
		Priority:    "system-wide",
		Description: "System-wide configuration paths",
	},
	"local-files": {
		Paths: []string{
			"./crosscodex.yml",
			"./config/crosscodex.yml",
			"./.crosscodex.yml",
		},
		Priority:    "local",
		Description: "Local project configuration files",
	},
}

StandardConfigPaths provides common configuration path scenarios

View Source 
var StandardErrors = map[string]ErrorCondition{
	"network-timeout": {
		Message:   "connection timeout after 30s",
		Category:  "network",
		Retryable: true,
		Code:      0,
	},
	"permission-denied": {
		Message:   "access denied: insufficient permissions",
		Category:  "authorization",
		Retryable: false,
		Code:      403,
	},
	"invalid-input": {
		Message:   "invalid input: field 'email' is required",
		Category:  "validation",
		Retryable: false,
		Code:      400,
	},
	"not-found": {
		Message:   "resource not found",
		Category:  "client",
		Retryable: false,
		Code:      404,
	},
}

StandardErrors provides common error scenarios for testing

View Source 
var StandardTenantContexts = map[string]TenantFixture{
	"valid-tenant": {
		TenantID:    "acme-corp",
		DisplayName: "Valid tenant with standard format",
		Valid:       true,
		ErrorType:   "",
	},
	"min-length": {
		TenantID:    "abc",
		DisplayName: "Minimum length valid tenant",
		Valid:       true,
		ErrorType:   "",
	},
	"max-length": {
		TenantID:    "a123456789b123456789c123456789d123456789e123456789f1",
		DisplayName: "Maximum length valid tenant",
		Valid:       true,
		ErrorType:   "",
	},
	"invalid-chars": {
		TenantID:    "bad-tenant!@#",
		DisplayName: "Invalid tenant with special characters",
		Valid:       false,
		ErrorType:   "invalid_format",
	},
	"too-short": {
		TenantID:    "ab",
		DisplayName: "Tenant ID shorter than minimum length",
		Valid:       false,
		ErrorType:   "too_short",
	},
	"too-long": {
		TenantID:    "a123456789b123456789c123456789d123456789e123456789f12",
		DisplayName: "Tenant ID longer than maximum length",
		Valid:       false,
		ErrorType:   "too_long",
	},
	"empty": {
		TenantID:    "",
		DisplayName: "Empty tenant ID",
		Valid:       false,
		ErrorType:   "empty",
	},
}

StandardTenantContexts provides common tenant test cases for all packages

Functions

func AssertErrorContains 

func AssertErrorContains(err error, expectedText string)

AssertErrorContains asserts that the error contains the expected text

func AssertNoError 

func AssertNoError(err error)

AssertNoError asserts that the error is nil using proper offset

func BeActionableError 

func BeActionableError() types.GomegaMatcher

BeActionableError returns a matcher that checks if an error provides actionable information

func BeSanitizedOutput 

func BeSanitizedOutput() types.GomegaMatcher

BeSanitizedOutput returns a matcher that checks if output has been sanitized of sensitive information

func BeSecureError 

func BeSecureError() types.GomegaMatcher

BeSecureError returns a matcher that checks if an error doesn't leak sensitive information

func BeValidDatabaseConnection 

func BeValidDatabaseConnection() types.GomegaMatcher

BeValidDatabaseConnection returns a matcher that checks if a value is a valid, active database connection

func BeValidTenantID 

func BeValidTenantID() types.GomegaMatcher

BeValidTenantID returns a matcher that checks if a string is a valid tenant ID

func BeWithinTolerance 

func BeWithinTolerance(expected, tolerance float64) types.GomegaMatcher

BeWithinTolerance returns a matcher that checks if a number is within a specified tolerance of an expected value

func ConfigurationComplianceBehavior 

func ConfigurationComplianceBehavior(subject ConfigurableComponent) func()

ConfigurationComplianceBehavior tests configuration loading compliance.

This shared behavior specification validates that components implementing ConfigurableComponent properly: - Follow XDG compliance precedence rules - Validate configuration schema - Reject invalid configuration with actionable errors

Usage: 

ginkgo.Describe("as a configurable component", ConfigurationComplianceBehavior(provider))

func ContainValidTimestamp 

func ContainValidTimestamp(fieldName string) types.GomegaMatcher

ContainValidTimestamp returns a matcher that checks if a structure contains a valid RFC3339 timestamp

func CreateCancelableContext 

func CreateCancelableContext() (context.Context, context.CancelFunc)

CreateCancelableContext creates a cancelable context

func CreateTimeoutContext 

func CreateTimeoutContext(timeout time.Duration) (context.Context, context.CancelFunc)

CreateTimeoutContext creates a context with the specified timeout

func ErrorHandlingBehavior 

func ErrorHandlingBehavior(subject ErrorHandlingComponent) func()

ErrorHandlingBehavior tests error resilience patterns.

This shared behavior specification validates that components implementing ErrorHandlingComponent properly: - Handle different error types appropriately - Categorize retryable vs non-retryable errors - Track error history for debugging

Usage: 

ginkgo.Describe("as an error-handling component", ErrorHandlingBehavior(provider))

func GinkgoLogger 

func GinkgoLogger() *slog.Logger

GinkgoLogger returns an *slog.Logger that writes to ginkgo.GinkgoWriter. Use this when injecting a logger into a component under test so its output is captured by Ginkgo and only shown on failure.

func GraphDBContractBehavior 

func GraphDBContractBehavior(setup GraphDBSetup) func()

GraphDBContractBehavior is the executable form of the contract documented on graphdb.GraphDB. Every driver runs it: memdriver in its unit suite, agedriver in its integration suite.

Three clauses are tested per driver instead of here:

  • ExecuteQuery's query-execution clauses (whole-token $name substitution; ErrNotSupported from drivers without an openCypher engine), because the in-memory driver cannot execute Cypher.
  • Concurrent first-time CreateGraph, because agedriver's restricted graph_user cannot create a graph (the tenant trigger does), so the shared specs run against a provisioned tenant.
  • The "ambiguous node ID" error for graphs written before #148, because the contract forbids building such a graph through the API; agedriver seeds one with raw Cypher.

A new driver must add its own specs for these three.

Usage: 

ginkgo.Describe("GraphDB contract", testspecs.GraphDBContractBehavior(setup))

func HaveIsolatedTenantData 

func HaveIsolatedTenantData() types.GomegaMatcher

HaveIsolatedTenantData returns a matcher that checks if data structures have proper tenant isolation

func HaveValidConfigStructure 

func HaveValidConfigStructure() types.GomegaMatcher

HaveValidConfigStructure returns a matcher that checks if a configuration has required sections

func HaveValidTenantPrefix 

func HaveValidTenantPrefix() types.GomegaMatcher

HaveValidTenantPrefix returns a matcher that checks if a string has a valid tenant prefix

func IgnoreOutput 

func IgnoreOutput() (restore func())

IgnoreOutput suppresses output to stdout/stderr and returns a restore function

func IsolateTelemetryGlobals 

func IsolateTelemetryGlobals() (restore func())

IsolateTelemetryGlobals snapshots the process-wide state that telemetry.Init replaces (otel tracer, meter and propagator, slog.Default, and the log package's output and flags), then installs an empty propagator so specs can assert whether Init ran. Returns a restore function for DeferCleanup.

The tracer and meter providers are restored only if they changed: otel logs an error when its default delegating provider is set to itself. The propagator is restored unconditionally; it was always replaced, and composite propagators are slices, so comparing them panics.

The log package's output and flags are restored after slog.Default because slog.SetDefault with any non-built-in handler (Init's trace handler) points log's output at that handler, and restoring slog's built-in default does not undo it. Left in place, the built-in handler would write through log into the wrapper and back into itself, deadlocking on log's mutex.

func LogTestProgress 

func LogTestProgress(message string, args ...interface{})

LogTestProgress logs test progress messages using Ginkgo's logging

func MatchConfigPrecedence 

func MatchConfigPrecedence() types.GomegaMatcher

MatchConfigPrecedence returns a matcher that checks if configuration follows XDG precedence rules

func RedirectLogsToGinkgo 

func RedirectLogsToGinkgo() (restore func())

RedirectLogsToGinkgo sets slog's default logger to write to ginkgo.GinkgoWriter. Ginkgo captures this output and only displays it when a spec fails, keeping the normal test output clean. Returns a restore function that reinstates the previous default logger.

Call in BeforeSuite or BeforeEach: 

var restoreLogs func()
BeforeSuite(func() { restoreLogs = testspecs.RedirectLogsToGinkgo() })
AfterSuite(func() { restoreLogs() })

func SecurityBoundaryBehavior 

func SecurityBoundaryBehavior(subject SecureComponent) func()

SecurityBoundaryBehavior tests security patterns and boundaries.

This shared behavior specification validates that components implementing SecureComponent properly: - Validate input at system boundaries using StandardErrorConditions - Sanitize output for external consumption - Operate in secure mode when required

Usage: 

ginkgo.Describe("as a secure component", SecurityBoundaryBehavior(provider))

func SetupTenantContext 

func SetupTenantContext(tenantID string) context.Context

SetupTenantContext creates a context with the specified tenant ID. Panics if the tenant ID is invalid — test helpers must use valid IDs.

func TenantIsolationBehavior 

func TenantIsolationBehavior(subject TenantIsolatedComponent) func()

TenantIsolationBehavior tests tenant boundary enforcement.

This shared behavior specification validates that components implementing TenantIsolatedComponent properly: - Prevent cross-tenant data access - Validate tenant ID format using StandardTenantContexts - Reject invalid tenant formats with secure errors

Usage: 

ginkgo.Describe("as a tenant-isolated component", TenantIsolationBehavior(provider))

func WaitForCondition 

func WaitForCondition(condition func() bool, timeout time.Duration, description string)

WaitForCondition waits for a condition to become true with timeout

Types

type ConfigPathFixture 

type ConfigPathFixture struct {
	Paths       []string // List of configuration file paths
	Priority    string   // Priority level: user-config, system-wide, local
	Description string   // Human-readable description
}

ConfigPathFixture represents test configuration paths with precedence metadata

type ConfigurableComponent 

type ConfigurableComponent interface {
	// LoadConfiguration loads configuration from the specified path.
	// The path should follow XDG conventions and support precedence resolution.
	// Returns an error if the configuration file is invalid or inaccessible.
	LoadConfiguration(path string) error

	// ValidateConfiguration validates the loaded configuration for completeness
	// and correctness. Returns an error with actionable feedback if validation fails.
	ValidateConfiguration() error

	// GetConfigValue retrieves a configuration value by key, applying precedence
	// rules (environment variables > config file > defaults).
	// Returns an error if the key is not found or the value is invalid.
	GetConfigValue(key string) (interface{}, error)
}

ConfigurableComponent defines contract for components that load and validate configuration.

Components implementing this interface must ensure that: - Configuration loading follows XDG Base Directory specification - Configuration validation is comprehensive and actionable - Configuration precedence is handled correctly (env vars > config file > defaults) - Invalid configuration produces clear error messages

This interface enables shared behavior specifications for configuration testing, ensuring consistent configuration handling across all components.

type DBPinger 

type DBPinger interface {
	Ping() error
}

DBPinger represents any type that can ping a database

type DeterministicLLM 

type DeterministicLLM struct {
	EmbeddingDim int
}

DeterministicLLM is a hermetic llmclient.Client for end-to-end tests. It lives in internal/testspecs, not pkg/, because its canned replies encode the internal analyzers' reply formats; deterministic_llm_test.go runs each reply through the real parsers so format drift fails there first. Embed returns EmbeddingDim-dimension vectors deterministically seeded from each input's fnv hash: identical text always yields the identical vector, so two controls with identical prepared text embed to cosine similarity 1.0 -> a stable candidate pair. Complete returns one canned, parseable response per analyzer, routed by CompletionRequest.PromptName, the prompt name the worker forwards from the task payload.

It is injected at the Go level rather than mocked over HTTP because PromptName is `json:"-"` (pkg/llmclient/types.go): it is never serialized in the outgoing HTTP request, so an httptest server would have to content-sniff raw prompt text to know which analyzer is calling.

EmbeddingDim must match the vector dimension of the store the pipeline writes to. Health and Close return nil.

func (DeterministicLLM) Close 

func (DeterministicLLM) Close() error

Close returns nil.

func (DeterministicLLM) Complete 

func (DeterministicLLM) Complete(_ context.Context, req *llmclient.CompletionRequest) (*llmclient.CompletionResponse, error)

Complete returns the canned response for req.PromptName, or an error for a prompt name no analyzer uses.

func (DeterministicLLM) Embed 

func (d DeterministicLLM) Embed(_ context.Context, req *llmclient.EmbeddingRequest) (*llmclient.EmbeddingResponse, error)

Embed returns one EmbeddingDim-dimension vector per input, a pure function of the input text. It errors when EmbeddingDim is not positive, which would otherwise yield empty vectors that every store rejects with a confusing dimension error.

func (DeterministicLLM) Health 

func (DeterministicLLM) Health(context.Context) error

Health returns nil.

type ErrorCondition 

type ErrorCondition struct {
	Message    string // Error message text
	Category   string // Error category: network, authorization, validation, client, server
	Retryable  bool   // Whether this error should be retried
	Code       int    // Optional error code
	Underlying error  // Optional underlying error for wrapping
}

ErrorCondition represents a test error scenario with categorization

type ErrorHandlingComponent 

type ErrorHandlingComponent interface {
	// HandleError processes the provided error according to the component's
	// error handling policy. May perform logging, metric collection, or
	// error transformation. Returns the processed error.
	HandleError(err error) error

	// IsRetryableError determines if the provided error represents a condition
	// that can be retried. Returns true for transient errors (network timeouts,
	// temporary resource unavailability), false for permanent errors.
	IsRetryableError(err error) bool

	// GetLastError returns the most recent error encountered by the component.
	// Used for debugging and error state inspection. Returns nil if no error.
	GetLastError() error
}

ErrorHandlingComponent defines contract for components that implement robust error handling.

Components implementing this interface must ensure that: - Errors are handled defensively with proper context preservation - Retryable errors are identified correctly - Error state is tracked for debugging and audit purposes - Error recovery follows established patterns

This interface enables shared behavior specifications for error handling testing, ensuring consistent error management across all components.

type GraphDBSetup 

type GraphDBSetup func() (db graphdb.GraphDB, tenant, other string)

GraphDBSetup returns a client and two tenants, tenant and other, whose graphs already exist on that same client. Both graphs must be fresh, empty, and private to the calling spec: the contract specs assert exact edge sets and emptiness, so a graph shared across specs, or one carrying leftover data, produces false failures. other exists only so the "tenant isolation" spec can prove tenant's data is invisible from it; no other spec writes to or reads from other. It runs inside a BeforeEach, so it should register teardown with ginkgo.DeferCleanup.

type SecureComponent 

type SecureComponent interface {
	// ValidateInput validates the provided input against security policies,
	// checking for injection attacks, malformed data, and size limits.
	// Returns an error if the input is invalid or potentially malicious.
	ValidateInput(input interface{}) error

	// SanitizeOutput sanitizes the provided output to prevent information
	// leakage, removing sensitive data and applying output encoding.
	// Returns the sanitized version of the output.
	SanitizeOutput(output interface{}) interface{}

	// IsSecureModeEnabled returns true if the component is operating in
	// secure mode (e.g., FIPS mode, enhanced validation, stricter policies).
	IsSecureModeEnabled() bool
}

SecureComponent defines contract for components that handle security boundaries.

Components implementing this interface must ensure that: - All inputs are validated against injection attacks and malformed data - Outputs are properly sanitized to prevent information leakage - Security mode enforcement is consistent - FIPS compliance is maintained when enabled

This interface enables shared behavior specifications for security testing, ensuring consistent security boundaries across all components.

type TenantFixture 

type TenantFixture struct {
	TenantID    string // The tenant identifier to test
	DisplayName string // Human-readable name for the test case
	Valid       bool   // Whether this tenant ID should pass validation
	ErrorType   string // Expected error type for invalid tenants
}

TenantFixture represents a test tenant with validation metadata

type TenantIsolatedComponent 

type TenantIsolatedComponent interface {
	// WithTenantContext returns a new instance of the component configured
	// for the tenant specified in the context. The returned component should
	// only operate on data belonging to that tenant.
	WithTenantContext(ctx context.Context) TenantIsolatedComponent

	// ValidateTenantAccess verifies that the given tenant ID is valid and
	// accessible by the current component. Returns an error if the tenant
	// ID is invalid, inaccessible, or violates isolation policies.
	ValidateTenantAccess(tenantID string) error

	// GetTenantID extracts and returns the tenant ID from the given context.
	// Returns an error if no tenant context is available or if the tenant
	// ID is invalid.
	GetTenantID(ctx context.Context) (string, error)
}

TenantIsolatedComponent defines contract for components that enforce tenant isolation.

Components implementing this interface must ensure that: - All operations are scoped to a specific tenant - Tenant context is properly propagated through the component - Cross-tenant data access is prevented - Tenant ID validation follows CrossCodex standards

This interface enables shared behavior specifications for tenant isolation testing, ensuring consistent multi-tenant boundaries across all components.

type TestCleanup 

type TestCleanup func()

TestCleanup represents a cleanup function that should be called after a test

func SetupTestConfig 

func SetupTestConfig(configData []byte) (string, TestCleanup)

SetupTestConfig creates a temporary config file and returns its path with cleanup

func SetupTestDatabase 

func SetupTestDatabase() (*sql.DB, TestCleanup)

SetupTestDatabase creates a test database connection with proper cleanup

func SetupTestNATS 

func SetupTestNATS() (natsbus.Client, TestCleanup)

SetupTestNATS creates a test NATS client with proper cleanup Returns the client itself for testing, not the underlying connection

func SetupTestStorage 

func SetupTestStorage() (storage.Provider, TestCleanup)

SetupTestStorage creates a test storage provider with proper cleanup

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.