integration

package
v1.0.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 14, 2026 License: MIT Imports: 11 Imported by: 0

README

ATLAS Compliance Integration Tests

Comprehensive test suite for validating ATLAS (MITRE ATT&CK for AI Systems) compliance patterns.

Overview

This test suite validates the AegisGate proxy's ability to detect and block 60+ ATLAS compliance patterns across 18 MITRE ATLAS techniques.

Files

File Description
atlas_compliance_test.go Main test file with 60+ ATLAS pattern tests
test_utils.go Test utilities, fixtures, and helpers
test_runner.go Test runner with configuration support
test_config.json Default test configuration
fixtures/atlas_fixtures.json Test data fixtures for ATLAS patterns
Makefile Make targets for running tests
run_tests.bat Windows test runner script
run_tests.sh Linux/Mac test runner script

Quick Start

Run All ATLAS Tests
# Using Go test directly
go test -v -run Atlas ./tests/integration/...

# Run the comprehensive test
go test -v -run TestAtlasComplianceComprehensive ./tests/integration/...

# Run specific technique tests
go test -v -run TestAtlasComplianceDirectPromptInjection ./tests/integration/...

ATLAS Techniques Covered

Technique Name Patterns
T1535 Direct Prompt Injection 5
T1484 LLM Jailbreak 5
T1632 System Prompt Extraction 5
T1589 Training Data Exposure 5
T1584 Indirect Prompt Injection 5
T1658 Adversarial Examples 5
T1648 Serverless Compute Exploitation 5
T1600 Vector Database Poisoning 3
T1613 Content Injection 3
T1563 Plugin Exploitation 3
T1622 Defense Evasion 3
T1606 Forge Web Credentials 2
T1621 MFA Request Generation 2
T1548 Abuse Elevation Control 2
T1490 Inhibit System Recovery 2
T1498 Network DoS 2
T1499 Endpoint DoS 2
T1602 Config Repository Exfiltration 2

Total: 18 techniques, 60+ patterns

Test Results

=== RUN   TestAtlasComplianceComprehensive
    Testing 18 ATLAS techniques
=== RUN   TestAtlasComplianceComprehensive/T1535
    Technique T1535 coverage verified
...
--- PASS: TestAtlasComplianceComprehensive (0.00s)
Test Categories
  • Technique Tests - Tests for each MITRE ATLAS technique
  • Benign Request Tests - Ensures legitimate requests are not blocked
  • Proxy Integration Tests - Tests the proxy's ATLAS compliance integration

Test Configuration

Edit test_config.json to customize test behavior:

{
  "atlas_enabled": true,
  "atlas_block_mode": true,
  "atlas_threshold": 0.75,
  "excluded_params": ["api_key", "token", "secret"],
  "timeout": 30,
  "parallel": false
}

Test Utilities

The test_utils.go provides:

  • TestLogger - Structured logging
  • TestFixture - Test data management
  • MockLLMServer - Configurable mock LLM server
  • CoverageTracker - Test coverage tracking
  • BenchmarkResult - Performance benchmarking
  • GetAtlasChecker() - Access to ATLAS compliance checker

Running Specific Tests

# Test specific technique
go test -v -run TestAtlasComplianceDirectPromptInjection ./tests/integration/...

# Test specific pattern
go test -v -run T1535.001 ./tests/integration/...

# Run benchmarks
go test -bench=. ./tests/integration/...

Test Development

Adding New Tests
  1. Add test cases to atlas_compliance_test.go
  2. Add fixtures to fixtures/atlas_fixtures.json
  3. Update the test count in coverage reports
Adding New Techniques
  1. Create new test function in atlas_compliance_test.go
  2. Add technique to TestAtlasComplianceComprehensive
  3. Update this README with new technique info

Expected Test Behavior

  • Tests use the compliance.NewAtlas() checker directly
  • Each test case validates detection of specific ATLAS patterns
  • Benign requests should NOT be blocked
  • Malicious patterns SHOULD be detected

Notes

  • Tests use httptest.Server for mock upstream servers
  • Tests utilize the existing compliance.AtlasManager for pattern detection
  • The test suite validates both detection capabilities and false positive prevention

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AssertAtlasAllowed 

func AssertAtlasAllowed(t TestingT, result AtlasTestResult, pattern string)

AssertAtlasAllowed verifies that a request was allowed through

func AssertAtlasBlocked 

func AssertAtlasBlocked(t TestingT, result AtlasTestResult, pattern string)

AssertAtlasBlocked verifies that an ATLAS pattern was blocked

func AssertScoreAboveThreshold 

func AssertScoreAboveThreshold(t TestingT, result AtlasTestResult, threshold float64)

AssertScoreAboveThreshold verifies that detection score is above threshold

func AssertTechniqueDetected 

func AssertTechniqueDetected(t TestingT, result AtlasTestResult, technique string)

AssertTechniqueDetected verifies that a technique was detected

func CreateBlockingMockLLMServer 

func CreateBlockingMockLLMServer() *httptest.Server

CreateBlockingMockLLMServer creates a mock server that blocks ATLAS patterns

func CreateMockLLMServer 

func CreateMockLLMServer(responses map[string]MockLLMResponse) *httptest.Server

CreateMockLLMServer creates a configurable mock LLM server for testing

func GetAtlasChecker 

func GetAtlasChecker() *compliance.ATLASFramework

GetAtlasChecker returns the ATLAS compliance checker

func GetTestPaths 

func GetTestPaths() (string, string, string)

GetTestPaths returns the paths to test resources

func PrintBenchmarkResults 

func PrintBenchmarkResults(results []BenchmarkResult)

PrintBenchmarkResults prints benchmark results

func PrintTestSummary 

func PrintTestSummary(tracker *CoverageTracker)

PrintTestSummary prints a summary of test results

func RunWithFixtures 

func RunWithFixtures(t *testing.T, fixtures []TestFixture)

RunWithFixtures runs ATLAS tests using fixtures

func SaveTestConfig 

func SaveTestConfig(config TestConfig, path string) error

SaveTestConfig saves test configuration to file

func SaveTestFixtures 

func SaveTestFixtures(fixtures []TestFixture, path string) error

SaveTestFixtures saves test fixtures to a JSON file

func TestMainWithConfig 

func TestMainWithConfig(m *testing.M)

TestMainWithConfig runs the test suite with configuration

func ValidateTestEnvironment 

func ValidateTestEnvironment() error

ValidateTestEnvironment validates the test environment

Types

type ATLASParams 

type ATLASParams struct {
	Technique string `json:"technique"`
	PatternID string `json:"pattern_id"`
	Severity  string `json:"severity"`
	BlockMode bool   `json:"block_mode"`
}

ATLASParams holds ATLAS-specific test parameters

type AtlasTestResult 

type AtlasTestResult struct {
	Blocked   bool
	Detected  bool
	Technique string
	Pattern   string
	Score     float64
}

AtlasTestResult represents the result of an ATLAS pattern test

type BenchmarkResult 

type BenchmarkResult struct {
	Name          string
	Iterations    int
	AvgDuration   time.Duration
	MinDuration   time.Duration
	MaxDuration   time.Duration
	TotalDuration time.Duration
	BytesAlloc    int64
	Allocs        int64
}

BenchmarkResult holds benchmark results

func RunBenchmark 

func RunBenchmark(name string, fn func(), iterations int) BenchmarkResult

RunBenchmark runs a benchmark test

type CoverageReport 

type CoverageReport struct {
	TechniquesCovered int      `json:"techniques_covered"`
	TechniquesTotal   int      `json:"techniques_total"`
	PatternsCovered   int      `json:"patterns_covered"`
	PatternsTotal     int      `json:"patterns_total"`
	Techniques        []string `json:"techniques"`
}

CoverageReport represents test coverage information

type CoverageTracker 

type CoverageTracker struct {
	Techniques  map[string]bool
	Patterns    map[string]bool
	TotalTests  int
	PassedTests int
	FailedTests int
}

CoverageTracker tracks test coverage

func NewCoverageTracker 

func NewCoverageTracker() *CoverageTracker

NewCoverageTracker creates a new coverage tracker

func (*CoverageTracker) GetCoverage 

func (c *CoverageTracker) GetCoverage() (techniqueCov, patternCov float64, total int)

GetCoverage returns coverage statistics

func (*CoverageTracker) PrintCoverage 

func (c *CoverageTracker) PrintCoverage()

PrintCoverage prints coverage report

func (*CoverageTracker) RecordTest 

func (c *CoverageTracker) RecordTest(technique, pattern string, passed bool)

RecordTest records a test result

type MockLLMResponse 

type MockLLMResponse struct {
	StatusCode int
	Body       string
	Headers    map[string]string
	Delay      time.Duration
}

MockLLMResponse defines a mock LLM response configuration

type TestConfig 

type TestConfig struct {
	// ATLAS Configuration
	ATLASEnabled   bool     `json:"atlas_enabled"`
	ATLASBlockMode bool     `json:"atlas_block_mode"`
	ATLASThreshold float64  `json:"atlas_threshold"`
	ExcludedParams []string `json:"excluded_params"`

	// Server Configuration
	UpstreamURL string        `json:"upstream_url"`
	Timeout     time.Duration `json:"timeout"`
	MaxRetries  int           `json:"max_retries"`

	// Test Configuration
	Parallel  bool          `json:"parallel"`
	Verbose   bool          `json:"verbose"`
	Seed      int64         `json:"seed"`
	TimeoutDS time.Duration `json:"test_timeout"`
}

TestConfig holds test configuration

func DefaultTestConfig 

func DefaultTestConfig() TestConfig

DefaultTestConfig returns default test configuration

func LoadConfig 

func LoadConfig(path string) (TestConfig, error)

LoadConfig loads the test configuration

func LoadTestConfig 

func LoadTestConfig(path string) (TestConfig, error)

LoadTestConfig loads test configuration from file

type TestFixture 

type TestFixture struct {
	Name        string                 `json:"name"`
	Category    string                 `json:"category"`
	Input       map[string]string      `json:"input"`
	Expected    map[string]interface{} `json:"expected"`
	ATLASParams ATLASParams            `json:"atlas_params"`
}

TestFixture represents a test fixture with expected results

func LoadFixtures 

func LoadFixtures(path string) ([]TestFixture, error)

LoadFixtures loads the test fixtures

func LoadTestFixtures 

func LoadTestFixtures(path string) ([]TestFixture, error)

LoadTestFixtures loads test fixtures from a JSON file

type TestLogger 

type TestLogger struct {
	Buffer strings.Builder
}

TestLogger provides structured logging for tests

func NewTestLogger 

func NewTestLogger() *TestLogger

NewTestLogger creates a new test logger

func (*TestLogger) GetLogs 

func (l *TestLogger) GetLogs() string

GetLogs returns the accumulated logs

func (*TestLogger) Log 

func (l *TestLogger) Log(format string, args ...interface{})

Log logs a message with timestamp

type TestReport 

type TestReport struct {
	Timestamp    time.Time      `json:"timestamp"`
	Duration     time.Duration  `json:"duration"`
	TotalTests   int            `json:"total_tests"`
	PassedTests  int            `json:"passed_tests"`
	FailedTests  int            `json:"failed_tests"`
	SkippedTests int            `json:"skipped_tests"`
	Coverage     CoverageReport `json:"coverage"`
	Results      []TestResult   `json:"results"`
}

TestReport represents the test execution report

type TestResult 

type TestResult struct {
	Name       string        `json:"name"`
	Category   string        `json:"category"`
	Status     string        `json:"status"`
	Duration   time.Duration `json:"duration"`
	Techniques []string      `json:"techniques"`
	Error      string        `json:"error,omitempty"`
}

TestResult represents an individual test result

type TestRunnerConfig 

type TestRunnerConfig struct {
	ConfigPath     string
	FixturesPath   string
	OutputPath     string
	Parallel       bool
	Verbose        bool
	GenerateReport bool
	Filter         string
}

TestRunnerConfig holds the test runner configuration

type TestingT 

type TestingT interface {
	Errorf(format string, args ...interface{})
}

TestingT is an interface for test helpers

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.