reporting

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 20, 2025 License: MIT Imports: 14 Imported by: 0

README

Comprehensive Reporting System for LLM Test Results

This package provides a flexible reporting system that generates detailed test results in multiple formats with support for pass/fail status, severity levels, detailed logs, and mapping to compliance frameworks like OWASP Top 10 for LLMs and ISO/IEC 42001.

Features

  • Multiple Output Formats: Generate reports in various formats including:

    • JSON
    • JSONL (JSON Lines)
    • CSV
    • Excel
    • Plain Text
    • Markdown
    • PDF
    • HTML
  • Comprehensive Test Results: Capture detailed information about tests:

    • Pass/Fail status
    • Severity levels (Critical, High, Medium, Low, Info)
    • Detailed logs
    • Input/Output data
    • Timing information
    • Tags and metadata
  • Compliance Framework Mapping: Map test results to industry standards:

    • OWASP Top 10 for LLMs
    • ISO/IEC 42001
    • Custom frameworks
  • Filtering and Customization: Customize reports with:

    • Filtering by severity level
    • Including/excluding tests by status
    • Filtering by tags
    • Custom templates for HTML and PDF outputs
  • Integration with Template Management: Seamlessly integrates with the existing template management system.

Usage

Command Line Interface

The reporting system can be used via the command line interface:

# Generate a report from test results
LLMrecon report generate --results results.json --format html --output report.html

# Generate a report from multiple test suites
LLMrecon report batch --suites suite1.json,suite2.json --format html --output report.html

# Convert a report from one format to another
LLMrecon report convert --input report.json --format html --output report.html
API Usage

The reporting system can also be used programmatically:

import (
    "context"
    "github.com/perplext/LLMrecon/src/reporting"
)

func main() {
    // Create a report generator
    factory := reporting.NewFormatterFactory()
    generator, _ := factory.CreateDefaultReportGenerator()

    // Register compliance providers
    generator.RegisterComplianceProvider(reporting.NewOWASPComplianceProvider())
    generator.RegisterComplianceProvider(reporting.NewISOComplianceProvider())

    // Create a test suite
    suite := &reporting.TestSuite{
        ID:          "example-suite",
        Name:        "Example Test Suite",
        Description: "An example test suite",
        Results:     []*reporting.TestResult{
            // Add test results here
        },
    }

    // Create report options
    options := &reporting.ReportOptions{
        Format:             reporting.HTMLFormat,
        Title:              "LLM Test Report",
        IncludePassedTests: true,
        MinimumSeverity:    reporting.InfoSeverity,
        OutputPath:         "report.html",
    }

    // Generate report
    ctx := context.Background()
    report, _ := generator.GenerateReport(ctx, []*reporting.TestSuite{suite}, options)

    // Get formatter for the specified format
    formatter, _ := generator.GetFormatter(options.Format)

    // Format report
    data, _ := formatter.Format(ctx, report, options)
}
Integration with Template Management

The reporting system integrates with the template management system:

import (
    "context"
    "github.com/perplext/LLMrecon/src/reporting"
    "github.com/perplext/LLMrecon/src/template/management"
)

func main() {
    // Get template results
    var templateResults []*management.TemplateResult
    
    // Create factory and report generator
    factory := reporting.NewFormatterFactory()
    generator, _ := factory.CreateDefaultReportGenerator()

    // Register compliance providers
    generator.RegisterComplianceProvider(reporting.NewOWASPComplianceProvider())
    generator.RegisterComplianceProvider(reporting.NewISOComplianceProvider())

    // Create converter
    converter := reporting.NewTemplateResultConverter([]reporting.ComplianceMappingProvider{
        reporting.NewOWASPComplianceProvider(),
        reporting.NewISOComplianceProvider(),
    })

    // Create reporting service
    service := reporting.NewTemplateReportingService(converter, generator)

    // Create report options
    options := &reporting.ReportOptions{
        Format:             reporting.HTMLFormat,
        Title:              "LLM Test Report",
        OutputPath:         "report.html",
    }

    // Generate report
    ctx := context.Background()
    data, _ := service.GenerateReport(ctx, templateResults, options)
}

Customization

Custom Templates

You can provide custom templates for HTML and PDF reports:

LLMrecon report generate --results results.json --format html --template custom_template.html --output report.html
Custom Compliance Frameworks

You can create custom compliance frameworks:

// Create custom compliance provider
customProvider := reporting.NewCustomComplianceProvider(map[string]reporting.ComplianceMapping{
    "CUSTOM01": {
        Framework:   reporting.CustomFramework,
        ID:          "CUSTOM01",
        Name:        "Custom Requirement 1",
        Description: "Description of custom requirement 1",
    },
})

// Register custom provider
generator.RegisterComplianceProvider(customProvider)

Example Reports

Example reports can be found in the examples/reporting directory:

  • test_suite.json: Example test suite with various test results

Architecture

The reporting system is built with a modular architecture:

  • Core Types: Defines the data model for test results, suites, and reports
  • Report Generator: Generates reports from test results
  • Formatters: Format reports in various output formats
  • Compliance Providers: Map test results to compliance frameworks
  • Integration Layer: Connects with the template management system

Contributing

To add a new output format:

  1. Create a new formatter in the formats package
  2. Implement the ReportFormatter interface
  3. Register the formatter with the FormatterFactory

To add a new compliance framework:

  1. Create a new compliance provider
  2. Implement the ComplianceMappingProvider interface
  3. Register the provider with the ReportGenerator

Documentation

Overview

Package reporting provides a comprehensive reporting system for LLM test results.

Index

Constants

View Source
const (
	TextFormat     = common.TextFormat
	MarkdownFormat = common.MarkdownFormat
	JSONFormat     = common.JSONFormat
	JSONLFormat    = common.JSONLFormat
	CSVFormat      = common.CSVFormat
	ExcelFormat    = common.ExcelFormat
	PDFFormat      = common.PDFFormat
	HTMLFormat     = common.HTMLFormat
)

Supported report formats

View Source
const (
	CriticalSeverity = common.Critical
	HighSeverity     = common.High
	MediumSeverity   = common.Medium
	LowSeverity      = common.Low
	InfoSeverity     = common.Info
)

Supported severity levels

Variables

View Source
var SeverityLevelMapping = map[string]common.SeverityLevel{
	"critical": common.Critical,
	"high":     common.High,
	"medium":   common.Medium,
	"low":      common.Low,
	"info":     common.Info,
}

SeverityLevelMapping maps string representations to SeverityLevel constants

Functions

func CreateFormatter

func CreateFormatter(format common.ReportFormat, options map[string]interface{}) (common.ReportFormatter, error)

CreateFormatter creates a formatter for the specified format

Types

type BatchReportingService

type BatchReportingService struct {
	// contains filtered or unexported fields
}

BatchReportingService provides reporting services for multiple test suites

func NewBatchReportingService

func NewBatchReportingService(generator common.ReportGenerator) *BatchReportingService

NewBatchReportingService creates a new batch reporting service

func (*BatchReportingService) GenerateReport

func (s *BatchReportingService) GenerateReport(ctx context.Context, suites []*TestSuite, options *ReportOptions) ([]byte, error)

GenerateReport generates a report from multiple test suites

type CSVRenderer

type CSVRenderer struct{}

CSVRenderer renders reports as CSV

func (*CSVRenderer) GetContentType

func (r *CSVRenderer) GetContentType() string

GetContentType returns the content type for CSV

func (*CSVRenderer) GetFileExtension

func (r *CSVRenderer) GetFileExtension() string

GetFileExtension returns the file extension for CSV

func (*CSVRenderer) Render

func (r *CSVRenderer) Render(report *SecurityReport, options RenderOptions) ([]byte, error)

Render renders a report as CSV

type CategoryCount

type CategoryCount struct {
	Category string `json:"category"`
	Count    int    `json:"count"`
}

type ComplianceFramework

type ComplianceFramework string

ComplianceFramework represents a compliance framework

const (
	OWASPFramework  ComplianceFramework = "owasp-top-10-llm"
	ISOFramework    ComplianceFramework = "iso-iec-42001"
	NISTFramework   ComplianceFramework = "nist-ai-risk"
	CustomFramework ComplianceFramework = "custom"
)

Supported compliance frameworks

type ComplianceMapping

type ComplianceMapping struct {
	// Framework is the compliance framework
	Framework ComplianceFramework `json:"framework"`
	// ID is the ID of the item within the framework
	ID string `json:"id"`
	// Name is the name of the item within the framework
	Name string `json:"name"`
	// Description is the description of the item within the framework
	Description string `json:"description,omitempty"`
	// URL is the URL to the item within the framework
	URL string `json:"url,omitempty"`
}

ComplianceMapping represents a mapping to a compliance framework

type ComplianceMappingProvider

type ComplianceMappingProvider interface {
	// GetMappings returns compliance mappings for a test result
	GetMappings(ctx context.Context, testResult *TestResult) ([]ComplianceMapping, error)
	// GetFrameworks returns a list of supported compliance frameworks
	GetFrameworks() []ComplianceFramework
}

ComplianceMappingProvider is the interface for compliance mapping providers

type ComplianceStatus

type ComplianceStatus struct {
	Compliant bool     `json:"compliant"`
	Coverage  float64  `json:"coverage"`
	Gaps      []string `json:"gaps"`
}

type CustomComplianceProvider

type CustomComplianceProvider struct {
	// contains filtered or unexported fields
}

CustomComplianceProvider provides custom compliance mappings

func NewCustomComplianceProvider

func NewCustomComplianceProvider(mappings map[string]ComplianceMapping) *CustomComplianceProvider

NewCustomComplianceProvider creates a new custom compliance provider

func (*CustomComplianceProvider) AddMapping

func (p *CustomComplianceProvider) AddMapping(id string, name string, description string, url string)

AddMapping adds a mapping to the provider

func (*CustomComplianceProvider) GetAllMappings

func (p *CustomComplianceProvider) GetAllMappings() map[string]ComplianceMapping

GetAllMappings gets all mappings from the provider

func (*CustomComplianceProvider) GetFrameworks

func (p *CustomComplianceProvider) GetFrameworks() []ComplianceFramework

GetFrameworks returns a list of supported compliance frameworks

func (*CustomComplianceProvider) GetMapping

GetMapping gets a mapping from the provider

func (*CustomComplianceProvider) GetMappings

func (p *CustomComplianceProvider) GetMappings(ctx context.Context, testResult *TestResult) ([]ComplianceMapping, error)

GetMappings returns compliance mappings for a test result

func (*CustomComplianceProvider) RemoveMapping

func (p *CustomComplianceProvider) RemoveMapping(id string)

RemoveMapping removes a mapping from the provider

type DateRange

type DateRange struct {
	Start time.Time `json:"start"`
	End   time.Time `json:"end"`
}

DateRange represents a time range

type DefaultReportGenerator

type DefaultReportGenerator struct {
	// contains filtered or unexported fields
}

DefaultReportGenerator is the default implementation of ReportGenerator

func CreateDefaultReportGenerator

func CreateDefaultReportGenerator() (*DefaultReportGenerator, error)

CreateDefaultReportGenerator creates a default report generator with all formatters registered

func NewReportGenerator

func NewReportGenerator() *DefaultReportGenerator

NewReportGenerator creates a new report generator

func (*DefaultReportGenerator) GenerateReport

func (g *DefaultReportGenerator) GenerateReport(ctx context.Context, testSuites []*TestSuite, options *ReportOptions) (*Report, error)

GenerateReport generates a report from test results

func (*DefaultReportGenerator) GetFormatter

GetFormatter returns a formatter for a specific format

func (*DefaultReportGenerator) ListFormats

func (g *DefaultReportGenerator) ListFormats() []common.ReportFormat

ListFormats returns a list of supported formats

func (*DefaultReportGenerator) RegisterComplianceProvider

func (g *DefaultReportGenerator) RegisterComplianceProvider(provider ComplianceMappingProvider)

RegisterComplianceProvider registers a compliance mapping provider

func (*DefaultReportGenerator) RegisterFormatter

func (g *DefaultReportGenerator) RegisterFormatter(formatter common.ReportFormatter)

RegisterFormatter registers a formatter for a specific format

type Evidence

type Evidence struct {
	Request  string `json:"request,omitempty"`
	Response string `json:"response,omitempty"`
}

type FilterFunc

type FilterFunc func(result *TestResult) bool

FilterFunc is a function that filters test results

type Finding

type Finding struct {
	ID           string                 `json:"id"`
	Timestamp    time.Time              `json:"timestamp"`
	Category     string                 `json:"category"`
	Subcategory  string                 `json:"subcategory"`
	Severity     string                 `json:"severity"`
	Confidence   float64                `json:"confidence"`
	Title        string                 `json:"title"`
	Description  string                 `json:"description"`
	Remediation  Remediation            `json:"remediation"`
	OWASPMapping []string               `json:"owasp_mapping"`
	References   []Reference            `json:"references"`
	Evidence     Evidence               `json:"evidence"`
	Tags         []string               `json:"tags"`
	Metadata     map[string]interface{} `json:"metadata,omitempty"`
}

type HTMLRenderer

type HTMLRenderer struct {
	// contains filtered or unexported fields
}

HTMLRenderer renders reports as HTML

func (*HTMLRenderer) GetContentType

func (r *HTMLRenderer) GetContentType() string

GetContentType returns the content type for HTML

func (*HTMLRenderer) GetFileExtension

func (r *HTMLRenderer) GetFileExtension() string

GetFileExtension returns the file extension for HTML

func (*HTMLRenderer) Render

func (r *HTMLRenderer) Render(report *SecurityReport, options RenderOptions) ([]byte, error)

Render renders a report as HTML

type ISOComplianceProvider

type ISOComplianceProvider struct {
	// contains filtered or unexported fields
}

ISOComplianceProvider provides compliance mappings for ISO/IEC 42001

func NewISOComplianceProvider

func NewISOComplianceProvider() *ISOComplianceProvider

NewISOComplianceProvider creates a new ISO compliance provider

func (*ISOComplianceProvider) GetFrameworks

func (p *ISOComplianceProvider) GetFrameworks() []ComplianceFramework

GetFrameworks returns a list of supported compliance frameworks

func (*ISOComplianceProvider) GetMappings

func (p *ISOComplianceProvider) GetMappings(ctx context.Context, testResult *TestResult) ([]ComplianceMapping, error)

GetMappings returns compliance mappings for a test result

type JSONRenderer

type JSONRenderer struct{}

JSONRenderer renders reports as JSON

func (*JSONRenderer) GetContentType

func (r *JSONRenderer) GetContentType() string

GetContentType returns the content type for JSON

func (*JSONRenderer) GetFileExtension

func (r *JSONRenderer) GetFileExtension() string

GetFileExtension returns the file extension for JSON

func (*JSONRenderer) Render

func (r *JSONRenderer) Render(report *SecurityReport, options RenderOptions) ([]byte, error)

Render renders a report as JSON

type MarkdownRenderer

type MarkdownRenderer struct{}

MarkdownRenderer renders reports as Markdown

func (*MarkdownRenderer) GetContentType

func (r *MarkdownRenderer) GetContentType() string

GetContentType returns the content type for Markdown

func (*MarkdownRenderer) GetFileExtension

func (r *MarkdownRenderer) GetFileExtension() string

GetFileExtension returns the file extension for Markdown

func (*MarkdownRenderer) Render

func (r *MarkdownRenderer) Render(report *SecurityReport, options RenderOptions) ([]byte, error)

Render renders a report as Markdown

type MultiFormatRenderer

type MultiFormatRenderer struct {
	// contains filtered or unexported fields
}

MultiFormatRenderer handles rendering reports in multiple formats

func NewMultiFormatRenderer

func NewMultiFormatRenderer() *MultiFormatRenderer

NewMultiFormatRenderer creates a new multi-format renderer

func (*MultiFormatRenderer) RegisterRenderer

func (r *MultiFormatRenderer) RegisterRenderer(format common.ReportFormat, renderer Renderer)

RegisterRenderer registers a renderer for a format

func (*MultiFormatRenderer) Render

func (r *MultiFormatRenderer) Render(report *SecurityReport, format common.ReportFormat, options RenderOptions) ([]byte, error)

Render renders a report in the specified format

type OWASPComplianceProvider

type OWASPComplianceProvider struct {
	// contains filtered or unexported fields
}

OWASPComplianceProvider provides compliance mappings for OWASP Top 10 for LLMs

func NewOWASPComplianceProvider

func NewOWASPComplianceProvider() *OWASPComplianceProvider

NewOWASPComplianceProvider creates a new OWASP compliance provider

func (*OWASPComplianceProvider) GetFrameworks

func (p *OWASPComplianceProvider) GetFrameworks() []ComplianceFramework

GetFrameworks returns a list of supported compliance frameworks

func (*OWASPComplianceProvider) GetMappings

func (p *OWASPComplianceProvider) GetMappings(ctx context.Context, testResult *TestResult) ([]ComplianceMapping, error)

GetMappings returns compliance mappings for a test result

type Reference

type Reference struct {
	Title string `json:"title"`
	URL   string `json:"url"`
}

type Remediation

type Remediation struct {
	Summary string   `json:"summary"`
	Steps   []string `json:"steps"`
}

type RenderOptions

type RenderOptions struct {
	Template       string                 `json:"template"`
	Locale         string                 `json:"locale"`
	TimeZone       string                 `json:"timezone"`
	IncludeRawData bool                   `json:"includeRawData"`
	Filters        ReportFilter           `json:"filters"`
	CustomFields   map[string]interface{} `json:"customFields"`
}

RenderOptions contains options for rendering

type Renderer

type Renderer interface {
	Render(report *SecurityReport, options RenderOptions) ([]byte, error)
	GetContentType() string
	GetFileExtension() string
}

Renderer interface for format-specific renderers

type Report

type Report struct {
	// ID is the unique identifier for the report
	ID string `json:"id"`
	// Title is the title of the report
	Title string `json:"title"`
	// Description is the description of the report
	Description string `json:"description,omitempty"`
	// GeneratedAt is the time the report was generated
	GeneratedAt time.Time `json:"generated_at"`
	// Summary is the summary of the report
	Summary ReportSummary `json:"summary"`
	// TestSuites is the list of test suites
	TestSuites []*TestSuite `json:"test_suites"`
	// Metadata is additional metadata for the report
	Metadata map[string]interface{} `json:"metadata,omitempty"`
}

Report represents a complete test report

type ReportBuilder

type ReportBuilder struct {
	// contains filtered or unexported fields
}

ReportBuilder builds reports with various options

func NewReportBuilder

func NewReportBuilder() *ReportBuilder

NewReportBuilder creates a new report builder

func (*ReportBuilder) AddFilter

func (rb *ReportBuilder) AddFilter(filter func(Finding) bool) *ReportBuilder

AddFilter adds a filter function

func (*ReportBuilder) AddFinding

func (rb *ReportBuilder) AddFinding(finding Finding) *ReportBuilder

AddFinding adds a finding to the report

func (*ReportBuilder) Build

func (rb *ReportBuilder) Build() *SecurityReport

Build builds the final report

func (*ReportBuilder) WithMetadata

func (rb *ReportBuilder) WithMetadata(metadata ReportMetadata) *ReportBuilder

WithMetadata sets report metadata

type ReportFilter

type ReportFilter struct {
	Severity   []string               `json:"severity"`
	Categories []string               `json:"categories"`
	DateRange  *DateRange             `json:"dateRange"`
	Tags       []string               `json:"tags"`
	Custom     map[string]interface{} `json:"custom"`
}

ReportFilter defines filtering options

type ReportFormat

type ReportFormat = common.ReportFormat

ReportFormat represents the format of a report

type ReportGenerator

type ReportGenerator interface {
	// GenerateReport generates a report from test results
	GenerateReport(ctx context.Context, testSuites []*TestSuite, options *ReportOptions) (*Report, error)
	// RegisterFormatter registers a formatter for a specific format
	RegisterFormatter(formatter common.ReportFormatter)
	// GetFormatter returns a formatter for a specific format
	GetFormatter(format common.ReportFormat) (common.ReportFormatter, bool)
	// ListFormats returns a list of supported formats
	ListFormats() []common.ReportFormat
}

ReportGenerator interface defines methods for report generation

type ReportMetadata

type ReportMetadata struct {
	ID        string                 `json:"id"`
	Title     string                 `json:"title"`
	CreatedAt time.Time              `json:"created_at"`
	Version   string                 `json:"version"`
	Metadata  map[string]interface{} `json:"metadata,omitempty"`
}

type ReportOptions

type ReportOptions struct {
	// Format is the format of the report
	Format common.ReportFormat `json:"format"`
	// Title is the title of the report
	Title string `json:"title"`
	// Description is the description of the report
	Description string `json:"description,omitempty"`
	// IncludePassedTests indicates whether to include passed tests
	IncludePassedTests bool `json:"include_passed_tests"`
	// IncludeSkippedTests indicates whether to include skipped tests
	IncludeSkippedTests bool `json:"include_skipped_tests"`
	// IncludePendingTests indicates whether to include pending tests
	IncludePendingTests bool `json:"include_pending_tests"`
	// MinimumSeverity is the minimum severity level to include
	MinimumSeverity common.SeverityLevel `json:"minimum_severity"`
	// IncludeTags is the list of tags to include
	IncludeTags []string `json:"include_tags,omitempty"`
	// ExcludeTags is the list of tags to exclude
	ExcludeTags []string `json:"exclude_tags,omitempty"`
	// SortBy is the field to sort by
	SortBy string `json:"sort_by,omitempty"`
	// SortOrder is the sort order (asc or desc)
	SortOrder string `json:"sort_order,omitempty"`
	// TemplatePath is the path to a custom template
	TemplatePath string `json:"template_path,omitempty"`
	// OutputPath is the path to write the report to
	OutputPath string `json:"output_path,omitempty"`
	// Metadata is additional metadata for the report
	Metadata map[string]interface{} `json:"metadata,omitempty"`
}

ReportOptions represents options for generating a report

type ReportStatistics

type ReportStatistics struct {
	StartTime         time.Time       `json:"start_time"`
	EndTime           time.Time       `json:"end_time"`
	FindingsPerMinute float64         `json:"findings_per_minute"`
	AverageConfidence float64         `json:"average_confidence"`
	TopCategories     []CategoryCount `json:"top_categories"`
}

type ReportSummary

type ReportSummary struct {
	// TotalTests is the total number of tests
	TotalTests int `json:"total_tests"`
	// PassedTests is the number of passed tests
	PassedTests int `json:"passed_tests"`
	// FailedTests is the number of failed tests
	FailedTests int `json:"failed_tests"`
	// ErrorTests is the number of tests with errors
	ErrorTests int `json:"error_tests"`
	// SkippedTests is the number of skipped tests
	SkippedTests int `json:"skipped_tests"`
	// PendingTests is the number of pending tests
	PendingTests int `json:"pending_tests"`
	// PassRate is the pass rate as a percentage
	PassRate float64 `json:"pass_rate"`
	// AverageScore is the average score of all tests
	AverageScore float64 `json:"average_score"`
	// AverageDuration is the average duration of all tests
	AverageDuration time.Duration `json:"average_duration"`
	// TotalDuration is the total duration of all tests
	TotalDuration time.Duration `json:"total_duration"`
	// SeverityCounts is the count of tests by severity
	SeverityCounts map[common.SeverityLevel]int `json:"severity_counts"`
	// TagCounts is the count of tests by tag
	TagCounts map[string]int `json:"tag_counts"`
}

ReportSummary represents a summary of a report

type SecurityReport

type SecurityReport struct {
	Metadata   ReportMetadata               `json:"metadata"`
	Summary    SecurityReportSummary        `json:"summary"`
	Findings   []Finding                    `json:"findings"`
	Statistics ReportStatistics             `json:"statistics"`
	Compliance map[string]*ComplianceStatus `json:"compliance,omitempty"`
}

Add type definitions that are missing

type SecurityReportSummary

type SecurityReportSummary struct {
	TotalFindings     int            `json:"total_findings"`
	RiskScore         float64        `json:"risk_score"`
	SeverityBreakdown map[string]int `json:"severity_breakdown"`
	CategoryBreakdown map[string]int `json:"category_breakdown"`
}

type SeverityLevel

type SeverityLevel = common.SeverityLevel

SeverityLevel represents the severity level of a test

type SortFunc

type SortFunc func(results []*TestResult) []*TestResult

SortFunc is a function that sorts test results

type StreamingRenderer

type StreamingRenderer interface {
	StartRender(w io.Writer, metadata ReportMetadata, options RenderOptions) error
	RenderFinding(w io.Writer, finding Finding) error
	FinishRender(w io.Writer, summary SecurityReportSummary) error
}

StreamingRenderer supports streaming large reports

type TemplateReportingService

type TemplateReportingService struct {
	// contains filtered or unexported fields
}

TemplateReportingService provides reporting services for template results

func NewTemplateReportingService

func NewTemplateReportingService(converter *TemplateResultConverter, generator common.ReportGenerator) *TemplateReportingService

NewTemplateReportingService creates a new template reporting service

func (*TemplateReportingService) GenerateReport

func (s *TemplateReportingService) GenerateReport(ctx context.Context, results []*types.TemplateResult, options *ReportOptions) ([]byte, error)

GenerateReport generates a report from template results

type TemplateResultConverter

type TemplateResultConverter struct {
	// contains filtered or unexported fields
}

TemplateResultConverter converts template results to test results

func NewTemplateResultConverter

func NewTemplateResultConverter(complianceProviders []ComplianceMappingProvider) *TemplateResultConverter

NewTemplateResultConverter creates a new template result converter

func (*TemplateResultConverter) ConvertToTestSuite

func (c *TemplateResultConverter) ConvertToTestSuite(ctx context.Context, results []*types.TemplateResult, suiteID string, suiteName string) (*TestSuite, error)

ConvertToTestSuite converts template results to a test suite

type TestResult

type TestResult struct {
	// ID is the unique identifier for the test result
	ID string `json:"id"`
	// Name is the name of the test
	Name string `json:"name"`
	// Description is the description of the test
	Description string `json:"description,omitempty"`
	// Status is the status of the test
	Status TestStatus `json:"status"`
	// Severity is the severity level of the test
	Severity common.SeverityLevel `json:"severity"`
	// Score is the score of the test (0-100)
	Score int `json:"score"`
	// StartTime is the time the test started
	StartTime time.Time `json:"start_time"`
	// EndTime is the time the test ended
	EndTime time.Time `json:"end_time"`
	// Duration is the duration of the test
	Duration time.Duration `json:"duration"`
	// Error is any error that occurred during the test
	Error string `json:"error,omitempty"`
	// Input is the input to the test
	Input string `json:"input,omitempty"`
	// ExpectedOutput is the expected output of the test
	ExpectedOutput string `json:"expected_output,omitempty"`
	// ActualOutput is the actual output of the test
	ActualOutput string `json:"actual_output,omitempty"`
	// ComplianceMappings is the list of compliance mappings for the test
	ComplianceMappings []ComplianceMapping `json:"compliance_mappings,omitempty"`
	// Tags is the list of tags for the test
	Tags []string `json:"tags,omitempty"`
	// Metadata is additional metadata for the test
	Metadata map[string]interface{} `json:"metadata,omitempty"`
}

TestResult represents a single test result

type TestStatus

type TestStatus string

TestStatus represents the status of a test

const (
	PassStatus    TestStatus = "pass"
	FailStatus    TestStatus = "fail"
	ErrorStatus   TestStatus = "error"
	SkippedStatus TestStatus = "skipped"
	PendingStatus TestStatus = "pending"
)

Supported test statuses

type TestSuite

type TestSuite struct {
	// ID is the unique identifier for the test suite
	ID string `json:"id"`
	// Name is the name of the test suite
	Name string `json:"name"`
	// Description is the description of the test suite
	Description string `json:"description,omitempty"`
	// StartTime is the time the test suite started
	StartTime time.Time `json:"start_time"`
	// EndTime is the time the test suite ended
	EndTime time.Time `json:"end_time"`
	// Duration is the duration of the test suite
	Duration time.Duration `json:"duration"`
	// Results is the list of test results
	Results []*TestResult `json:"results"`
	// Metadata is additional metadata for the test suite
	Metadata map[string]interface{} `json:"metadata,omitempty"`
}

TestSuite represents a collection of test results

Directories

Path Synopsis
Package api provides common types and interfaces for the reporting system
Package api provides common types and interfaces for the reporting system
Package common provides common types and interfaces for the reporting system
Package common provides common types and interfaces for the reporting system
backup
Package common provides common types and interfaces for the reporting system
Package common provides common types and interfaces for the reporting system
consolidated
Package common provides common types and interfaces for the reporting system
Package common provides common types and interfaces for the reporting system
types
Package types provides common types for the reporting formatters
Package types provides common types for the reporting formatters
Package interfaces provides common interfaces for the reporting system
Package interfaces provides common interfaces for the reporting system
Package types provides common types for the reporting system
Package types provides common types for the reporting system

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL