testrun

package
v0.13.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 4, 2026 License: BSD-3-Clause Imports: 21 Imported by: 0

Documentation

Overview

Package testrun runs test cases and checks their responses. See docs/testcases-design.md.

Index

Constants

View Source
const (
	SourceAssert = "assert"
	SourceScript = "script"
)

Where an assertion came from.

View Source
const (
	SectionSetup    = "setup"
	SectionSteps    = "steps"
	SectionTeardown = "teardown"
)

Sections of a test case, in run order.

Variables

This section is empty.

Functions

func Bundle

func Bundle(name string, cases []*domain.TestCase, src RequestSource, collection func(id string) *domain.Collection, o BundleOptions) (*domain.TestBundle, error)

Bundle packs cases with the requests they send, including the ones their pre-request actions trigger, so they run without the workspace. collection finds the collection a request belongs to.

func NewSender

func NewSender(requests sender.RequestLookup, colls sender.CollectionLookup, scripts sender.Scripts, scriptingOn func() bool) func() Sender

NewSender returns a factory of senders for test runs. Each one keeps cookies in memory for its run and never saves the environment. scriptingOn decides whether request scripts run; nil follows the scripting setting, as the app does.

func RefOf

func RefOf(req *domain.Request) string

RefOf returns the ref that names req: "Collection/Request", or the request name for one outside a collection.

func Resolve

func Resolve(src RequestSource, ref domain.TestRequestRef) (*domain.Request, error)

Resolve finds the request ref points at: by ID first, then by name.

Types

type AssertionResult

type AssertionResult struct {
	Source   string `json:"source"`
	Target   string `json:"target,omitempty"`
	Key      string `json:"key,omitempty"`
	Path     string `json:"path,omitempty"`
	Op       string `json:"op,omitempty"`
	Expected any    `json:"expected,omitempty"`
	Actual   any    `json:"actual,omitempty"`
	Passed   bool   `json:"passed"`
	// Message says why it failed.
	Message string `json:"message,omitempty"`
}

AssertionResult is one assertion checked against a response.

func Evaluate

func Evaluate(res *egress.Response, asserts []domain.TestAssertion) []AssertionResult

Evaluate checks asserts against res.

func ScriptAssertions

func ScriptAssertions(tests []scripting.TestResult) []AssertionResult

ScriptAssertions reports the chapar.test() calls of a request's scripts as assertions.

type BundleOptions

type BundleOptions struct {
	// Env is exported with the cases when set.
	Env *domain.Environment
	// IncludeSecrets exports secret values in plain text. Off, they are
	// left out and named in SecretsLeftOut.
	IncludeSecrets bool
}

BundleOptions says what goes into a bundle besides the test cases.

type CaptureResult

type CaptureResult struct {
	Var   string `json:"var"`
	Value string `json:"value,omitempty"`
	Error string `json:"error,omitempty"`
}

CaptureResult is one value a step stored in a run variable.

func Capture

func Capture(res *egress.Response, caps []domain.TestCapture) (results []CaptureResult, vars map[string]string)

Capture reads the values caps store from res. vars holds the ones that were read; a capture that fails leaves its variable unset.

type Config

type Config struct {
	// NewSender returns the sender for one run, see NewSender.
	NewSender func() Sender
	Requests  RequestSource
	// SaveEnv writes an environment back for cases with persistEnv; nil
	// ignores persistEnv.
	SaveEnv func(env *domain.Environment) error
	// Getenv reads OS environment variables; nil means os.Getenv.
	Getenv func(key string) string
}

type Event

type Event struct {
	Kind EventKind
	Step *StepResult
	Run  *Run
}

Event reports progress while a run is going. Step is set for step events, Run for EventRunFinished.

type EventKind

type EventKind int

EventKind says what happened in an Event.

const (
	EventStepStarted EventKind = iota
	EventStepFinished
	EventRunFinished
)

type Options

type Options struct {
	// Env is the environment to run with, with secrets already resolved.
	// It is copied and never changed; nil runs without one.
	Env *domain.Environment
	// Only runs just these steps (by id) of the steps section; setup and
	// teardown still run. Empty runs all.
	Only []string
	// OnEvent reports progress. It is called on the run's goroutine.
	OnEvent func(Event)
}

type Problem

type Problem struct {
	Where   string
	Message string
}

Problem is something wrong with a test case file. Where is a path into the spec, such as "steps[1].assert[0]".

func Validate

func Validate(tc *domain.TestCase, find RequestFinder) []Problem

Validate lists the problems in tc. find checks that each step's request exists; nil skips that check.

func (Problem) Error

func (p Problem) Error() string

type RequestFinder

type RequestFinder func(ref domain.TestRequestRef) error

RequestFinder reports why a step's request cannot be found, or nil.

func Finder

func Finder(src RequestSource) RequestFinder

Finder checks step requests against src, for Validate.

type RequestSource

type RequestSource interface {
	RequestByID(id string) *domain.Request
	// AllRequests returns standalone requests and those in collections,
	// with CollectionName set on the latter.
	AllRequests() []*domain.Request
}

RequestSource is where a runner finds the requests steps send.

type RequestSummary

type RequestSummary struct {
	ID       string `json:"id"`
	Name     string `json:"name"`
	Protocol string `json:"protocol"`
	Method   string `json:"method,omitempty"`
	URL      string `json:"url,omitempty"`
}

RequestSummary is what a step sent, before variables were substituted.

type ResponseSummary

type ResponseSummary struct {
	Status  int               `json:"status"`
	Size    int               `json:"size"`
	Time    time.Duration     `json:"time"`
	Headers map[string]string `json:"headers,omitempty"`
	Body    string            `json:"body,omitempty"`
	// Truncated is set when Body holds only the first maxBody bytes.
	Truncated bool `json:"truncated,omitempty"`
}

ResponseSummary is what a step got back.

type Run

type Run struct {
	ID       string        `json:"id"`
	CaseID   string        `json:"caseId"`
	CaseName string        `json:"caseName"`
	EnvName  string        `json:"envName,omitempty"`
	Status   Status        `json:"status"`
	Started  time.Time     `json:"started"`
	Duration time.Duration `json:"duration"`
	Steps    []StepResult  `json:"steps"`
	// Message is a run-level problem, such as failing to save the env.
	Message string `json:"message,omitempty"`
}

Run is the result of running one test case.

func (*Run) Counts

func (r *Run) Counts() map[Status]int

Counts returns how many steps ended in each status.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

Runner runs test cases. It is safe to run several cases at once.

func New

func New(cfg Config) *Runner

func (*Runner) Run

func (r *Runner) Run(ctx context.Context, tc *domain.TestCase, o Options) *Run

Run runs tc and returns its result. Cancelling ctx stops the run after the current send; teardown still runs.

type Sender

type Sender interface {
	Send(req *domain.Request, env *domain.Environment) (*egress.Response, error)
}

Sender sends one request with an environment. *sender.Service is one.

type Status

type Status string

Status of a run or a step.

const (
	StatusRunning Status = "running"
	StatusPassed  Status = "passed"
	// StatusFailed means the step ran and an assertion or capture failed.
	StatusFailed Status = "failed"
	// StatusError means the step could not run: its request is missing,
	// the send failed, timed out, or a post-request action raised.
	StatusError     Status = "error"
	StatusSkipped   Status = "skipped"
	StatusCancelled Status = "cancelled"
)

type StepResult

type StepResult struct {
	Section  string           `json:"section"`
	StepID   string           `json:"stepId"`
	Name     string           `json:"name"`
	Status   Status           `json:"status"`
	Attempts int              `json:"attempts,omitempty"`
	Duration time.Duration    `json:"duration,omitempty"`
	Request  *RequestSummary  `json:"request,omitempty"`
	Response *ResponseSummary `json:"response,omitempty"`

	Assertions []AssertionResult `json:"assertions,omitempty"`
	Captures   []CaptureResult   `json:"captures,omitempty"`
	// Message says why the step failed, errored or was skipped.
	Message string `json:"message,omitempty"`
}

StepResult is the result of one step.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL