e2e

dmt end-to-end test framework

This package runs dmt against concrete module fixtures and asserts on the structured findings it produces. Unlike the per-rule unit tests (which call a single rule in isolation), each case here exercises config loading, the helm render, and every linter together — the same path a user hits when running dmt lint <module>.

Most cases run the full dmt lint pipeline. Cases can also run the dmt test conversions testers by setting kind: conversions in expected.yaml; their results are surfaced through the same matching machinery under the linter ID conversions.

How it works

Cases live under testdata/<linter>/<case>/, where <linter> is the name of the linter the case primarily exercises. TestE2E (in e2e_test.go) treats each linter folder as a parent subtest and every case inside it as a nested subtest (e.g. TestE2E/templates/vpa-pdb-absent). A directory is a case when it contains an expected.yaml file. For each case it:

1. copies the case's module/ directory into an isolated temp dir (so the run is hermetic — no .dmtlint.yaml inherited from parent dirs, no artifacts written back into testdata/),
2. runs the lint manager (or, for kind: conversions, the conversions testers) against the copy,
3. matches the collected findings against the expectations in expected.yaml.

Adding a case

Create a new directory under the relevant linter folder, testdata/<linter>/<your-case>/:

testdata/<linter>/<your-case>/
├── expected.yaml          # the case specification
└── module/                # the Deckhouse module that gets linted
    ├── module.yaml
    ├── openapi/
    │   ├── values.yaml
    │   └── config-values.yaml
    └── templates/         # optional
        └── ...

A minimal lint-able module needs at least:

• module.yaml with name and namespace (or Chart.yaml + a .namespace file), and
• an openapi/ directory containing values.yaml and config-values.yaml.

expected.yaml

description: >
  Human-readable summary of what this case verifies.
kind: lint              # "lint" (default) or "conversions"
module: module          # subdir to lint, defaults to "module"
expectClean: false      # assert the run produced zero findings
exhaustive: false       # assert there are NO findings beyond those listed
expect:
  - linter: container             # required, matched case-insensitively
    rule: env-variables-duplicates # optional
    level: error                   # optional: ignored | warn | error
    textContains: "same name"      # optional, case-sensitive substring
    count: 1                       # optional; 0/omitted means "at least one"

Matching semantics:

• linter is required and compared case-insensitively to the finding's linter ID.
• rule, level and textContains are optional filters; when present they all must match.
• count is the expected number of matching findings. 0 (or omitting it) means "at least one".
• expectClean: true asserts the module produced no findings at all.
• exhaustive: true asserts that every produced finding is matched by some entry in expect (use it to lock down the complete output of a fixture).

Discovering the findings for a new fixture

The quickest way to author expectations is to temporarily set expectClean: true in expected.yaml and run the case — the failure message prints every finding the fixture produced, which you can then copy into expect:

go test ./test/e2e/ -run 'TestE2E/<linter>/<your-case>' -v

Current cases

conversions tester (kind: conversions)

templates linter (comprehensive)

Running

# all cases
go test ./test/e2e/

# every case for one linter
go test ./test/e2e/ -run 'TestE2E/templates' -v

# a single case, verbose
go test ./test/e2e/ -run 'TestE2E/templates/vpa-pdb-absent' -v