Documentation
¶
Overview ¶
Package expect evaluates captured observations of a ceph cluster against the conditions a test states, and explains each mismatch.
It is optional: constructing, querying and watching a cluster needs only package ceph. A test that wants a precise verdict on one observation, such as "these pinned OSDs are up and in and no other owned OSD exists", states it as an expectation and evaluates the observation it already has:
states, err := observer.OSDStates(ctx)
if err != nil {
t.Fatal(err) // a failed query is never a mismatch
}
result, err := expect.OSDs{Select: baseline, Up: true, In: true, Exact: true}.Evaluate(states)
if err != nil {
t.Fatal(err) // an invalid expectation or a malformed observation
}
if !result.Matched {
t.Fatal(result) // names each OSD and the field that differs
}
Every Evaluate is pure: it reads only its arguments, performs no query, wait or retry, and returns a result that shares no slice or map with them. A mismatch is a result; an error means there is no verdict. Polling, retries and assertion failures stay with the test and its own tools. Evaluations of service observations live in the service's own expect package, such as cephfs/expect, rbd/expect or rgw/expect.
Index ¶
- type Health
- type HealthCheck
- type HealthEvaluation
- type HealthMismatch
- type HealthMismatchReason
- type HealthMuteState
- type ManagerEvaluation
- type ManagerMismatch
- type ManagerMismatchReason
- type Managers
- type MonitorEvaluation
- type MonitorMismatch
- type MonitorMismatchReason
- type Monitors
- type OSDEvaluation
- type OSDMismatch
- type OSDMismatchReason
- type OSDs
- type PGEvaluation
- type PGMismatch
- type PGMismatchReason
- type PGs
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Health ¶
type Health struct {
FSID string
Status string
Require []HealthCheck
Forbid []string
Mutes []string
NoMutes []string
Exact bool
}
Health states the health checks and stored mutes a test expects in one ceph.Observer.HealthDetails observation. FSID, when set, pins the cluster, so an observation of another cluster does not match. Status, when set, is the overall status the observation must report; Ceph derives it from the worst unmuted check, so HEALTH_OK can accompany muted warnings. Require lists checks that must be present and Forbid codes that must be absent. Mutes lists codes that must have a stored mute and NoMutes codes that must not; a stored sticky mute can outlive its check. Exact also requires no check outside Require and no stored mute outside Mutes.
A match proves only what this one observation reported. Ceph raises many checks after a delay and clears them at a later monitor tick, so a missing check is not proof that the condition behind it is gone, and a present check says nothing about the client data path. Unlike the readiness waits of package ceph, such as ceph.Observer.WaitForClean, an expectation never polls and names each check that differs.
func (Health) Evaluate ¶
func (e Health) Evaluate(observed ceph.HealthSnapshot) (HealthEvaluation, error)
Evaluate checks one successful ceph.Observer.HealthDetails observation against the expectation. It reads only its arguments: no query, wait or retry, so a test composes it with its own polling or assertion helpers. A mismatch is a result; an error means no verdict. An expectation that states nothing, an invalid FSID, status, severity or mute state, an empty or repeated code, and a contradiction (a code both required and forbidden, both in Mutes and NoMutes, a muted required check without a stored mute or an unmuted one with a stored mute, or an unmuted required severity worse than Status) are ceph.ErrInvalidRequest. An observation with an invalid status or severity, an empty check code, an empty or repeated mute code, or an invalid FSID when the expectation pins one is a malformed observation, never a mismatch. The result shares no slice or map with its arguments.
Example ¶
A health expectation compares check codes, severities, counts and mute state, never the messages, and names each check that differs.
package main
import (
"fmt"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph/expect"
)
func main() {
observed := ceph.HealthSnapshot{
FSID: "44444444-4444-4444-8444-444444444444",
Status: "HEALTH_WARN",
Checks: map[string]ceph.HealthCheck{
"OSD_DOWN": {Severity: "HEALTH_WARN", Summary: "1 osds down", Count: 1, Details: []string{"osd.1 is down"}},
"SLOW_OPS": {Severity: "HEALTH_WARN", Summary: "2 slow ops", Count: 2},
},
Mutes: []ceph.HealthMute{},
}
want := expect.Health{
Require: []expect.HealthCheck{{Code: "OSD_DOWN", Severity: "HEALTH_WARN", Muted: expect.HealthUnmuted, Count: 1}},
Forbid: []string{"SLOW_OPS"},
NoMutes: []string{"OSD_DOWN"},
}
result, err := want.Evaluate(observed)
if err != nil {
panic(err) // invalid expectation or malformed observation: no verdict
}
fmt.Println(result.Matched)
fmt.Println(result)
}
Output: false health want require=[OSD_DOWN] forbid=[SLOW_OPS] nomutes=[OSD_DOWN] in HEALTH_WARN with 2 checks, 0 mutes: SLOW_OPS present: HEALTH_WARN count=2 unmuted
type HealthCheck ¶
type HealthCheck struct {
Code string
Severity string
Muted HealthMuteState
Count int64
}
HealthCheck is one health check that a Health expectation requires. Code is the native, case-sensitive check code such as "OSD_DOWN". Severity, when set, is the native severity the check must report (HEALTH_OK, HEALTH_WARN or HEALTH_ERR). Muted states the reported mute flag. Count, when nonzero, is the native summary count the check must report; a count of zero cannot be required. The free-form summary and detail messages are never compared: they carry pool names, paths and other application data.
type HealthEvaluation ¶
type HealthEvaluation struct {
Expectation Health
Observed ceph.HealthSnapshot
Mismatches []HealthMismatch
Matched bool
}
HealthEvaluation is the result of evaluating one health observation. Expectation and Observed are deep copies of what was evaluated, so the diagnostics describe that observation even when the caller later reuses its slices and maps. Mismatches list HealthMismatchFSID and HealthMismatchStatus first, then the codes in ascending order, each code's reasons in the order of the reason constants; Matched means there are none.
func (HealthEvaluation) String ¶
func (e HealthEvaluation) String() string
String summarizes the expectation and at most eight mismatches.
type HealthMismatch ¶
type HealthMismatch struct {
Reason HealthMismatchReason
Code string
Want, Observed string
}
HealthMismatch is one reason a health observation does not satisfy an expectation. Code is the check or mute code, empty for HealthMismatchFSID and HealthMismatchStatus. Want and Observed are the compared values as text, both empty where the reason alone says it: Want is empty for HealthMismatchPresent, HealthMismatchUnexpected, HealthMismatchMutePresent and HealthMismatchMuteUnexpected, whose Observed describes what was reported, and Observed is empty for HealthMismatchAbsent and HealthMismatchMuteAbsent.
func (HealthMismatch) String ¶
func (m HealthMismatch) String() string
String names the code, the reason and the evaluated values.
type HealthMismatchReason ¶
type HealthMismatchReason string
HealthMismatchReason names why a health observation does not satisfy a Health expectation.
const ( // HealthMismatchFSID: the observation belongs to another cluster. HealthMismatchFSID HealthMismatchReason = "fsid" // HealthMismatchStatus: the overall status differs. HealthMismatchStatus HealthMismatchReason = "status" // HealthMismatchAbsent: a required check is not reported. Its severity, // mute flag and count are not compared. HealthMismatchAbsent HealthMismatchReason = "absent" // HealthMismatchSeverity: a required check reports another severity. HealthMismatchSeverity HealthMismatchReason = "severity" // HealthMismatchMuted: a required check reports another mute flag. HealthMismatchMuted HealthMismatchReason = "muted" // HealthMismatchCount: a required check reports another summary count. HealthMismatchCount HealthMismatchReason = "count" // HealthMismatchPresent: a forbidden check is reported. HealthMismatchPresent HealthMismatchReason = "present" // HealthMismatchUnexpected: an Exact expectation found a check that // neither Require nor Forbid names (a forbidden check is // HealthMismatchPresent). HealthMismatchUnexpected HealthMismatchReason = "unexpected" // HealthMismatchMuteAbsent: a code listed in Mutes has no stored mute. HealthMismatchMuteAbsent HealthMismatchReason = "mute absent" // HealthMismatchMutePresent: a code listed in NoMutes has a stored mute. HealthMismatchMutePresent HealthMismatchReason = "mute present" // HealthMismatchMuteUnexpected: an Exact expectation found a stored mute // that neither Mutes nor NoMutes names (a mute listed in NoMutes is // HealthMismatchMutePresent). HealthMismatchMuteUnexpected HealthMismatchReason = "mute unexpected" )
The reasons a HealthEvaluation reports.
type HealthMuteState ¶
type HealthMuteState string
HealthMuteState states whether a required health check must be muted.
const ( // HealthMuteAny does not compare the check's Muted flag. HealthMuteAny HealthMuteState = "" // HealthMuted requires the check to be reported as muted. HealthMuted HealthMuteState = "muted" // HealthUnmuted requires the check to be reported as not muted, so it // counts towards the overall status. HealthUnmuted HealthMuteState = "unmuted" )
The mute states a HealthCheck can require.
type ManagerEvaluation ¶
type ManagerEvaluation struct {
Expectation Managers
Observed ceph.ManagerStatus
Mismatches []ManagerMismatch
Matched bool
}
ManagerEvaluation is the result of evaluating one observation. Expectation and Observed are copies of what was evaluated, so the diagnostics describe that observation even when the caller later reuses its slices. Mismatches list the reasons about availability, the active MGR and the standby count first, in the order of the reason constants, then the per-standby reasons ordered by name and then in the order of the reason constants; Matched means there are none.
func (ManagerEvaluation) String ¶
func (e ManagerEvaluation) String() string
String summarizes the expectation, the observed MGRMap and at most eight mismatches.
type ManagerMismatch ¶
type ManagerMismatch struct {
Reason ManagerMismatchReason
Name string
GID uint64
Count int
ObservedAvailable bool
ObservedName string
ObservedGID uint64
ObservedCount int
}
ManagerMismatch is one reason an observation does not satisfy a Managers expectation. The expected side is Name for ManagerMismatchActive and the standby reasons, GID for ManagerMismatchActiveGID and ManagerMismatchFormerGID, and Count for ManagerMismatchStandbyCount. The observed side is ObservedAvailable for ManagerMismatchAvailable, ObservedName for ManagerMismatchActive (empty without an active MGR), ObservedGID for the GID reasons (zero without an active MGR) and for ManagerMismatchStandbyUnexpected (that standby's GID), and ObservedCount for ManagerMismatchStandbyCount.
func (ManagerMismatch) String ¶
func (m ManagerMismatch) String() string
String names the reason and the evaluated values.
type ManagerMismatchReason ¶
type ManagerMismatchReason string
ManagerMismatchReason names why an observation does not satisfy a Managers expectation.
const ( // ManagerMismatchAvailable: the reported availability differs from the // expectation. ManagerMismatchAvailable ManagerMismatchReason = "available" // ManagerMismatchActive: the active MGR has another name, or there is // none. ManagerMismatchActive ManagerMismatchReason = "active" // ManagerMismatchActiveGID: the active MGR carries another GID than the // pinned one, or there is none. ManagerMismatchActiveGID ManagerMismatchReason = "active-gid" // ManagerMismatchFormerGID: the active MGR still carries the former GID, // or there is none. ManagerMismatchFormerGID ManagerMismatchReason = "former-gid" // ManagerMismatchStandbyCount: the MGRMap lists another number of // standbys. ManagerMismatchStandbyCount ManagerMismatchReason = "standby-count" // ManagerMismatchStandbyAbsent: a name of Standbys is not a standby. ManagerMismatchStandbyAbsent ManagerMismatchReason = "standby-absent" // ManagerMismatchStandbyUnexpected: a standby that Standbys does not name. ManagerMismatchStandbyUnexpected ManagerMismatchReason = "standby-unexpected" )
The reasons a ManagerEvaluation reports.
type Managers ¶
type Managers struct {
Available bool
Active string
ActiveGID uint64
FormerGID uint64
Standbys []string
StandbyCount int
}
Managers states the MGRMap ownership a test expects in one ceph.Observer.ManagerStatus observation, such as the status of a ceph.ManagerWatchEvent.
Available is always compared: the zero value expects an MGRMap with no available MGR. Active, when set, is the name the active MGR must have, and ActiveGID, when set, pins its registration: a GID from an earlier observation does not match an MGR that re-registered under the same name. FormerGID, when set, is the GID of an earlier active MGR that the current active one must not carry, which states a failover or a restart of the active MGR; it also requires an active MGR. Standbys, when not nil, is the exact set of standby names, so an empty non-nil slice requires that there is no standby. StandbyCount, when positive, is the number of standbys, for a test that does not know their names.
A match proves only what that one MGRMap reported: it does not prove that a module serves requests, that a standby process runs, or that a stopped MGR has exited. Unlike ceph.Observer.WaitForClean, which requires only an available MGR, an expectation names the MGR registrations the test requires.
func (Managers) Evaluate ¶
func (e Managers) Evaluate(observed ceph.ManagerStatus) (ManagerEvaluation, error)
Evaluate checks one successful ceph.Observer.ManagerStatus observation against the expectation. It reads only its arguments: no query, wait or retry, so a test composes it with its own polling or assertion helpers. A mismatch is a result; an error means no verdict.
A negative StandbyCount, a StandbyCount that contradicts Standbys, an empty or repeated standby name, or a FormerGID equal to ActiveGID is ceph.ErrInvalidRequest. An observation whose active name and GID disagree about whether there is an active MGR, that is available without one, or that lists a standby with no name or GID or with a GID that another registration carries is a malformed observation, never a mismatch. The result shares no slice with its arguments.
Example ¶
A failover expectation names the standby that must take over and the GID of the active MGR it must replace, taken from an earlier observation.
package main
import (
"encoding/json"
"fmt"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph/expect"
)
func main() {
var before, after ceph.ManagerStatus // as observer.ManagerStatus decodes them
if err := json.Unmarshal([]byte(`{"available":true,"active_name":"a","active_gid":4100,
"standbys":[{"name":"b","gid":4200}]}`), &before); err != nil {
panic(err)
}
// ... fail the active MGR while the SUT runs ...
if err := json.Unmarshal([]byte(`{"available":false,"active_name":"a","active_gid":4100,
"standbys":[{"name":"b","gid":4200}]}`), &after); err != nil {
panic(err)
}
want := expect.Managers{Available: true, Active: before.Standbys[0].Name, FormerGID: before.ActiveGID}
result, err := want.Evaluate(after)
if err != nil {
panic(err) // invalid expectation or malformed observation: no verdict
}
fmt.Println(result.Matched)
fmt.Println(result)
}
Output: false mgrs want available=true active mgr.b gid other than 4100 in available=false active mgr.a gid 4100 standbys [b]: available: want true, observed false; active: want mgr.b, observed mgr.a; active gid: want other than 4100, observed 4100
type MonitorEvaluation ¶
type MonitorEvaluation struct {
Expectation Monitors
Observed ceph.QuorumStatus
Mismatches []MonitorMismatch
Matched bool
}
MonitorEvaluation is the result of evaluating one observation. Expectation and Observed are copies of what was evaluated, so the diagnostics describe that observation even when the caller later reuses its slices. Mismatches list the reasons without a MON name first, in the order of the reason constants, then the per-member reasons ordered by name and then in the order of the reason constants; Matched means there are none.
func (MonitorEvaluation) String ¶
func (e MonitorEvaluation) String() string
String summarizes the expectation, the observed membership and at most eight mismatches.
type MonitorMismatch ¶
type MonitorMismatch struct {
Reason MonitorMismatchReason
Name string
FSID, ObservedFSID string
Count, ObservedCount int
}
MonitorMismatch is one reason an observation does not satisfy a Monitors expectation. Name is the MON for the per-member reasons and empty for the others. FSID and ObservedFSID are the pinned and the observed FSID for MonitorMismatchFSID. Count and ObservedCount are the expected and the observed number for MonitorMismatchMemberCount and MonitorMismatchQuorumSize.
func (MonitorMismatch) String ¶
func (m MonitorMismatch) String() string
String names the reason and the evaluated values.
type MonitorMismatchReason ¶
type MonitorMismatchReason string
MonitorMismatchReason names why an observation does not satisfy a Monitors expectation.
const ( // MonitorMismatchFSID: the observation is of a cluster with another FSID. MonitorMismatchFSID MonitorMismatchReason = "fsid" // MonitorMismatchMemberCount: the monmap lists another number of members. MonitorMismatchMemberCount MonitorMismatchReason = "member-count" // MonitorMismatchQuorumSize: the quorum holds another number of members. MonitorMismatchQuorumSize MonitorMismatchReason = "quorum-size" // MonitorMismatchAbsent: a name of Members or Quorum is not in the monmap. // Its quorum membership is not reported separately. MonitorMismatchAbsent MonitorMismatchReason = "absent" // MonitorMismatchUnexpected: the monmap lists a name that Members does not. MonitorMismatchUnexpected MonitorMismatchReason = "unexpected" // MonitorMismatchOutOfQuorum: a name of Quorum is in the monmap but not in // the quorum. MonitorMismatchOutOfQuorum MonitorMismatchReason = "out-of-quorum" // MonitorMismatchInQuorum: the quorum holds a name that Quorum does not. MonitorMismatchInQuorum MonitorMismatchReason = "in-quorum" )
The reasons a MonitorEvaluation reports.
type Monitors ¶
type Monitors struct {
FSID string
Members []string
MemberCount int
Quorum []string
QuorumSize int
}
Monitors states the monmap membership and quorum a test expects in one ceph.Observer.QuorumStatus observation. Every set field is a condition and a zero field is not checked; at least one must be set.
FSID pins the cluster: a QuorumStatus read from another cluster, such as a rebuilt one, does not match. Members is the exact set of MON names the monmap must list and MemberCount the number of members it must list, for a test whose members get new names. Quorum is the exact set of MON names the quorum must hold and QuorumSize the number it must hold. Names compare as sets, so their order is irrelevant.
A match proves only what that one quorum_status answer reported: it does not prove that a member outside the quorum is stopped, that one inside it accepts new sessions, or which MON leads. Unlike ceph.Observer.WaitForQuorum, which accepts any majority of the current monmap, an expectation names the members and the quorum that the test requires.
func (Monitors) Evaluate ¶
func (e Monitors) Evaluate(observed ceph.QuorumStatus) (MonitorEvaluation, error)
Evaluate checks one successful ceph.Observer.QuorumStatus observation against the expectation. It reads only its arguments: no query, wait or retry, so a test composes it with its own polling or assertion helpers. A mismatch is a result; an error means no verdict.
An expectation that sets no field, a negative count, an empty or repeated name, an unparsable FSID, a Quorum name outside Members, a count that contradicts its name list, or a quorum that is not a majority of the expected members is ceph.ErrInvalidRequest: no observation could match it. An observation with no monmap member, an empty or repeated name, an empty quorum, a quorum name outside the monmap or an unparsable FSID is a malformed observation, never a mismatch. The result shares no slice with its arguments.
Example ¶
A stopped MON leaves the quorum but stays in the monmap, and the evaluation names it.
package main
import (
"encoding/json"
"fmt"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph/expect"
)
func main() {
var observed ceph.QuorumStatus // as observer.QuorumStatus decodes it
if err := json.Unmarshal([]byte(`{"quorum_names":["a","b"],"monmap":{"fsid":"44444444-4444-4444-8444-444444444444",
"mons":[{"name":"a"},{"name":"b"},{"name":"c"}]}}`), &observed); err != nil {
panic(err)
}
want := expect.Monitors{
FSID: "44444444-4444-4444-8444-444444444444",
Members: []string{"a", "b", "c"},
Quorum: []string{"a", "b", "c"},
}
result, err := want.Evaluate(observed)
if err != nil {
panic(err) // invalid expectation or malformed observation: no verdict
}
fmt.Println(result.Matched)
fmt.Println(result)
}
Output: false mons want fsid 44444444-4444-4444-8444-444444444444 members [a b c] quorum [a b c] in monmap [a b c] quorum [a b]: mon.c out of quorum
type OSDEvaluation ¶
type OSDEvaluation struct {
Expectation OSDs
Observed []ceph.OSDState
Mismatches []OSDMismatch
Matched bool
}
OSDEvaluation is the result of evaluating one observation. Expectation and Observed are copies of what was evaluated, so the diagnostics describe that observation even when the caller later reuses its slices. Mismatches are ordered by OSD ID, then in the order of the reason constants; Matched means there are none.
func (OSDEvaluation) String ¶
func (e OSDEvaluation) String() string
String summarizes the expectation and at most eight mismatches.
type OSDMismatch ¶
type OSDMismatch struct {
Reason OSDMismatchReason
ID int
UUID string
Observed ceph.OSDState
}
OSDMismatch is one reason one OSD does not satisfy an expectation. UUID is the selected UUID, empty for a selection by ID alone and for OSDMismatchUnselected. Observed is the OSD's state in the evaluated observation and the zero value for OSDMismatchAbsent. For OSDMismatchUp and OSDMismatchIn the expected flag is the negation of the observed one.
func (OSDMismatch) String ¶
func (m OSDMismatch) String() string
String names the OSD, the reason and the evaluated values.
type OSDMismatchReason ¶
type OSDMismatchReason string
OSDMismatchReason names why one OSD does not satisfy an OSDs expectation.
const ( // OSDMismatchAbsent: a selected ID is not in the observation. OSDMismatchAbsent OSDMismatchReason = "absent" // OSDMismatchReplaced: the ID now carries a different native UUID than the // selection pinned. Up and In of the replacement are not compared. OSDMismatchReplaced OSDMismatchReason = "replaced" // OSDMismatchUp: the reported up flag differs from the expectation. OSDMismatchUp OSDMismatchReason = "up" // OSDMismatchIn: the reported in flag differs from the expectation. OSDMismatchIn OSDMismatchReason = "in" // OSDMismatchUnselected: an Exact expectation found an owned OSD that the // selection does not name. OSDMismatchUnselected OSDMismatchReason = "unselected" )
The reasons an OSDEvaluation reports.
type OSDs ¶
OSDs states the OSDMap membership a test expects of selected owned OSDs. Select names the OSDs by ID and, when UUID is set, by native UUID: states from an earlier ceph.Observer.OSDStates baseline pin the exact registrations, so an OSD that a replacement re-registered under the same ID does not match. Only ID and UUID of a Select entry are read. Up and In are the reported OSDMap flags every selected OSD must have; they are not proof of process liveness or client reachability. Exact also requires the observation to contain no owned OSD outside the selection.
Unlike ceph.Observer.WaitForClean, which compares native OSD counts from the cluster status, an expectation checks individual identities, so it can tell which OSD is missing, replaced, down or out.
func (OSDs) Evaluate ¶
func (e OSDs) Evaluate(observed []ceph.OSDState) (OSDEvaluation, error)
Evaluate checks one successful ceph.Observer.OSDStates observation against the expectation. It reads only its arguments: no query, wait or retry, so a test composes it with its own polling or assertion helpers. A mismatch is a result; an error means no verdict. An empty or duplicate selection or an unparsable selected UUID is ceph.ErrInvalidRequest, and an observation with a negative or repeated ID or an unparsable UUID is a malformed observation, never a mismatch. The result shares no slice with its arguments.
Example ¶
An evaluation reads only the observation it is given, so its diagnostics name the field that failed in that observation.
package main
import (
"fmt"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph/expect"
)
func main() {
baseline := []ceph.OSDState{
{ID: 0, UUID: "11111111-1111-4111-8111-111111111111", Up: true, In: true, Weight: 1},
{ID: 1, UUID: "22222222-2222-4222-8222-222222222222", Up: true, In: true, Weight: 1},
}
later := []ceph.OSDState{
baseline[0],
{ID: 1, UUID: "22222222-2222-4222-8222-222222222222", Up: false, In: true, Weight: 1},
}
result, err := expect.OSDs{Select: baseline, Up: true, In: true, Exact: true}.Evaluate(later)
if err != nil {
panic(err) // invalid expectation or malformed observation: no verdict
}
fmt.Println(result.Matched)
fmt.Println(result)
}
Output: false osds [0 1] want up=true in=true exact in 2 observed: osd.1 up: want true, observed false
Example (Polling) ¶
The test owns the polling: a failed query is kept apart from the last mismatch, so a timeout reports both instead of an empty "not ready". This example is compiled, not run.
package main
import (
"context"
"fmt"
"time"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph/expect"
)
func main() {
ctx := context.Background()
cluster, err := ceph.Run(ctx, ceph.DefaultImage)
if cluster != nil {
defer func() { _ = cluster.Terminate(context.Background()) }()
}
if err != nil {
panic(err)
}
observer, err := ceph.Observe(cluster)
if err != nil {
panic(err)
}
baseline, err := observer.OSDStates(ctx)
if err != nil {
panic(err)
}
// ... stop and restart an OSD while the SUT runs ...
want := expect.OSDs{Select: baseline, Up: true, In: true, Exact: true}
ctx, cancel := context.WithTimeout(ctx, 2*time.Minute)
defer cancel()
var last expect.OSDEvaluation
var queryErr error
for {
states, err := observer.OSDStates(ctx)
if err == nil {
last, err = want.Evaluate(states)
}
if err == nil && last.Matched {
return
}
if err != nil {
queryErr = err
}
select {
case <-ctx.Done():
panic(fmt.Sprintf("OSDs not restored: last %v, last error %v", last, queryErr))
case <-time.After(time.Second):
}
}
}
Output:
type PGEvaluation ¶
type PGEvaluation struct {
Expectation PGs
Observed ceph.PoolPGSnapshot
Mismatches []PGMismatch
Matched bool
}
PGEvaluation is the result of evaluating one pool PG observation. Expectation and Observed are deep copies of what was evaluated, so the diagnostics describe that observation even when the caller later reuses its slices. Mismatches list PGMismatchFSID, PGMismatchPool and PGMismatchCount first, then the PGs by numeric PG number, each PG's reasons in the order of the reason constants and, within one reason, in the order of the expectation's list; Matched means there are none.
func (PGEvaluation) String ¶
func (e PGEvaluation) String() string
String summarizes the expectation and at most eight mismatches.
type PGMismatch ¶
type PGMismatch struct {
Reason PGMismatchReason
PGID ceph.PGID
Want, Observed string
}
PGMismatch is one reason a pool PG observation does not satisfy an expectation. PGID names the PG, empty for PGMismatchFSID, PGMismatchPool and PGMismatchCount. Want and Observed are the compared values as text: the token for PGMismatchRequired and PGMismatchForbidden and the OSD ID for the missing and present reasons in Want, with the reported state or vector in Observed. Both are empty for PGMismatchAbsent.
func (PGMismatch) String ¶
func (m PGMismatch) String() string
String names the PG, the reason and the evaluated values.
type PGMismatchReason ¶
type PGMismatchReason string
PGMismatchReason names why a pool PG observation does not satisfy a PGs expectation.
const ( // PGMismatchFSID: the observation belongs to another cluster. PGMismatchFSID PGMismatchReason = "fsid" // PGMismatchPool: the observation reports another pool name, or the // pinned name now carries another native pool ID. PGMismatchPool PGMismatchReason = "pool" // PGMismatchCount: the pool reports another number of PGs. PGMismatchCount PGMismatchReason = "count" // PGMismatchAbsent: a selected PG is not reported. Its conditions are not // compared. PGMismatchAbsent PGMismatchReason = "absent" // PGMismatchState: the PG reports another exact state. PGMismatchState PGMismatchReason = "state" // PGMismatchRequired: the PG state lacks a required token. PGMismatchRequired PGMismatchReason = "required" // PGMismatchForbidden: the PG state carries a forbidden token. PGMismatchForbidden PGMismatchReason = "forbidden" // PGMismatchUp: the PG reports another up vector. PGMismatchUp PGMismatchReason = "up" // PGMismatchActing: the PG reports another acting vector. PGMismatchActing PGMismatchReason = "acting" // PGMismatchActingSize: the acting vector has another length. PGMismatchActingSize PGMismatchReason = "acting size" // PGMismatchUpMissing: the up vector lacks an OSD of InUp. PGMismatchUpMissing PGMismatchReason = "up missing" // PGMismatchUpPresent: the up vector contains an OSD of NotInUp. PGMismatchUpPresent PGMismatchReason = "up present" // PGMismatchActingMissing: the acting vector lacks an OSD of InActing. PGMismatchActingMissing PGMismatchReason = "acting missing" // PGMismatchActingPresent: the acting vector contains an OSD of // NotInActing. PGMismatchActingPresent PGMismatchReason = "acting present" // PGMismatchPrimary: the acting primary is not one of ActingPrimary. PGMismatchPrimary PGMismatchReason = "primary" )
The reasons a PGEvaluation reports.
type PGs ¶
type PGs struct {
FSID string
Pool ceph.PoolState
Select []ceph.PGID
Count int
State string
Require, Forbid []string
Up, Acting []int
ActingSize int
InUp, NotInUp, InActing, NotInActing []int
ActingPrimary []int
}
PGs states the placement groups a test expects in one ceph.Pool.PGs observation of one pool.
FSID, when set, pins the cluster. Only Name and ID of Pool are read: Name, when set, must be the observed pool name, and ID, when nonzero, pins the native pool ID (Ceph numbers pools from 1), so a pool that was removed and recreated under the same name does not match. A PoolState from an earlier observation, such as the PoolAfter of a baseline, pins both.
Select names the PGs to check; when it is empty every reported PG is checked. Count, when nonzero, is the exact number of PGs the pool must report. An expectation selects no PG unless it sets Count or Select, so an empty report never matches by having nothing to check.
Each checked PG must satisfy every condition that is set:
- State, when set, is the exact native state string, such as "active+clean"; "active+clean+scrubbing" does not match it.
- Require and Forbid list native state tokens (the parts of the state between "+") the PG must and must not report, such as "active" or "degraded". A token matches only whole: "backfill" does not match "backfill_wait".
- Up and Acting, when non-nil, are the exact ordered vectors the PG must report, including missing EC shard slots (2147483647); an empty non-nil vector requires an empty report.
- ActingSize, when nonzero, is the length of the acting vector.
- InUp and InActing list OSD IDs the vector must contain; NotInUp and NotInActing OSD IDs it must not contain. NotInActing can list 2147483647 to forbid a missing EC shard slot.
- ActingPrimary, when non-empty, lists the reported acting primaries that are acceptable; -1 is the native value for no primary.
A match proves only what this one observation reported: the PG reports arrive from the MGR asynchronously, are not an atomic placement snapshot and are no read-after-write barrier, and the native PGReady flag is not compared. Unlike ceph.Observer.WaitForClean, which waits for the whole cluster to report clean, an expectation checks one pool's PGs once and names each PG and condition that differs.
func (PGs) Evaluate ¶
func (e PGs) Evaluate(observed ceph.PoolPGSnapshot) (PGEvaluation, error)
Evaluate checks one successful ceph.Pool.PGs observation against the expectation. It reads only its arguments: no query, wait or retry, so a test composes it with its own polling or assertion helpers. A mismatch is a result; an error means no verdict. An expectation that selects no PG (neither Count nor Select), a negative count, size, pool ID or OSD ID, an invalid FSID, PGID or state token, a repeated entry, a selected PG of another pinned pool, and a contradiction (more selected PGs than Count, a token both required and forbidden or disagreeing with State, an OSD both wanted and forbidden or disagreeing with Up or Acting, an ActingSize other than the length of Acting) are ceph.ErrInvalidRequest. An observation whose two pool reads disagree on name or ID, a PG of another pool or with an invalid or repeated PGID, an empty state, a negative OSD in a vector, a primary below -1, or an invalid FSID when the expectation pins one is a malformed observation, never a mismatch. The result shares no slice with its arguments.
Example ¶
A PG expectation pins the pool of a baseline, so a recreated pool of the same name does not pass, and names each PG and condition that differs.
package main
import (
"fmt"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph/expect"
)
func main() {
pool := ceph.PoolState{ID: 3, Name: "orders", Type: "replicated", Size: 2, MinSize: 1, PGNum: 2}
observed := ceph.PoolPGSnapshot{
FSID: "44444444-4444-4444-8444-444444444444",
PoolBefore: pool, PoolAfter: pool,
PGs: []ceph.PGState{
{PGID: "3.0", State: "active+clean", Up: []int{0, 1}, Acting: []int{0, 1}, UpPrimary: 0, ActingPrimary: 0},
{PGID: "3.1", State: "active+undersized+degraded", Up: []int{2}, Acting: []int{2}, UpPrimary: 2, ActingPrimary: 2},
},
}
// Every PG of the baseline pool is clean on two OSDs, none on osd.2.
want := expect.PGs{Pool: pool, Count: 2, State: "active+clean", ActingSize: 2, NotInActing: []int{2}}
result, err := want.Evaluate(observed)
if err != nil {
panic(err) // invalid expectation or malformed observation: no verdict
}
fmt.Println(result.Matched)
for _, m := range result.Mismatches {
fmt.Println(m)
}
}
Output: false pg 3.1 state: want active+clean, observed active+undersized+degraded pg 3.1 acting size: want 2, observed [2] pg 3.1 acting present osd.2: observed [2]