Documentation
¶
Overview ¶
Package gate runs the trained F5 steganalysis CNN models in pure Go (no cgo), over the standalone ONNX engine github.com/0verkilll/onnx.
It is the neural complement to the detect package's Fridrich histogram attack and JPEG-format signatures: where those are analytic, gate runs a learned detector that is far stronger at moderate embed rates (β≥0.1 ⇒ ~90–94% TPR). It is a SEPARATE, heavier package on purpose — importing it pulls in the ONNX engine and (via gate/embedded) a multi-megabyte model, so the light embed/extract/encode/detect paths never compile it.
Single-image inference flow:
- DetectJPEG takes raw JPEG bytes (progressive inputs are losslessly transcoded to baseline first).
- decodeCoeffs + extractGrid produce the 65,536-int16 luma DCT coefficient grid (train/inference parity is guaranteed at the extractor level).
- buildInputTensor formats the grid into whichever wire the loaded .onnx declares (flat-int16, one-hot, or multi-stream), encoding the one-hot / stream features Go-side because the engine does not implement OneHot / Round / AvgPool.
- engine.Run dispatches the graph; the two trained outputs (logits_detect, beta_raw) are softmaxed to P(stego) and inverted to the F5 modification rate β.
This is the Phase-1 detection pipeline (F5_DETECTED / CLEAN / SKIP). The Phase-2 recompression gate (ARTIFACT routing for lossy web re-saves) is not yet wired here — see VerdictArtifact.
A Detector is reusable but NOT safe for concurrent use across goroutines; construct one per goroutine or guard with a mutex.
Index ¶
- Constants
- Variables
- type Bundle
- type Detector
- func (d *Detector) BuildCalTensor(stego, cover []int16) (*tensor.Tensor, error)
- func (d *Detector) BuildInputTensor(coeffs []int16) (*tensor.Tensor, error)
- func (d *Detector) Close() error
- func (d *Detector) DetectCoeffs(coeffs []int16) (Result, error)
- func (d *Detector) DetectCoeffsCal(stego, cover []int16) (Result, error)
- func (d *Detector) DetectJPEG(data []byte) (Result, error)
- func (d *Detector) InferShared(inT *tensor.Tensor, wire string) (res Result, ok bool, err error)
- func (d *Detector) NeedsCover() bool
- func (d *Detector) Wire() string
- type EnsembleDetector
- type EnsembleResult
- type NamedScore
- type Option
- type Result
- type ScanResult
- type Verdict
Constants ¶
const DefaultThreshold = 0.5
DefaultThreshold is the conventional P(stego) cutoff for the binary verdict — the BALANCED-benchmark operating point (50/50 clean:stego). It is the WRONG point for real-world use, where almost every image is clean and any false-positive rate dominates — see SingleModelThreshold / DeploymentThreshold.
const DeploymentThreshold = 0.6
DeploymentThreshold is the high-precision P(stego) cutoff for the multi-member ensemble, where the base rate of F5 stego is ~0% and false positives must be rare. At 0.6 the clean false-positive rate is ~1.7% (COCO) / ~0% (web) while heavy-embed recall holds. It applies to the ENSEMBLE; a single model is not calibrated enough for any safe threshold (see SingleModelThreshold).
const F5T = 0.71
F5T is the redesign_v5 detection operating point (m_student_m / the v6 ensemble): p_stego ≥ this ⇒ F5_DETECTED, calibrated to a 1% false-positive rate. It is the Stage-2 threshold of the 2-stage bundle pipeline.
const NCoeffs = 65536
NCoeffs is the model's fixed input size: a 32x32 grid of 8x8 DCT blocks, 64 frequencies per block (32 * 32 * 64 = 65536). The training pipeline emits exactly this many int16 coefficients per sample, so inference must match. It corresponds to a luminance image of at least 256x256.
const SingleModelThreshold = 0.58
SingleModelThreshold is the operating cutoff for the embedded single model — the soft-label-distilled student, which IS calibrated. Measured on real clean images: clean p_stego peaks ≈0.61, so at 0.58 the false-positive rate is ~0.5% (COCO) / ~3% (web) while recall(β>0.1) holds ≈0.75. This single model is embeddable on every platform (incl. WASM).
Variables ¶
var ( // ErrWeightFileMissing is returned by NewDetector when the path it is asked // to load does not exist, or by NewDetectorFromBytes when given empty bytes. ErrWeightFileMissing = errors.New("gate: weight file not found") // ErrInvalidInput is returned by DetectJPEG / DetectCoeffs when the input // cannot be decoded into a 256x256-luma-minimum baseline JPEG, or when the // loaded graph's input name is unrecognised. ErrInvalidInput = errors.New("gate: invalid JPEG input") // ErrModelOutput is returned when the engine produced no value for the // expected output name — typically a sign the .onnx was exported without the // trained detect / beta heads. ErrModelOutput = errors.New("gate: model produced no detect / beta output") )
Sentinel errors so callers can match on errors.Is.
var ErrEmptyInput = errors.New("gate: empty JPEG input")
ErrEmptyInput is returned by decodeCoeffs when called with no bytes; an empty slice is never a valid JPEG and is rejected before the decoder is invoked.
var ErrTooFewCoefficients = errors.New("gate: too few usable coefficients")
ErrTooFewCoefficients is returned by extractGrid when the raw stream yields fewer than NCoeffs usable coefficients — the image is smaller than the 256x256-luma minimum the model requires.
Functions ¶
This section is empty.
Types ¶
type Bundle ¶
type Bundle struct {
// contains filtered or unexported fields
}
Bundle is the redesign_v5 detection pipeline: a recompression gate (Stage 1, OR-rule) in front of an F5 detector (Stage 2). The detector is either a single distilled model (Tier 1, m_student_m) or the full ensemble (Tier 2, mean of the luma + calibration + histogram members). Loaded from a directory of ONNX files + thresholds.
Tier-1 layout: detector.onnx Tier-2 layout: detect_luma_{0..}.onnx, detect_cal_{0..}.onnx,
detect_cooc.onnx + detect_cooc.meta.json
Both layouts: recompb_{a..e}.onnx, recompdet_sharp.onnx(+ .meta.json),
recompo_thresholds.json
A Bundle is NOT safe for concurrent use across goroutines (its detectors each own an engine pool); construct one per goroutine or guard with a mutex.
func LoadBundle ¶
LoadBundle loads a bundle directory, auto-detecting Tier-1 (detector.onnx) vs Tier-2 (detect_luma_0.onnx present). The gate models are optional (a detector-only bundle still scans, without ARTIFACT routing).
func LoadBundleFromBytes ¶
LoadBundleFromBytes builds a bundle from a name→bytes map (the .onnx / .json files keyed by base name). This backs both LoadBundle (from a directory) and the binary-embedded bundle (from go:embed).
func (*Bundle) NumDetectModels ¶
NumDetectModels reports the Stage-2 member count (1 for Tier-1).
func (*Bundle) NumGateModels ¶
NumGateModels reports how many Stage-1 gate models loaded (CNNs + cooc).
type Detector ¶
type Detector struct {
// contains filtered or unexported fields
}
Detector is a reusable inference handle owning one ONNX engine. Construction is the expensive step (parses + tensor-loads the weights); per-image inference thereafter is cheap.
Detector is NOT safe for concurrent use across goroutines — the underlying engine pool would race. Guard with a sync.Mutex or construct one Detector per goroutine.
func NewDetector ¶
NewDetector loads an .onnx model file and returns a ready Detector. The file is read in full at construction; subsequent inferences do not touch disk. Distinguish "wrong path" from "wrong content" with errors.Is(err, ErrWeightFileMissing).
func NewDetectorFromBytes ¶
NewDetectorFromBytes mirrors NewDetector but loads from an in-memory .onnx byte slice — used by the //go:embed path where the model ships inside the binary. No temporary file is written.
func (*Detector) BuildCalTensor ¶
BuildCalTensor exposes the cal-mode (2-plane) stream builder for the same shared-build purpose across the calibration members.
func (*Detector) BuildInputTensor ¶
BuildInputTensor exposes the stream-tensor builder so a bundle can build the shared single-plane tensor once (from a representative model) and feed it to every same-wire member via InferShared.
func (*Detector) Close ¶
Close releases the engine's CPU provider resources. After Close the Detector must not be used.
func (*Detector) DetectCoeffs ¶
DetectCoeffs is the lower-level entry point: classify from an already- extracted 65,536-element int16 DCT coefficient grid. Calibration models (NeedsCover) reject this path — use DetectCoeffsCal.
func (*Detector) DetectCoeffsCal ¶
DetectCoeffsCal classifies a calibration model from the stego grid and its crop-4 cover estimate (both 65,536-element int16). For a non-cal model it falls back to DetectCoeffs(stego).
func (*Detector) DetectJPEG ¶
DetectJPEG classifies one JPEG. data is the raw JPEG bytes (any size with luminance ≥ 256x256; only the top-left 256x256 block grid is used). Progressive inputs are losslessly transcoded to baseline first.
func (*Detector) InferShared ¶
InferShared runs the model on a PRE-BUILT input tensor when that tensor's wire matches this model, so identical-wire ensemble members share one tensor build. Returns ok=false when the wire differs (the caller builds per-model). The engine treats the input as read-only, so reuse across members is safe.
func (*Detector) NeedsCover ¶
NeedsCover reports whether this model is a calibration member that requires the crop-4 cover estimate (run it via DetectCoeffsCal, not DetectCoeffs).
type EnsembleDetector ¶
type EnsembleDetector struct {
// Available is true only when at least one member was loaded.
Available bool
// contains filtered or unexported fields
}
EnsembleDetector wraps a slice of *Detector and combines their per-image outputs via arithmetic mean (uniform-mean rule).
EnsembleDetector is NOT safe for concurrent use across goroutines — each member Detector owns an engine with the same constraint. Guard with a sync.Mutex or construct one EnsembleDetector per goroutine.
func NewEnsembleDetectorFromBytes ¶
func NewEnsembleDetectorFromBytes(members [][]byte, opts ...Option) (*EnsembleDetector, error)
NewEnsembleDetectorFromBytes loads members from in-memory .onnx byte slices. An empty or nil members slice returns Available=false with nil error. If any member fails, already-opened members are closed and the error is returned.
func NewEnsembleDetectorFromDir ¶
func NewEnsembleDetectorFromDir(dir string, opts ...Option) (*EnsembleDetector, error)
NewEnsembleDetectorFromDir loads all member .onnx files from dir. It first attempts to read manifest.json for ordered member names; if absent it falls back to glob member_0.onnx … member_4.onnx. If dir is empty or no .onnx files are found, the returned detector has Available=false and nil error — callers must check Available before using DetectJPEG. If any member fails to load, already-opened members are closed and the error is returned (fail-fast).
func (*EnsembleDetector) Close ¶
func (e *EnsembleDetector) Close() error
Close releases all member detectors in order. Errors from each member are joined and returned; a nil return means all closed cleanly.
func (*EnsembleDetector) DetectJPEG ¶
func (e *EnsembleDetector) DetectJPEG(data []byte) (EnsembleResult, error)
DetectJPEG classifies one JPEG using all member detectors and averages their outputs. When Available is false it returns EnsembleResult{Available: false} and nil error. If all members error, the first error is returned; if a subset error, those are skipped and the rest are averaged.
func (*EnsembleDetector) NumMembers ¶
func (e *EnsembleDetector) NumMembers() int
NumMembers reports how many member detectors were loaded.
type EnsembleResult ¶
type EnsembleResult struct {
MemberCount int
PStego float32
BetaEst float32
Threshold float32
IsStego bool
Available bool
}
EnsembleResult is the per-image verdict emitted by EnsembleDetector. PStego and BetaEst are arithmetic means over all member Results. IsStego is the threshold-applied binary call on the averaged PStego.
When Available is false the numeric fields are zero-valued and must not be used to make any inference decision — the model was not loaded.
type NamedScore ¶
type NamedScore struct {
Name string `json:"name"`
Kind string `json:"kind"` // luma|robust|cal|cooc|gate_cnn|gate_cooc
Score float64 `json:"score"`
Threshold float64 `json:"threshold,omitempty"`
Fired bool `json:"fired,omitempty"`
}
NamedScore is one model's raw output, exposed so the scoring system can be fine-tuned with full per-model visibility. For gate models Score is p(re-save) and Threshold/Fired apply; for detection members Score is p_stego.
type Option ¶
type Option func(*Detector)
Option configures a Detector at construction time.
func WithThreshold ¶
WithThreshold sets the P(stego) cutoff for IsStego. Values outside [0, 1] are clamped.
type Result ¶
Result is the per-image verdict the detector emits. PStego is the posterior probability from the trained detect head. BetaEst is the F5 modification-rate estimate from the beta head (only meaningful when PStego is high). IsStego is the threshold-applied binary call.
type ScanResult ¶
type ScanResult struct {
Verdict Verdict
Reason string
GateModels []NamedScore // every Stage-1 gate model's p(re-save)
DetectModels []NamedScore // every Stage-2 detection member's p_stego
PStego float64 // mean Stage-2 p_stego
BetaEst float64
GateScore float64 // max Stage-1 re-save score across gate models
Members int // Stage-2 ensemble member count
Detected bool
Gated bool
}
ScanResult is the bundle's per-image verdict, including every model's raw score (GateModels + DetectModels) for tuning.
type Verdict ¶
type Verdict string
Verdict is the human-facing call the neural gate emits for one image.
const ( // VerdictF5Detected means the detector's P(stego) met or exceeded its // threshold under a reliable input. VerdictF5Detected Verdict = "F5_DETECTED" // VerdictClean means the detector ran and P(stego) was below threshold. VerdictClean Verdict = "CLEAN" // VerdictSkip means the image could not be analysed — smaller than the // 256x256-luma minimum (264 px with the Phase-2 crop margin) or undecodable. VerdictSkip Verdict = "SKIP" // VerdictArtifact is reserved for the Phase-2 recompression gate: a lossy // web re-save whose second compression makes F5 unreliable. It is never // returned by the Phase-1 detection-only pipeline. VerdictArtifact Verdict = "ARTIFACT" )
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package embedded ships the redesign_v5 detection models inside the binary (compiled in with the standard library's embed package), so the full pipeline runs turnkey with no external files.
|
Package embedded ships the redesign_v5 detection models inside the binary (compiled in with the standard library's embed package), so the full pipeline runs turnkey with no external files. |