compat

compat

The two-layer compatibility gate for the pure-Go port:

1. Committed vectors (vectors/*.json) — deterministic test vectors generated by the Rust harness (rust-harness/, pinned to upstream libsignal v0.96.4) and committed to the repo. vectors_test.go consumes them under a plain go test ./compat/ — no Rust toolchain is needed at test time, because the vectors are checked-in files.
2. Live interop (added in T13) — a CI job that runs the Go code against the harness binary over JSON-RPC.

This directory covers layer 1.

Vectors

Each curve signing case and each sender_key_message case records the 64-byte XEdDSA signing nonce, so the Go consumer can reproduce the signature byte-for-byte (XEdDSA draws a random nonce; replaying it makes signing deterministic).

Regenerating the vectors

The vectors are regenerated by the Rust harness. Building the harness requires protoc (the Protocol Buffers compiler): upstream's build.rs compiles .proto files via prost-build 0.14, which does not vendor protoc. Install it first:

• macOS: brew install protobuf
• Debian/Ubuntu: apt-get install -y protobuf-compiler

See rust-harness/README.md for full harness/toolchain details.

# from compat/
cargo build --release --manifest-path rust-harness/Cargo.toml
BIN=rust-harness/target/release/rust-harness
for d in curve kem-decaps hkdf messages fingerprint sessions groups sealedsender username-links; do
  "$BIN" gen-vectors "$d" > "vectors/$d.json"
done

Generation is deterministic (seeded ChaCha20, seed recorded in each file's header), so regenerating without changing the harness produces byte-identical files.

What vectors_test.go asserts

coverage_manifest.json records which Signal Workflow-adjacent domains are vector-backed in this module and which are explicitly deferred to proof-system work. ProofInventory() exposes the same data to Go callers with stable domain, status, reason, and next-upstream-input fields. The manifest is tested so automation cannot forget to regenerate harness-backed domains such as username-links when the upstream pin changes, and so non-harness vectors such as account-keys stay explicitly tracked.

• curve — Go verifies each upstream signature; Go re-signs with the recorded nonce and matches the upstream signature bytes; Go ECDH equals the upstream shared secret (and is symmetric).
• kem-decaps — Go Decapsulate(sk, ct) equals the upstream shared secret for every triple. This closes design assumption A1 (circl round-3 Kyber == upstream libcrux Kyber).
• hkdf — Go reproductions of the chain-key, message-keys, root-key, and pqxdh-secret derivations equal upstream for all four sub-domains. The Go ratchet/ package does not exist yet (T14, PR 5); the derivations are implemented in the test against crypto/{hkdf,hmac,sha256}, with formulas from rust/protocol/src/ratchet/keys.rs. When T14 lands they move into ratchet/ and these committed vectors become its contract.
• messages — all four wire message types (SignalMessage, PreKeySignalMessage, SenderKeyMessage, SenderKeyDistributionMessage) are consumed both directions: deserialize the upstream golden bytes and check field equality, then re-serialize and check byte equality with upstream. SenderKeyMessage re-signs by replaying the recorded nonce; SignalMessage rebuilds from the recorded mac key + identities (its MAC is keyed and cannot be recovered from the message); PreKeySignalMessage re-wraps its deserialized inner SignalMessage. The PreKeySignalMessage cases carry a Kyber payload, as a v4 message requires.
• fingerprint — the Go fingerprint package does not exist yet (T25); the vectors are asserted to parse with cases present, with a TODO(T25) consuming test.
• sessions — the PQXDH master-secret derivation for the no-one-time-prekey case (DH4 absent): Go re-derives the secret as 0xFF*32 ‖ DH1 ‖ DH2 ‖ DH3 ‖ kyber_ss (one agreement shorter than the with-DH4 secret) and ratchet.DeriveInitialKeys reproduces upstream's root/chain/pqr. This is the committed-vector counterpart to the live no-OPK session interop; the hkdf.json pqxdh-secret sub-domain only covers the with-DH4 path.
• username-links — Go creates the same IV || ciphertext || HMAC bytes from the recorded username, entropy, and IV as upstream rust/usernames, computes the same username reservation hash, then decrypts the bytes back to the recorded username. This covers the username-link and reserve-hash surfaces without claiming support for username proofs, which remain tied to the deferred zk/poksho phase.

Live interop (session_interop_test.go, interop build tag)

The session interop suite drives a full PQXDH / Double Ratchet conversation between the Go session layer and the genuine upstream session API (v0.96.4) over the harness JSON-RPC loop. It covers both role assignments (Go=Alice/Rust=Bob and Rust=Alice/Go=Bob), with and without a one-time pre-key (the no-OPK case is the cross-impl proof of the optional-DH4 path), a 20-message exchange with out-of-order + skipped-key delivery, and a serialize→reload→continue persistence check. Run it with the harness binary:

cargo build --release --manifest-path rust-harness/Cargo.toml
COMPAT_HARNESS_BIN=$(pwd)/rust-harness/target/release/rust-harness \
  go test ./ -tags=interop -run Session -v   # from compat/

Documented limitation: no v3 session decrypt vectors

v0.96.4 sessions are PQXDH/v4 only. X3DH/v3 is removed upstream: every session is created at CIPHERTEXT_MESSAGE_CURRENT_VERSION (4), and the decrypt path rejects a v3 PreKey message outright ("X3DH no longer supported"). The public API therefore cannot produce a v3 session or a v3 SignalMessage, so a v3-format decrypt-vector suite is not achievable against the pinned upstream. The v4 interop above is the full session surface v0.96.4 supports. At v0.96.4 the Sparse Post-Quantum Ratchet (SPQR) is mandatory for new sessions (min_version: V1), so every SignalMessage in the interop carries a non-empty pq_ratchet field — the suite asserts that both directions, proving the Go port mixes the SPQR key correctly cross-impl (the T28 triple ratchet).

The Go port's own v3 decrypt-compat capability (the version-floor=3 deserialize path) is unit-tested directly — see protocol/signal_message_test.go:TestSignalMessageVersionFloorAndCeiling, which deserializes a v3-version-byte SignalMessage and asserts it is accepted (with v2 below the floor and v5 above the ceiling rejected). That capability exists and is covered; it simply cannot be cross-checked against the v0.96.4 harness, because the harness's upstream cannot emit a v3 message.

• groups — groups.json carries two arrays. The cases array drives the full sender-key cipher both directions: Go Encrypt from the recorded pre-encrypt sending record (replaying the recorded signing nonce) must match the upstream SenderKeyMessage bytes, and Go ProcessSenderKeyDistributionMessage + Decrypt of the upstream SKM recovers the recorded plaintext. The derivations array is the byte-exact sender-key primitive KAT — upstream chain-key ratchet (HMAC(chain_key,[0x02])) and senderMessageKey iv/cipher_key (HKDF(HMAC(chain_key,[0x01]), "WhisperGroup")) — consumed by groups/derivation_kat_test.go (a white-box test, since those primitives are unexported), pinning the primitive independently of the cipher wire framing, the same pattern ratchet/ uses against hkdf.json.

The live interop leg (interop_test.go, -tags=interop) additionally drives the group cipher both directions against the running harness: Go-distribute + encrypt -> Rust process + decrypt (group.process_distribution / group.decrypt), and Rust-distribute + encrypt -> Go process + decrypt (group.create_distribution / group.encrypt). The harness threads the serialized SenderKeyRecord through each call so its dispatch stays stateless.

• account-keys — account-keys.json carries upstream v0.96.4 known values for account entropy parsing, SVR key derivation, backup key and backup ID derivation, Argon2id PIN hash access keys, and Argon2i PHC local PIN hash verification.