README
¶
pkg/keccak — SIMD Keccak-256 binaries
This package wraps the SIMD-accelerated, legacy Keccak-256 (Ethereum-compatible,
0x01 padding suffix — not FIPS 202 SHA3-256) primitives produced by
XKCP. The actual permutation code is shipped as
two pre-linked, relocation-free .syso blobs that the Go linker pulls in
directly — no CGO, no toolchain dependency at go build time:
keccak_times4_linux_amd64.syso— AVX2, 4-way parallelkeccak_times8_linux_amd64.syso— AVX-512, 8-way parallel
The .syso files are checked in and verified against CHECKSUM by
TestSysoChecksums in keccak_test.go, so bee builds reproducibly without
the XKCP toolchain. Re-build the blobs only when XKCP itself changes (or when
porting to a new platform).
Rebuilding the syso files
The build lives in our XKCP fork — the upstream repo does not carry the Go bindings out of the box.
1. Check out the fork
git clone https://github.com/ethersphere/XKCP.git ~/repos/XKCP
cd ~/repos/XKCP
git submodule update --init
(If you already have a clone at ~/repos/XKCP, just git pull.)
2. Install the build dependencies
The build script needs a system C toolchain and xsltproc (XKCP's own build
system uses XSLT for Makefile generation):
# Debian/Ubuntu
sudo apt-get install build-essential xsltproc
# Arch / Manjaro
sudo pacman -S base-devel libxslt
# macOS
brew install xsltproc
3. Run the build
From the root of the XKCP checkout:
./build_go_asm.sh
This script (described in detail at the top of the file) does the work in
six steps: builds the XKCP AVX2/AVX-512 libraries, compiles C wrappers that
adapt the XKCP API to a Go-friendly layout, fully links each combined object
to remove every relocation, re-wraps the result as an ET_REL ELF, and emits
both .syso blobs and the matching Plan 9 assembly glue under
~/repos/XKCP/go_keccak/.
4. Copy the artifacts into bee
The build script writes files named keccak_times4_amd64.syso /
keccak_times8_amd64.syso. We rename them on the way in so Go's build
constraints scope them to linux/amd64 (!linux builds must not link these
blobs):
KECCAK=/home/acud/go/src/github.com/ethersphere/bee/pkg/keccak
cp ~/repos/XKCP/go_keccak/keccak_times4_amd64.syso \
"$KECCAK/keccak_times4_linux_amd64.syso"
cp ~/repos/XKCP/go_keccak/keccak_times8_amd64.syso \
"$KECCAK/keccak_times8_linux_amd64.syso"
The Plan 9 assembly stubs (keccak_times{4,8}_linux_amd64.s) and the Go
wrappers are hand-maintained in this directory; they should not need
re-generation unless the XKCP entry-symbol names change.
5. Refresh CHECKSUM and run the guard test
The CHECKSUM file in this directory pins the SHA-256 of each .syso. Update
it whenever the blobs are regenerated:
cd "$KECCAK"
sha256sum keccak_times4_linux_amd64.syso keccak_times8_linux_amd64.syso > CHECKSUM
go test ./...
TestSysoChecksums parses CHECKSUM, recomputes the digests of the on-disk
.syso files, and fails the build if either drifts — that's the CI tripwire
that catches an accidental rebuild or a corrupted check-in.
Documentation
¶
Overview ¶
Package keccak provides legacy Keccak-256 (Ethereum-compatible) hashing with SIMD acceleration via XKCP.
Sum256x4 uses AVX2 (4-way parallel). Sum256x8 uses AVX-512 (8-way parallel). The caller is responsible for dispatching to the correct function based on CPU capabilities; Sum256x8 will crash on non-AVX-512 hardware.
Platform availability ¶
Sum256x4 and Sum256x8 are only compiled on linux/amd64 (with the !purego build tag). On every other platform (darwin, windows, linux/arm64, or any build configured with -tags=purego) the symbols do not exist; callers must route around them using the HasSIMD / BatchWidth helpers below, which are available on all platforms and return false / 0 where no SIMD hasher is linked in.
Equal-length (or nil) input constraint ¶
All non-nil inputs passed to Sum256x4 / Sum256x8 in the same call MUST have the same length. This is an intrinsic limit of the batched XKCP primitive: KeccakP1600timesN_PermuteAll_24rounds advances the sponge state of every lane in lockstep, so any lane that has already absorbed its final padding block gets extra, unwanted permutations every time a longer lane absorbs another block — silently corrupting its output.
Nil (or zero-length) lanes are allowed for partial batches: they are treated as no-op fillers so the caller can populate N_real < N lanes with real data and ignore the remaining N - N_real output digests. The digests produced for nil lanes are not meaningful and must not be consumed. Mixing distinct non-zero lengths within one call is unsupported and will produce wrong digests for the shorter lanes, even when every individual length would work on its own in an all-same-length call.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BatchWidth ¶
func BatchWidth() int
BatchWidth returns the SIMD batch width: 8 for AVX-512, 4 for AVX2, or 0 if no SIMD acceleration is available.
Types ¶
type Hash256 ¶
type Hash256 [32]byte
Hash256 represents a 32-byte Keccak-256 hash
func Sum256x4 ¶
Sum256x4 computes 4 Keccak-256 hashes in parallel using AVX2.
All non-nil inputs MUST have the same length; nil (or zero-length) lanes are allowed as partial-batch fillers whose output must be ignored. See the package doc for the rationale.
Only compiled on linux/amd64 (the only platform where the XKCP .syso is linkable). Call sites that may run on other platforms must be gated on the same build tag or on keccak.HasSIMD() at runtime.
func Sum256x8 ¶
Sum256x8 computes 8 Keccak-256 hashes in parallel using AVX-512. Must only be called on AVX-512-capable hardware.
All non-nil inputs MUST have the same length; nil (or zero-length) lanes are allowed as partial-batch fillers whose output must be ignored. See the package doc for the rationale.
Only compiled on linux/amd64 (the only platform where the XKCP .syso is linkable). Call sites that may run on other platforms must be gated on the same build tag or on keccak.HasAVX512() at runtime.
Source Files