Documentation
¶
Overview ¶
Package uploads provides helpers for client-side-encrypted upload storage. Both formats it implements wrap a fresh AES-256 data encryption key with RSA-OAEP-SHA256 under the org's public key.
SealEnvelope / ParseEnvelope / OpenEnvelope are the v1 JSON envelope, which mirrors the browser's encryption.ts: one AES-GCM ciphertext base64-encoded into a JSON object, so the whole payload is in memory at once.
SealStream / OpenStream are the v2 binary stream envelope, for payloads too large to buffer. A header carries the format version, key version, wrapped key and chunk size, followed by AES-GCM chunks in the STREAM construction: each chunk's nonce derives from its index and a final-chunk flag, so a truncated, reordered or spliced stream fails to open rather than decrypting to a plausible prefix. NewCRC32CWriter gives a caller the ciphertext's CRC32C without buffering it.
Both Open functions accept an unwrap callback so an org's RSA private key can stay in KMS. OpenStream passes the stored CryptoKeyVersion to that callback, allowing uploads to remain readable after key rotation.
The stream nonce is a 7-byte random per-stream prefix, a 4-byte big-endian chunk index and a 1-byte final flag. The prefix defends against one thing, and it is not a live threat in this package: because SealStream generates a fresh DEK per stream, the index and the flag alone already make every nonce unique under that key, and the prefix adds nothing an attacker can reach. It is carried so that a caller who ever seals two streams under one DEK — a supplied key, a cached KMS key — gets a 56-bit birthday bound on nonce reuse instead of a guaranteed repeat on chunk 0. Read it as nothing else; in particular it is not what keeps chunks from moving between streams, which is the distinct DEK.
The 4-byte index is what the prefix costs: 2^32 chunks, 4 PiB at the 1 MiB chunk size, enforced on both the sealing and the opening side. Spending all 11 remaining bytes on a counter would lift that cap, and a cap is the point — 2^32 chunks of 1 MiB is 2^48 AES blocks under one DEK, which still sits inside AES-GCM's bounds for deterministic nonces, while an uncapped counter does not.
Index ¶
- Constants
- func OpenEnvelope(payload string, ...) ([]byte, error)
- func OpenStream(r io.Reader, ...) (io.Reader, error)
- func SealEnvelope(plaintext []byte, publicKeyPEM, algorithm, keyVersion string) (string, error)
- func SealStream(w io.Writer, orgPub *rsa.PublicKey, algorithm, keyVersion string) (io.WriteCloser, error)
- type CRC32CWriter
- type Envelope
Examples ¶
Constants ¶
const EncryptionAlgorithm = "RSA_DECRYPT_OAEP_3072_SHA256"
EncryptionAlgorithm is the KMS algorithm string the client is hardcoded against. The org public key endpoint advertises this; SealEnvelope refuses to run against any other value rather than silently producing payloads no reader can decrypt.
const StreamChunkSize = 1 << 20
StreamChunkSize is the plaintext chunk size SealStream seals under a single AES-GCM nonce. 1 MiB keeps the in-memory working set small enough to stream an arbitrarily large upload while staying far below the per-key, per-nonce birthday bounds for AES-GCM.
Variables ¶
This section is empty.
Functions ¶
func OpenEnvelope ¶
func OpenEnvelope(payload string, unwrapAESKey func(wrappedKey []byte) (aesKey []byte, err error)) ([]byte, error)
OpenEnvelope is the inverse of SealEnvelope. The caller supplies an unwrap function that decrypts the RSA-OAEP-wrapped AES key (typically a KMS AsymmetricDecrypt call against the version recorded in env.KeyVersion); this package stays GCP-free.
The split lets break-glass / reader code reuse the same AES-GCM / base64 logic the upload side uses, without forcing every SDK consumer to pull in cloud.google.com/go/kms.
func OpenStream ¶ added in v0.1.343
func OpenStream(r io.Reader, unwrapAESKey func(keyVersion string, wrappedKey []byte) ([]byte, error)) (io.Reader, error)
OpenStream parses the header and calls unwrapAESKey with the stored key version and RSA-OAEP-SHA256-wrapped DEK. As with OpenEnvelope, the callback can use KMS without exposing the RSA private key to this package.
The key version is untrusted until a chunk authenticates; the callback must scope key lookup to the caller's authorized org. It must not modify wrappedKey, which is part of the authenticated header.
Read the returned reader to EOF to verify the complete stream. Each chunk authenticates before its plaintext is returned, and modification, deletion, reordering, splicing or a missing final chunk causes a read error.
The returned reader is not safe for concurrent use.
func SealEnvelope ¶
SealEnvelope produces the JSON-encoded payload string the upload service expects in the request's payload field. It mirrors the browser's hybrid AES-GCM / RSA-OAEP encryption minus the network call:
- Generate a fresh AES-256 key + 12-byte IV.
- AES-256-GCM encrypt the plaintext.
- Parse the SPKI PEM into an RSA public key.
- RSA-OAEP-SHA256 wrap the AES key.
- Base64-encode each binary field, marshal the envelope.
The algorithm parameter is the value advertised by the server's GetOrgPublicKey response; mismatch against the client's hardcoded EncryptionAlgorithm is a hard failure rather than silently producing payloads no reader can decrypt.
keyVersion is the CryptoKeyVersion (e.g. "1") that wrapped this envelope, recorded so post-rotation readers know which private key to call AsymmetricDecrypt against.
func SealStream ¶ added in v0.1.343
func SealStream(w io.Writer, orgPub *rsa.PublicKey, algorithm, keyVersion string) (io.WriteCloser, error)
SealStream returns a writer that encrypts everything written to it into w as a stream envelope: a binary header followed by AES-GCM chunks in the STREAM construction.
A fresh AES-256 data encryption key is generated per stream and RSA-OAEP-SHA256-wrapped under orgPub in the header, mirroring SealEnvelope's hybrid scheme. The header is written before SealStream returns, and is bound into every chunk as additional authenticated data.
algorithm and keyVersion are the values advertised with orgPub by the server's GetOrgPublicKey response. As in SealEnvelope, a mismatch against the client's hardcoded EncryptionAlgorithm is a hard failure rather than silently producing payloads no reader can decrypt. keyVersion is the positive decimal CryptoKeyVersion, recorded so readers can select the same key after rotation.
The caller must Close the returned writer to emit the final chunk; a stream whose final chunk is missing does not open. Close does not close w, which the caller still owns.
The returned writer is not safe for concurrent use.
Example ¶
ExampleSealStream demonstrates sealing a payload as a stream envelope while capturing the CRC32C of the ciphertext, then opening it again.
package main
import (
"bytes"
"crypto/rand"
"crypto/rsa"
"crypto/sha256"
"fmt"
"hash/crc32"
"io"
"chainguard.dev/sdk/uploads"
)
func main() {
priv, err := rsa.GenerateKey(rand.Reader, 3072)
if err != nil {
panic(err)
}
// In production the destination is an object-storage writer and the
// ciphertext is never held in memory; a buffer stands in here so the
// example can check the checksum against the finished bytes.
var sealed bytes.Buffer
cw := uploads.NewCRC32CWriter(&sealed)
sw, err := uploads.SealStream(cw, &priv.PublicKey, uploads.EncryptionAlgorithm, "1")
if err != nil {
panic(err)
}
if _, err := io.WriteString(sw, "hello argos"); err != nil {
panic(err)
}
// Close seals the final chunk. A stream missing it does not open.
if err := sw.Close(); err != nil {
panic(err)
}
// The incrementally accumulated checksum is ready as soon as the last
// byte is written, which is what a caller that cannot buffer needs.
fmt.Println(cw.Sum32() == crc32.Checksum(sealed.Bytes(), crc32.MakeTable(crc32.Castagnoli)))
// A production callback selects the stored key version in Cloud KMS.
r, err := uploads.OpenStream(&sealed, func(keyVersion string, wrapped []byte) ([]byte, error) {
if keyVersion != "1" {
return nil, fmt.Errorf("unknown key version %q", keyVersion)
}
return rsa.DecryptOAEP(sha256.New(), rand.Reader, priv, wrapped, nil)
})
if err != nil {
panic(err)
}
plaintext, err := io.ReadAll(r)
if err != nil {
panic(err)
}
fmt.Printf("%s\n", plaintext)
}
Output: true hello argos
Types ¶
type CRC32CWriter ¶ added in v0.1.343
type CRC32CWriter struct {
// contains filtered or unexported fields
}
CRC32CWriter passes bytes through to an underlying writer while accumulating their CRC32C (Castagnoli) checksum. It exists so a caller streaming a sealed upload straight to object storage can hand the checksum over at the end without ever holding the ciphertext:
cw := uploads.NewCRC32CWriter(objectWriter) sw, err := uploads.SealStream(cw, orgPub, algorithm, keyVersion) // ... io.Copy(sw, src); sw.Close() ... checksum := cw.Sum32()
Sum32 is meaningful at any point and covers every byte written so far, header included. A CRC32CWriter is not safe for concurrent use.
func NewCRC32CWriter ¶ added in v0.1.343
func NewCRC32CWriter(w io.Writer) *CRC32CWriter
NewCRC32CWriter returns a CRC32CWriter that forwards to w.
func (*CRC32CWriter) Sum32 ¶ added in v0.1.343
func (c *CRC32CWriter) Sum32() uint32
Sum32 returns the CRC32C of everything written so far.
func (*CRC32CWriter) Write ¶ added in v0.1.343
func (c *CRC32CWriter) Write(p []byte) (int, error)
Write forwards p to the underlying writer and folds the bytes that reached it into the running checksum. A short write contributes only the accepted prefix, so the checksum always describes what the underlying writer actually holds.
type Envelope ¶
type Envelope struct {
Ciphertext string `json:"ciphertext"`
EncryptedKey string `json:"encryptedKey"`
IV string `json:"iv"`
Timestamp string `json:"timestamp"`
// KeyVersion is the integer CryptoKeyVersion the client used for
// RSA-OAEP wrapping (matches Cloud KMS' version numbering, e.g. "1").
// Carried in the envelope so decrypters can pick the right version
// regardless of what's currently the published primary — required
// for any post-rotation read of pre-rotation blobs. Empty means "1"
// for backward compatibility with envelopes uploaded before this
// field existed.
KeyVersion string `json:"keyVersion,omitempty"`
}
Envelope is the wire shape the client encrypts and the server stores as-is. All binary fields are base64-encoded; the server treats the JSON-encoded envelope as opaque and never inspects its contents.
func ParseEnvelope ¶
ParseEnvelope decodes the JSON envelope without performing the RSA-OAEP/AES-GCM decryption. Useful on the upload path when the server wants to read non-sensitive metadata (Timestamp) without touching the plaintext.
Example ¶
ExampleParseEnvelope demonstrates decoding a JSON envelope without performing any cryptographic operations.
package main
import (
"fmt"
"chainguard.dev/sdk/uploads"
)
func main() {
payload := `{"ciphertext":"abc=","encryptedKey":"def=","iv":"ghi=","timestamp":"2024-01-01T00:00:00Z","keyVersion":"1"}`
env, err := uploads.ParseEnvelope(payload)
fmt.Println(err)
fmt.Println(env.KeyVersion)
fmt.Println(env.Timestamp)
}
Output: <nil> 1 2024-01-01T00:00:00Z
Example (Invalid) ¶
ExampleParseEnvelope_invalid demonstrates the error returned when the payload is not a valid JSON envelope.
package main
import (
"fmt"
"chainguard.dev/sdk/uploads"
)
func main() {
_, err := uploads.ParseEnvelope("not-json")
fmt.Println(err != nil)
}
Output: true