Documentation
¶
Overview ¶
Package keyforge manages the lifecycle of encryption and signing keys: envelope encryption with versioned key-encryption keys, automatic rotation without re-encrypting stored data, and controlled retirement and destruction.
The design is in docs/specs/initiatives/INIT-KEYFORGE-001.
Model ¶
Data is encrypted with a data-encryption key (DEK) under a named purpose. DEKs are stored only wrapped by a key-encryption key (KEK) that a provider.Provider holds. A Client encrypts and decrypts; the registry (Store) records which DEK and which KEK version protect what.
Data keys are unwrapped once and cached in process, so encrypting and decrypting make no network call.
Example ¶
A client needs a provider that holds key-encryption keys, a registry that records which keys protect what, and the purposes it may use. The in-memory provider and registry used here are for tests and examples only.
package main
import (
"context"
"fmt"
"log"
"time"
"github.com/plexusone/keyforge"
"github.com/plexusone/keyforge/algorithm"
"github.com/plexusone/keyforge/provider"
providermemory "github.com/plexusone/keyforge/provider/memory"
registrymemory "github.com/plexusone/keyforge/registry/memory"
)
func main() {
ctx := context.Background()
prov := providermemory.NewKeyring()
reg := registrymemory.NewStore()
// Create a key-encryption key and make it the active one for a purpose.
kek, err := prov.CreateKey(ctx, provider.KeySpec{Purpose: "session-secrets", Algorithm: algorithm.AES256GCM})
if err != nil {
log.Fatal(err)
}
if err := reg.Bootstrap(ctx, "session-secrets", kek, 1, time.Now()); err != nil {
log.Fatal(err)
}
client, err := keyforge.New(keyforge.Options{
Store: reg,
Providers: []provider.Provider{prov},
Purposes: map[keyforge.Purpose]keyforge.PurposeConfig{"session-secrets": {}},
})
if err != nil {
log.Fatal(err)
}
// The context binds the ciphertext to a record: it cannot be moved.
sealed, err := client.Encrypt(ctx, "session-secrets", []byte("token-value"), []byte("user:42"))
if err != nil {
log.Fatal(err)
}
text := sealed.String() // "kf1." followed by base64url, safe for a text column
plaintext, info, err := client.Decrypt(ctx, "session-secrets", sealed, []byte("user:42"))
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s (stale: %v)\n", plaintext, info.Stale)
fmt.Println(text[:4])
}
Output: token-value (stale: false) kf1.
Index ¶
- Constants
- Variables
- func DEKWrapAAD(purpose Purpose, subject string, dekID, kekID envelope.ID) []byte
- func InlineWrapAAD(purpose Purpose, kekID envelope.ID) []byte
- func ValidateSubject(subject string) error
- type Client
- func (c *Client) AdvanceEpoch(ctx context.Context, purpose Purpose, subject string) (DEKRecord, error)
- func (c *Client) Decrypt(ctx context.Context, purpose Purpose, e *Envelope, bound []byte) (pt []byte, info Info, err error)
- func (c *Client) Encrypt(ctx context.Context, purpose Purpose, plaintext, context []byte) (e *Envelope, err error)
- func (c *Client) EncryptFor(ctx context.Context, purpose Purpose, subject string, ...) (e *Envelope, err error)
- func (c *Client) EncryptInline(ctx context.Context, purpose Purpose, plaintext, bound []byte) (e *Envelope, err error)
- func (c *Client) Purge()
- func (c *Client) Reencrypt(ctx context.Context, purpose Purpose, e *Envelope, bound []byte) (_ *Envelope, err error)
- func (c *Client) Verify(ctx context.Context, purposes ...Purpose) []VerifyResult
- type Clock
- type DEKRecord
- type DEKState
- type Envelope
- type Info
- type KEKVersion
- type KEKVersionRef
- type KeyRef
- type KeySpec
- type Options
- type Purpose
- type PurposeConfig
- type Store
- type VerifyCheck
- type VerifyResult
Examples ¶
Constants ¶
const ( // DefaultCacheTTL is how long an unwrapped data key stays cached. DefaultCacheTTL = 5 * time.Minute // DefaultCacheSize is the most data keys held unwrapped at once. DefaultCacheSize = 1024 // DefaultEncryptionCap is the most encryptions made under one data key // before it is replaced by a new epoch. It stays well below the NIST limit // of 2^32 invocations for random 96-bit nonces. DefaultEncryptionCap uint64 = 1 << 30 // MaxEncryptionCap is the largest cap a configuration may set. MaxEncryptionCap uint64 = 1 << 31 // DefaultReserveBlock is how many encryptions a process reserves from the // registry at a time. DefaultReserveBlock uint64 = 1 << 16 // DefaultMaxAge is how long a data key epoch is used for writes before a // new epoch is created. DefaultMaxAge = 90 * 24 * time.Hour )
Defaults and limits.
const MaxSubjectLen = 255
MaxSubjectLen is the longest subject in bytes.
Variables ¶
var ( // ErrUnknownPurpose is returned for a purpose the client is not // configured for. ErrUnknownPurpose = errors.New("keyforge: unknown purpose") // ErrPurposeMismatch is returned when an envelope names a different // purpose than the one the caller asked to decrypt under. ErrPurposeMismatch = errors.New("keyforge: envelope purpose does not match") // registry or provider failed and the key is not cached. Operations fail // closed; they never fall back to a weaker path. ErrKeyUnavailable = errors.New("keyforge: key unavailable") // ErrKeyUnusable is returned for a data key whose state forbids the // operation, such as a destroyed key. ErrKeyUnusable = errors.New("keyforge: key unusable") // ErrInvalidEnvelope is returned when an envelope is inconsistent with the // registry's record of its key. ErrInvalidEnvelope = errors.New("keyforge: envelope inconsistent with its key") // ErrDEKNotFound is returned by a Store for a data key it does not hold. ErrDEKNotFound = errors.New("keyforge: data key not found") // ErrKEKNotFound is returned by a Store for a KEK it does not hold. ErrKEKNotFound = errors.New("keyforge: key-encryption key not found") // ErrNoActiveKEK is returned by a Store for a purpose with no active KEK. ErrNoActiveKEK = errors.New("keyforge: no active key-encryption key") // ErrSubjectShredded is returned when a subject has been shredded and a // new key would be created for it. A shredded subject stays shredded: an // erasure that quietly re-enrolled the subject would not be one. ErrSubjectShredded = errors.New("keyforge: the subject has been shredded") // ErrSubjectRequired is returned when a purpose is per-subject and no // subject was given. ErrSubjectRequired = errors.New("keyforge: this purpose is per-subject and needs a subject") // ErrSubjectNotAllowed is returned when a subject is given for a purpose // that is not per-subject. ErrSubjectNotAllowed = errors.New("keyforge: this purpose is not per-subject") // ErrInvalidSubject is returned for a subject that is empty, too long, not // valid UTF-8, or contains control characters. ErrInvalidSubject = errors.New("keyforge: invalid subject") )
Typed errors. Callers match them with errors.Is.
Functions ¶
func DEKWrapAAD ¶
DEKWrapAAD is the additional data a data key is wrapped with. It binds the wrapped key to its purpose, subject, and its own and its KEK's identity, so a wrapped key cannot be moved to another record. The layout is part of the stored format: changing it makes existing wrapped keys unreadable.
The client uses it when it creates a data key, and the re-wrap job uses it to move a key to a new KEK version, so both must produce the same bytes.
func InlineWrapAAD ¶
InlineWrapAAD is the additional data an inline data key is wrapped with. Like DEKWrapAAD it is stored format.
func ValidateSubject ¶
ValidateSubject checks a subject: 1 to MaxSubjectLen bytes of valid UTF-8 with no control characters, so it is unambiguous in logs and in the additional data a data key is wrapped with.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client encrypts and decrypts under named purposes. It is safe for concurrent use.
func New ¶
New returns a client. It validates every purpose against the selector's profile, so a configuration that widens it fails here and not on first use.
func (*Client) AdvanceEpoch ¶
func (c *Client) AdvanceEpoch(ctx context.Context, purpose Purpose, subject string) (DEKRecord, error)
AdvanceEpoch creates the next data key epoch for a purpose and subject now, rather than waiting for the volume or age trigger, and returns the record that is current afterwards. The previous current key becomes decrypt-only. It is the first step of replacing an exposed key: writes move to the new epoch at once, and the old one stays readable so what it protects can be re-encrypted.
func (*Client) Decrypt ¶
func (c *Client) Decrypt(ctx context.Context, purpose Purpose, e *Envelope, bound []byte) (pt []byte, info Info, err error)
Decrypt authenticates and decrypts an envelope under purpose, with the same context that was given to Encrypt. A wrong key, a changed header or context and a modified ciphertext all fail with envelope.ErrAuthentication.
Info.Stale reports an envelope that is not under the current key, so the caller can rewrite it on its next write. No rewrite is needed for correctness.
func (*Client) Encrypt ¶
func (c *Client) Encrypt(ctx context.Context, purpose Purpose, plaintext, context []byte) (e *Envelope, err error)
Encrypt encrypts plaintext under purpose in reference mode: the envelope names the data key, and rotating the KEK does not touch it. context is bound into the ciphertext, so it cannot be moved to another record.
A per-subject purpose is written with Client.EncryptFor; Encrypt on one is ErrSubjectRequired.
func (*Client) EncryptFor ¶
func (c *Client) EncryptFor(ctx context.Context, purpose Purpose, subject string, plaintext, context []byte) (e *Envelope, err error)
EncryptFor encrypts plaintext under a per-subject purpose with the subject's own data key, creating the subject's first epoch if it has none. The subject is not part of the envelope: the envelope names its data key, and the registry knows whose key that is.
A subject that has been shredded is ErrSubjectShredded. A purpose that is not per-subject is ErrSubjectNotAllowed.
func (*Client) EncryptInline ¶
func (c *Client) EncryptInline(ctx context.Context, purpose Purpose, plaintext, bound []byte) (e *Envelope, err error)
EncryptInline encrypts plaintext in inline mode: a fresh data key per object, wrapped under the purpose's active KEK and carried in the envelope. It suits standalone objects such as exports and snapshots.
Unlike reference mode it contacts the provider once per object, to wrap the key, so it is not for a hot path against a managed key service.
func (*Client) Purge ¶
func (c *Client) Purge()
Purge drops every cached key and view. Use it when a key must stop being usable at once, for example after destroying it or shredding a subject; otherwise a cached key can outlive its destruction by up to the cache TTL. That holds for writes as well as reads: a process holding a cached key for a shredded subject goes on writing under it, and nothing it writes can be read by another process or after a restart. Operations already in flight when Purge is called do not repopulate the caches.
func (*Client) Reencrypt ¶
func (c *Client) Reencrypt(ctx context.Context, purpose Purpose, e *Envelope, bound []byte) (_ *Envelope, err error)
Reencrypt decrypts an envelope and encrypts it again under the current key, in the same mode, so a caller can bring old data up to date. It always produces a new envelope; use the Stale report from Decrypt to skip the ones that are already current.
func (*Client) Verify ¶
func (c *Client) Verify(ctx context.Context, purposes ...Purpose) []VerifyResult
Verify round-trips a fresh random test value through each purpose's real key path, reading it back from cold: the current data key and the provider for reference mode, and a wrap and an unwrap by the provider for inline mode. It proves that the keys a purpose depends on can be used now, which is what an operator wants to know before a rotation or after a change, and what a quiet week would otherwise hide.
A per-subject purpose is checked in inline mode only, so verification creates no data key record for a subject that does not exist. The test value and its context are random and are zeroed afterwards; nothing is returned but outcomes.
type DEKRecord ¶
type DEKRecord struct {
ID envelope.ID
Purpose Purpose
Subject string // empty for a purpose-wide key
Epoch uint32 // starts at 1 and increases per purpose and subject
Algorithm algorithm.ID
KEKID envelope.ID
KEKVersion uint32
Wrapped []byte
State DEKState
Reserved uint64 // high-water mark of reserved encryptions
CreatedAt time.Time
// WrappedUnder lists every KEK version this key has been wrapped under, in
// order, ending with the current one. A backup of the registry taken at any
// point holds the key wrapped under one of them, which is what decides when
// destroying the key is complete.
WrappedUnder []KEKVersionRef
}
DEKRecord is the registry's record of a data key. It never holds the unwrapped key.
type DEKState ¶
type DEKState uint8
DEKState is the lifecycle state of a data key.
const ( // DEKActive is the current epoch: used for new writes and for reads. DEKActive DEKState = iota + 1 // DEKDecryptOnly is an earlier epoch: read, never written. DEKDecryptOnly // DEKCompromised is marked by an operator and is read only as policy // allows. DEKCompromised // DEKDestroyed has no usable wrapped key. DEKDestroyed )
type Envelope ¶
Envelope is a sealed ciphertext. See envelope.Envelope.
type Info ¶
type Info struct {
Mode envelope.Mode
Subject string
// Epoch is the data key's epoch (reference mode).
Epoch uint32
// KEKVersion is the KEK version that wraps the key (inline mode).
KEKVersion uint32
// Stale reports that the envelope was written under a data key that is no
// longer current, or an inline key under an older KEK version, so the
// caller can rewrite it on its next write. It is advisory: it is computed
// from a cached view, and a registry outage can hide it.
Stale bool
}
Info describes how an envelope was protected, as reported by Decrypt.
type KEKVersion ¶
KEKVersion names one version of a KEK.
type KEKVersionRef ¶
KEKVersionRef names a KEK version by identifier, without the provider reference. A data key's wrapping history is a list of these.
type KeySpec ¶
KeySpec describes a KEK to create. See provider.KeySpec.
type Options ¶
type Options struct {
// Store is the registry. Required.
Store Store
// Providers are the KEK holders, matched to a key by [provider.Provider]
// Name. At least one is required.
Providers []provider.Provider
// Selector chooses algorithms and enforces the profile. The zero value
// uses [algorithm.DefaultProfile].
Selector *algorithm.Selector
// Purposes lists the purposes this client may use. Anything else fails
// with [ErrUnknownPurpose].
Purposes map[Purpose]PurposeConfig
// Clock defaults to the system clock.
Clock Clock
// Entropy supplies data keys and identifiers, and defaults to
// crypto/rand. Tests inject a deterministic reader. Nonces come from the
// AEAD's own generator and are not drawn from it.
Entropy io.Reader
// CacheTTL and CacheSize bound the unwrapped-key cache.
CacheTTL time.Duration
CacheSize int
// EncryptionCap and ReserveBlock set the per-key cap and the size of the
// blocks reserved from the registry. Zero uses the defaults.
EncryptionCap uint64
ReserveBlock uint64
// MaxAge is how long a data key epoch is used for writes. Zero means
// [DefaultMaxAge].
MaxAge time.Duration
// Metrics receives operation rates, failures and cache behavior. The
// default discards them.
Metrics metrics.Metrics
}
Options configures a Client.
type Purpose ¶
type Purpose string
Purpose is a named use of encryption, such as "session-secrets". Keys are scoped to a purpose, and the purpose is bound into every ciphertext.
type PurposeConfig ¶
type PurposeConfig struct {
// Algorithms narrows the algorithms used for this purpose. See
// [algorithm.Selection].
Algorithms algorithm.Selection
// EncryptionCap overrides [Options.EncryptionCap] for this purpose. Zero
// uses the client's.
EncryptionCap uint64
// MaxAge overrides [Options.MaxAge] for this purpose. Zero uses the
// client's.
MaxAge time.Duration
// PerSubject gives each subject (a user, a tenant) data key epochs of its
// own, so that destroying one subject's keys makes that subject's data
// unrecoverable wherever it was copied. A per-subject purpose is written
// with [Client.EncryptFor]; any other purpose is written with
// [Client.Encrypt].
PerSubject bool
}
PurposeConfig is the configuration of one purpose.
type Store ¶
type Store interface {
// ActiveKEK returns the KEK version new data keys for purpose are wrapped
// under. It returns [ErrNoActiveKEK] if the purpose has none.
ActiveKEK(ctx context.Context, purpose Purpose) (KEKVersion, error)
// KEK returns the provider reference for a KEK identifier. It returns
// [ErrKEKNotFound] for an identifier the registry does not hold.
KEK(ctx context.Context, id envelope.ID) (provider.KeyRef, error)
// KEKVersionUsable reports whether data keys wrapped under a KEK version
// may be unwrapped: it is active, decrypt-only, retire-requested or
// compromised. A pending, retired or destroyed version is not usable, even
// while the provider still holds its material. It returns
// [ErrKEKNotFound] for a KEK the registry does not hold, and false for a
// version it does not hold.
KEKVersionUsable(ctx context.Context, id envelope.ID, version uint32) (bool, error)
// CurrentDEK returns the data key that new writes for purpose and subject
// use. It returns [ErrDEKNotFound] if none exists yet.
CurrentDEK(ctx context.Context, purpose Purpose, subject string) (DEKRecord, error)
// GetDEK returns a data key by identifier, in any state. It returns
// [ErrDEKNotFound] for an identifier the registry does not hold.
GetDEK(ctx context.Context, id envelope.ID) (DEKRecord, error)
// AdvanceDEK installs next as the current data key for purpose and
// subject if the current epoch is still from (zero when there is none),
// and marks the previous one decrypt-only, atomically. If the current
// epoch is no longer from, it changes nothing and returns the record that
// is current, so concurrent callers converge on one key.
AdvanceDEK(ctx context.Context, purpose Purpose, subject string, from uint32, next DEKRecord) (DEKRecord, error)
// ReserveEncryptions grants up to n more encryptions on a data key,
// advancing its reserved high-water mark, but never past limit. It grants
// zero when the key has no capacity left or is no longer the current
// epoch, which tells the caller to advance.
ReserveEncryptions(ctx context.Context, id envelope.ID, n, limit uint64) (granted uint64, err error)
}
Store is what the Client needs from the registry. It is defined here, where it is used, and kept to the operations the data path performs. The registry's own interface satisfies it.
A Store is safe for concurrent use. Operations that change state are atomic: two replicas calling them at once see a single winner.
type VerifyCheck ¶
type VerifyCheck struct {
// Mode is "reference" or "inline".
Mode string `json:"mode"`
// Outcome is "ok" or one of the failure classes a metric uses; it names no
// detail.
Outcome string `json:"outcome"`
}
VerifyCheck is one round trip.
type VerifyResult ¶
type VerifyResult struct {
Purpose Purpose `json:"purpose"`
// OK is true when every check passed.
OK bool `json:"ok"`
// Checks lists what was exercised and how each went.
Checks []VerifyCheck `json:"checks"`
}
VerifyResult is the outcome of verifying one purpose.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package algorithm holds keyforge's algorithm registry: the identifiers written in envelopes, their exact names, and the construction of the primitive for each.
|
Package algorithm holds keyforge's algorithm registry: the identifiers written in envelopes, their exact names, and the construction of the primitive for each. |
|
Package audit defines the events keyforge emits for key lifecycle changes and the sink that receives them.
|
Package audit defines the events keyforge emits for key lifecycle changes and the sink that receives them. |
|
Package cli is keyforge's command line: Cobra commands over the HTTP control plane.
|
Package cli is keyforge's command line: Cobra commands over the HTTP control plane. |
|
cmd
|
|
|
keyforge
command
Command keyforge is the key lifecycle command line.
|
Command keyforge is the key lifecycle command line. |
|
Package embedded wires keyforge for one host: a keyring file, a SQLite registry, the lifecycle jobs and, optionally, the HTTP control plane, all from a directory and a root key source.
|
Package embedded wires keyforge for one host: a keyring file, a SQLite registry, the lifecycle jobs and, optionally, the HTTP control plane, all from a directory and a root key source. |
|
Package envelope implements keyforge's self-describing ciphertext format: its binary and text forms, the authenticated header, and sealing and opening of the payload.
|
Package envelope implements keyforge's self-describing ciphertext format: its binary and text forms, the authenticated header, and sealing and opening of the payload. |
|
Package guard defines the veto a host application holds over retiring and destroying keys.
|
Package guard defines the veto a host application holds over retiring and destroying keys. |
|
Package jobs runs keyforge's lifecycle work as durable jobs on OmniWorkflow.
|
Package jobs runs keyforge's lifecycle work as durable jobs on OmniWorkflow. |
|
Package metrics defines the small interface through which keyforge reports rates, failures and overdue keys, and the names it reports them under.
|
Package metrics defines the small interface through which keyforge reports rates, failures and overdue keys, and the names it reports them under. |
|
Package policy holds the rotation, epoch and retirement policy of a purpose.
|
Package policy holds the rotation, epoch and retirement policy of a purpose. |
|
Package provider defines the interface to a key-encryption-key (KEK) holder: the local keyring, or a managed key service.
|
Package provider defines the interface to a key-encryption-key (KEK) holder: the local keyring, or a managed key service. |
|
local
Package local is the keyring-file provider: KEKs held in an encrypted file on the local disk, for development and single-host deployments.
|
Package local is the keyring-file provider: KEKs held in an encrypted file on the local disk, for development and single-host deployments. |
|
memory
Package memory provides in-memory providers for tests and examples.
|
Package memory provides in-memory providers for tests and examples. |
|
providertest
Package providertest is the conformance suite every provider.Provider must pass.
|
Package providertest is the conformance suite every provider.Provider must pass. |
|
Package registry defines the record of keys: key-encryption keys, their versions and states, and the data keys wrapped under them.
|
Package registry defines the record of keys: key-encryption keys, their versions and states, and the data keys wrapped under them. |
|
memory
Package memory provides an in-memory registry.Store for tests, examples and single-process use.
|
Package memory provides an in-memory registry.Store for tests, examples and single-process use. |
|
storetest
Package storetest is the conformance suite every registry.Store must pass.
|
Package storetest is the conformance suite every registry.Store must pass. |
|
Package rotate rotates key-encryption keys: it creates and activates the next version, re-wraps the data keys that still sit under the old one, rolls back, and says when a version may be retired.
|
Package rotate rotates key-encryption keys: it creates and activates the next version, re-wraps the data keys that still sit under the old one, rolls back, and says when a version may be retired. |
|
Package service is the HTTP control plane: an http.Handler for status, key listing, rotation, retirement, destruction, backup acknowledgement and the emergency procedures.
|
Package service is the HTTP control plane: an http.Handler for status, key listing, rotation, retirement, destruction, backup acknowledgement and the emergency procedures. |
|
Package sqlstore is the SQL-backed registry.Store, built on Ent.
|
Package sqlstore is the SQL-backed registry.Store, built on Ent. |
|
ent/schema
Package schema is keyforge's Ent schema: the registry's records.
|
Package schema is keyforge's Ent schema: the registry's records. |
|
Package status assembles what an operator needs to see: the state of every key, when each is next due, what is overdue, whether the keyring is backed up, whether each provider is healthy, and what the scheduler is struggling with.
|
Package status assembles what an operator needs to see: the state of every key, when each is next due, what is overdue, whether the keyring is backed up, whether each provider is healthy, and what the scheduler is struggling with. |