knowledge

package
v0.7.14-dev.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 10, 2026 License: MIT Imports: 32 Imported by: 0

Documentation

Overview

Package knowledge is the registry's store of knowledge packs: the packs a signed-in person, or an organization, keeps in Agentry, each held as its tree sealed under a data key of its own (pkg/packseal) with that key wrapped for its owner by a KeyWrapper, scanned when it arrives, and destroyed with its owner. The core carries all of it without certificates: its wrapper is the deployment's knowledge sealing key; an extension (a certificate wrapper) replaces the wrapper and nothing else.

Index

Constants

View Source
const (
	JobScan      = "knowledge.scan"
	JobErase     = "knowledge.erase"
	JobScanSweep = "knowledge.scan.sweep"
)

The job kinds: the pack scan runs on the sandbox queue, as a version's scan does, because it reads what an account uploaded; the retry of an erasure's last step and the sweep that queues a lost scan again on the core queue, since they read no content.

View Source
const (
	WikiDir    = "wiki"
	SourcesDir = "sources"
)

The two directories a pack's tree holds (agentry-cli pkg/knowledge).

View Source
const DeploymentKind = "deployment"

DeploymentKind is the core wrapper's kind.

MaxBytes caps a pack: every file of its tree, in MiB as the CLI counts it (agentry-cli pkg/knowledge.MaxBytes).

MaxFiles caps the files of one pack's tree.

View Source
const MaxTreeBytes = MaxBytes + MaxFiles*2048 + 1024

MaxTreeBytes bounds a tree's tar: its files, and for each at most a PAX header, a ustar header and a block of padding, and the end blocks.

View Source
const SaltSize = 32

SaltSize is an owner salt's length.

View Source
const ScanStallAfter = 2 * time.Hour

ScanStallAfter is how long a pack waits scanning since its tree was stored, or since the sweep last queued its scan, before the sweep queues the scan again: past a scan job's whole retry budget (three attempts at the queue's job timeout, about 75 minutes at worst), so a scan still being tried is not doubled, with room for a sandbox queue that is behind.

Variables

View Source
var (
	// ErrNotOwner is a wrapped key that does not open for the owner who
	// presents it: wrapped for someone else, under a key since destroyed,
	// or altered. Which of those is not the caller's to learn.
	ErrNotOwner = errors.New("knowledge: the wrapped key is not this owner's")
	// ErrUnconfigured is a deployment with no knowledge sealing key and no
	// extension wrapper: it stores no packs.
	ErrUnconfigured = errors.New("knowledge: no key wrapper is configured")
)
View Source
var (
	// ErrNotFound is a pack the caller does not own, or that does not
	// exist: the two are one answer.
	ErrNotFound = errors.New("knowledge: no such pack")
	// ErrNameTaken is a second pack of one name for one owner.
	ErrNameTaken = errors.New("knowledge: the owner already has a pack of this name")
	// ErrName is a name that is not a pack's.
	ErrName = errors.New("knowledge: a pack name is 1-64 lowercase letters, digits, '.', '-' or '_', starting with a letter or a digit")
	// ErrScanning is a pack whose tree is being scanned: a pack is
	// unusable only then.
	ErrScanning = errors.New("knowledge: the pack is being scanned")
)
View Source
var (
	// ErrTooLarge is a tree past MaxBytes or MaxFiles.
	ErrTooLarge = fmt.Errorf("knowledge: a pack is at most %d MB and %d files", MaxBytes>>20, MaxFiles)
	// ErrDigest is a tree whose bytes are not the digest it names.
	ErrDigest = errors.New("knowledge: the tree does not match its digest")
	// ErrTree is a tree that is not a pack's canonical tar.
	ErrTree = errors.New("knowledge: not a knowledge pack's tree")
)
View Source
var ErrChanged = errors.New("knowledge: the pack's tree changed since it was read")

ErrChanged is a replace whose pack no longer holds the tree it was read with: another upload or build landed in between.

View Source
var ErrNotExportable = errors.New("knowledge: the pack is not marked exportable")

ErrNotExportable is an organization's shared pack its admins have not marked exportable, exported by a member.

Functions

func DecodeSealingKey

func DecodeSealingKey(b64 string) ([]byte, error)

DecodeSealingKey decodes a base64 knowledge sealing key: 32 bytes.

func EraseOwner

func EraseOwner(ctx context.Context, q *storegen.Queries, owner Owner) ([]string, error)

EraseOwner deletes every pack of owner on q, the caller's transaction — the account erasure, a data clear, an organization's decommission — and the salt the core's wrapper derives the owner's keys from, so that the crypto-erasure commits with the erasure itself. It returns the blob keys, which FinishErasure takes once the transaction has committed. It runs whether or not the store is on, so that turning it off strands nothing an erasure should remove.

func MaxUpload

func MaxUpload() int64

MaxUpload bounds the body of a tree upload: a sealed tree of the largest tar a pack can be.

func ValidName

func ValidName(name string) bool

ValidName is the CLI's rule for a pack name (agentry-cli pkg/knowledge.ValidName): the name is the pack's directory.

func WriteTree

func WriteTree(w io.Writer, files []scan.PackFile) error

WriteTree writes files as a pack's canonical tar, in byte order of their paths: the form agentry-cli's TreeDigest hashes, field by field (mode 0644, uid and gid 0, mtime 0, a PAX header only when a name needs one, two zero blocks and no record padding after the last file).

Types

type Admit

type Admit func(ctx context.Context, a UploadAdmission) (ticket string, release func(stored bool), err error)

Admit admits an upload that will queue a scan. It is asked once, after the tree is read and checked against the allocation, and before anything is stored; an error refuses the upload with nothing stored. ticket, when not empty, is recorded as stored (admitted_stores, kind "knowledge") in the transaction that stores the tree. release is called once with whether that transaction committed, so a ticket for an upload the core then refused (the allocation cap, a lost race, a panic) can be voided.

type AllocationError

type AllocationError struct {
	Bytes int64
	Max   int64
}

AllocationError is an upload past the owner's allocation: what they would keep and the cap.

func (*AllocationError) Error

func (e *AllocationError) Error() string

type Also

type Also func(q *storegen.Queries, p storegen.KnowledgePack) error

Also is what a caller records in the same transaction as a change to a pack — an organization's audit event — so the change and its record commit or roll back together. It sees the pack as the change left it.

type DeploymentWrapper

type DeploymentWrapper struct {
	// contains filtered or unexported fields
}

DeploymentWrapper is the core's KeyWrapper: AES-256-GCM under a key derived, per owner, from the deployment's knowledge sealing key (AGENTRY_KNOWLEDGE_SEALING_KEY, a key of its own) and the owner's random salt, with the owner named in the additional data. A core-only registry so stores trees it can open only for their owners, and an owner's erasure deletes the salt, after which no copy of their wrapped keys — not the pack rows', not one on a laptop — opens again.

func NewDeploymentWrapper

func NewDeploymentWrapper(key []byte, salts OwnerSalts) (*DeploymentWrapper, error)

NewDeploymentWrapper builds the core wrapper over a 32-byte key.

func (*DeploymentWrapper) Destroy

func (d *DeploymentWrapper) Destroy(ctx context.Context, owner Owner) error

Destroy implements KeyWrapper: the owner's salt goes, and with it every key-encryption key their wrapped keys need.

func (*DeploymentWrapper) Recipient

func (d *DeploymentWrapper) Recipient() string

Recipient is the id the wrapper names its wrapped keys with: a digest of the key, never the key.

func (*DeploymentWrapper) Unwrap

func (d *DeploymentWrapper) Unwrap(ctx context.Context, owner Owner, w WrappedKey) ([]byte, error)

Unwrap implements KeyWrapper.

func (*DeploymentWrapper) Wrap

func (d *DeploymentWrapper) Wrap(ctx context.Context, owner Owner, dataKey []byte) (WrappedKey, error)

Wrap implements KeyWrapper.

type Deps

type Deps struct {
	Store   *store.Store
	Blobs   registry.BlobStore
	Wrapper KeyWrapper
	Scanner Scanner
	Jobs    queue.Client
	// Entitlements answers the owner's allocation; read on every upload,
	// so a resolver swapped in later (an extension's) is the one asked.
	Entitlements func() entitlements.Resolver
	Log          *slog.Logger
}

Deps is what the service is built from.

type EraseArgs

type EraseArgs struct {
	Kind OwnerKind `json:"kind"`
	ID   uuid.UUID `json:"id"`
	Keys []string  `json:"keys,omitempty"`
}

EraseArgs names an erasure's last step: the owner whose keys the wrapper destroys, and the sealed blobs to delete.

type Info

type Info struct {
	ID     uuid.UUID
	Name   string
	State  State
	Digest string // "" for an empty pack
	Bytes  int64
	Files  int
	Pages  int
}

Info is what an extension reads of a pack: never a key.

type KeyWrapper

type KeyWrapper interface {
	// Wrap seals dataKey for owner.
	Wrap(ctx context.Context, owner Owner, dataKey []byte) (WrappedKey, error)
	// Unwrap opens a wrapped key for owner, or refuses with ErrNotOwner.
	Unwrap(ctx context.Context, owner Owner, wrapped WrappedKey) ([]byte, error)
	// Destroy makes every key ever wrapped for owner unopenable, wherever
	// a copy of it is kept. It is the owner's erasure, a data clear, or an
	// organization's decommission; it is idempotent.
	Destroy(ctx context.Context, owner Owner) error
}

KeyWrapper wraps a pack's data key for its owner and unwraps it again, and destroys what makes an owner's wrapped keys open. The registry is the only party that unwraps: a data key leaves it over TLS to its owner, and no route ever returns what wraps it.

func NewCoreWrapper

func NewCoreWrapper(sealingKey string, devMode bool, salts OwnerSalts) (KeyWrapper, error)

NewCoreWrapper is the core's wrapper from configuration: the knowledge sealing key (base64), or in dev mode without one a fixed insecure key, as the auth sealer has. With neither it is nil, which leaves the store off unless an extension wraps keys.

type Owner

type Owner struct {
	Kind OwnerKind
	ID   uuid.UUID
}

Owner is a pack's owner, by the id of its account or its organization.

func Org

func Org(id uuid.UUID) Owner

Org is the owner that is the organization with this id.

func User

func User(id uuid.UUID) Owner

User is the owner that is the person with this account id.

func (Owner) String

func (o Owner) String() string

func (Owner) Valid

func (o Owner) Valid() bool

Valid says whether the owner names a kind and an id.

type OwnerKind

type OwnerKind string

OwnerKind is what owns a pack: a person or an organization.

const (
	OwnerUser OwnerKind = "user"
	OwnerOrg  OwnerKind = "org"
)

The owners a pack can have.

type OwnerSalts

type OwnerSalts interface {
	// Salt returns the owner's salt, creating it first when create is set;
	// with create unset, an owner with none gets ErrNotOwner.
	Salt(ctx context.Context, owner Owner, create bool) ([]byte, error)
	// DeleteSalt removes the owner's salt, if any.
	DeleteSalt(ctx context.Context, owner Owner) error
}

OwnerSalts keeps the random salt each owner's key-encryption key is derived with. Deleting an owner's salt is what destroys their keys.

type Pool

type Pool struct {
	// Key names the pool: every upload into it is reserved under one
	// transaction-scoped lock on the key, so two owners who share it
	// cannot both pass its cap.
	Key string
	// Owners are every owner whose packs count against the pool, the one
	// asking included.
	Owners []Owner
	// Cap is the pool's allocation; a cap never deletes.
	Cap entitlements.Cap
}

Pool is the allocation an owner's knowledge packs draw on: the owner alone in the core, or, where an extension pools owners, a paid organization's seats shared by its packs and its members'.

type Pools

type Pools interface {
	KnowledgePool(ctx context.Context, owner Owner) (Pool, error)
}

Pools is the optional interface of an entitlements resolver that pools owners' knowledge. The core asks the resolver it holds, so an extension's resolver answers; an error is not permission. The contract the core cannot check: every owner a pool lists must be answered with that same pool (the same Key), or two of them are not serialised on one lock. Pooling discloses the pool's total to every owner in it — the meter and the refusal name it — which a shared cap cannot avoid.

type ScanArgs

type ScanArgs struct {
	Pack   uuid.UUID `json:"pack"`
	Object string    `json:"object"`
}

ScanArgs names the tree a scan job reads.

type Scanner

type Scanner interface {
	ScanPack(ctx context.Context, files []scan.PackFile) scan.PackReport
}

Scanner is what reads a pack: the scan orchestrator.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service is the registry's knowledge store.

func NewService

func NewService(d Deps) *Service

NewService builds the store. A nil Wrapper leaves it off: every call answers ErrUnconfigured.

func (*Service) Configured

func (s *Service) Configured() bool

Configured says whether the store is on.

func (*Service) Create

func (s *Service) Create(ctx context.Context, owner Owner, name string, w WrappedKey, also Also) (storegen.KnowledgePack, error)

Create makes an empty pack whose data key is w, which must open for owner: a pack is never created with a key its owner cannot use. also, when given, runs in the same transaction.

func (*Service) CreateSealed

func (s *Service) CreateSealed(ctx context.Context, owner Owner, name string) (Info, error)

CreateSealed makes an empty pack for owner under a data key the registry makes and its wrapper wraps for owner; the key is kept only wrapped.

func (*Service) Delete

func (s *Service) Delete(ctx context.Context, owner Owner, id uuid.UUID, also Also) error

Delete removes one of owner's packs, its objects and their blobs. The pack row is locked first, so an upload racing the delete either lands before it (and its object goes with the pack) or finds no pack. also, when given, runs in the same transaction with the pack as it stood before it went.

func (*Service) Export

func (s *Service) Export(ctx context.Context, owner Owner, id uuid.UUID) (storegen.KnowledgePack, *Tree, error)

Export is one of owner's packs' tree, decrypted: the canonical tar whose digest is the pack's. It is opened and checked in full before a byte is written, so a tree that does not open writes nothing.

func (*Service) ExportIf

func (s *Service) ExportIf(ctx context.Context, owner Owner, id uuid.UUID, allow func(storegen.KnowledgePack) error) (storegen.KnowledgePack, *Tree, error)

ExportIf is Export, refused when allow answers an error for the pack as the export read it (nil: always allowed), so that the rule and the tree come from one read of the row.

func (*Service) FinishErasure

func (s *Service) FinishErasure(ctx context.Context, owner Owner, keys []string)

FinishErasure is an erasure's last step, after its transaction committed: the wrapper destroys what opens the owner's keys (an extension's wrapper holds its own; the core's salt is already gone), then the sealed blobs go. A step that fails is handed to a retrying job and logged, never hidden, and undoes nothing: a blob left behind is sealed under a data key no one can unwrap any more.

func (*Service) Get

func (s *Service) Get(ctx context.Context, owner Owner, id uuid.UUID) (storegen.KnowledgePack, error)

Get is one of owner's packs.

func (*Service) Info

func (s *Service) Info(ctx context.Context, owner Owner, id uuid.UUID) (Info, error)

Info is one of owner's packs as an extension reads it.

func (*Service) List

func (s *Service) List(ctx context.Context, owner Owner) ([]storegen.KnowledgePack, error)

List is owner's packs, by name.

func (*Service) RegisterHandlers

func (s *Service) RegisterHandlers(q queue.Client)

RegisterHandlers wires the scan job, the sweep of stalled scans and the erasure's retry.

func (*Service) Replace

func (s *Service) Replace(ctx context.Context, owner Owner, id uuid.UUID, from string, files []scan.PackFile) (info Info, changed bool, err error)

Replace makes files the tree of one of owner's packs, provided the pack still holds the tree whose digest is from ("" for an empty pack): the files written as the canonical tar, sealed under the pack's own data key with the stored header's pack id kept, and stored as an upload is — opened and checked, counted against its pool under the pool's lock, the tree it replaces deleted, its scan queued. It answers ErrChanged when the pack moved on, the upload's errors otherwise (ErrTree, ErrTooLarge, *AllocationError). A tree the pack already holds is not stored again, and changed is false, whatever from says: the pack is already what the caller asked for, so a replace retried after it landed is not a conflict.

func (*Service) SealedTree

func (s *Service) SealedTree(ctx context.Context, owner Owner, id uuid.UUID) (storegen.KnowledgePack, io.ReadCloser, error)

SealedTree is the stored sealed tree of one of owner's packs.

func (*Service) SetExportable

func (s *Service) SetExportable(ctx context.Context, owner Owner, id uuid.UUID, exportable bool, also Also) (storegen.KnowledgePack, error)

SetExportable marks whether one of an organization's packs may be exported by its members, with also run in its transaction. A person's own pack has no such mark: its owner exports it.

func (*Service) SetWrapper

func (s *Service) SetWrapper(w KeyWrapper)

SetWrapper replaces the key wrapper: an extension's (pkg/extension WrapsKeys) in place of the deployment key.

func (*Service) Sizes

Sizes is what platform staff see of owner's packs: names and sizes.

func (*Service) SweepScans

func (s *Service) SweepScans(ctx context.Context, cutoff time.Time) (int, error)

SweepScans queues the scan again of every pack still scanning since before cutoff, its tree stored or its scan last queued then, and reports how many. It is how a pack whose scan was never queued, or whose job was lost, reaches ready or failed: a scan's own last attempt fails the pack rather than leave it scanning.

func (*Service) TreeOf

func (s *Service) TreeOf(ctx context.Context, owner Owner, id uuid.UUID) (Info, []byte, error)

TreeOf is one of owner's packs' tree, decrypted, as the canonical tar whose SHA-256 is the pack's digest. A pack being scanned is ErrScanning, an empty one ErrNotFound, as the export answers.

func (*Service) Unwrap

func (s *Service) Unwrap(ctx context.Context, owner Owner, w WrappedKey) ([]byte, error)

Unwrap opens a wrapped key for owner, or refuses with ErrNotOwner.

func (*Service) Upload

func (s *Service) Upload(ctx context.Context, owner Owner, id uuid.UUID, sealed []byte) (storegen.KnowledgePack, error)

Upload stores a pack's new sealed tree and queues its scan. The registry opens it first — the data key unwrapped for owner, the tree read and its digest checked — so that what it keeps is always a pack, and counts it against the allocation owner's packs draw on, under a lock on that pool, then stores it sealed as it came.

func (*Service) UploadAdmitted

func (s *Service) UploadAdmitted(ctx context.Context, owner Owner, id uuid.UUID, sealed []byte, admit Admit) (storegen.KnowledgePack, error)

UploadAdmitted is Upload with admit asked first. An upload of the tree the pack already holds, scanned or being scanned, stores nothing and queues no scan, so it is never admitted; one that failed to scan is scanned again unadmitted, since the failure was the registry's.

func (*Service) Usage

func (s *Service) Usage(ctx context.Context, owner Owner) (Usage, error)

Usage is owner's meter.

func (*Service) Wrap

func (s *Service) Wrap(ctx context.Context, owner Owner, dataKey []byte) (WrappedKey, error)

Wrap wraps a data key for owner.

func (*Service) Wrapper

func (s *Service) Wrapper() KeyWrapper

Wrapper is the key wrapper in use, nil when the store is off.

type State

type State string

State is where a pack is in its life.

const (
	StateEmpty    State = "empty"
	StateScanning State = "scanning"
	StateReady    State = "ready"
	StateFailed   State = "failed"
)

The states a pack is in: empty until a tree is uploaded, scanning from each upload until its scan ends, ready after it whatever it found, failed when the stored tree could not be read to scan.

type StoreSalts

type StoreSalts struct {
	// contains filtered or unexported fields
}

StoreSalts keeps owner salts in knowledge_owner_keys.

func NewStoreSalts

func NewStoreSalts(st *store.Store) *StoreSalts

NewStoreSalts is the database's OwnerSalts.

func (*StoreSalts) DeleteSalt

func (s *StoreSalts) DeleteSalt(ctx context.Context, owner Owner) error

DeleteSalt implements OwnerSalts.

func (*StoreSalts) Salt

func (s *StoreSalts) Salt(ctx context.Context, owner Owner, create bool) ([]byte, error)

Salt implements OwnerSalts.

type Tree

type Tree struct {
	Files []scan.PackFile
	// Bytes is the files' size, Pages the wiki's pages (the index and the
	// log aside), as the CLI counts them.
	Bytes int64
	Pages int
	// Digest is the SHA-256 of the tar, "sha256:<hex>".
	Digest string
}

Tree is a pack's tree as the registry read it.

func ReadTree

func ReadTree(r io.Reader, digest string) (*Tree, error)

ReadTree reads a pack's tree from the canonical tar its digest is taken over (agentry-cli pkg/knowledge.TreeDigest: regular files under wiki/ and sources/, in byte order of their paths) and checks that its bytes are digest. It refuses anything else a tar can carry — a link, a device, a directory entry, a path that climbs or is not the pack's, a name twice — and a tree past the pack's caps.

type UploadAdmission

type UploadAdmission struct {
	Owner Owner
	Pack  uuid.UUID
	Bytes int64
	Files int
}

UploadAdmission is what an upload that will queue a scan is admitted on: the pack and the tree read from the body.

type Usage

type Usage struct {
	// Bytes and Packs are the owner's own packs.
	Bytes int64
	Packs int
	// Used is what counts against the allocation: Bytes, or the pool's
	// total when Pooled.
	Used int64
	// Max is the allocation; 0 is no cap.
	Max    int64
	Pooled bool
}

Usage is an owner's meter: what the owner keeps, and the allocation it draws on — the owner's own, or a pool an extension shares among owners, whose total every owner in it sees.

type WrappedKey

type WrappedKey struct {
	Kind      string
	Recipient string
	Blob      []byte
}

WrappedKey is a pack's data key as its owner's wrapper sealed it: which wrapper (Kind), to which key (Recipient: the deployment key's id, or, with an extension's wrapper, a certificate's serial), and the sealed bytes. It is what a client keeps beside a sealed pack and what a pack's row holds.

func WrappedKeyOf

func WrappedKeyOf(p storegen.KnowledgePack) WrappedKey

WrappedKeyOf is a pack's data key as its row holds it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL