librarymanager

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 19, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// DatabaseFileName is the name of the database file created by bbolt.
	DatabaseFileName = "datadog-csi-driver.db"

	// VolumesBucket maps a volume to the library it uses. Key = volume ID,
	// value = JSON volumeRecord. The relationship is 1:1 (a volume mounts
	// exactly one library), so a flat bucket is all that is needed.
	VolumesBucket = "volumes"
	// LibrariesBucket holds one record per cached library. Key = library ID
	// (the image digest), value = JSON libraryRecord. It is the single
	// source of truth for the package label, the on-disk size and the number
	// of volumes currently using the library.
	LibrariesBucket = "libraries"
)
View Source 
const (
	// StoreDirectory is the subdirectory where active libraries are stored.
	StoreDirectory = "store"
	// DatabaseDirectory is the subdirectory where the databse file will be stored.
	DatabaseDirectory = "db"
	// ScratchDirectory is the subdirectory used for scratch download space for libraries.
	ScratchDirectory = "scratch"
	// DefaultImageCacheTTL is the max amount of time before we fetch a new image digest.
	DefaultImageCacheTTL = 1 * time.Hour
)

Variables

View Source 
var ErrItemNotFound = errors.New("item not found in store")

Functions

This section is empty.

Types

type ArchiveExtractor 

type ArchiveExtractor struct {
	// contains filtered or unexported fields
}

ArchiveExtractor extracts directories from a tar archive.

func NewArchiveExtractor 

func NewArchiveExtractor(src string, dst string) (*ArchiveExtractor, error)

NewArchiveExtractor initializes a new archive extractor.

func (*ArchiveExtractor) Extract 

func (fp *ArchiveExtractor) Extract(ctx context.Context, reader io.Reader) (int64, error)

Extract will copy files from the configured source directory inside the archive to the desitnation directory outside of the archive through the reader provided. Returns the cumulative size of the regular files written; symlinks and directories are not counted.

type CleanupFunc added in v1.2.0 

type CleanupFunc func(libraryID string) error

CleanupFunc is a function that performs cleanup for a library. It receives the libraryID and should re-check if cleanup is still needed.

type CleanupStrategy added in v1.2.0 

type CleanupStrategy interface {
	// ScheduleCleanup is called when a library has no more volumes using it.
	// The cleanupFunc will be called either immediately or after a delay,
	// depending on the strategy implementation.
	ScheduleCleanup(libraryID string, cleanupFunc CleanupFunc)

	// Stop stops the strategy and executes all pending cleanups.
	Stop()

	// Name returns the short identifier of the strategy (e.g. "immediate", "delayed").
	// Used for metric labels.
	Name() string
}

CleanupStrategy defines how libraries are cleaned up when no longer in use.

type Database 

type Database struct {
	// contains filtered or unexported fields
}

Database is a thin wrapper around bbolt.

Transaction consistency

bbolt provides serializable isolation: write transactions are mutually exclusive, reads see a consistent snapshot, and every transaction is atomic. As a result each method here is a single self-contained transaction and the database keeps no in-memory bookkeeping: the per-package aggregates the metrics listener needs are derived on demand by scanning LibrariesBucket (see Snapshot), which is cheap because a node only ever caches a handful of libraries.

External locking

Operations that combine a database write with a filesystem operation (e.g. LinkVolume followed by store.Add) still require external synchronisation; the LibraryManager uses a per-library Locker for that.

func NewDatabase 

func NewDatabase(basePath string) (*Database, error)

NewDatabase initializes a new database. If a database file exists it is reused (and migrated from the legacy schema if necessary). Call Close when you are done.

func (*Database) AddLibrary added in v1.3.0 

func (db *Database) AddLibrary(libraryID, packageName string, sizeBytes int64) error

AddLibrary records a freshly-cached library by persisting its package name and on-disk size. It is idempotent and preserves the volume count of an existing record, so it can safely be called again (for instance when the size changed) without disturbing the link bookkeeping.

func (*Database) Close 

func (db *Database) Close() error

Close cleans up the database and should be called before exiting.

func (*Database) GetLibrary added in v1.3.0 

func (db *Database) GetLibrary(libraryID string) (LibraryInfo, bool, error)

GetLibrary returns the stored information for a library. The boolean is false when the library has no record (for instance a legacy entry on disk that was never tracked with metadata).

func (*Database) GetLibraryForVolume 

func (db *Database) GetLibraryForVolume(volumeID string) (string, error)

GetLibraryForVolume returns the library ID a volume is linked to, or an empty string when the volume is not tracked.

func (*Database) LinkVolume 

func (db *Database) LinkVolume(libraryID, volumeID string, fromCache bool) error

LinkVolume records that volumeID uses libraryID and increments the library's volume count. fromCache notes whether the publish reused an already-cached library or had to download it; it is persisted on the record together with a creation timestamp.

A volume maps to exactly one library for its whole lifetime: callers resolve an already-linked volume from its existing record instead of re-resolving the image, so LinkVolume is only reached for volumes that are not yet linked. Linking a volume that is already tracked is therefore treated as an idempotent no-op rather than re-pointing it, which keeps the per-library counts from drifting even if the function is called twice.

func (*Database) RemoveLibrary added in v1.3.0 

func (db *Database) RemoveLibrary(libraryID string) error

RemoveLibrary deletes the record for a library. It is a no-op when the library is unknown.

func (*Database) Snapshot added in v1.3.0 

func (db *Database) Snapshot() (libraryevents.Snapshot, error)

Snapshot derives the per-package aggregates the metrics listener needs by scanning LibrariesBucket. It is cheap because a node only ever caches a small number of libraries. Libraries without a package label (legacy entries) are left out because they were never published as gauges.

func (*Database) UnlinkVolume 

func (db *Database) UnlinkVolume(volumeID string) (libraryID, packageName string, err error)

UnlinkVolume removes the link for a volume and decrements the owning library's volume count. It returns the library ID and package name the volume was linked to (both empty when the volume was not tracked, in which case it is a no-op). The package is read off the library record that is loaded to decrement the count, so callers get the metric label for free without an extra lookup.

type DelayedCleanupStrategy added in v1.2.0 

type DelayedCleanupStrategy struct {
	// contains filtered or unexported fields
}

DelayedCleanupStrategy waits for a configurable delay before executing cleanup. This allows rolling updates to reuse libraries without re-downloading them.

func NewDelayedCleanupStrategy added in v1.2.0 

func NewDelayedCleanupStrategy(delay time.Duration) *DelayedCleanupStrategy

NewDelayedCleanupStrategy creates a new delayed cleanup strategy. The delay parameter specifies how long to wait before cleaning up unused libraries.

func (*DelayedCleanupStrategy) Name added in v1.3.0 

func (s *DelayedCleanupStrategy) Name() string

func (*DelayedCleanupStrategy) ScheduleCleanup added in v1.2.0 

func (s *DelayedCleanupStrategy) ScheduleCleanup(libraryID string, cleanupFunc CleanupFunc)

func (*DelayedCleanupStrategy) Stop added in v1.2.0 

func (s *DelayedCleanupStrategy) Stop()

type Downloader 

type Downloader struct {
	// contains filtered or unexported fields
}

Downloader enables downloading and extracting directories from container images.

func NewDownloader 

func NewDownloader() *Downloader

NewDownloader creates a new downloader with the default settings.

func NewDownloaderWithKeychain added in v1.4.0 

func NewDownloaderWithKeychain(keychain authn.Keychain) *Downloader

NewDownloaderWithKeychain creates a downloader with driver-scoped registry credentials.

func NewDownloaderWithRoundTripper 

func NewDownloaderWithRoundTripper(roundTripper http.RoundTripper) *Downloader

NewDownloaderWithRoundTripper creates a new downloader with the provided round tripper.

func (*Downloader) Download 

func (d *Downloader) Download(ctx context.Context, image string, dst string) (int64, error)

Download will stream a container image and extract the source directory from inside of the image to the destination directory on disk. Returns the cumulative size of the regular files written.

func (*Downloader) FetchDigest 

func (d *Downloader) FetchDigest(ctx context.Context, image string) (string, error)

FetchDigest will fetch a sha256 sum of the image and return it.

type ImageCache 

type ImageCache struct {
	// contains filtered or unexported fields
}

ImageCache provides an in memory cache of container image digests so we don't have to resolve a container tag to sha256sum each time.

func NewImageCache 

func NewImageCache(d *Downloader, ttl time.Duration) *ImageCache

NewImageChace initializes a new, empty image cache.

func (*ImageCache) FetchDigest 

func (ic *ImageCache) FetchDigest(ctx context.Context, image string, pull bool) (string, error)

FetchDigest returns the sha256 digest for a container image, using the cache when possible.

The image parameter must be a valid container image reference as accepted by crane (https://pkg.go.dev/github.com/google/go-containerregistry/pkg/crane). Examples:

  • "gcr.io/datadoghq/dd-lib-java-init:v1.2.3"
  • "gcr.io/datadoghq/dd-lib-java-init@sha256:abc123..."
  • "nginx:latest" (defaults to docker.io registry)

If the image already contains a digest (@sha256:...), this function will still resolve the full digest from the registry to ensure it exists and is valid.

If pull is true, the cache is bypassed and a fresh digest is always fetched from the registry. If pull is false, the cache is checked first and a remote call is only made on cache miss.

type ImmediateCleanupStrategy added in v1.2.0 

type ImmediateCleanupStrategy struct{}

ImmediateCleanupStrategy executes cleanup immediately when a library is no longer used. This is the default behavior.

func NewImmediateCleanupStrategy added in v1.2.0 

func NewImmediateCleanupStrategy() *ImmediateCleanupStrategy

NewImmediateCleanupStrategy creates a new immediate cleanup strategy.

func (*ImmediateCleanupStrategy) Name added in v1.3.0 

func (s *ImmediateCleanupStrategy) Name() string

func (*ImmediateCleanupStrategy) ScheduleCleanup added in v1.2.0 

func (s *ImmediateCleanupStrategy) ScheduleCleanup(libraryID string, cleanupFunc CleanupFunc)

func (*ImmediateCleanupStrategy) Stop added in v1.2.0 

func (s *ImmediateCleanupStrategy) Stop()

type Library 

type Library struct {
	// contains filtered or unexported fields
}

Library represents a Datadog package to download and mount as part of a DatadogLibrary volume request.

func NewLibrary 

func NewLibrary(name string, registry string, version string, pull bool) (*Library, error)

NewLibrary instatiates a new library from the provided fields and ensures they are valid.

func (*Library) Image 

func (l *Library) Image() string

Image provides a container image path pullable by crane. Handles tag, digest, and tag@digest versions:

  • Tags: registry/name:v1.0.0
  • Digests: registry/name@sha256:abc123...
  • Tag+Digest: registry/name:v1.0.0@sha256:abc123...

func (*Library) Name added in v1.3.0 

func (l *Library) Name() string

Name returns the package name of the library (e.g. dd-lib-java-init, apm-inject).

func (*Library) Pull 

func (l *Library) Pull() bool

Pull returns if this library should be pulled or not based on the pull policy.

func (*Library) Registry added in v1.3.0 

func (l *Library) Registry() string

Registry returns the registry the library is pulled from.

type LibraryInfo added in v1.3.0 

type LibraryInfo struct {
	// Package is the canonical package name used as the metric label. It is
	// empty for legacy entries that predate per-library metadata.
	Package string
	// SizeBytes is the on-disk size of the library, in bytes.
	SizeBytes int64
	// VolumeCount is the number of volumes currently linked to the library.
	VolumeCount int
}

LibraryInfo is the public, read-only view of a library record returned by GetLibrary.

type LibraryManager 

type LibraryManager struct {
	// contains filtered or unexported fields
}

LibraryManager is a high level object to manage fetching libraries for volumes. It will download, extract, store, and track libraries and how they map to a volume.

func NewLibraryManager 

func NewLibraryManager(basePath string, opts ...LibraryManagerOption) (*LibraryManager, error)

NewLibraryManager creates a new library manager with all of the required dependencies. The basePath is required as an absolute path (rather than using afero.NewBasePathFs) because bind mounts need absolute paths.

func (*LibraryManager) GetLibraryForVolume 

func (lm *LibraryManager) GetLibraryForVolume(ctx context.Context, volumeID string, lib *Library) (string, error)

GetLibraryForVolume fetches the remote library if it doesn't exist, records its usage, and returns the path on disk that can be mounted for the volume.

func (*LibraryManager) HasVolume added in v1.2.0 

func (lm *LibraryManager) HasVolume(volumeID string) (bool, error)

HasVolume returns true if the volume is managed by the library manager.

func (*LibraryManager) RemoveVolume 

func (lm *LibraryManager) RemoveVolume(_ context.Context, volumeID string) error

RemoveVolume removes the link between the LibraryID and the VolumeID in the database. If there are no more uses of the library, it is also removed from disk. Calling RemoveVolume for a volume that was never linked is a no-op.

func (*LibraryManager) Stop 

func (lm *LibraryManager) Stop() error

Stop ensures all dependencies are stopped correctly.

type LibraryManagerOption 

type LibraryManagerOption func(*LibraryManager)

LibraryManagerOption is a functional option for configuring a LibraryManager.

func WithCleanupStrategy added in v1.2.0 

func WithCleanupStrategy(s CleanupStrategy) LibraryManagerOption

WithCleanupStrategy sets the cleanup strategy to use. If not set, ImmediateCleanupStrategy is used by default.

func WithDownloader 

func WithDownloader(d *Downloader) LibraryManagerOption

WithDownloader sets the downloader to use. Useful for testing.

func WithEventListener added in v1.3.0 

func WithEventListener(l libraryevents.Listener) LibraryManagerOption

WithEventListener injects a Listener. Without this option the manager uses a no-op listener; the production wiring should always pass an implementation that publishes metrics (or any other observability signal).

func WithFilesystem 

func WithFilesystem(fs afero.Afero) LibraryManagerOption

WithFilesystem sets the filesystem to use. Useful for testing.

type Locker 

type Locker struct {
	// contains filtered or unexported fields
}

Locker is a sharded mutex to be able to perform concurrent operations on unrelated keys.

func NewLocker 

func NewLocker() *Locker

NewLocker initializes a new locker.

func (*Locker) Lock 

func (l *Locker) Lock(id string)

Lock will acquire a lock for the given ID. The caller MUST call unlock.

func (*Locker) Unlock 

func (l *Locker) Unlock(id string)

Unlock will release a lock for the given ID.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store provides a file based storage solution for packages. It is not thread safe and it is up to the caller to manage concurrency.

func NewStore 

func NewStore(afs afero.Afero, basePath string) (*Store, error)

NewStore creates a new store and ensures the base path exists.

func (*Store) Add 

func (s *Store) Add(id string, src string) (string, error)

Add will move a source directory into the store. This is intended to be used with a downloader and scratch space. If a package already exists at the provided ID, it will not be re-added.

func (*Store) Exists 

func (s *Store) Exists(id string) (bool, error)

Exists determines if an item exists in the store.

func (*Store) Get 

func (s *Store) Get(id string) (string, error)

Get returns an item in the store if it exists.

func (*Store) Remove 

func (s *Store) Remove(id string) error

Remove deletes an item from the store.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.