Documentation
¶
Overview ¶
Package rbd prepares RBD server fixtures in disposable Ceph clusters: initialized image pools, namespaces and full or incremental backup transport. Mirroring between two clusters is in the rbd/mirror package.
Run starts a cluster with at least one initialized RBD pool. InitPool prepares a pool on a cluster created by ceph.Run or another service package's Run, and WithPools selects RBD pools for any of those Run functions.
Index ¶
- Constants
- func ExportBackup(ctx context.Context, client testcontainers.Container, image string, ...) error
- func ExportIncremental(ctx context.Context, client testcontainers.Container, ...) error
- func InitPool(ctx context.Context, c *ceph.Container, pool string) error
- func ListNamespaces(ctx context.Context, c *ceph.Container, pool string) ([]string, error)
- func RemoveNamespace(ctx context.Context, c *ceph.Container, ns *Namespace) error
- func RestoreBackup(ctx context.Context, client testcontainers.Container, image string, ...) error
- func RestoreIncremental(ctx context.Context, client testcontainers.Container, image string, ...) error
- func WithPools(configs ...ceph.PoolConfig) ceph.Option
- type Container
- type ImageClientStatus
- type ImageLock
- type ImageWatcher
- type ImmutableObjectCache
- type ImmutableObjectCacheLease
- func (l *ImmutableObjectCacheLease) ClientOptions() map[string]string
- func (l *ImmutableObjectCacheLease) Restore(ctx context.Context) error
- func (l *ImmutableObjectCacheLease) Status(ctx context.Context) (ImmutableObjectCacheStatus, error)
- func (l *ImmutableObjectCacheLease) Stop(ctx context.Context) error
- type ImmutableObjectCacheStatus
- type Namespace
Examples ¶
Constants ¶
const ( DefaultImmutableObjectCachePath = "/var/cache/ceph/immutable-object-cache" DefaultImmutableObjectCacheSocket = "/var/run/ceph/immutable-object-cache.sock" DefaultImmutableObjectCacheMaxSize = 1 << 30 )
Defaults of ImmutableObjectCache.
Variables ¶
This section is empty.
Functions ¶
func ExportBackup ¶
func ExportBackup(ctx context.Context, client testcontainers.Container, image string, destination io.Writer) error
ExportBackup writes a CLI-generated format-2 archive to destination. The archive includes image metadata and snapshots. No mirror daemon is required. The caller owns the writer and decides where to retain the backup.
client is any container that has the rbd CLI and credentials for its own cluster, such as ceph.Container.ClientContainer or a container attached with WithClient, and image is an image spec such as volumes/disk or volumes/tenant-a/disk. The archive is written to a temporary file under /tmp in the client container and removed afterwards. The backup functions never close the caller's reader or writer, and they check cancellation between reads, so a reader or writer that can block indefinitely must support its own cancellation.
func ExportIncremental ¶
func ExportIncremental(ctx context.Context, client testcontainers.Container, image, fromSnapshot string, destination io.Writer) error
ExportIncremental writes the CLI diff from fromSnapshot to image (which can include @snapshot). Restore requires the matching baseline snapshot. fromSnapshot is a bare snapshot name. With image@snapshot the diff ends at that snapshot, and restoring it creates that snapshot on the target. The client and stream rules are those of ExportBackup.
func InitPool ¶
InitPool initializes an existing replicated pool for RBD metadata. Pools registered to other applications are rejected; no force option is used. An existing RBD initialization is safe to repeat. EC pools may supply image data but cannot hold RBD metadata. Create the pool first, then wait for clean PGs before client I/O. Partial initialization never deletes pool data. Success is verified natively, by the pool's application tag and its rbd_trash object, not only by the CLI exit code. Pool names may contain letters, digits, _, . and -, must not start with . or -, and are at most 128 characters.
Example ¶
package main
import (
"context"
"log"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
"github.com/JSYoo5B/ceph-testcontainers-go/rbd"
"github.com/testcontainers/testcontainers-go"
)
func main() {
ctx := context.Background()
cluster, err := ceph.Run(ctx, ceph.DefaultImage,
ceph.WithClusterConfig(ceph.ClusterConfig{}.WithPools(ceph.PoolConfig{Name: "images", Application: "rbd"})),
)
defer func() {
if err := testcontainers.TerminateContainer(cluster); err != nil {
log.Printf("failed to terminate container: %s", err)
}
}()
if err != nil {
panic(err)
}
if err := rbd.InitPool(ctx, cluster, "images"); err != nil {
panic(err)
}
namespace, err := rbd.CreateNamespace(ctx, cluster, "images", "tenant-a")
if err != nil {
panic(err)
}
// Pass namespace.PoolName() and namespace.Name() to a native RBD client.
// Images, snapshots and trash must be removed before deleting the namespace.
if err := rbd.RemoveNamespace(ctx, cluster, namespace); err != nil {
panic(err)
}
}
Output:
func ListNamespaces ¶
ListNamespaces returns the pool's native namespace names in sorted order, including names created outside this fixture. The unnamed default namespace is omitted by Ceph. Listing does not grant ownership for removal.
func RemoveNamespace ¶
RemoveNamespace removes a confirmed namespace owned by this cluster. Ceph refuses nonempty namespaces, including images in trash. No image purge or force option is used. Successful removal is idempotent; failed CLI removal can be retried with the same descriptor and a new context. Replacing its underlying pool is rejected. Do not delete and recreate the same named namespace through another client while its descriptor is in use: native RBD namespaces have no separate generation identity to distinguish replacements.
func RestoreBackup ¶
func RestoreBackup(ctx context.Context, client testcontainers.Container, image string, archive io.Reader) error
RestoreBackup imports a format-2 archive into a new image on the client's own cluster. Pool creation and archive retention belong to the caller. It first spools the reader into a private temporary file on the test host, so a large archive is not held in memory, then copies it into the container; the client and stream rules are those of ExportBackup.
func RestoreIncremental ¶
func RestoreIncremental(ctx context.Context, client testcontainers.Container, image string, archive io.Reader) error
RestoreIncremental applies a CLI diff to an existing image. Ceph checks that its baseline snapshot exists; this function does not invent a baseline. The check is by snapshot name only: nothing proves that the baseline has the same content.
func WithPools ¶
func WithPools(configs ...ceph.PoolConfig) ceph.Option
WithPools creates replicated pools for RBD images during Run and initializes their RBD metadata. Application defaults to rbd and must not be another application; erasure-coded data pools are created with ceph.ClusterConfig.WithPools and referenced as an image's data pool. Repeated options replace the list, and passing the option replaces the default rbd pool of Run. The other ceph.PoolConfig fields keep their ceph meaning: a zero PGNum selects 8 PGs with the autoscaler off, zero Replicas and MinSize select the cluster defaults (2/1, or 1/1 with one OSD), and the placement fields are validated against the OSDs like those of any other pool.
Types ¶
type Container ¶
Container is a cluster started by Run. It embeds *ceph.Container, so every cluster method is available on it; pass its Container field to functions that take a *ceph.Container, such as those of this and the other service packages.
func Run ¶
func Run(ctx context.Context, img string, opts ...testcontainers.ContainerCustomizer) (*Container, error)
Run starts a disposable cluster for RBD clients. It accepts every ceph.Run option and the options of other service packages. When no WithPools option is given, it creates and initializes a replicated pool named rbd. Bootstrap options that leave the cluster without OSDs for the pool, such as ceph.ClusterConfig.WithNoInitialOSDs, are rejected; use ceph.Run for those stages. A non-nil Container returned with an error must still be terminated.
Run returns after the pools exist and their RBD metadata is initialized. PG peering may still be finishing, so call WaitForClean before the first client I/O.
Example ¶
package main
import (
"context"
"log"
"github.com/JSYoo5B/ceph-testcontainers-go/ceph"
"github.com/JSYoo5B/ceph-testcontainers-go/rbd"
"github.com/testcontainers/testcontainers-go"
)
func main() {
ctx := context.Background()
// Without WithPools, Run creates and initializes the replicated pool rbd.
cluster, err := rbd.Run(ctx, ceph.DefaultImage,
rbd.WithPools(ceph.PoolConfig{Name: "volumes"}),
)
defer func() {
if err := testcontainers.TerminateContainer(cluster); err != nil {
log.Printf("failed to terminate container: %s", err)
}
}()
if err != nil {
panic(err)
}
// Connect a native RBD client with cluster.ConnectionConfig().
}
Output:
type ImageClientStatus ¶
type ImageClientStatus struct {
Watchers []ImageWatcher
Locks []ImageLock
}
ImageClientStatus reports an image's watchers and locks. The two lists come from separate native reads, so a lock handoff between them can show a new owner without its watch or the reverse; poll for a stable answer.
func ImageClients ¶
func ImageClients(ctx context.Context, c *ceph.Container, pool, namespace, image string) (*ImageClientStatus, error)
ImageClients reads which clients watch an image and which hold its locks, including clients outside this fixture. Namespace is empty for the default namespace. It reads native state only and grants no ownership.
func (*ImageClientStatus) ExclusiveOwner ¶
func (status *ImageClientStatus) ExclusiveOwner() *ImageLock
ExclusiveOwner returns the managed exclusive lock, or nil if no client holds it.
type ImageLock ¶
type ImageLock struct {
// ID is the lock ID; for a managed lock it is "auto <watch cookie>".
ID string
// Locker is the owner's entity name, such as client.4235.
Locker string
// Address is the owner's IP:port/nonce, the form that
// override.AddBlocklist takes.
Address string
// Managed marks the lock that librbd takes for exclusive-lock.
Managed bool
}
ImageLock is one lock on an image header. Managed marks the lock librbd takes for the exclusive-lock feature; its ID carries the owner's watch cookie. Other locks are advisory locks added through rbd lock add.
type ImageWatcher ¶
ImageWatcher is one client watching an image header. Address is the client's IP:port/nonce, the same form BlocklistEntries reports. A client that stops answering keeps its watch until the OSD's osd_client_watch_timeout expires, 30 seconds by default.
type ImmutableObjectCache ¶ added in v0.4.0
type ImmutableObjectCache struct {
// ID is the cephx ID, client.<ID>, that the daemon uses to read the
// parent objects; empty means admin. The container's ceph.conf must name
// its keyring, as ceph.ClientConfig.WithIdentity does.
ID string
// Path is the cache directory; empty means
// DefaultImmutableObjectCachePath. It must be absent or empty.
Path string
// Socket is the path of the daemon's client socket; empty means
// DefaultImmutableObjectCacheSocket. It must not exist, and it must not
// be under Path.
Socket string
// MaxSize is immutable_object_cache_max_size in bytes; 0 means
// DefaultImmutableObjectCacheMaxSize. The daemon treats it as a target,
// not a hard limit.
MaxSize int64
}
ImmutableObjectCache configures the RBD immutable object cache daemon, ceph-immutable-object-cache, that StartImmutableObjectCache starts in a client container. The daemon caches whole parent-image objects of clones for the librbd clients in that container that enable the parent cache.
type ImmutableObjectCacheLease ¶ added in v0.4.0
type ImmutableObjectCacheLease struct {
// Config is the request with its defaults filled in.
Config ImmutableObjectCache
// contains filtered or unexported fields
}
ImmutableObjectCacheLease owns one daemon that StartImmutableObjectCache started, with its socket and cache directory. It signals the process only while its command line holds the lease's socket, so a reused PID is never stopped. To count cache use, read Status.RADOSReads: rados df read counters lag behind and do not show it.
func StartImmutableObjectCache ¶ added in v0.4.0
func StartImmutableObjectCache(ctx context.Context, client testcontainers.Container, config ImmutableObjectCache) (*ImmutableObjectCacheLease, error)
StartImmutableObjectCache starts the RBD immutable object cache daemon in client, a container that the caller owns, and returns its lease. A librbd client in the same container uses the cache when it sets the lease's ClientOptions; other clients read the parents from RADOS. The container must have ceph-immutable-object-cache on its PATH and the librbd parent_cache plugin in its plugin directory: the official Ceph image has both, and an image without them is refused with ceph.ErrImageCapability. An existing socket, a non-empty cache directory or an invalid path is refused with ceph.ErrInvalidRequest, and nothing is changed. A daemon that exits or never opens its socket fails the call with ceph.ErrNativeFailure, and what it left is removed.
func (*ImmutableObjectCacheLease) ClientOptions ¶ added in v0.4.0
func (l *ImmutableObjectCacheLease) ClientOptions() map[string]string
ClientOptions returns the Ceph options that make a librbd client in the lease's container use the cache, for a ceph.conf section, the rbd CLI (as --name=value) or rados.Rados(conf=...).
func (*ImmutableObjectCacheLease) Restore ¶ added in v0.4.0
func (l *ImmutableObjectCacheLease) Restore(ctx context.Context) error
Restore stops the daemon if it runs and removes its socket, admin socket, PID file and cached objects. It removes the cache directory only when the lease created it. A repeated call after success does nothing.
func (*ImmutableObjectCacheLease) Status ¶ added in v0.4.0
func (l *ImmutableObjectCacheLease) Status(ctx context.Context) (ImmutableObjectCacheStatus, error)
Status reports the daemon and the cache directory.
func (*ImmutableObjectCacheLease) Stop ¶ added in v0.4.0
func (l *ImmutableObjectCacheLease) Stop(ctx context.Context) error
Stop stops the daemon, with SIGTERM and then SIGKILL, and keeps the cache directory and socket file: librbd clients that open an image afterwards fail to register with the cache and read the parents from RADOS. A daemon that already exited needs no stop.
type ImmutableObjectCacheStatus ¶ added in v0.4.0
type ImmutableObjectCacheStatus struct {
// Running means that the daemon the lease started still runs.
Running bool
// PID is the daemon's process ID in the client container.
PID int
// CachedObjects and CachedBytes count the files under the cache
// directory: one per cached parent object.
CachedObjects int
CachedBytes int64
// RADOSReads is the daemon's objecter read count while Running: the
// parent objects it read from RADOS to cache them. The daemon has no
// cache hit counters of its own.
RADOSReads uint64
}
ImmutableObjectCacheStatus is what the client container reports for the daemon and its cache directory.
type Namespace ¶
type Namespace struct {
// contains filtered or unexported fields
}
Namespace identifies an image namespace created by this cluster. Names and native pool identity are immutable. Namespace removal never deletes images, snapshots or trash; callers must remove those through their RBD client first. The cluster disposes of namespaces with all its data when it terminates.
func CreateNamespace ¶
CreateNamespace creates a fresh image namespace in an initialized, replicated RBD pool. Existing names are rejected without changing their data. A non-nil descriptor returned with an error identifies an attempted creation, but cannot authorize removal unless creation was confirmed. Inspect native state or dispose of the cluster after an uncertain creation failure. Namespace names follow the rules of InitPool's pool names. Namespaces need not be removed for cleanup: terminating the cluster disposes of them with all their data.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
resourcename
Package resourcename validates the pool, namespace and image names accepted by standalone RBD operations.
|
Package resourcename validates the pool, namespace and image names accepted by standalone RBD operations. |
|
Package mirror connects two Ceph clusters with native RBD mirroring: setup CLI containers, one-way pool or image mirroring in snapshot or journal mode, any number of rbd-mirror receiver daemons, and typed readiness observation.
|
Package mirror connects two Ceph clusters with native RBD mirroring: setup CLI containers, one-way pool or image mirroring in snapshot or journal mode, any number of rbd-mirror receiver daemons, and typed readiness observation. |