backuptest

package
v1.35.21 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 21, 2026 License: BSD-3-Clause Imports: 41 Imported by: 0

Documentation

Overview

Package backuptest provides self-contained test infrastructure for backup integration tests. It handles automatic provisioning and cleanup of storage backend emulators (MinIO for S3, GCS emulator, Azurite for Azure) with zero required configuration.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNotStarted is returned when methods requiring a running backend
	// are called before Start() succeeds.
	ErrNotStarted = errors.New("backend not started")

	// ErrAlreadyStarted is returned when Start() is called on an already
	// running backend.
	ErrAlreadyStarted = errors.New("backend already started")

	// ErrAlreadyStopped is returned when operations are attempted on a
	// stopped backend.
	ErrAlreadyStopped = errors.New("backend already stopped")

	// ErrBucketExists is returned when CreateBucket is called with a
	// bucket name that already exists.
	ErrBucketExists = errors.New("bucket already exists")
)

Common errors returned by BackupBackend implementations.

Functions

func AssertBackupSucceeded 

func AssertBackupSucceeded(t *testing.T, backend, backupID string)

AssertBackupSucceeded is a helper to verify a backup completed successfully.

func AssertRestoreSucceeded 

func AssertRestoreSucceeded(t *testing.T, backend, backupID string)

AssertRestoreSucceeded is a helper to verify a restore completed successfully.

func RunAzureBackupTests 

func RunAzureBackupTests(t *testing.T, compose *docker.DockerCompose, testCase BackupTestCase)

RunAzureBackupTests runs Azure backup tests using a shared compose and test case configuration.

func RunFilesystemBackupTests 

func RunFilesystemBackupTests(t *testing.T, compose *docker.DockerCompose, testCase BackupTestCase)

RunFilesystemBackupTests runs filesystem backup tests using a shared compose and test case configuration. Note: Filesystem backup only works on single-node clusters. The shared compose must be a single-node cluster.

func RunGCSBackupTests 

func RunGCSBackupTests(t *testing.T, compose *docker.DockerCompose, testCase BackupTestCase)

RunGCSBackupTests runs GCS backup tests using a shared compose and test case configuration.

func RunS3BackupTests 

func RunS3BackupTests(t *testing.T, compose *docker.DockerCompose, minioEndpoint, region string, testCase BackupTestCase)

RunS3BackupTests runs S3 backup tests using a shared compose and test case configuration.

func UniqueBucketName 

func UniqueBucketName(prefix string) string

UniqueBucketName generates a valid S3 bucket name (lowercase, dashes allowed, no underscores).

func UniqueClassName 

func UniqueClassName(prefix string) string

UniqueClassName generates a valid Weaviate class name (underscores allowed, no dashes).

func UniqueTestID 

func UniqueTestID(prefix string) string

UniqueTestID generates a unique identifier for test isolation. Returns just the timestamp portion - callers should format appropriately: - Class names: use underscores (dashes invalid in Weaviate) - Bucket names: use dashes (underscores invalid in S3)

Types

type AzureBackend 

type AzureBackend struct {
	*BaseBackend
	// contains filtered or unexported fields
}

AzureBackend implements BackupBackend for Azure Blob Storage using Azurite.

func NewAzureBackend 

func NewAzureBackend(opts ...BackendOption) *AzureBackend

NewAzureBackend creates a new AzureBackend with Azurite.

func (*AzureBackend) AzureClient 

func (b *AzureBackend) AzureClient() *azblob.Client

AzureClient returns the underlying Azure Blob client for advanced operations. Returns nil if the backend is not running.

func (*AzureBackend) ConnectionString 

func (b *AzureBackend) ConnectionString() string

ConnectionString returns the Azure connection string for external access.

func (*AzureBackend) ContainerName 

func (b *AzureBackend) ContainerName() string

ContainerName returns the Azurite container name (Docker container, not Azure container).

func (*AzureBackend) CreateBucket 

func (b *AzureBackend) CreateBucket(ctx context.Context, name string) error

CreateBucket creates a new Azure container (Azure's term for bucket).

func (*AzureBackend) CreateBucketInternal 

func (b *AzureBackend) CreateBucketInternal(ctx context.Context, name string) error

CreateBucketInternal creates a container without state checks (used during Start).

func (*AzureBackend) DeleteBucket 

func (b *AzureBackend) DeleteBucket(ctx context.Context, name string) error

DeleteBucket removes an Azure container from Azurite.

func (*AzureBackend) Endpoint 

func (b *AzureBackend) Endpoint() string

Endpoint returns the Azurite blob HTTP endpoint for external access.

func (*AzureBackend) GetWeaviateEnv 

func (b *AzureBackend) GetWeaviateEnv() map[string]string

GetWeaviateEnv returns environment variables for Weaviate Azure backup configuration.

func (*AzureBackend) InternalEndpoint 

func (b *AzureBackend) InternalEndpoint() string

InternalEndpoint returns the endpoint for use within the Docker network.

func (*AzureBackend) ListBuckets 

func (b *AzureBackend) ListBuckets(ctx context.Context) ([]string, error)

ListBuckets returns all Azure containers (buckets) in Azurite.

func (*AzureBackend) Name 

func (b *AzureBackend) Name() string

Name returns "azure".

func (*AzureBackend) NetworkName 

func (b *AzureBackend) NetworkName() string

NetworkName returns the Docker network name used by this backend.

func (*AzureBackend) Start 

func (b *AzureBackend) Start(ctx context.Context) error

Start provisions Azurite and creates the default container.

func (*AzureBackend) Stop 

func (b *AzureBackend) Stop(ctx context.Context) error

Stop terminates the Azurite container and cleans up resources.

type BackendOption 

type BackendOption func(*backendConfig)

BackendOption configures a BackupBackend.

func WithBucketName 

func WithBucketName(name string) BackendOption

WithBucketName sets the default bucket/container name. Default is "backups".

func WithContainerReuse 

func WithContainerReuse(reuse bool) BackendOption

WithContainerReuse enables/disables container reuse. When enabled, existing containers with matching names will be reused. Default is true.

func WithNetworkName 

func WithNetworkName(name string) BackendOption

WithNetworkName sets the Docker network name to use. If empty, a new network will be created.

type BackendState 

type BackendState int

BackendState tracks the lifecycle state of a backend. 

const (
	// StateNotStarted is the initial state before Start() is called.
	StateNotStarted BackendState = iota
	// StateStarting indicates Start() is in progress.
	StateStarting
	// StateRunning indicates the backend is fully operational.
	StateRunning
	// StateStopping indicates Stop() is in progress.
	StateStopping
	// StateStopped indicates the backend has been stopped.
	StateStopped
)

func (BackendState) String 

func (s BackendState) String() string

type BackendTestCase 

type BackendTestCase struct {
	Name        string
	BackendType string
	BucketName  string
	Region      string // Only for S3
}

BackendTestCase defines a test case for a specific backend.

func StandardBackendTestCases 

func StandardBackendTestCases() []BackendTestCase

StandardBackendTestCases returns test cases for all standard backends.

type BackupBackend 

type BackupBackend interface {
	// Name returns the backend identifier used by Weaviate (s3, gcs, azure).
	Name() string

	// Start provisions all required infrastructure:
	// - Starts the emulator container (MinIO, GCS emulator, or Azurite)
	// - Creates the default bucket/container
	// - Waits for the service to be healthy
	// Returns an error if provisioning fails.
	Start(ctx context.Context) error

	// Stop tears down all infrastructure created by Start.
	// This should be called in a defer after Start succeeds.
	// It's safe to call Stop multiple times.
	Stop(ctx context.Context) error

	// GetWeaviateEnv returns environment variables that Weaviate needs to
	// connect to this backend. These should be passed to the Weaviate container.
	// Must only be called after Start() succeeds.
	GetWeaviateEnv() map[string]string

	// BucketName returns the default bucket/container name used for backups.
	BucketName() string

	// CreateBucket creates an additional bucket/container for override testing.
	// Must only be called after Start() succeeds.
	CreateBucket(ctx context.Context, name string) error

	// Endpoint returns the emulator's endpoint URI for external access.
	// Must only be called after Start() succeeds.
	Endpoint() string
}

BackupBackend encapsulates all setup/teardown for a backup storage backend. Implementations handle container lifecycle, bucket/container creation, and provide environment variables for Weaviate configuration.

type BackupTestCase 

type BackupTestCase struct {
	// Name is a short identifier for the test case (used in class/backup naming)
	Name string

	// MultiTenant enables multi-tenant testing
	MultiTenant bool

	// NumTenants overrides the default number of tenants (default: 3, or 50 for multi-tenant)
	NumTenants int

	// ObjectsPerTenant overrides the default objects per tenant (default: 10)
	ObjectsPerTenant int

	// WithMidBackupActivations specifies whether to activate/deactivate tenants in the middle of backup to test activity status handling
	WithMidBackupActivations bool

	// WithCompression specifies the compression algorithm to enable (CompressionPQ or CompressionRQ)
	WithCompression CompressionType

	// TestCancellation tests backup cancellation instead of full backup/restore
	TestCancellation bool

	// TestIncremental tests incremental backup and restore
	TestIncremental bool
}

BackupTestCase defines a test case configuration that can be extended with new options. This allows adding new test parameters without changing function signatures.

func CancellationTestCase 

func CancellationTestCase() BackupTestCase

CancellationTestCase returns a test case for testing backup cancellation.

func DefaultTestCase 

func DefaultTestCase() BackupTestCase

DefaultTestCase returns a test case with default settings (single-tenant).

func IncrementalTestCase added in v1.34.18 

func IncrementalTestCase() BackupTestCase

IncrementalTestCase returns a test case for testing incremental backups.

func MultiTenantTestCase 

func MultiTenantTestCase() BackupTestCase

MultiTenantTestCase returns a test case for multi-tenant backup testing.

func MultiTenantTestCaseWithMidBackupActivations added in v1.35.17 

func MultiTenantTestCaseWithMidBackupActivations() BackupTestCase

MultiTenantTestCaseWithMidBackupActivations returns a test case for multi-tenant backup testing with tenant activations/deactivations in the middle of backup.

func MultiTenantWithPQTestCase 

func MultiTenantWithPQTestCase() BackupTestCase

MultiTenantWithPQTestCase returns a test case for multi-tenant backup with PQ compression.

func MultiTenantWithRQTestCase 

func MultiTenantWithRQTestCase() BackupTestCase

MultiTenantWithRQTestCase returns a test case for multi-tenant backup with RQ compression.

func SingleTenantTestCase 

func SingleTenantTestCase() BackupTestCase

SingleTenantTestCase returns a test case for single-tenant backup testing.

func SingleTenantWithPQTestCase 

func SingleTenantWithPQTestCase() BackupTestCase

SingleTenantWithPQTestCase returns a test case for single-tenant backup with PQ compression.

func SingleTenantWithRQTestCase 

func SingleTenantWithRQTestCase() BackupTestCase

SingleTenantWithRQTestCase returns a test case for single-tenant backup with RQ compression.

type BackupTestSuite 

type BackupTestSuite struct {
	// contains filtered or unexported fields
}

BackupTestSuite provides a reusable test suite for backup functionality.

func NewBackupTestSuite 

func NewBackupTestSuite(config *BackupTestSuiteConfig) *BackupTestSuite

NewBackupTestSuite creates a new backup test suite with the given configuration. Any zero-value fields in config will be filled with defaults.

func (*BackupTestSuite) ActivateSomeTestTenants added in v1.35.17 

func (s *BackupTestSuite) ActivateSomeTestTenants(t *testing.T)

ActivateSomeTestTenants activates random tenants in the middle of backup to test activity status handling.

func (*BackupTestSuite) CreateBackup 

func (s *BackupTestSuite) CreateBackup(t *testing.T, backupID, baseBackupID string)

CreateBackup creates a backup and waits for it to complete.

func (*BackupTestSuite) CreateTestClass 

func (s *BackupTestSuite) CreateTestClass(t *testing.T)

CreateTestClass creates the test class schema.

func (*BackupTestSuite) CreateTestObjects 

func (s *BackupTestSuite) CreateTestObjects(t *testing.T, saveObjects bool)

CreateTestObjects creates test objects and stores their IDs.

func (*BackupTestSuite) CreateTestTenants 

func (s *BackupTestSuite) CreateTestTenants(t *testing.T)

CreateTestTenants creates tenants for multi-tenant tests.

func (*BackupTestSuite) DeactivateSomeTestTenants added in v1.35.17 

func (s *BackupTestSuite) DeactivateSomeTestTenants(t *testing.T)

DeactivateSomeTestTenants deactivates random tenants in the middle of backup to test activity status handling.

func (*BackupTestSuite) DeactivateTestTenants added in v1.35.17 

func (s *BackupTestSuite) DeactivateTestTenants(t *testing.T)

DeactivateTestTenants deactivates tenants to test backup/restore with inactive tenants.

func (*BackupTestSuite) DeleteTestClass 

func (s *BackupTestSuite) DeleteTestClass(t *testing.T)

DeleteTestClass removes the test class.

func (*BackupTestSuite) EnablePQ 

func (s *BackupTestSuite) EnablePQ(t *testing.T)

EnablePQ enables Product Quantization compression on the class.

func (*BackupTestSuite) EnableRQ 

func (s *BackupTestSuite) EnableRQ(t *testing.T)

EnableRQ enables Rotational Quantization compression on the class.

func (*BackupTestSuite) GetActiveTenants added in v1.35.17 

func (s *BackupTestSuite) GetActiveTenants(t *testing.T) []string

GetActiveTenants returns the list of active tenants (those not deactivated) for multi-tenant tests.

func (*BackupTestSuite) GetInactiveTenants added in v1.35.17 

func (s *BackupTestSuite) GetInactiveTenants(t *testing.T) []string

GetInactiveTenants returns the list of tenants that should be deactivated for testing.

func (*BackupTestSuite) GetObjectCount 

func (s *BackupTestSuite) GetObjectCount() int

GetObjectCount returns the number of objects created by this test suite.

func (*BackupTestSuite) GetObjectIDs 

func (s *BackupTestSuite) GetObjectIDs() []string

GetObjectIDs returns the IDs of all objects created by this test suite.

func (*BackupTestSuite) RestoreBackup 

func (s *BackupTestSuite) RestoreBackup(t *testing.T, backupID string)

RestoreBackup restores a backup and waits for it to complete.

func (*BackupTestSuite) RunBasicBackupRestoreTest 

func (s *BackupTestSuite) RunBasicBackupRestoreTest(t *testing.T)

RunBasicBackupRestoreTest runs a complete backup and restore test cycle.

func (*BackupTestSuite) RunCancellationTest 

func (s *BackupTestSuite) RunCancellationTest(t *testing.T)

RunCancellationTest tests that backups can be cancelled.

func (*BackupTestSuite) RunIncrementalTestAndRestore added in v1.34.18 

func (s *BackupTestSuite) RunIncrementalTestAndRestore(t *testing.T)

func (*BackupTestSuite) RunIncrementalTestAndRestoreErrors added in v1.34.18 

func (s *BackupTestSuite) RunIncrementalTestAndRestoreErrors(t *testing.T)

func (*BackupTestSuite) RunIncrementalTestAndRestoreSuccess added in v1.34.18 

func (s *BackupTestSuite) RunIncrementalTestAndRestoreSuccess(t *testing.T)

func (*BackupTestSuite) RunTestsWithSharedCompose 

func (s *BackupTestSuite) RunTestsWithSharedCompose(t *testing.T, compose *docker.DockerCompose)

RunTestsWithSharedCompose runs all backup test scenarios using a pre-existing compose. This is optimized for test suites that share a single cluster across multiple tests. The compose lifecycle is NOT managed by this method - caller is responsible for setup/teardown.

func (*BackupTestSuite) SetupCompose 

func (s *BackupTestSuite) SetupCompose(ctx context.Context) error

SetupCompose starts the Docker compose environment with the appropriate backend. If ExternalCompose is set in the config, it uses that instead of creating a new one.

func (*BackupTestSuite) TeardownCompose 

func (s *BackupTestSuite) TeardownCompose(ctx context.Context) error

TeardownCompose stops the Docker compose environment. If using an external compose (set via ExternalCompose config), this is a no-op since the external compose lifecycle is managed by the caller.

func (*BackupTestSuite) VerifyCompressedVectorsRestored 

func (s *BackupTestSuite) VerifyCompressedVectorsRestored(t *testing.T, sampleSize int)

VerifyCompressedVectorsRestored samples random objects and verifies their vectors are present. This is used to validate that compressed vectors are correctly restored after backup/restore.

func (*BackupTestSuite) VerifyObjectDataIntegrity added in v1.35.17 

func (s *BackupTestSuite) VerifyObjectDataIntegrity(t *testing.T)

VerifyObjectDataIntegrity retrieves every object after restore and compares its properties against the originals stored during creation. This catches corruption from mutable-file issues (WAL append, BoltDB mmap, HNSW commitlog) that a count-only check would miss.

Properties are compared via their JSON representation to avoid Go type mismatches (e.g. int vs float64, []string vs []interface{}) that occur between the stored objects and the JSON-decoded API response.

func (*BackupTestSuite) VerifyObjectsDoNotExist 

func (s *BackupTestSuite) VerifyObjectsDoNotExist(t *testing.T)

VerifyObjectsDoNotExist checks that objects no longer exist (after class deletion). Uses class existence check since the class should be deleted.

func (*BackupTestSuite) VerifyObjectsExist 

func (s *BackupTestSuite) VerifyObjectsExist(t *testing.T)

VerifyObjectsExist checks that all created objects still exist using aggregate count. This is much faster than checking individual objects.

func (*BackupTestSuite) VerifyTenantActivitiesRestored added in v1.35.17 

func (s *BackupTestSuite) VerifyTenantActivitiesRestored(t *testing.T)

VerifyTenantActivitiesRestored checks that tenant activity statuses are correctly restored after backup/restore.

type BackupTestSuiteConfig 

type BackupTestSuiteConfig struct {
	// BackendType is the backup backend type: "s3", "gcs", "azure", or "filesystem"
	BackendType string

	// BucketName is the bucket/container name for backups
	BucketName string

	// Region is the region for S3 (ignored for other backends)
	Region string

	// ClassName is the test class name
	ClassName string

	// BackupID is the base backup ID
	BackupID string

	// MultiTenant configures multi-tenant testing
	MultiTenant BackupTestSuiteMultiTenancyConfig

	// ClusterSize is the number of Weaviate nodes (1 for single-node, 3 for cluster)
	ClusterSize int

	// WithVectorizer enables text2vec-contextionary vectorizer
	WithVectorizer bool

	// TestTimeout is the overall test timeout
	TestTimeout time.Duration

	// BackupTimeout is the timeout for backup operations
	BackupTimeout time.Duration

	// RestoreTimeout is the timeout for restore operations
	RestoreTimeout time.Duration

	// ExternalCompose allows using a pre-existing Docker compose instead of creating a new one.
	// When set, SetupCompose() will use this compose and TeardownCompose() becomes a no-op.
	ExternalCompose *docker.DockerCompose

	// MinioEndpoint is the endpoint for the MinIO server (required when using ExternalCompose with S3).
	MinioEndpoint string

	// WithCompression specifies the compression algorithm to enable after class creation.
	// Use CompressionPQ for Product Quantization or CompressionRQ for Rotational Quantization.
	WithCompression CompressionType

	// TestCancellation runs backup cancellation test instead of full backup/restore
	TestCancellation bool

	TestIncremental bool
}

BackupTestSuiteConfig configures the backup test suite.

func DefaultSuiteConfig 

func DefaultSuiteConfig() *BackupTestSuiteConfig

DefaultSuiteConfig returns a default configuration for the backup test suite. By default, WithVectorizer is enabled to use text2vec-contextionary for consistency.

func NewSuiteConfigFromTestCase 

func NewSuiteConfigFromTestCase(sharedConfig SharedComposeConfig, testCase BackupTestCase) *BackupTestSuiteConfig

NewSuiteConfigFromTestCase creates a BackupTestSuiteConfig from a test case and shared compose. It generates unique class names and backup IDs for test isolation.

type BackupTestSuiteMultiTenancyConfig added in v1.35.17 

type BackupTestSuiteMultiTenancyConfig struct {
	Enabled                  bool
	NumTenants               int
	ObjectsPerTenant         int
	WithMidBackupActivations bool
}

type BaseBackend 

type BaseBackend struct {
	// contains filtered or unexported fields
}

BaseBackend provides common functionality for all backend implementations. Embed this in concrete backend types.

func NewBaseBackend 

func NewBaseBackend(opts ...BackendOption) *BaseBackend

NewBaseBackend creates a new BaseBackend with the given options.

func (*BaseBackend) BucketName 

func (b *BaseBackend) BucketName() string

BucketName returns the configured bucket name.

func (*BaseBackend) CheckNotStarted 

func (b *BaseBackend) CheckNotStarted() error

CheckNotStarted returns ErrAlreadyStarted if the backend is already running.

func (*BaseBackend) CheckRunning 

func (b *BaseBackend) CheckRunning() error

CheckRunning returns ErrNotStarted if the backend is not in StateRunning.

func (*BaseBackend) Config 

func (b *BaseBackend) Config() *backendConfig

Config returns the backend configuration.

func (*BaseBackend) SetState 

func (b *BaseBackend) SetState(s BackendState)

SetState updates the backend state.

func (*BaseBackend) State 

func (b *BaseBackend) State() BackendState

State returns the current backend state.

type CompressionType 

type CompressionType string

CompressionType specifies the vector compression algorithm to use. 

const (
	// CompressionNone indicates no compression
	CompressionNone CompressionType = ""
	// CompressionPQ indicates Product Quantization compression
	CompressionPQ CompressionType = "pq"
	// CompressionRQ indicates Rotational Quantization compression
	CompressionRQ CompressionType = "rq"
)

type FixedTestData 

type FixedTestData struct {
	ClassName string
	Objects   []*models.Object
}

FixedTestData contains pre-defined test data with known IDs for deterministic tests.

func NewFixedTestData 

func NewFixedTestData(className string) *FixedTestData

NewFixedTestData creates test data with known, fixed IDs.

func (*FixedTestData) IDs 

func (f *FixedTestData) IDs() []strfmt.UUID

IDs returns all object IDs in the fixed test data.

type GCSBackend 

type GCSBackend struct {
	*BaseBackend
	// contains filtered or unexported fields
}

GCSBackend implements BackupBackend for Google Cloud Storage using the GCS emulator.

func NewGCSBackend 

func NewGCSBackend(opts ...BackendOption) *GCSBackend

NewGCSBackend creates a new GCSBackend with the GCS emulator.

func NewGCSBackendWithOptions 

func NewGCSBackendWithOptions(baseOpts []BackendOption, gcsOpts ...GCSOption) *GCSBackend

NewGCSBackendWithOptions creates a new GCSBackend with both base and GCS-specific options.

func (*GCSBackend) ContainerName 

func (b *GCSBackend) ContainerName() string

ContainerName returns the GCS emulator container name.

func (*GCSBackend) CreateBucket 

func (b *GCSBackend) CreateBucket(ctx context.Context, name string) error

CreateBucket creates a new GCS bucket.

func (*GCSBackend) CreateBucketInternal 

func (b *GCSBackend) CreateBucketInternal(ctx context.Context, name string) error

CreateBucketInternal creates a bucket without state checks (used during Start).

func (*GCSBackend) DeleteBucket 

func (b *GCSBackend) DeleteBucket(ctx context.Context, name string) error

DeleteBucket removes a bucket from the GCS emulator. The bucket must be empty.

func (*GCSBackend) Endpoint 

func (b *GCSBackend) Endpoint() string

Endpoint returns the GCS emulator HTTP endpoint for external access.

func (*GCSBackend) GCSClient 

func (b *GCSBackend) GCSClient() *storage.Client

GCSClient returns the underlying GCS client for advanced operations. Returns nil if the backend is not running.

func (*GCSBackend) GetWeaviateEnv 

func (b *GCSBackend) GetWeaviateEnv() map[string]string

GetWeaviateEnv returns environment variables for Weaviate GCS backup configuration.

func (*GCSBackend) InternalEndpoint 

func (b *GCSBackend) InternalEndpoint() string

InternalEndpoint returns the endpoint for use within the Docker network.

func (*GCSBackend) ListBuckets 

func (b *GCSBackend) ListBuckets(ctx context.Context) ([]string, error)

ListBuckets returns all buckets in the GCS emulator.

func (*GCSBackend) Name 

func (b *GCSBackend) Name() string

Name returns "gcs".

func (*GCSBackend) NetworkName 

func (b *GCSBackend) NetworkName() string

NetworkName returns the Docker network name used by this backend.

func (*GCSBackend) ProjectID 

func (b *GCSBackend) ProjectID() string

ProjectID returns the configured GCP project ID.

func (*GCSBackend) Start 

func (b *GCSBackend) Start(ctx context.Context) error

Start provisions the GCS emulator and creates the default bucket.

func (*GCSBackend) Stop 

func (b *GCSBackend) Stop(ctx context.Context) error

Stop terminates the GCS emulator container and cleans up resources.

type GCSOption 

type GCSOption func(*GCSBackend)

GCSOption configures the GCSBackend.

func WithGCSProjectID 

func WithGCSProjectID(projectID string) GCSOption

WithGCSProjectID sets the GCP project ID.

type S3Backend 

type S3Backend struct {
	*BaseBackend
	// contains filtered or unexported fields
}

S3Backend implements BackupBackend for S3-compatible storage using MinIO.

func NewS3Backend 

func NewS3Backend(opts ...BackendOption) *S3Backend

NewS3Backend creates a new S3Backend with MinIO.

func NewS3BackendWithOptions 

func NewS3BackendWithOptions(baseOpts []BackendOption, s3Opts ...S3Option) *S3Backend

NewS3BackendWithOptions creates a new S3Backend with both base and S3-specific options.

func (*S3Backend) ContainerName 

func (b *S3Backend) ContainerName() string

ContainerName returns the MinIO container name.

func (*S3Backend) CreateBucket 

func (b *S3Backend) CreateBucket(ctx context.Context, name string) error

CreateBucket creates a new S3 bucket.

func (*S3Backend) CreateBucketInternal 

func (b *S3Backend) CreateBucketInternal(ctx context.Context, name string) error

CreateBucketInternal creates a bucket without state checks (used during Start).

func (*S3Backend) DeleteBucket 

func (b *S3Backend) DeleteBucket(ctx context.Context, name string) error

DeleteBucket removes a bucket from MinIO.

func (*S3Backend) Endpoint 

func (b *S3Backend) Endpoint() string

Endpoint returns the MinIO HTTP endpoint for external access.

func (*S3Backend) GetWeaviateEnv 

func (b *S3Backend) GetWeaviateEnv() map[string]string

GetWeaviateEnv returns environment variables for Weaviate S3 backup configuration.

func (*S3Backend) InternalEndpoint 

func (b *S3Backend) InternalEndpoint() string

InternalEndpoint returns the endpoint for use within the Docker network.

func (*S3Backend) ListBuckets 

func (b *S3Backend) ListBuckets(ctx context.Context) ([]string, error)

ListBuckets returns all buckets in MinIO.

func (*S3Backend) Name 

func (b *S3Backend) Name() string

Name returns "s3".

func (*S3Backend) NetworkName 

func (b *S3Backend) NetworkName() string

NetworkName returns the Docker network name used by this backend.

func (*S3Backend) Region 

func (b *S3Backend) Region() string

Region returns the configured AWS region.

func (*S3Backend) S3Client 

func (b *S3Backend) S3Client() *s3.Client

S3Client returns the underlying S3 client for advanced operations. Returns nil if the backend is not running.

func (*S3Backend) Start 

func (b *S3Backend) Start(ctx context.Context) error

Start provisions MinIO and creates the default bucket.

func (*S3Backend) Stop 

func (b *S3Backend) Stop(ctx context.Context) error

Stop terminates the MinIO container and cleans up resources.

type S3Option 

type S3Option func(*S3Backend)

S3Option configures the S3Backend.

func WithS3Region 

func WithS3Region(region string) S3Option

WithS3Region sets the AWS region for the S3 backend.

type SharedComposeConfig 

type SharedComposeConfig struct {
	// Compose is the shared Docker compose instance
	Compose *docker.DockerCompose
	// BackendType is the backup backend type ("s3", "gcs", "azure", "filesystem")
	BackendType string
	// MinioEndpoint is the MinIO endpoint (for S3 tests)
	MinioEndpoint string
	// Region is the S3 region (for S3 tests)
	Region string
	// GCSEndpoint is the GCS emulator endpoint (for GCS tests)
	GCSEndpoint string
	// GCSProjectID is the GCS project ID (for GCS tests)
	GCSProjectID string
}

SharedComposeConfig holds configuration for the shared Docker compose environment.

type TestDataConfig 

type TestDataConfig struct {
	// ClassName is the name of the class to create.
	ClassName string

	// MultiTenant enables multi-tenancy for the class.
	MultiTenant bool

	// NumTenants is the number of tenants to create (only used if MultiTenant is true).
	NumTenants int

	// ObjectsPerTenant is the number of objects to create per tenant.
	// If MultiTenant is false, this is the total number of objects.
	ObjectsPerTenant int

	// Seed for random number generator. Use 0 for time-based seed.
	Seed int64

	// UseVectorizer specifies which vectorizer module to use.
	// Empty string means no vectorizer (manual vectors).
	UseVectorizer string
}

TestDataConfig contains configuration for generating test data.

func DefaultTestDataConfig 

func DefaultTestDataConfig() *TestDataConfig

DefaultTestDataConfig returns a default configuration for test data generation.

type TestDataGenerator 

type TestDataGenerator struct {
	// contains filtered or unexported fields
}

TestDataGenerator generates test classes, objects, and tenants.

func NewTestDataGenerator 

func NewTestDataGenerator(config *TestDataConfig) *TestDataGenerator

NewTestDataGenerator creates a new TestDataGenerator with the given config.

func (*TestDataGenerator) Config 

func (g *TestDataGenerator) Config() *TestDataConfig

Config returns the current configuration.

func (*TestDataGenerator) GenerateAllObjects 

func (g *TestDataGenerator) GenerateAllObjects() []*models.Object

GenerateAllObjects creates objects for all tenants (or a single batch if not multi-tenant).

func (*TestDataGenerator) GenerateClass 

func (g *TestDataGenerator) GenerateClass() *models.Class

GenerateClass creates a class schema for backup testing.

func (*TestDataGenerator) GenerateObject 

func (g *TestDataGenerator) GenerateObject(tenantName string) *models.Object

GenerateObject creates a single test object with random data.

func (*TestDataGenerator) GenerateObjects 

func (g *TestDataGenerator) GenerateObjects(tenantName string, count int) []*models.Object

GenerateObjects creates a batch of test objects.

func (*TestDataGenerator) GenerateTenantModels 

func (g *TestDataGenerator) GenerateTenantModels() []*models.Tenant

GenerateTenantModels creates tenant models for the Weaviate API.

func (*TestDataGenerator) GenerateTenants 

func (g *TestDataGenerator) GenerateTenants() []string

GenerateTenants creates a list of tenant names.

func (*TestDataGenerator) ObjectsByTenant 

func (g *TestDataGenerator) ObjectsByTenant(objects []*models.Object) map[string][]*models.Object

ObjectsByTenant groups objects by tenant name. For non-multi-tenant classes, all objects are under the empty string key.

func (*TestDataGenerator) TotalObjectCount 

func (g *TestDataGenerator) TotalObjectCount() int

TotalObjectCount returns the total number of objects that will be generated.

func (*TestDataGenerator) WithClassName 

func (g *TestDataGenerator) WithClassName(name string) *TestDataGenerator

WithClassName returns a modified generator with a new class name.

func (*TestDataGenerator) WithMultiTenant 

func (g *TestDataGenerator) WithMultiTenant(enabled bool) *TestDataGenerator

WithMultiTenant returns a modified generator with multi-tenancy enabled/disabled.

func (*TestDataGenerator) WithNumTenants 

func (g *TestDataGenerator) WithNumTenants(count int) *TestDataGenerator

WithNumTenants returns a modified generator with a different tenant count.

func (*TestDataGenerator) WithObjectsPerTenant 

func (g *TestDataGenerator) WithObjectsPerTenant(count int) *TestDataGenerator

WithObjectsPerTenant returns a modified generator with a different object count.

func (*TestDataGenerator) WithSeed 

func (g *TestDataGenerator) WithSeed(seed int64) *TestDataGenerator

WithSeed returns a modified generator with a specific seed for reproducibility.

func (*TestDataGenerator) WithVectorizer 

func (g *TestDataGenerator) WithVectorizer(vectorizer string) *TestDataGenerator

WithVectorizer returns a modified generator with a vectorizer module.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.