serverutils

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 18, 2022 License: Apache-2.0 Imports: 26 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetJSONProto

func GetJSONProto(ts TestServerInterface, path string, response protoutil.Message) error

GetJSONProto uses the supplied client to GET the URL specified by the parameters and unmarshals the result into response.

func GetJSONProtoWithAdminOption

func GetJSONProtoWithAdminOption(
	ts TestServerInterface, path string, response protoutil.Message, isAdmin bool,
) error

GetJSONProtoWithAdminOption is like GetJSONProto but the caller can customize whether the request is performed with admin privilege

func InitTestClusterFactory

func InitTestClusterFactory(impl TestClusterFactory)

InitTestClusterFactory should be called once to provide the implementation of the service. It will be called from a xx_test package that can import the server package.

func InitTestServerFactory

func InitTestServerFactory(impl TestServerFactory)

InitTestServerFactory should be called once to provide the implementation of the service. It will be called from a xx_test package that can import the server package.

func OpenDBConn

func OpenDBConn(
	t testing.TB, sqlAddr string, useDatabase string, insecure bool, stopper *stop.Stopper,
) *gosql.DB

OpenDBConn sets up a gosql DB connection to the given server.

func OpenDBConnE

func OpenDBConnE(
	sqlAddr string, useDatabase string, insecure bool, stopper *stop.Stopper,
) (*gosql.DB, error)

OpenDBConnE is like OpenDBConn, but returns an error.

func PostJSONProto

func PostJSONProto(ts TestServerInterface, path string, request, response protoutil.Message) error

PostJSONProto uses the supplied client to POST the URL specified by the parameters and unmarshals the result into response.

func PostJSONProtoWithAdminOption

func PostJSONProtoWithAdminOption(
	ts TestServerInterface, path string, request, response protoutil.Message, isAdmin bool,
) error

PostJSONProtoWithAdminOption is like PostJSONProto but the caller can customize whether the request is performed with admin privilege.

func SetClusterSetting

func SetClusterSetting(t testutils.TB, c TestClusterInterface, name string, value interface{})

SetClusterSetting executes set cluster settings statement, and then ensures that all nodes in the test cluster see that setting update.

func TestTenantID

func TestTenantID() roachpb.TenantID

TestTenantID returns a roachpb.TenantID that can be used when starting a test Tenant. The returned tenant IDs match those built into the test certificates.

Types

type KeyAndTargets

type KeyAndTargets struct {
	StartKey roachpb.Key
	Targets  []roachpb.ReplicationTarget
}

KeyAndTargets contains replica startKey and targets.

type TestClusterFactory

type TestClusterFactory interface {
	// NewTestCluster creates a test cluster without starting it.
	NewTestCluster(t testing.TB, numNodes int, args base.TestClusterArgs) TestClusterInterface
}

TestClusterFactory encompasses the actual implementation of the shim service.

type TestClusterInterface

type TestClusterInterface interface {
	// Start is used to start up the servers that were instantiated when
	// creating this cluster.
	Start(t testing.TB)

	// NumServers returns the number of servers this test cluster is configured
	// with.
	NumServers() int

	// Server returns the TestServerInterface corresponding to a specific node.
	Server(idx int) TestServerInterface

	// ServerConn returns a gosql.DB connection to a specific node.
	ServerConn(idx int) *gosql.DB

	// StopServer stops a single server.
	StopServer(idx int)

	// Stopper retrieves the stopper for this test cluster. Tests should call or
	// defer the Stop() method on this stopper after starting a test cluster.
	Stopper() *stop.Stopper

	// AddVoters adds voter replicas for a range on a set of stores.
	// It's illegal to have multiple replicas of the same range on stores of a single
	// node.
	// The method blocks until a snapshot of the range has been copied to all the
	// new replicas and the new replicas become part of the Raft group.
	AddVoters(
		startKey roachpb.Key, targets ...roachpb.ReplicationTarget,
	) (roachpb.RangeDescriptor, error)

	// AddVotersMulti is the same as AddVoters but will execute multiple jobs.
	AddVotersMulti(
		kts ...KeyAndTargets,
	) ([]roachpb.RangeDescriptor, []error)

	// AddVotersOrFatal is the same as AddVoters but will Fatal the test on
	// error.
	AddVotersOrFatal(
		t testing.TB, startKey roachpb.Key, targets ...roachpb.ReplicationTarget,
	) roachpb.RangeDescriptor

	// RemoveVoters removes one or more voter replicas from a range.
	RemoveVoters(
		startKey roachpb.Key, targets ...roachpb.ReplicationTarget,
	) (roachpb.RangeDescriptor, error)

	// RemoveVotersOrFatal is the same as RemoveVoters but will Fatal the test on
	// error.
	RemoveVotersOrFatal(
		t testing.TB, startKey roachpb.Key, targets ...roachpb.ReplicationTarget,
	) roachpb.RangeDescriptor

	// AddNonVoters adds non-voting replicas for a range on a set of stores.
	//
	//This method blocks until the new replicas become a part of the Raft group.
	AddNonVoters(
		startKey roachpb.Key,
		targets ...roachpb.ReplicationTarget,
	) (roachpb.RangeDescriptor, error)

	// AddNonVotersOrFatal is the same as AddNonVoters but will fatal if it fails.
	AddNonVotersOrFatal(
		t testing.TB, startKey roachpb.Key, targets ...roachpb.ReplicationTarget,
	) roachpb.RangeDescriptor

	// RemoveNonVoters removes one or more non-voters from a range.
	RemoveNonVoters(
		startKey roachpb.Key, targets ...roachpb.ReplicationTarget,
	) (roachpb.RangeDescriptor, error)

	// RemoveNonVotersOrFatal is the same as RemoveNonVoters but will fatal if it
	// fails.
	RemoveNonVotersOrFatal(
		t testing.TB, startKey roachpb.Key, targets ...roachpb.ReplicationTarget,
	) roachpb.RangeDescriptor

	// SwapVoterWithNonVoter atomically "swaps" the voting replica located on
	// `voterTarget` with the non-voting replica located on `nonVoterTarget`. A
	// sequence of ReplicationChanges is considered to have "swapped" a voter on
	// s1 with a non-voter on s2 iff the resulting state after the execution of
	// these changes is such that s1 has a non-voter and s2 has a voter.
	SwapVoterWithNonVoter(
		startKey roachpb.Key, voterTarget, nonVoterTarget roachpb.ReplicationTarget,
	) (*roachpb.RangeDescriptor, error)

	// SwapVoterWithNonVoterOrFatal is the same as SwapVoterWithNonVoter but will
	// fatal if it fails.
	SwapVoterWithNonVoterOrFatal(
		t *testing.T, startKey roachpb.Key, voterTarget, nonVoterTarget roachpb.ReplicationTarget,
	) *roachpb.RangeDescriptor

	// FindRangeLeaseHolder returns the current lease holder for the given range.
	// In particular, it returns one particular node's (the hint, if specified) view
	// of the current lease.
	// An error is returned if there's no active lease.
	//
	// Note that not all nodes have necessarily applied the latest lease,
	// particularly immediately after a TransferRangeLease() call. So specifying
	// different hints can yield different results. The one server that's guaranteed
	// to have applied the transfer is the previous lease holder.
	FindRangeLeaseHolder(
		rangeDesc roachpb.RangeDescriptor,
		hint *roachpb.ReplicationTarget,
	) (roachpb.ReplicationTarget, error)

	// TransferRangeLease transfers the lease for a range from whoever has it to
	// a particular store. That store must already have a replica of the range. If
	// that replica already has the (active) lease, this method is a no-op.
	//
	// When this method returns, it's guaranteed that the old lease holder has
	// applied the new lease, but that's about it. It's not guaranteed that the new
	// lease holder has applied it (so it might not know immediately that it is the
	// new lease holder).
	TransferRangeLease(
		rangeDesc roachpb.RangeDescriptor, dest roachpb.ReplicationTarget,
	) error

	// MoveRangeLeaseNonCooperatively performs a non-cooperative transfer of the
	// lease for a range from whoever has it to a particular store. That store
	// must already have a replica of the range. If that replica already has the
	// (active) lease, this method is a no-op.
	//
	// The transfer is non-cooperative in that the lease is made to expire by
	// advancing the manual clock. The target is then instructed to acquire the
	// ownerless lease. Most tests should use the cooperative version of this
	// method, TransferRangeLease.
	//
	// Returns the new lease.
	//
	// If the lease starts out on dest, this is a no-op and the current lease is
	// returned.
	MoveRangeLeaseNonCooperatively(
		ctx context.Context,
		rangeDesc roachpb.RangeDescriptor,
		dest roachpb.ReplicationTarget,
		manual *hlc.HybridManualClock,
	) (*roachpb.Lease, error)

	// LookupRange returns the descriptor of the range containing key.
	LookupRange(key roachpb.Key) (roachpb.RangeDescriptor, error)

	// LookupRangeOrFatal is the same as LookupRange but will Fatal the test on
	// error.
	LookupRangeOrFatal(t testing.TB, key roachpb.Key) roachpb.RangeDescriptor

	// Target returns a roachpb.ReplicationTarget for the specified server.
	Target(serverIdx int) roachpb.ReplicationTarget

	// ReplicationMode returns the ReplicationMode that the test cluster was
	// configured with.
	ReplicationMode() base.TestClusterReplicationMode

	// ScratchRange returns the start key of a span of keyspace suitable for use
	// as kv scratch space (it doesn't overlap system spans or SQL tables). The
	// range is lazily split off on the first call to ScratchRange.
	ScratchRange(t testing.TB) roachpb.Key

	// WaitForFullReplication waits until all stores in the cluster
	// have no ranges with replication pending.
	WaitForFullReplication() error
}

TestClusterInterface defines TestCluster functionality used by tests.

func NewTestCluster

func NewTestCluster(t testing.TB, numNodes int, args base.TestClusterArgs) TestClusterInterface

NewTestCluster creates TestCluster made up of numNodes in-memory testing servers. It can be started using the return type.

func StartNewTestCluster

func StartNewTestCluster(
	t testing.TB, numNodes int, args base.TestClusterArgs,
) TestClusterInterface

StartNewTestCluster creates and starts up a TestCluster made up of numNodes in-memory testing servers. The cluster should be stopped using Stopper().Stop().

type TestServerFactory

type TestServerFactory interface {
	// New instantiates a test server.
	New(params base.TestServerArgs) (interface{}, error)
}

TestServerFactory encompasses the actual implementation of the shim service.

type TestServerInterface

type TestServerInterface interface {
	Start(context.Context) error

	// TestTenantInterface embeds SQL-only APIs that tests need to interact with
	// the host tenant.
	//
	// TODO(irfansharif): Audit the remaining symbols in TestServerInterface to
	// see if they're better suited to TestTenantInterface.
	TestTenantInterface

	// Node returns the server.Node as an interface{}.
	Node() interface{}

	// NodeID returns the ID of this node within its cluster.
	NodeID() roachpb.NodeID

	// ClusterID returns the cluster ID as understood by this node in the
	// cluster.
	ClusterID() uuid.UUID

	// ServingRPCAddr returns the server's advertised address.
	ServingRPCAddr() string

	// ServingSQLAddr returns the server's advertised SQL address.
	ServingSQLAddr() string

	// RPCAddr returns the server's RPC address.
	// Note: use ServingRPCAddr() instead unless specific reason not to.
	RPCAddr() string

	// DB returns a *client.DB instance for talking to this KV server.
	DB() *kv.DB

	// LeaseManager() returns the *sql.LeaseManager as an interface{}.
	LeaseManager() interface{}

	// InternalExecutor returns a *sql.InternalExecutor as an interface{} (which
	// also implements sqlutil.InternalExecutor if the test cannot depend on sql).
	InternalExecutor() interface{}

	// TracerI returns a *tracing.Tracer as an interface{}.
	TracerI() interface{}

	// GossipI returns the gossip used by the TestServer.
	// The real return type is *gossip.Gossip.
	GossipI() interface{}

	// DistSenderI returns the DistSender used by the TestServer.
	// The real return type is *kv.DistSender.
	DistSenderI() interface{}

	// MigrationServer returns the internal *migrationServer as in interface{}
	MigrationServer() interface{}

	// SQLServer returns the *sql.Server as an interface{}.
	SQLServer() interface{}

	// SQLLivenessProvider returns the sqlliveness.Provider as an interface{}.
	SQLLivenessProvider() interface{}

	// StartupMigrationsManager returns the *startupmigrations.Manager as an interface{}.
	StartupMigrationsManager() interface{}

	// NodeLiveness exposes the NodeLiveness instance used by the TestServer as an
	// interface{}.
	NodeLiveness() interface{}

	// HeartbeatNodeLiveness heartbeats the server's NodeLiveness record.
	HeartbeatNodeLiveness() error

	// NodeDialer exposes the NodeDialer instance used by the TestServer as an
	// interface{}.
	NodeDialer() interface{}

	// SetDistSQLSpanResolver changes the SpanResolver used for DistSQL inside the
	// server's executor. The argument must be a physicalplan.SpanResolver
	// instance.
	//
	// This method exists because we cannot pass the fake span resolver with the
	// server or cluster params: the fake span resolver needs the node IDs and
	// addresses of the servers in a cluster, which are not available before we
	// start the servers.
	//
	// It is the caller's responsibility to make sure no queries are being run
	// with DistSQL at the same time.
	SetDistSQLSpanResolver(spanResolver interface{})

	// MustGetSQLCounter returns the value of a counter metric from the server's
	// SQL Executor. Runs in O(# of metrics) time, which is fine for test code.
	MustGetSQLCounter(name string) int64
	// MustGetSQLNetworkCounter returns the value of a counter metric from the
	// server's SQL server. Runs in O(# of metrics) time, which is fine for test
	// code.
	MustGetSQLNetworkCounter(name string) int64
	// WriteSummaries records summaries of time-series data, which is required for
	// any tests that query server stats.
	WriteSummaries() error

	// GetFirstStoreID is a utility function returning the StoreID of the first
	// store on this node.
	GetFirstStoreID() roachpb.StoreID

	// GetStores returns the collection of stores from this TestServer's node.
	// The return value is of type *kvserver.Stores.
	GetStores() interface{}

	// Decommission idempotently sets the decommissioning flag for specified nodes.
	Decommission(ctx context.Context, targetStatus livenesspb.MembershipStatus, nodeIDs []roachpb.NodeID) error

	// SplitRange splits the range containing splitKey.
	SplitRange(splitKey roachpb.Key) (left roachpb.RangeDescriptor, right roachpb.RangeDescriptor, err error)

	// MergeRanges merges the range containing leftKey with the following adjacent
	// range.
	MergeRanges(leftKey roachpb.Key) (merged roachpb.RangeDescriptor, err error)

	// ExpectedInitialRangeCount returns the expected number of ranges that should
	// be on the server after initial (asynchronous) splits have been completed,
	// assuming no additional information is added outside of the normal bootstrap
	// process.
	ExpectedInitialRangeCount() (int, error)

	// ForceTableGC sends a GCRequest for the ranges corresponding to a table.
	//
	// An error will be returned if the same table name exists in multiple schemas
	// inside the specified database.
	ForceTableGC(ctx context.Context, database, table string, timestamp hlc.Timestamp) error

	// UpdateChecker returns the server's *diagnostics.UpdateChecker as an
	// interface{}. The UpdateChecker periodically phones home to check for new
	// updates that are available.
	UpdateChecker() interface{}

	// StartTenant spawns off tenant process connecting to this TestServer.
	StartTenant(ctx context.Context, params base.TestTenantArgs) (TestTenantInterface, error)

	// ScratchRange splits off a range suitable to be used as KV scratch space.
	// (it doesn't overlap system spans or SQL tables).
	//
	// Calling this multiple times is undefined (but see
	// TestCluster.ScratchRange() which is idempotent).
	ScratchRange() (roachpb.Key, error)

	// Engines returns the TestServer's engines.
	Engines() []storage.Engine

	// MetricsRecorder periodically records node-level and store-level metrics.
	MetricsRecorder() *status.MetricsRecorder

	// CollectionFactory returns a *descs.CollectionFactory.
	CollectionFactory() interface{}

	// SystemTableIDResolver returns a catalog.SystemTableIDResolver.
	SystemTableIDResolver() interface{}

	// SpanConfigKVSubscriber returns the embedded spanconfig.KVSubscriber for
	// the server.
	SpanConfigKVSubscriber() interface{}
}

TestServerInterface defines test server functionality that tests need; it is implemented by server.TestServer.

func NewServer

func NewServer(params base.TestServerArgs) (TestServerInterface, error)

NewServer creates a test server.

func StartServer

func StartServer(
	t testing.TB, params base.TestServerArgs,
) (TestServerInterface, *gosql.DB, *kv.DB)

StartServer creates and starts a test server, and sets up a gosql DB connection to it. The server should be stopped by calling server.Stopper().Stop().

func StartServerRaw

func StartServerRaw(args base.TestServerArgs) (TestServerInterface, error)

StartServerRaw creates and starts a TestServer. Generally StartServer() should be used. However this function can be used directly when opening a connection to the server is not desired.

type TestTenantInterface

type TestTenantInterface interface {
	// SQLInstanceID is the ephemeral ID assigned to a running instance of the
	// SQLServer. Each tenant can have zero or more running SQLServer instances.
	SQLInstanceID() base.SQLInstanceID

	// SQLAddr returns the tenant's SQL address.
	SQLAddr() string

	// HTTPAddr returns the tenant's http address.
	HTTPAddr() string

	// RPCAddr returns the tenant's RPC address.
	RPCAddr() string

	// PGServer returns the tenant's *pgwire.Server as an interface{}.
	PGServer() interface{}

	// DiagnosticsReporter returns the tenant's *diagnostics.Reporter as an
	// interface{}. The DiagnosticsReporter periodically phones home to report
	// diagnostics and usage.
	DiagnosticsReporter() interface{}

	// StatusServer returns the tenant's *server.SQLStatusServer as an
	// interface{}.
	StatusServer() interface{}

	// TenantStatusServer returns the tenant's *server.TenantStatusServer as an
	// interface{}.
	TenantStatusServer() interface{}

	// DistSQLServer returns the *distsql.ServerImpl as an interface{}.
	DistSQLServer() interface{}

	// JobRegistry returns the *jobs.Registry as an interface{}.
	JobRegistry() interface{}

	// RPCContext returns the *rpc.Context used by the test tenant.
	RPCContext() *rpc.Context

	// AnnotateCtx annotates a context.
	AnnotateCtx(context.Context) context.Context

	// ExecutorConfig returns a copy of the tenant's ExecutorConfig.
	// The real return type is sql.ExecutorConfig.
	ExecutorConfig() interface{}

	// RangeFeedFactory returns the range feed factory used by the tenant.
	// The real return type is *rangefeed.Factory.
	RangeFeedFactory() interface{}

	// ClusterSettings returns the ClusterSettings shared by all components of
	// this tenant.
	ClusterSettings() *cluster.Settings

	// Stopper returns the stopper used by the tenant.
	Stopper() *stop.Stopper

	// Clock returns the clock used by the tenant.
	Clock() *hlc.Clock

	// SpanConfigKVAccessor returns the underlying spanconfig.KVAccessor as an
	// interface{}.
	SpanConfigKVAccessor() interface{}

	// SpanConfigReconciler returns the underlying spanconfig.Reconciler as an
	// interface{}.
	SpanConfigReconciler() interface{}

	// SpanConfigSQLTranslatorFactory returns the underlying
	// spanconfig.SQLTranslatorFactory as an interface{}.
	SpanConfigSQLTranslatorFactory() interface{}

	// SpanConfigSQLWatcher returns the underlying spanconfig.SQLWatcher as an
	// interface{}.
	SpanConfigSQLWatcher() interface{}

	// TestingKnobs returns the TestingKnobs in use by the test server.
	TestingKnobs() *base.TestingKnobs

	// AmbientCtx retrieves the AmbientContext for this server,
	// so that a test can instantiate additional one-off components
	// using the same context details as the server. This should not
	// be used in non-test code.
	AmbientCtx() log.AmbientContext

	// AdminURL returns the URL for the admin UI.
	AdminURL() string
	// GetUnauthenticatedHTTPClient returns an http client configured with the client TLS
	// config required by the TestServer's configuration.
	// Discourages implementer from using unauthenticated http connections
	// with verbose method name.
	GetUnauthenticatedHTTPClient() (http.Client, error)
	// GetAdminHTTPClient returns an http client which has been
	// authenticated to access Admin API methods (via a cookie).
	// The user has admin privileges.
	GetAdminHTTPClient() (http.Client, error)
	// GetAuthenticatedHTTPClient returns an http client which has been
	// authenticated to access Admin API methods (via a cookie).
	GetAuthenticatedHTTPClient(isAdmin bool) (http.Client, error)

	// DrainClients shuts down client connections.
	DrainClients(ctx context.Context) error

	// SystemConfigProvider provides access to the system config.
	SystemConfigProvider() config.SystemConfigProvider
}

TestTenantInterface defines SQL-only tenant functionality that tests need; it is implemented by server.Test{Tenant,Server}. Tests written against this interface are effectively agnostic to the type of tenant (host or secondary) they're dealing with.

func StartTenant

func StartTenant(
	t testing.TB, ts TestServerInterface, params base.TestTenantArgs,
) (TestTenantInterface, *gosql.DB)

StartTenant starts a tenant SQL server connecting to the supplied test server. It uses the server's stopper to shut down automatically. However, the returned DB is for the caller to close.

Note: log.Scope() should always be used in tests that start a tenant (otherwise, having more than one test in a package which uses StartTenant without log.Scope() will cause a a "clusterID already set" panic).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL