setup

package
v0.64.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 25, 2025 License: Apache-2.0 Imports: 43 Imported by: 0

README

Setup

Utility functions useful for setting tests up

Doc

Functions for generating documents for tests and benchmarks.

You can generate random data, or fill it by hand, and then convert them to jsons to pass for bulk/check search results

Env

Functions for env config and creating the nodes from config

Methods

Useful methods for calling ingestor/store handles

  • Bulk/BulkBuffer/GenBuffer/...: generates buffer from docs and send them to bulk handle. Also asserts, that everything went well
  • Search/SearchHTTP: convenient ways to search

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Bulk 

func Bulk(t *testing.T, addr string, docs []string)

func BulkBuffer 

func BulkBuffer(t *testing.T, addr string, b *bytes.Buffer)

func ChanceValue 

func ChanceValue(prob int, value string) string

ChanceValue returns given string with probability prob%, or empty string instead

func DocsToStrings 

func DocsToStrings(docs []ExampleDoc) []string

func FetchHTTP 

func FetchHTTP(t *testing.T, addr string, ids []string) []*seqproxyapi.Document

func GenBuffer 

func GenBuffer(docs []string) *bytes.Buffer

func GenerateDocsJSON 

func GenerateDocsJSON(size int, additionalFields bool) [][]byte

GenerateDocsJSON generates random docs straight in json just pass the output to `setup.BulkBytes` to store it faster ~2x if additionalFields=false

func MergeJSONs 

func MergeJSONs(a, b []byte) []byte

func RandomChanceString 

func RandomChanceString(prob, minSize, maxSize int) string

RandomChanceString returns random string with probability prob%, or empty string instead

func RandomDocJSON 

func RandomDocJSON(sizeScale, addFields int) []byte

RandomDocJSON creates random doc and adds to it some additional fields

func RandomJSON 

func RandomJSON(size int) string

RandomJSON generates json with random string keys and another random json or random string as values

func RandomService 

func RandomService() string

func RandomString 

func RandomString(minSize, maxSize int) string

RandomString calls common.RandomString it is here, so you could call `setup.RandomSomething`, without exceptions

func RandomSymbol 

func RandomSymbol() byte

func RandomText 

func RandomText(size int) string

RandomText generates string with random words and symbols inbetween

func RandomWord 

func RandomWord() string

func SearchHTTP 

func SearchHTTP(t *testing.T, addr string, request *seqproxyapi.SearchRequest) *seqproxyapi.SearchResponse

Types

type ExampleDoc 

type ExampleDoc struct {
	Service        string     `json:"service,omitempty"`
	Message        string     `json:"message,omitempty"`
	TraceID        string     `json:"traceID,omitempty"`
	Source         string     `json:"source,omitempty"`
	Zone           string     `json:"zone,omitempty"`
	RequiestObject InlineJSON `json:"requestObject,omitempty"`
	Level          int        `json:"level,omitempty"`
	Timestamp      time.Time  `json:"timestamp,omitempty"`
}

ExampleDoc is useful for testing and benchmarking instead of hardcoding json docs in code, you can hardcode struct which will be turned in json

func GenerateDocs 

func GenerateDocs(size int, generator func(int, *ExampleDoc)) []ExampleDoc

GenerateDocs creates slice of docs and calls generator for each doc to will it with data If timestamp after call is zero, then this function will fill it with deterministic timestamp so you could query each doc by range, if needed

func RandomDoc 

func RandomDoc(sizeScale int) *ExampleDoc

type Ingestor 

type Ingestor struct {
	*proxyapi.Ingestor
	HTTPAddr string
}

func MakeIngestors 

func MakeIngestors(cfg *TestingEnvConfig, hot, cold [][]string) []*Ingestor

type InlineJSON 

type InlineJSON string

InlineJSON is a string representing valid json similar as json.RawMessage, but it's a string

func (*InlineJSON) MarshalJSON 

func (j *InlineJSON) MarshalJSON() ([]byte, error)

func (*InlineJSON) UnmarshalJSON 

func (j *InlineJSON) UnmarshalJSON(v []byte) error

type SearchOption 

type SearchOption func(sr *search.SearchRequest)

func NoFetch 

func NoFetch() SearchOption

func WithAggQuery 

func WithAggQuery(aggQueries ...any) SearchOption

WithAggQuery adds aggregation query to search request. Aggregations parameters are passed as consequent strings of aggregation fields and filters: aggField1, aggFilter1, aggField2, aggFilter2, ..., aggFieldN, aggFilterN.

For example, if two aggregations without filters are needed, this function should be called with args: "agg1", "", "agg2".

If called with single empty string, no aggregation query is added.

func WithInterval 

func WithInterval(interval time.Duration) SearchOption

func WithOffset 

func WithOffset(offset int) SearchOption

func WithOrder 

func WithOrder(o seq.DocsOrder) SearchOption

func WithTimeRange 

func WithTimeRange(from, to time.Time) SearchOption

func WithTotal 

func WithTotal(f bool) SearchOption

type Stores 

type Stores [][]*storeapi.Store

func MakeStores 

func MakeStores(cfg *TestingEnvConfig, replicas int, cold bool) (Stores, [][]string)

func (Stores) CountInstances 

func (s Stores) CountInstances() int

func (Stores) SealAll 

func (s Stores) SealAll()

func (Stores) Stop 

func (s Stores) Stop()

func (Stores) WaitIdle 

func (s Stores) WaitIdle()

type TestingEnv 

type TestingEnv struct {
	Ingestors  []*Ingestor
	HotStores  Stores
	ColdStores Stores

	Config *TestingEnvConfig
	// contains filtered or unexported fields
}

func NewTestingEnv 

func NewTestingEnv(cfg *TestingEnvConfig) *TestingEnv

func (*TestingEnv) Fetch 

func (t *TestingEnv) Fetch(ids []seq.ID) ([][]byte, error)

func (*TestingEnv) Ingestor 

func (t *TestingEnv) Ingestor() *Ingestor

Ingestor returns "random" ingestor managed by TestingEnv but guarantees that each store will return at least once

func (*TestingEnv) IngestorAddr 

func (t *TestingEnv) IngestorAddr() string

func (*TestingEnv) IngestorBulkAddr 

func (t *TestingEnv) IngestorBulkAddr() string

IngestorBulkAddr returns "random" ingestor HTTP address but guarantees that each store will return at least once

func (*TestingEnv) IngestorFetchAddr 

func (t *TestingEnv) IngestorFetchAddr() string

func (*TestingEnv) IngestorSearchAddr 

func (t *TestingEnv) IngestorSearchAddr() string

IngestorSearchAddr returns "random" ingestor HTTP address but guarantees that each store will return at least once

func (*TestingEnv) SealAll 

func (t *TestingEnv) SealAll()

func (*TestingEnv) Search 

func (t *TestingEnv) Search(q string, size int, options ...SearchOption) (*seq.QPR, [][]byte, time.Duration, error)

func (*TestingEnv) StopAll 

func (t *TestingEnv) StopAll()

func (*TestingEnv) StopIngestor 

func (t *TestingEnv) StopIngestor()

func (*TestingEnv) StopStore 

func (t *TestingEnv) StopStore()

func (*TestingEnv) Store 

func (t *TestingEnv) Store(hot bool) *storeapi.Store

Store returns random store managed by TestingEnv but guarantees that each store will return at least once

func (*TestingEnv) WaitIdle 

func (t *TestingEnv) WaitIdle()

type TestingEnvConfig 

type TestingEnvConfig struct {
	Name              string
	DataDir           string
	IngestorCount     int
	ColdShards        int // number of replicaSets (aka shards)
	ColdFactor        int // number of replicas in each replicaSet
	HotShards         int // number of replicaSets (aka shards)
	HotFactor         int // number of replicas in each replicaSet
	HotModeEnabled    bool
	QueryRateLimit    *float64
	FracManagerConfig *fracmanager.Config

	Mapping        seq.Mapping
	IndexAllFields bool

	S3Cli *seqs3.Client
}

func (*TestingEnvConfig) GetColdFactor 

func (cfg *TestingEnvConfig) GetColdFactor() int

func (*TestingEnvConfig) GetColdStoresConfs 

func (cfg *TestingEnvConfig) GetColdStoresConfs() []storeapi.StoreConfig

func (*TestingEnvConfig) GetFracManagerConfig 

func (cfg *TestingEnvConfig) GetFracManagerConfig(replicaID string) fracmanager.Config

func (*TestingEnvConfig) GetHotFactor 

func (cfg *TestingEnvConfig) GetHotFactor() int

func (*TestingEnvConfig) GetHotStoresConfs 

func (cfg *TestingEnvConfig) GetHotStoresConfs() []storeapi.StoreConfig

func (*TestingEnvConfig) GetStoreConfig 

func (cfg *TestingEnvConfig) GetStoreConfig(replicaID string, cold bool) storeapi.StoreConfig

func (*TestingEnvConfig) MakeStores 

func (cfg *TestingEnvConfig) MakeStores(
	confs []storeapi.StoreConfig, replicas int, s3cli *seqs3.Client,
) (Stores, [][]string)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.