index

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 2, 2025 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Overview

Package index implements the index system for searching data.

Index

Constants

View Source
const (
	// AnalyzerUnspecified represents an unspecified analyzer.
	AnalyzerUnspecified = ""
	// AnalyzerKeyword is a “noop” analyzer which returns the entire input string as a single token.
	AnalyzerKeyword = "keyword"
	// AnalyzerSimple breaks text into tokens at any non-letter character.
	AnalyzerSimple = "simple"
	// AnalyzerStandard provides grammar based tokenization.
	AnalyzerStandard = "standard"
	// AnalyzerURL breaks test into tokens at any non-letter and non-digit character.
	AnalyzerURL = "url"
	// IndexModeName is the name in the index mode.
	IndexModeName = "_im_name"
	// IndexModeEntityTagPrefix is the entity tag prefix in the index mode.
	IndexModeEntityTagPrefix = "_im_entity_tag_"
)

Variables

View Source
var DummyFieldIterator = &dummyIterator{}

DummyFieldIterator never iterates.

Functions

This section is empty.

Types

type Batch added in v0.6.0

type Batch struct {
	Documents Documents
}

Batch is a collection of documents.

type BytesTermValue added in v0.8.0

type BytesTermValue struct {
	Value []byte
}

BytesTermValue represents a byte term value.

func (BytesTermValue) String added in v0.8.0

func (b BytesTermValue) String() string

type Document added in v0.6.0

type Document struct {
	Fields       []Field
	EntityValues []byte
	Timestamp    int64
	DocID        uint64
	Version      int64
}

Document represents a document in an index.

type DocumentResult added in v0.7.0

type DocumentResult struct {
	EntityValues []byte
	Values       map[string][]byte
	SortedValue  []byte
	SeriesID     common.SeriesID
	DocID        uint64
	Timestamp    int64
	Version      int64
}

DocumentResult represents a document in an index.

func (DocumentResult) SortedField added in v0.7.0

func (ir DocumentResult) SortedField() []byte

SortedField returns the value of the sorted field.

type Documents added in v0.6.0

type Documents []Document

Documents is a collection of documents.

type Field

type Field struct {
	Key    FieldKey
	NoSort bool
	Store  bool
	Index  bool
	// contains filtered or unexported fields
}

Field is a indexed item in a document.

func NewBytesField added in v0.8.0

func NewBytesField(key FieldKey, value []byte) Field

NewBytesField creates a new bytes field.

func NewIntField added in v0.8.0

func NewIntField(key FieldKey, value int64) Field

NewIntField creates a new int field.

func NewStringField added in v0.8.0

func NewStringField(key FieldKey, value string) Field

NewStringField creates a new string field.

func (*Field) GetBytes added in v0.8.0

func (f *Field) GetBytes() []byte

GetBytes returns the byte value of the field.

func (*Field) GetFloat added in v0.8.0

func (f *Field) GetFloat() float64

GetFloat returns the float value of the field.

func (*Field) GetTerm added in v0.8.0

func (f *Field) GetTerm() IsTermValue

GetTerm returns the term value of the field.

func (*Field) MarshalJSON added in v0.8.0

func (f *Field) MarshalJSON() ([]byte, error)

MarshalJSON encodes f to JSON.

func (*Field) String added in v0.8.0

func (f *Field) String() string

String returns a string representation of the field.

type FieldIterable

type FieldIterable interface {
	BuildQuery(seriesMatchers []SeriesMatcher, secondaryQuery Query, timeRange *timestamp.TimeRange) (Query, error)
	Iterator(ctx context.Context, fieldKey FieldKey, termRange RangeOpts, order modelv1.Sort,
		preLoadSize int) (iter FieldIterator[*DocumentResult], err error)
	Sort(ctx context.Context, sids []common.SeriesID, fieldKey FieldKey,
		order modelv1.Sort, timeRange *timestamp.TimeRange, preLoadSize int) (FieldIterator[*DocumentResult], error)
}

FieldIterable allows building a FieldIterator.

type FieldIterator

type FieldIterator[T sort.Comparable] interface {
	Next() bool
	Val() T
	Close() error
	Query() Query
}

FieldIterator allows iterating over a field's posting values.

type FieldKey

type FieldKey struct {
	TimeRange   *RangeOpts
	Analyzer    string
	TagName     string
	SeriesID    common.SeriesID
	IndexRuleID uint32
}

FieldKey is the key of field in a document.

func (FieldKey) Marshal

func (f FieldKey) Marshal() string

Marshal encodes f to string.

type Filter added in v0.2.0

type Filter interface {
	fmt.Stringer
	Execute(getSearcher GetSearcher, seriesID common.SeriesID, timeRange *RangeOpts) (posting.List, posting.List, error)
}

Filter is a node in the filter tree.

type FloatTermValue added in v0.8.0

type FloatTermValue struct {
	Value float64
}

FloatTermValue represents a float term value.

func (FloatTermValue) String added in v0.8.0

func (f FloatTermValue) String() string

type GetSearcher added in v0.2.0

type GetSearcher func(location databasev1.IndexRule_Type) (Searcher, error)

GetSearcher returns a searcher associated with input index rule type.

type IsTermValue added in v0.8.0

type IsTermValue interface {
	String() string
	// contains filtered or unexported methods
}

IsTermValue is the interface for term value.

type OrderBy added in v0.8.0

type OrderBy struct {
	Index *databasev1.IndexRule
	Sort  modelv1.Sort
	Type  OrderByType
}

OrderBy is the order by rule.

type OrderByType added in v0.8.0

type OrderByType int

OrderByType is the type of order by.

const (
	// OrderByTypeTime is the order by time.
	OrderByTypeTime OrderByType = iota
	// OrderByTypeIndex is the order by index.
	OrderByTypeIndex
	// OrderByTypeSeries is the order by series.
	OrderByTypeSeries
)

type Query added in v0.7.0

type Query interface {
	fmt.Stringer
}

Query is an abstract of an index query.

type RangeOpts

type RangeOpts struct {
	Upper         IsTermValue
	Lower         IsTermValue
	IncludesUpper bool
	IncludesLower bool
}

RangeOpts contains options to performance a continuous scan.

func NewBytesRangeOpts added in v0.8.0

func NewBytesRangeOpts(lower, upper []byte, includesLower, includesUpper bool) RangeOpts

NewBytesRangeOpts creates a new bytes range option.

func NewIntRangeOpts added in v0.8.0

func NewIntRangeOpts(lower, upper int64, includesLower, includesUpper bool) RangeOpts

NewIntRangeOpts creates a new int range option.

func NewStringRangeOpts added in v0.8.0

func NewStringRangeOpts(lower, upper string, includesLower, includesUpper bool) RangeOpts

NewStringRangeOpts creates a new string range option.

func (RangeOpts) IsEmpty added in v0.8.0

func (r RangeOpts) IsEmpty() bool

IsEmpty returns true if the range is empty.

func (RangeOpts) Valid added in v0.8.0

func (r RangeOpts) Valid() bool

Valid returns true if the range is valid.

type Searcher

type Searcher interface {
	FieldIterable
	Match(fieldKey FieldKey, match []string, opts *modelv1.Condition_MatchOption) (list posting.List, timestamps posting.List, err error)
	MatchField(fieldKey FieldKey) (list posting.List, timestamps posting.List, err error)
	MatchTerms(field Field) (list posting.List, timestamps posting.List, err error)
	Range(fieldKey FieldKey, opts RangeOpts) (list posting.List, timestamps posting.List, err error)
}

Searcher allows searching a field either by its key or by its key and term.

type Series added in v0.6.0

type Series struct {
	EntityValues []byte
}

Series represents a series in an index.

func (Series) SortedField added in v0.7.1

func (s Series) SortedField() []byte

SortedField returns the value of the sorted field.

func (Series) String added in v0.6.1

func (s Series) String() string

type SeriesDocument added in v0.7.0

type SeriesDocument struct {
	Fields    map[string][]byte
	Key       Series
	Timestamp int64
	Version   int64
}

SeriesDocument represents a series document in an index.

type SeriesMatcher added in v0.6.0

type SeriesMatcher struct {
	Match []byte
	Type  SeriesMatcherType
}

SeriesMatcher represents a series matcher.

func (SeriesMatcher) String added in v0.6.1

func (s SeriesMatcher) String() string

String returns a string representation of the series matcher.

type SeriesMatcherType added in v0.6.0

type SeriesMatcherType int

SeriesMatcherType represents the type of series matcher.

const (
	// SeriesMatcherTypeExact represents an exact matcher.
	SeriesMatcherTypeExact SeriesMatcherType = iota
	// SeriesMatcherTypePrefix represents a prefix matcher.
	SeriesMatcherTypePrefix
	// SeriesMatcherTypeWildcard represents a wildcard matcher.
	SeriesMatcherTypeWildcard
)

type SeriesStore added in v0.6.0

type SeriesStore interface {
	Store
	// Search returns a list of series that match the given matchers.
	Search(context.Context, []FieldKey, Query, int) ([]SeriesDocument, error)
	SeriesIterator(context.Context) (FieldIterator[Series], error)
	SeriesSort(ctx context.Context, indexQuery Query, orderBy *OrderBy,
		preLoadSize int, fieldKeys []FieldKey) (iter FieldIterator[*DocumentResult], err error)
}

SeriesStore is an abstract of a series repository.

type Store

type Store interface {
	io.Closer
	Writer
	Searcher
	CollectMetrics(...string)
	Reset()
	TakeFileSnapshot(dst string) error
}

Store is an abstract of an index repository.

type Writer

type Writer interface {
	Batch(batch Batch) error
	InsertSeriesBatch(batch Batch) error
	UpdateSeriesBatch(batch Batch) error
	Delete(docID [][]byte) error
}

Writer allows writing fields and docID in a document to an index.

Directories

Path Synopsis
Package inverted implements an inverted index repository.
Package inverted implements an inverted index repository.
Package posting implements a posting list contains a list of document ids.
Package posting implements a posting list contains a list of document ids.
roaring
Package roaring implements the posting list by a roaring bitmap.
Package roaring implements the posting list by a roaring bitmap.
Package testcases implements common helpers for testing inverted and lsm indices.
Package testcases implements common helpers for testing inverted and lsm indices.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL