oomstore

package
v0.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 7, 2022 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	Day       = 24 * time.Hour
	Capacity  = 1000
	Period    = 5 * time.Minute
	MinPeriod = 2 * time.Minute
)
View Source 
const (
	FeatureFullNameSeparator = "."
)

Variables

This section is empty.

Functions

func GetEntityRowsFromInputFile added in v0.0.4 

func GetEntityRowsFromInputFile(inputFilePath string) (<-chan types.EntityRow, []string, error)

Types

type GroupBuffer added in v0.0.4 

type GroupBuffer struct {
	// contains filtered or unexported fields
}

type OomStore 

type OomStore struct {
	// contains filtered or unexported fields
}

Combine online store, offline store, and metadata store instances in one place.

func Create 

func Create(ctx context.Context, opt types.OomStoreConfig) (*OomStore, error)

Create a new OomStore instance.

func Open 

func Open(ctx context.Context, opt types.OomStoreConfig) (*OomStore, error)

Return an OomStore instance given the configuration. Under the hood, it setups connections to the underlying databases. You should always use this method to create a new OomStore instance in code.

func TEST__New added in v0.0.3 

func TEST__New(online online.Store, offline offline.Store, metadata metadata.Store) *OomStore

Return an OomStore instance for internal test purpose ONLY. You should NOT use this method directly in any of your code.

func (*OomStore) Apply added in v0.0.4 

func (s *OomStore) Apply(ctx context.Context, opt apply.ApplyOpt) error

func (*OomStore) ChannelExport added in v0.0.4 

func (s *OomStore) ChannelExport(ctx context.Context, opt types.ChannelExportOpt) (*types.ExportResult, error)

Export feature values of a particular revision. Usage Example: 

exportResult, err := store.Export(ctx, opt)
if err != nil {
	return err
}
for row := range exportResult.Data {
	fmt.Println(cast.ToStringSlice([]interface{}(row)))
}
// Attention: call CheckStreamError after consuming exportResult.Data channel
return exportResult.CheckStreamError()

func (*OomStore) ChannelJoin added in v0.0.4 

func (s *OomStore) ChannelJoin(ctx context.Context, opt types.ChannelJoinOpt) (*types.JoinResult, error)

ChannelJoin gets point-in-time correct feature values for each entity row. Currently, this API only supports batch features.

func (*OomStore) Close 

func (s *OomStore) Close() error

Close the connections to underlying databases.

func (*OomStore) CreateEntity 

func (s *OomStore) CreateEntity(ctx context.Context, opt types.CreateEntityOpt) (int, error)

Create metadata for an entity.

func (*OomStore) CreateFeature added in v0.0.4 

func (s *OomStore) CreateFeature(ctx context.Context, opt types.CreateFeatureOpt) (int, error)

Create metadata of a feature.

func (*OomStore) CreateGroup added in v0.0.3 

func (s *OomStore) CreateGroup(ctx context.Context, opt types.CreateGroupOpt) (int, error)

Create metadata of a feature group.

func (*OomStore) Export added in v0.0.4 

func (s *OomStore) Export(ctx context.Context, opt types.ExportOpt) error

func (*OomStore) GetEntity 

func (s *OomStore) GetEntity(ctx context.Context, id int) (*types.Entity, error)

Get metadata of an entity by ID.

func (*OomStore) GetEntityByName added in v0.0.2 

func (s *OomStore) GetEntityByName(ctx context.Context, name string) (*types.Entity, error)

Get metadata of an entity by name.

func (*OomStore) GetFeature 

func (s *OomStore) GetFeature(ctx context.Context, id int) (*types.Feature, error)

Get metadata of a feature by ID.

func (*OomStore) GetFeatureByName added in v0.0.2 

func (s *OomStore) GetFeatureByName(ctx context.Context, fullName string) (*types.Feature, error)

Get metadata of a feature by full name.

func (*OomStore) GetGroup added in v0.0.3 

func (s *OomStore) GetGroup(ctx context.Context, id int) (*types.Group, error)

Get metadata of a feature group by ID.

func (*OomStore) GetGroupByName added in v0.0.3 

func (s *OomStore) GetGroupByName(ctx context.Context, name string) (*types.Group, error)

Get metadata of a feature group by name.

func (*OomStore) GetRevision 

func (s *OomStore) GetRevision(ctx context.Context, id int) (*types.Revision, error)

Get metadata of a revision by ID.

func (*OomStore) GetRevisionBy added in v0.0.2 

func (s *OomStore) GetRevisionBy(ctx context.Context, groupID int, revision int64) (*types.Revision, error)

Get metadata of a revision by group ID and revision.

func (*OomStore) Import added in v0.0.3 

func (s *OomStore) Import(ctx context.Context, opt types.ImportOpt) (int, error)

Import data into the offline feature store as a new revision. In the future we want to support more diverse data sources.

func (*OomStore) InitStreamPushProcessor added in v0.0.4 

func (s *OomStore) InitStreamPushProcessor(ctx context.Context)

func (*OomStore) Join added in v0.0.3 

func (s *OomStore) Join(ctx context.Context, opt types.JoinOpt) error

Join gets point-in-time correct feature values for each entity row. The method is similar to Join, except that both input and output are files on disk. Input File should contain header, the first two columns of Input File should be entity_key, unix_milli, then followed by other real-time feature values.

func (*OomStore) ListEntity 

func (s *OomStore) ListEntity(ctx context.Context) (types.EntityList, error)

List metadata of all entities.

func (*OomStore) ListFeature 

func (s *OomStore) ListFeature(ctx context.Context, opt types.ListFeatureOpt) (types.FeatureList, error)

List metadata of features meeting particular criteria.

func (*OomStore) ListGroup added in v0.0.3 

func (s *OomStore) ListGroup(ctx context.Context, entityID *int) (types.GroupList, error)

List metadata of feature groups under the same entity.

func (*OomStore) ListRevision 

func (s *OomStore) ListRevision(ctx context.Context, groupID *int) (types.RevisionList, error)

List metadata of revisions of a same group.

func (*OomStore) OnlineGet added in v0.0.3 

func (s *OomStore) OnlineGet(ctx context.Context, opt types.OnlineGetOpt) (*types.FeatureValues, error)

OnlineGet gets online features of a particular entity instance.

func (*OomStore) OnlineMultiGet added in v0.0.3 

func (s *OomStore) OnlineMultiGet(ctx context.Context, opt types.OnlineMultiGetOpt) (map[string]*types.FeatureValues, error)

OnlineMultiGet gets online features of multiple entity instances.

func (*OomStore) Ping added in v0.0.4 

func (s *OomStore) Ping(ctx context.Context) error

Ping verifies the connections to the backend stores are still alive

func (*OomStore) Push added in v0.0.4 

func (s *OomStore) Push(ctx context.Context, opt types.PushOpt) error

Push inserts stream feature values to online store and offline store

func (*OomStore) Snapshot added in v0.0.4 

func (s *OomStore) Snapshot(ctx context.Context, groupName string) error

func (*OomStore) Sync added in v0.0.2 

func (s *OomStore) Sync(ctx context.Context, opt types.SyncOpt) error

Sync a particular revision of a feature group from offline to online store. It is a streaming process - it writes to online store while reading from offline store. This helps get rid of unwanted out-of-memory errors, where size of the particular revision outgrows memory limit of your machine.

func (*OomStore) UpdateEntity 

func (s *OomStore) UpdateEntity(ctx context.Context, opt types.UpdateEntityOpt) error

Update metadata for an entity.

func (*OomStore) UpdateFeature 

func (s *OomStore) UpdateFeature(ctx context.Context, opt types.UpdateFeatureOpt) error

Update metadata of a feature.

func (*OomStore) UpdateGroup added in v0.0.3 

func (s *OomStore) UpdateGroup(ctx context.Context, opt types.UpdateGroupOpt) error

Update metadata of a feature group.

type StreamPushProcessor added in v0.0.4 

type StreamPushProcessor struct {
	// contains filtered or unexported fields
}

func (*StreamPushProcessor) Close added in v0.0.4 

func (p *StreamPushProcessor) Close() error

func (*StreamPushProcessor) Push added in v0.0.4 

func (p *StreamPushProcessor) Push(record types.StreamRecord)

Source Files

View all Source files

Directories

Path Synopsis
apply

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.