storage

package
v1.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 30, 2024 License: Apache-2.0 Imports: 11 Imported by: 8

Documentation

Overview

Package storage contains storage interfaces and implementations.

Index

Constants

View Source
const (
	// DefaultMaxTuplesPerWrite specifies the default maximum number of tuples that can be written
	// in a single write operation. This constant is used to limit the batch size in write operations
	// to maintain performance and avoid overloading the system. The value is set to 100 tuples,
	// which is a balance between efficiency and resource usage.
	DefaultMaxTuplesPerWrite = 100

	// DefaultMaxTypesPerAuthorizationModel defines the default upper limit on the number of distinct
	// types that can be included in a single authorization model. This constraint helps in managing
	// the complexity and ensuring the maintainability of the authorization models. The limit is
	// set to 100 types, providing ample flexibility while keeping the model manageable.
	DefaultMaxTypesPerAuthorizationModel = 100

	// DefaultPageSize sets the default number of items to be returned in a single page when paginating
	// through a set of results. This constant is used to standardize the pagination size across various
	// parts of the system, ensuring a consistent and manageable volume of data per page. The default
	// value is set to 50, balancing detail per page with the overall number of pages.
	DefaultPageSize = 50
)

Variables

View Source
var (
	// ErrCollision is returned when an item already exists within the store.
	ErrCollision = errors.New("item already exists")

	// ErrInvalidContinuationToken is returned when the continuation token is invalid.
	ErrInvalidContinuationToken = errors.New("invalid continuation token")

	// ErrMismatchObjectType is returned when there is a type discrepancy between the requested
	// object in the ReadChanges API and the type indicated by the continuation token.
	ErrMismatchObjectType = errors.New("mismatched types in request and continuation token")

	// ErrInvalidWriteInput is returned when the tuple to be written
	// already existed or the tuple to be deleted did not exist.
	ErrInvalidWriteInput = errors.New("invalid write input")

	// ErrTransactionalWriteFailed is returned when two writes attempt to write the same tuple at the same time.
	ErrTransactionalWriteFailed = errors.New("transactional write failed due to conflict")

	// ErrExceededWriteBatchLimit is returned when MaxTuplesPerWrite is exceeded.
	ErrExceededWriteBatchLimit = errors.New("number of operations exceeded write batch limit")

	// ErrCancelled is returned when the request has been cancelled.
	ErrCancelled = errors.New("request has been cancelled")

	// ErrDeadlineExceeded is returned when the request's deadline is exceeded.
	ErrDeadlineExceeded = errors.New("request deadline exceeded")

	// ErrNotFound is returned when the object does not exist.
	ErrNotFound = errors.New("not found")
)
View Source
var ErrIteratorDone = errors.New("iterator done")

ErrIteratorDone is returned when the iterator has finished iterating through all the items.

Functions

func ContextWithRelationshipTupleReader added in v1.5.0

func ContextWithRelationshipTupleReader(
	parent context.Context,
	reader RelationshipTupleReader,
) context.Context

ContextWithRelationshipTupleReader sets the provided [RelationshipTupleReader] in the context. The context returned is a new context derived from the parent context provided.

func ExceededMaxTypeDefinitionsLimitError

func ExceededMaxTypeDefinitionsLimitError(limit int) error

ExceededMaxTypeDefinitionsLimitError constructs an error indicating that the maximum allowed limit for type definitions has been exceeded.

func InvalidWriteInputError

func InvalidWriteInputError(tk tuple.TupleWithoutCondition, operation openfgav1.TupleOperation) error

InvalidWriteInputError generates an error for invalid operations in a tuple store. This function is invoked when an attempt is made to write or delete a tuple with invalid conditions. Specifically, it addresses two scenarios: 1. Attempting to delete a non-existent tuple. 2. Attempting to write a tuple that already exists.

Types

type AssertionsBackend

type AssertionsBackend interface {
	// WriteAssertions overwrites the assertions for a store and modelID.
	WriteAssertions(ctx context.Context, store, modelID string, assertions []*openfgav1.Assertion) error

	// ReadAssertions returns the assertions for a store and modelID.
	// If no assertions were ever written, it must return an empty list.
	ReadAssertions(ctx context.Context, store, modelID string) ([]*openfgav1.Assertion, error)
}

AssertionsBackend is an interface that defines the set of methods for reading and writing assertions.

type AuthorizationModelBackend

type AuthorizationModelBackend interface {
	AuthorizationModelReadBackend
	TypeDefinitionWriteBackend
}

AuthorizationModelBackend provides an read/write interface for managing models and their type definitions.

type AuthorizationModelReadBackend

type AuthorizationModelReadBackend interface {
	// ReadAuthorizationModel reads the model corresponding to store and model ID.
	// If it's not found, it must return ErrNotFound.
	ReadAuthorizationModel(ctx context.Context, store string, id string) (*openfgav1.AuthorizationModel, error)

	// ReadAuthorizationModels reads all models for the supplied store and returns them in descending order of ULID (from newest to oldest).
	ReadAuthorizationModels(ctx context.Context, store string, options ReadAuthorizationModelsOptions) ([]*openfgav1.AuthorizationModel, []byte, error)

	// FindLatestAuthorizationModel returns the last model for the store.
	// If none were ever written, it must return ErrNotFound.
	FindLatestAuthorizationModel(ctx context.Context, store string) (*openfgav1.AuthorizationModel, error)
}

AuthorizationModelReadBackend provides a read interface for managing type definitions.

type CachedResult added in v1.5.7

type CachedResult[T any] struct {
	Value   T
	Expired bool
}

type ChangelogBackend

type ChangelogBackend interface {
	// ReadChanges returns the writes and deletes that have occurred for tuples within a store,
	// in the order that they occurred.
	// You can optionally provide a filter to filter out changes for objects of a specific type.
	// The horizonOffset should be specified using a unit no more granular than a millisecond.
	// It should always return a non-empty continuation token so readers can continue reading later, except the case where
	// if no changes are found, it should return storage.ErrNotFound and an empty continuation token.
	// It the objectType and the type in the continuation token don't match, it should return ErrMismatchObjectType.
	ReadChanges(ctx context.Context, store, objectType string, options ReadChangesOptions, horizonOffset time.Duration) ([]*openfgav1.TupleChange, []byte, error)
}

ChangelogBackend is an interface for interacting with and managing changelogs.

type ConditionsFilteredTupleKeyIterator added in v1.5.6

type ConditionsFilteredTupleKeyIterator struct {
	// contains filtered or unexported fields
}

func NewConditionsFilteredTupleKeyIterator added in v1.5.6

func NewConditionsFilteredTupleKeyIterator(iter TupleKeyIterator, filter TupleKeyConditionFilterFunc) *ConditionsFilteredTupleKeyIterator

NewConditionsFilteredTupleKeyIterator returns a TupleKeyIterator that filters out all *openfgav1.Tuple(s) that don't meet the conditions of the provided TupleKeyFilterFunc.

func (*ConditionsFilteredTupleKeyIterator) Head added in v1.5.9

Head returns the next most tuple in the underlying iterator that meets the filter function this iterator was constructed with. The underlying iterator may advance but calling consecutive Head will yield consistent result. Further, calling Head following by Next will also yield consistent result. This function is not thread-safe.

func (*ConditionsFilteredTupleKeyIterator) Next added in v1.5.6

Next returns the next most tuple in the underlying iterator that meets the filter function this iterator was constructed with. This function is not thread-safe.

func (*ConditionsFilteredTupleKeyIterator) Stop added in v1.5.6

Stop see [Iterator.Stop].

type ConsistencyOptions added in v1.5.7

type ConsistencyOptions struct {
	Preference openfgav1.ConsistencyPreference
}

ConsistencyOptions represents the options that can be used for methods that accept a consistency preference.

type Deletes

Deletes is a typesafe alias for Delete arguments.

type InMemoryCache added in v1.5.7

type InMemoryCache[T any] interface {

	// Get If the key exists, returns the value. If the key didn't exist, returns nil.
	Get(key string) *CachedResult[T]
	Set(key string, value T, ttl time.Duration)

	// Stop cleans resources.
	Stop()
}

InMemoryCache is a general purpose cache to store things in memory.

type InMemoryLRUCache added in v1.5.7

type InMemoryLRUCache[T any] struct {
	// contains filtered or unexported fields
}

func NewInMemoryLRUCache added in v1.5.7

func NewInMemoryLRUCache[T any](opts ...InMemoryLRUCacheOpt[T]) *InMemoryLRUCache[T]

func (InMemoryLRUCache[T]) Get added in v1.5.7

func (i InMemoryLRUCache[T]) Get(key string) *CachedResult[T]

func (InMemoryLRUCache[T]) Set added in v1.5.7

func (i InMemoryLRUCache[T]) Set(key string, value T, ttl time.Duration)

func (InMemoryLRUCache[T]) Stop added in v1.5.7

func (i InMemoryLRUCache[T]) Stop()

type InMemoryLRUCacheOpt added in v1.5.7

type InMemoryLRUCacheOpt[T any] func(i *InMemoryLRUCache[T])

func WithMaxCacheSize added in v1.5.7

func WithMaxCacheSize[T any](maxElements int64) InMemoryLRUCacheOpt[T]

type Iterator

type Iterator[T any] interface {
	// Next will return the next available
	// item or ErrIteratorDone if no more
	// items are available.
	Next(ctx context.Context) (T, error)

	// Stop terminates iteration over
	// the underlying iterator.
	Stop()

	// Head will return the first item or ErrIteratorDone if the iterator
	// is empty.
	// It's possible for this method to advance the iterator internally, but a subsequent call to Next will not miss any results.
	// Calling Head() continuously without calling Next() will yield the same result (the first one) over and over.
	Head(ctx context.Context) (T, error)
}

Iterator is a generic interface defining methods for iterating over a collection of items of type T.

func NewCombinedIterator

func NewCombinedIterator[T any](iters ...Iterator[T]) Iterator[T]

NewCombinedIterator takes generic iterators of a given type T and combines them into a single iterator that yields all the values from all iterators. Duplicates can be returned.

type ListStoresOptions added in v1.5.7

type ListStoresOptions struct {
	Pagination PaginationOptions
}

ListStoresOptions represents the options that can be used with the ListStores method.

type OpenFGADatastore

type OpenFGADatastore interface {
	TupleBackend
	AuthorizationModelBackend
	StoresBackend
	AssertionsBackend
	ChangelogBackend

	// IsReady reports whether the datastore is ready to accept traffic.
	IsReady(ctx context.Context) (ReadinessStatus, error)

	// Close closes the datastore and cleans up any residual resources.
	Close()
}

OpenFGADatastore is an interface that defines a set of methods for interacting with and managing data in an OpenFGA (Fine-Grained Authorization) system.

type PaginationOptions

type PaginationOptions struct {
	PageSize int
	From     string
}

PaginationOptions should not be instantiated directly. Use NewPaginationOptions.

func NewPaginationOptions

func NewPaginationOptions(ps int32, contToken string) PaginationOptions

NewPaginationOptions creates a new PaginationOptions instance with a specified page size and continuation token. If the input page size is empty, it uses DefaultPageSize.

type ReadAuthorizationModelsOptions added in v1.5.7

type ReadAuthorizationModelsOptions struct {
	Pagination PaginationOptions
}

ReadAuthorizationModelOptions represents the options that can be used with the ReadAuthorizationModels method.

type ReadChangesOptions added in v1.5.7

type ReadChangesOptions struct {
	Pagination PaginationOptions
}

ReadChangesOptions represents the options that can be used with the ReadChanges method.

type ReadOptions added in v1.5.7

type ReadOptions struct {
	Consistency ConsistencyOptions
}

ReadOptions represents the options that can be used with the Read method.

type ReadPageOptions added in v1.5.7

type ReadPageOptions struct {
	Pagination  PaginationOptions
	Consistency ConsistencyOptions
}

ReadPageOptions represents the options that can be used with the ReadPage method.

type ReadStartingWithUserFilter

type ReadStartingWithUserFilter struct {
	// Mandatory.
	ObjectType string
	// Mandatory.
	Relation string
	// Mandatory.
	UserFilter []*openfgav1.ObjectRelation

	// Optional. It can be nil. If present, it will be sorted in ascending order.
	// The datastore should return the intersection between this filter and what is in the database.
	ObjectIDs SortedSet
}

ReadStartingWithUserFilter specifies the filter options that will be used to constrain the [RelationshipTupleReader.ReadStartingWithUser] query.

type ReadStartingWithUserOptions added in v1.5.7

type ReadStartingWithUserOptions struct {
	Consistency ConsistencyOptions
}

ReadStartingWithUserOptions represents the options that can be used with the ReadStartingWithUser method.

type ReadUserTupleOptions added in v1.5.7

type ReadUserTupleOptions struct {
	Consistency ConsistencyOptions
}

ReadUserTupleOptions represents the options that can be used with the ReadUserTuple method.

type ReadUsersetTuplesFilter added in v0.4.0

type ReadUsersetTuplesFilter struct {
	Object                      string                         // Required.
	Relation                    string                         // Required.
	AllowedUserTypeRestrictions []*openfgav1.RelationReference // Optional.
}

ReadUsersetTuplesFilter specifies the filter options that will be used to constrain the ReadUsersetTuples query.

type ReadUsersetTuplesOptions added in v1.5.7

type ReadUsersetTuplesOptions struct {
	Consistency ConsistencyOptions
}

ReadUsersetTuplesOptions represents the options that can be used with the ReadUsersetTuples method.

type ReadinessStatus added in v1.3.8

type ReadinessStatus struct {
	// Message is a human-friendly status message for the current datastore status.
	Message string

	IsReady bool
}

ReadinessStatus represents the readiness status of the datastore.

type RedBlackTreeSet added in v1.5.6

type RedBlackTreeSet struct {
	// contains filtered or unexported fields
}

func NewSortedSet added in v1.5.6

func NewSortedSet() *RedBlackTreeSet

func (*RedBlackTreeSet) Add added in v1.5.6

func (r *RedBlackTreeSet) Add(value string)

func (*RedBlackTreeSet) Exists added in v1.5.6

func (r *RedBlackTreeSet) Exists(value string) bool

func (*RedBlackTreeSet) Max added in v1.5.6

func (r *RedBlackTreeSet) Max() string

func (*RedBlackTreeSet) Min added in v1.5.6

func (r *RedBlackTreeSet) Min() string

func (*RedBlackTreeSet) Size added in v1.5.6

func (r *RedBlackTreeSet) Size() int

func (*RedBlackTreeSet) Values added in v1.5.6

func (r *RedBlackTreeSet) Values() []string

type RelationshipTupleReader

type RelationshipTupleReader interface {
	// Read the set of tuples associated with `store` and `tupleKey`, which may be nil or partially filled. If nil,
	// Read will return an iterator over all the tuples in the given `store`. If the `tupleKey` is partially filled,
	// it will return an iterator over those tuples which match the `tupleKey`. Note that at least one of `Object`
	// or `User` (or both), must be specified in this case.
	//
	// The caller must be careful to close the [TupleIterator], either by consuming the entire iterator or by closing it.
	// There is NO guarantee on the order of the tuples returned on the iterator.
	Read(ctx context.Context, store string, tupleKey *openfgav1.TupleKey, options ReadOptions) (TupleIterator, error)

	// ReadPage functions similarly to Read but includes support for pagination. It takes
	// mandatory ReadPageOptions options. PageSize will always be greater than zero.
	// It returns a slice of tuples along with a continuation token. This token can be used for retrieving subsequent pages of data.
	// There is NO guarantee on the order of the tuples in one page.
	ReadPage(
		ctx context.Context,
		store string,
		tupleKey *openfgav1.TupleKey,
		options ReadPageOptions,
	) ([]*openfgav1.Tuple, []byte, error)

	// ReadUserTuple tries to return one tuple that matches the provided key exactly.
	// If none is found, it must return [ErrNotFound].
	ReadUserTuple(
		ctx context.Context,
		store string,
		tupleKey *openfgav1.TupleKey,
		options ReadUserTupleOptions,
	) (*openfgav1.Tuple, error)

	// ReadUsersetTuples returns all userset tuples for a specified object and relation.
	// For example, given the following relationship tuples:
	//	document:doc1, viewer, user:*
	//	document:doc1, viewer, group:eng#member
	// and the filter
	//	object=document:1, relation=viewer, allowedTypesForUser=[group#member]
	// this method would return the tuple (document:doc1, viewer, group:eng#member)
	// If allowedTypesForUser is empty, both tuples would be returned.
	// There is NO guarantee on the order returned on the iterator.
	ReadUsersetTuples(
		ctx context.Context,
		store string,
		filter ReadUsersetTuplesFilter,
		options ReadUsersetTuplesOptions,
	) (TupleIterator, error)

	// ReadStartingWithUser performs a reverse read of relationship tuples starting at one or
	// more user(s) or userset(s) and filtered by object type and relation and possibly a list of object IDs.
	//
	// For example, given the following relationship tuples:
	//   document:doc1, viewer, user:jon
	//   document:doc2, viewer, group:eng#member
	//   document:doc3, editor, user:jon
	//   document:doc4, viewer, group:eng#member
	//
	// ReadStartingWithUser for ['user:jon', 'group:eng#member'] filtered by 'document#viewer'
	// and 'document:doc1, document:doc2' would
	// return ['document:doc1#viewer@user:jon', 'document:doc2#viewer@group:eng#member'].
	// There is NO guarantee on the order returned on the iterator.
	ReadStartingWithUser(
		ctx context.Context,
		store string,
		filter ReadStartingWithUserFilter,
		options ReadStartingWithUserOptions,
	) (TupleIterator, error)
}

RelationshipTupleReader is an interface that defines the set of methods required to read relationship tuples from a data store.

func RelationshipTupleReaderFromContext added in v1.5.0

func RelationshipTupleReaderFromContext(ctx context.Context) (RelationshipTupleReader, bool)

RelationshipTupleReaderFromContext extracts a [RelationshipTupleReader] from the provided context (if any). If no such value is in the context a boolean false is returned, otherwise the RelationshipTupleReader is returned.

type RelationshipTupleWriter

type RelationshipTupleWriter interface {
	// Write updates data in the tuple backend, performing all delete operations in
	// `deletes` before adding new values in `writes`, returning the time of the transaction, or an error.
	// If there are more than MaxTuplesPerWrite, it must return ErrExceededWriteBatchLimit.
	// If two requests attempt to write the same tuple at the same time, it must return ErrTransactionalWriteFailed.
	// If the tuple to be written already existed or the tuple to be deleted didn't exist, it must return ErrInvalidWriteInput.
	Write(ctx context.Context, store string, d Deletes, w Writes) error

	// MaxTuplesPerWrite returns the maximum number of items (writes and deletes combined)
	// allowed in a single write transaction.
	MaxTuplesPerWrite() int
}

RelationshipTupleWriter is an interface that defines the set of methods required for writing relationship tuples in a data store.

type SortedSet added in v1.5.6

type SortedSet interface {
	Size() int

	// Min returns an empty string if the set is empty.
	Min() string

	// Max returns an empty string if the set is empty.
	Max() string
	Add(value string)
	Exists(value string) bool

	// Values returns the elements in the set in sorted order (ascending).
	Values() []string
}

SortedSet stores a set (no duplicates allowed) of string IDs in memory in a way that also provides fast sorted access.

type StoresBackend

type StoresBackend interface {
	CreateStore(ctx context.Context, store *openfgav1.Store) (*openfgav1.Store, error)
	DeleteStore(ctx context.Context, id string) error
	GetStore(ctx context.Context, id string) (*openfgav1.Store, error)
	ListStores(ctx context.Context, options ListStoresOptions) ([]*openfgav1.Store, []byte, error)
}

StoresBackend is an interface that defines the set of methods required for interacting with and managing different types of storage backends.

type TupleBackend

type TupleBackend interface {
	RelationshipTupleReader
	RelationshipTupleWriter
}

A TupleBackend provides a read/write interface for managing tuples.

type TupleIterator

type TupleIterator = Iterator[*openfgav1.Tuple]

TupleIterator is an iterator for *openfgav1.Tuple(s). It is closed by explicitly calling [Iterator.Stop] or by calling [Iterator.Next] until it returns an ErrIteratorDone error.

func NewStaticTupleIterator

func NewStaticTupleIterator(tuples []*openfgav1.Tuple) TupleIterator

NewStaticTupleIterator returns a TupleIterator that iterates over the provided slice.

type TupleKeyConditionFilterFunc added in v1.5.6

type TupleKeyConditionFilterFunc func(tupleKey *openfgav1.TupleKey) (bool, error)

TupleKeyConditionFilterFunc is a filter function that is used to filter out tuples from a TupleKeyIterator that don't meet the tuple the conditions provided by the request. Implementations should return true if the tuple should be returned and false if it should be filtered out. Errors will be treated as false. If none of the tuples are valid AND there are errors, Next() will return the last error.

type TupleKeyFilterFunc

type TupleKeyFilterFunc func(tupleKey *openfgav1.TupleKey) bool

TupleKeyFilterFunc is a filter function that is used to filter out tuples from a TupleKeyIterator that don't meet certain criteria. Implementations should return true if the tuple should be returned and false if it should be filtered out.

type TupleKeyIterator

type TupleKeyIterator = Iterator[*openfgav1.TupleKey]

TupleKeyIterator is an iterator for *openfgav1.TupleKey(s). It is closed by explicitly calling [Iterator.Stop] or by calling [Iterator.Next] until it returns an ErrIteratorDone error.

func NewFilteredTupleKeyIterator

func NewFilteredTupleKeyIterator(iter TupleKeyIterator, filter TupleKeyFilterFunc) TupleKeyIterator

NewFilteredTupleKeyIterator returns a TupleKeyIterator that filters out all *openfgav1.Tuple(s) that don't meet the conditions of the provided TupleKeyFilterFunc.

func NewStaticTupleKeyIterator

func NewStaticTupleKeyIterator(tupleKeys []*openfgav1.TupleKey) TupleKeyIterator

NewStaticTupleKeyIterator returns a TupleKeyIterator that iterates over the provided slice.

func NewTupleKeyIteratorFromTupleIterator

func NewTupleKeyIteratorFromTupleIterator(iter TupleIterator) TupleKeyIterator

NewTupleKeyIteratorFromTupleIterator takes a TupleIterator and yields all the *openfgav1.TupleKey(s) from it as a TupleKeyIterator.

type TupleRecord added in v1.3.8

type TupleRecord struct {
	Store            string
	ObjectType       string
	ObjectID         string
	Relation         string
	User             string
	ConditionName    string
	ConditionContext *structpb.Struct
	Ulid             string
	InsertedAt       time.Time
}

TupleRecord represents a record structure used to store information about a specific tuple.

func (*TupleRecord) AsTuple added in v1.3.8

func (t *TupleRecord) AsTuple() *openfgav1.Tuple

AsTuple converts a TupleRecord into a *openfgav1.Tuple.

type TypeDefinitionWriteBackend

type TypeDefinitionWriteBackend interface {
	// MaxTypesPerAuthorizationModel returns the maximum number of type definition rows/items per model.
	MaxTypesPerAuthorizationModel() int

	// WriteAuthorizationModel writes an authorization model for the given store.
	WriteAuthorizationModel(ctx context.Context, store string, model *openfgav1.AuthorizationModel) error
}

TypeDefinitionWriteBackend provides a write interface for managing typed definition.

type Writes

type Writes = []*openfgav1.TupleKey

Writes is a typesafe alias for Write arguments.

Directories

Path Synopsis
Package memory contains an implementation of the storage interface that lives in memory.
Package memory contains an implementation of the storage interface that lives in memory.
Package mysql contains an implementation of the storage interface that works with MySQL.
Package mysql contains an implementation of the storage interface that works with MySQL.
Package postgres contains an implementation of the storage interface that works with Postgres.
Package postgres contains an implementation of the storage interface that works with Postgres.
Package sqlcommon contains utility functions shared among all SQL data stores.
Package sqlcommon contains utility functions shared among all SQL data stores.
Package storagewrappers contains decorators for storage implementations.
Package storagewrappers contains decorators for storage implementations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL