Documentation
¶
Overview ¶
Package datastore offers common datastore API with multiple store implementations:
- CloudStore is a Google datastore implementation.
- FileStore is a simple file-based store implementation.
Index ¶
- Constants
- Variables
- func DeleteMulti(ctx context.Context, store Store, keys []*Key) (int, error)
- func IDKey(id, ts, st int64) int64
- func RegisterEntity(kind string, construct func() Entity)
- func SplitIDKey(id int64) (int64, int64, int64)
- type Cache
- type CloudQuery
- type CloudStore
- func (s *CloudStore) Create(ctx context.Context, key *Key, src Entity) error
- func (s *CloudStore) Delete(ctx context.Context, key *Key) error
- func (s *CloudStore) DeleteMulti(ctx context.Context, keys []*Key) error
- func (s *CloudStore) Get(ctx context.Context, key *Key, dst Entity) error
- func (s *CloudStore) GetAll(ctx context.Context, query Query, dst interface{}) ([]*Key, error)
- func (s *CloudStore) IDKey(kind string, id int64) *Key
- func (s *CloudStore) IncompleteKey(kind string) *Key
- func (s *CloudStore) NameKey(kind, name string) *Key
- func (s *CloudStore) NewQuery(kind string, keysOnly bool, keyParts ...string) Query
- func (s *CloudStore) Put(ctx context.Context, key *Key, src Entity) (*Key, error)
- func (s *CloudStore) Update(ctx context.Context, key *Key, fn func(Entity), dst Entity) error
- type Entity
- type EntityCache
- type EntityDecoder
- type EntityEncoder
- type ErrCacheMiss
- type FileQuery
- type FileStore
- func (s *FileStore) Create(ctx context.Context, key *Key, src Entity) error
- func (s *FileStore) Delete(ctx context.Context, key *Key) error
- func (s *FileStore) DeleteMulti(ctx context.Context, keys []*Key) error
- func (s *FileStore) Get(ctx context.Context, key *Key, dst Entity) error
- func (s *FileStore) GetAll(ctx context.Context, query Query, dst interface{}) ([]*Key, error)
- func (s *FileStore) IDKey(kind string, id int64) *Key
- func (s *FileStore) IncompleteKey(kind string) *Key
- func (s *FileStore) NameKey(kind, name string) *Key
- func (s *FileStore) NewQuery(kind string, keysOnly bool, keyParts ...string) Query
- func (s *FileStore) Put(ctx context.Context, key *Key, src Entity) (*Key, error)
- func (s *FileStore) Update(ctx context.Context, key *Key, fn func(Entity), dst Entity) error
- type Key
- type NoCache
- type Property
- type Query
- type Store
Constants ¶
const ( MaxKeys = 500 // Maximum number of datastore keys to retrieve in one go. MaxBlob = 1000000 // 1MB )
CloudStore limits.
const ( EpochStart = 1483228800 // Start of the AusOcean epoch, namely 2017-01-01 00:00:00+Z. EpochEnd = math.MaxInt64 // End of the epoch. SubTimeBits = 3 // Sub-second time bits. )
FileStore constants.
Variables ¶
var ( ErrDecoding = errors.New("decoding error") ErrUnimplemented = errors.New("unimplemented feature") ErrInvalidField = errors.New("invalid field") ErrInvalidFilter = errors.New("invalid filter") ErrInvalidOperator = errors.New("invalid operator") ErrInvalidValue = errors.New("invalid value") ErrOperatorMissing = errors.New("operator missing") ErrNoSuchEntity = datastore.ErrNoSuchEntity ErrEntityExists = errors.New("entity exists") ErrWrongType = errors.New("wrong type") ErrInvalidType = datastore.ErrInvalidEntityType ErrInvalidStoreID = errors.New("invalid datastore id") )
Functions ¶
func DeleteMulti ¶
DeleteMulti is a wrapper for store.DeleteMulti which returns the number of deletions.
func IDKey ¶
IDKey makes a datastore ID key by combining an ID, a Unix timestamp and an optional subtime (st). The latter is used to disambiguate otherwise-identical timestamps.
- The least significant 32 bits of the ID.
- The timestamp from the start of the "AusOcean epoch", 2017-01-01 00:00:00+Z (29 bits).
- The subtime (3 bits).
func RegisterEntity ¶
RegisterEntity registers a new kind of entity and its constructor.
Types ¶
type Cache ¶
type Cache interface {
Set(key *Key, src Entity) error // Set adds or updates a value to the cache.
Get(key *Key, dst Entity) error // Get retrieves a value from the cache, or returns ErrCacheMiss.
Delete(key *Key) // Delete removes a value from the cache.
Reset() // Reset resets (clears) the cache.
}
Cache defines the (optional) caching interface used by Entity.
type CloudQuery ¶
type CloudQuery struct {
// contains filtered or unexported fields
}
CloudQuery implements Query for the Google Cloud Datastore.
func (*CloudQuery) Filter ¶
func (q *CloudQuery) Filter(filterStr string, value interface{}) error
func (*CloudQuery) FilterField ¶
func (q *CloudQuery) FilterField(fieldName string, operator string, value interface{}) error
FilterField filters a query.
func (*CloudQuery) Limit ¶
func (q *CloudQuery) Limit(limit int)
Limit limits the number of results returned.
func (*CloudQuery) Offset ¶
func (q *CloudQuery) Offset(offset int)
Offset sets the number of keys to skip before returning results.
func (*CloudQuery) Order ¶
func (q *CloudQuery) Order(fieldName string)
type CloudStore ¶
type CloudStore struct {
// contains filtered or unexported fields
}
CloudStore implements Store for the Google Cloud Datastore.
func (*CloudStore) DeleteMulti ¶
func (s *CloudStore) DeleteMulti(ctx context.Context, keys []*Key) error
func (*CloudStore) IDKey ¶
func (s *CloudStore) IDKey(kind string, id int64) *Key
IDKey returns an ID key given a kind and an int64 ID.
func (*CloudStore) IncompleteKey ¶
func (s *CloudStore) IncompleteKey(kind string) *Key
IncompleteKey returns an incomplete key given a kind.
func (*CloudStore) NameKey ¶
func (s *CloudStore) NameKey(kind, name string) *Key
NameKey returns an name key given a kind and a (string) name.
func (*CloudStore) NewQuery ¶
func (s *CloudStore) NewQuery(kind string, keysOnly bool, keyParts ...string) Query
NewQuery returns a new CloudQuery and is a wrapper for datastore.NewQuery. If keysOnly is true the query is set to keys only, but keyParts are ignored.
type Entity ¶
type Entity interface {
Copy(dst Entity) (Entity, error) // Copy an entity to dst, or return a copy of the entity when dst is nil.
GetCache() Cache // Returns a cache, or nil for no caching.
}
Entity defines the common interface for our datastore entities.
func CopyEntity ¶
CopyEntity copies src into dst (if non-nil) or allocates a new *T. It also enforces that dst is of the correct concrete type.
It is a generic helper function to reduce boilerplate code when implementing the Entity interface. T is the concrete struct type (e.g. Plan), and PT is a pointer to T (e.g. *Plan). PT must implement the Entity interface.
type EntityCache ¶
type EntityCache struct {
// contains filtered or unexported fields
}
EntityCache, which implements Cache, represents a cache for holding datastore entities indexed by key.
func (*EntityCache) Delete ¶
func (c *EntityCache) Delete(key *Key)
Delete removes a value from the cache.
type EntityDecoder ¶
EntityDecoder defines an Entity with a custom decoding function.
type EntityEncoder ¶
type EntityEncoder interface {
Encode() []byte // Encode an entity into bytes.
}
EntityEncoder defines an Entity with a custom encoding function.
type ErrCacheMiss ¶
type ErrCacheMiss struct {
// contains filtered or unexported fields
}
ErrCacheMiss is the type of error returned when a key is not found in the cache.
func (ErrCacheMiss) Error ¶
func (e ErrCacheMiss) Error() string
Error returns an error string for errors of type ErrCacheMiss.
type FileQuery ¶
type FileQuery struct {
// contains filtered or unexported fields
}
FileQuery implements Query for FileStore.
func (*FileQuery) Filter ¶
Filter implements FileQuery matching against key parts. This function transforms certain properties commonly used in queries:
- Properties named ID or MID are reduced to their least-significant 32 bits.
- Properties named Timestamp are converted from the Unix epoch to the AusOcean epoch.
func (*FileQuery) FilterField ¶
FilterField filters a query by field name and value using the specified operator. If the field name matches one of the declared keyParts, the query is handled efficiently using filename-based key filtering via Filter. Otherwise, the query falls back to reflection-based entity field filtering, which reads and inspects each entity file.
type FileStore ¶
type FileStore struct {
// contains filtered or unexported fields
}
FileStore implements Store for file storage. Each entity is represented as a file named <key> under the directory <dir>/<id>/<kind>, where <kind> and <key> are the entity kind and key respectively.
FileStore implements a simple form of indexing based on the period-separated parts of key names. For example, a User has the key structure <Skey>.<Email>. Filestore queries that utilize key-based indexes must specify the optional key parts when constructing the query.
q = store.NewQuery(ctx, "User", "Skey", "Email")
All but the last part of the key must not contain periods. In this example, only Email may contain periods.
The key "671314941988.test@ausocean.org" would match 671314941988 or "test@ausocean.org" or both.
To match both Skey and Email:
q.Filter("Skey =", 671314941988)
q.Filter("Email =", "test@ausocean.org)
To match just Skey, i.e., return all entities for a given site, simply omit the Email part as follows.
q.Filter("Skey =", 671314941988)
Alternatively, specify nil for the Email which is a wild card.
q.Filter("Skey =", 671314941988)
q.Filter("Email =", nil)
Using nil is most useful however when ignoring a key part which has key parts before and after that must be matched.
To match just Email, i.e., return all entities for a given user:
q.Filter("Email =", "test@ausocean.org")
The following query would however fail due to the wrong order:
q.Filter("Email =", "test@ausocean.org")
q.Filter("Skey =", 671314941988)
FileStore represents all keys as strings. To facilitate substring comparisons, 64-bit ID keys are represented as two 32-bit integers separated by a period and the keyParts in NewQuery must reflect this.
In addition to key-based filtering, FileStore supports content-based filtering using FilterField. If the field name matches a key part, the query is evaluated efficiently using the file name. Otherwise, the entity file is read, parsed, and the field is evaluated using reflection. This allows for flexible filtering even on fields not encoded in the key.
func (*FileStore) DeleteMulti ¶
func (*FileStore) GetAll ¶
GetAll implements Store.GetAll for FileStore. Unlike CloudStore queries are only valid against key parts. If the entity has a property named Key, it is automatically populated with its Key value.
func (*FileStore) IDKey ¶
IDKey returns a FileStore ID key for the given kind, setting Name to the file name. A 64-bit ID is represented as two 32-bit unsigned integers separated by a period.
func (*FileStore) IncompleteKey ¶
IncompleteKey returns an incomplete key given a kind.
func (*FileStore) NameKey ¶
NameKey returns a FileStore name key for the given kind, setting Name to the file name.
type NoCache ¶
type NoCache struct{}
NoCache is a helper struct to reduce boilerplate code when implementing the Entity interface for entities that do not require caching.
type Query ¶
type Query interface {
Filter(filterStr string, value interface{}) error // Filters a query (deprecated).
FilterField(fieldName string, operator string, value interface{}) error // Filters a query.
Order(fieldName string) // Orders a query.
Limit(limit int) // Limits the number of results returned.
Offset(offset int) // How many keys to skip before returning results.
}
Query defines the query interface, which is a subset of Google Cloud's datastore.Query. Note that unlike datastore.Query methods, these methods modify the Query value without returning it. A nil value denotes a wild card (match anything).
See also Google Cloud datastore.Query.Filter and datastore.Query.Order.
type Store ¶
type Store interface {
IDKey(kind string, id int64) *Key // Returns an ID key.
NameKey(kind, name string) *Key // Returns a name key.
IncompleteKey(kind string) *Key // Returns an incomplete key.
NewQuery(kind string, keysOnly bool, keyParts ...string) Query // Returns a new query.
Get(ctx context.Context, key *Key, dst Entity) error // Gets a single entity by its key.
GetAll(ctx context.Context, q Query, dst interface{}) ([]*Key, error) // Runs a query and returns all matching entities.
Create(ctx context.Context, key *Key, src Entity) error // Creates a single entity by its key.
Put(ctx context.Context, key *Key, src Entity) (*Key, error) // Put or creates a single entity by its key.
Update(ctx context.Context, key *Key, fn func(Entity), dst Entity) error // Atomically updates a single entity by its key.
Delete(ctx context.Context, key *Key) error // Deletes a single entity by its key.
DeleteMulti(ctx context.Context, keys []*Key) error // Deletes multiple entities by their keys.
}
Store defines the datastore interface. It is a blend and subset of Google Cloud datastore functions and datastore.Client methods.
See also https://godoc.org/cloud.google.com/go.
Source Files