Documentation
¶
Overview ¶
Package storage provides the底层 storage layer for gograph using Pebble as the underlying key-value store. It handles data marshaling, key generation, and basic CRUD operations.
Package storage provides the底层 storage layer for gograph using Pebble as the underlying key-value store.
Package storage provides a key-value storage layer for gograph using PebbleDB. It wraps the Pebble database to provide a simplified interface for storing and retrieving graph data including nodes, relationships, and indices.
The storage layer uses the following key prefixes:
- "node:" - Node data
- "rel:" - Relationship data
- "label:" - Label index data
- "prop:" - Property index data
- "adj:" - Adjacency list data
- "meta:" - Metadata
Basic Usage:
db, err := storage.Open("/path/to/db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Store data
err = db.Put([]byte("key"), []byte("value"))
// Retrieve data
value, err := db.Get([]byte("key"))
Thread Safety:
DB is safe for concurrent use by multiple goroutines.
Index ¶
- Constants
- func AdjGroupKey(nodeID, relType, direction string) []byte
- func AdjKey(nodeID, relType, direction, relID string) []byte
- func AdjKeyPrefix(nodeID string) []byte
- func AdjKeyPrefixNodeAndType(nodeID, relType string) []byte
- func AdjKeyPrefixNodeAndTypeAndDir(nodeID, relType, direction string) []byte
- func DecodeLabel(code string) (string, bool)
- func DecodeLabelKey(key string) (string, bool)
- func EncodeLabel(label string) string
- func EncodeLabelKey(label string) string
- func LabelKey(labelName, nodeID string) []byte
- func LabelKeyPrefix(labelName string) []byte
- func Marshal(v interface{}) ([]byte, error)
- func MustDecodeLabelKey(key string) string
- func NodeKey(nodeID string) []byte
- func PropertyKey(labelName, propName, propValue string) []byte
- func PropertyKeyPrefix(labelName, propName string) []byte
- func RelKey(relID string) []byte
- func ResetLabelCoder()
- func SetLabelCoder(coder *LabelCoder)
- func Unmarshal(data []byte, v interface{}) error
- type Batch
- type DB
- type Iterator
- func (i *Iterator) Close() error
- func (i *Iterator) Error() error
- func (i *Iterator) Key() []byte
- func (i *Iterator) Next() bool
- func (i *Iterator) Prev() bool
- func (i *Iterator) SeekGE(key []byte) bool
- func (i *Iterator) SeekLT(key []byte) bool
- func (i *Iterator) Valid() bool
- func (i *Iterator) Value() []byte
- type LabelCoder
- type MemoryBatch
- type MemoryDB
Constants ¶
const ( // KeyPrefixNode is the prefix for node data keys. // Format: "node:<nodeID>" KeyPrefixNode = "node:" // KeyPrefixRel is the prefix for relationship data keys. // Format: "rel:<relID>" KeyPrefixRel = "rel:" // KeyPrefixLabel is the prefix for label index keys. // Format: "label:<labelName>:<nodeID>" KeyPrefixLabel = "label:" // KeyPrefixProp is the prefix for property index keys. // Format: "prop:<labelName>:<propName>:<propValue>:<nodeID>" KeyPrefixProp = "prop:" // KeyPrefixAdj is the prefix for adjacency list keys. // Format: "adj:<nodeID>:<relType>:<direction>:<relID>" KeyPrefixAdj = "adj:" // KeyPrefixMeta is the prefix for metadata keys. KeyPrefixMeta = "meta:" // KeyPrefixIndexCount is the key for storing the index count. KeyPrefixIndexCount = "meta:index_count" )
Key prefixes for different types of data in the storage layer. These prefixes ensure that different data types are stored in separate key ranges for efficient iteration and querying.
Variables ¶
This section is empty.
Functions ¶
func AdjGroupKey ¶ added in v0.2.3
AdjGroupKey returns the storage key for a merged adjacency group. A group contains all relationships of the same type and direction for a node. This reduces the number of small keys in the storage layer.
Parameters:
- nodeID: The ID of the node
- relType: The type of relationship
- direction: The direction ("out" or "in")
Returns the storage key as a byte slice.
Example:
key := storage.AdjGroupKey("node:a1", "KNOWS", "out")
// key = []byte("adj:node:a1:KNOWS:out")
func AdjKey ¶
AdjKey returns the storage key for an adjacency entry.
Parameters:
- nodeID: The ID of the node
- relType: The type of relationship
- direction: The direction ("out" or "in")
- relID: The ID of the relationship
Returns the storage key as a byte slice.
Example:
key := storage.AdjKey("node:a1", "KNOWS", "out", "rel:b2")
// key = []byte("adj:node:a1:KNOWS:out:rel:b2")
func AdjKeyPrefix ¶
AdjKeyPrefix returns the key prefix for all adjacency entries of a node. This is used for finding all relationships connected to a node.
Parameters:
- nodeID: The ID of the node
Returns the key prefix as a byte slice.
Example:
prefix := storage.AdjKeyPrefix("node:a1")
// prefix = []byte("adj:node:a1:")
func AdjKeyPrefixNodeAndType ¶
AdjKeyPrefixNodeAndType returns the key prefix for all adjacency entries of a node with a specific relationship type. This is used for finding all relationships of a specific type connected to a node.
Parameters:
- nodeID: The ID of the node
- relType: The type of relationship
Returns the key prefix as a byte slice.
Example:
prefix := storage.AdjKeyPrefixNodeAndType("node:a1", "KNOWS")
// prefix = []byte("adj:node:a1:KNOWS:")
func AdjKeyPrefixNodeAndTypeAndDir ¶
AdjKeyPrefixNodeAndTypeAndDir returns the key prefix for all adjacency entries of a node with a specific relationship type and direction. This is used for finding relationships in a specific direction.
Parameters:
- nodeID: The ID of the node
- relType: The type of relationship
- direction: The direction ("out" or "in")
Returns the key prefix as a byte slice.
Example:
prefix := storage.AdjKeyPrefixNodeAndTypeAndDir("node:a1", "KNOWS", "out")
// prefix = []byte("adj:node:a1:KNOWS:out:")
func DecodeLabel ¶ added in v0.2.3
DecodeLabel returns the original label for a short code using the global coder.
func DecodeLabelKey ¶ added in v0.2.3
DecodeLabelKey decodes a key-safe short code back to the original label.
func EncodeLabel ¶ added in v0.2.3
EncodeLabel returns the short code for a label using the global coder.
func EncodeLabelKey ¶ added in v0.2.3
EncodeLabelKey encodes a label into a key-safe short code with a prefix. This ensures encoded keys are distinguishable from raw labels.
func LabelKey ¶
LabelKey returns the storage key for a label index entry. Uses short label encoding to reduce key size.
Parameters:
- labelName: The name of the label
- nodeID: The ID of the node with this label
Returns the storage key as a byte slice.
Example:
key := storage.LabelKey("Person", "node:a1")
// key = []byte("label:La:node:a1") (if Person maps to code 'a')
func LabelKeyPrefix ¶
LabelKeyPrefix returns the key prefix for all entries with the given label. This is used for iterating over all nodes with a specific label. Uses short label encoding to reduce key size.
Parameters:
- labelName: The name of the label
Returns the key prefix as a byte slice.
Example:
prefix := storage.LabelKeyPrefix("Person")
// prefix = []byte("label:La:") (if Person maps to code 'a')
func Marshal ¶
Marshal encodes a value to bytes using MessagePack encoding. MessagePack is more compact and faster than Gob/JSON.
func MustDecodeLabelKey ¶ added in v0.2.3
MustDecodeLabelKey is like DecodeLabelKey but panics on failure.
func NodeKey ¶
NodeKey returns the storage key for a node with the given ID.
Parameters:
- nodeID: The unique identifier of the node
Returns the storage key as a byte slice.
Example:
key := storage.NodeKey("node:a1")
// key = []byte("node:node:a1")
func PropertyKey ¶
PropertyKey returns the storage key for a property index entry. Uses short label encoding to reduce key size.
Parameters:
- labelName: The name of the label
- propName: The name of the property
- propValue: The value of the property (encoded as string)
Returns the storage key as a byte slice.
Example:
key := storage.PropertyKey("Person", "name", "Alice")
// key = []byte("prop:La:name:Alice") (if Person maps to code 'a')
func PropertyKeyPrefix ¶
PropertyKeyPrefix returns the key prefix for all entries with the given label and property name. This is used for iterating over all nodes with a specific label and property. Uses short label encoding to reduce key size.
Parameters:
- labelName: The name of the label
- propName: The name of the property
Returns the key prefix as a byte slice.
Example:
prefix := storage.PropertyKeyPrefix("Person", "name")
// prefix = []byte("prop:La:name:") (if Person maps to code 'a')
func RelKey ¶
RelKey returns the storage key for a relationship with the given ID.
Parameters:
- relID: The unique identifier of the relationship
Returns the storage key as a byte slice.
Example:
key := storage.RelKey("rel:b2")
// key = []byte("rel:rel:b2")
func ResetLabelCoder ¶ added in v0.2.3
func ResetLabelCoder()
ResetLabelCoder resets the global label coder to a fresh state (for testing).
func SetLabelCoder ¶ added in v0.2.3
func SetLabelCoder(coder *LabelCoder)
SetLabelCoder replaces the global label coder (mainly for testing).
Types ¶
type Batch ¶
type Batch struct {
// contains filtered or unexported fields
}
Batch represents a collection of write operations that can be committed atomically. Use NewBatch to create a batch, add operations to it, then call Commit.
func (*Batch) Close ¶
Close releases resources associated with the batch. This should be called after the batch is no longer needed.
Returns an error if closing fails.
func (*Batch) Commit ¶
Commit applies all operations in the batch atomically. If any operation fails, none of the operations will be applied.
Returns an error if the commit fails.
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB wraps a Pebble database instance. It provides a simplified interface for key-value operations and manages the underlying Pebble database lifecycle.
DB is safe for concurrent use by multiple goroutines.
func Open ¶
Open opens a database at the given path and returns a DB instance.
Parameters:
- path: The directory path where the database files will be stored. If the directory doesn't exist, it will be created.
Returns a new DB instance or an error if the database cannot be opened.
Example:
db, err := storage.Open("/var/lib/gograph/mydb")
if err != nil {
log.Fatal(err)
}
defer db.Close()
func (*DB) Close ¶
Close closes the database and releases all associated resources. Any pending writes will be flushed before closing.
Returns an error if the close operation fails.
func (*DB) Delete ¶
Delete removes the value for the given key. If the key doesn't exist, this is a no-op.
Parameters:
- key: The key to delete
Returns an error if the operation fails.
Example:
err := db.Delete([]byte("mykey"))
if err != nil {
log.Fatal(err)
}
func (*DB) DumpAll ¶ added in v0.2.0
DumpAll returns all key-value pairs in the database. This is primarily useful for debugging and testing.
Returns a map of all keys to their values, or an error if the operation fails.
Warning: This method loads all data into memory and should not be used on large databases.
Example:
data, err := db.DumpAll()
if err != nil {
log.Fatal(err)
}
for key, value := range data {
fmt.Printf("%s: %s\n", key, value)
}
func (*DB) Get ¶
Get retrieves the value for the given key.
Parameters:
- key: The key to look up
Returns the value as a byte slice, or an error if the key is not found or if a database error occurs.
Example:
value, err := db.Get([]byte("mykey"))
if err != nil {
log.Printf("Key not found: %v", err)
} else {
fmt.Printf("Value: %s\n", value)
}
func (*DB) NewBatch ¶
NewBatch creates a new batch for atomic write operations. Batches allow multiple operations to be committed atomically.
Returns a new Batch instance.
Example:
batch := db.NewBatch()
batch.Put([]byte("key1"), []byte("value1"))
batch.Put([]byte("key2"), []byte("value2"))
batch.Delete([]byte("key3"))
err := batch.Commit()
func (*DB) NewIter ¶
NewIter creates a new iterator for scanning the database.
Parameters:
- opts: Iterator options. Pass nil for default options.
Returns a new Iterator or an error if creation fails.
Example:
iter, err := db.NewIter(nil)
if err != nil {
log.Fatal(err)
}
defer iter.Close()
for iter.SeekGE([]byte("prefix")); iter.Valid(); iter.Next() {
fmt.Printf("%s: %s\n", iter.Key(), iter.Value())
}
func (*DB) Put ¶
Put stores a key-value pair in the database. If the key already exists, its value will be overwritten.
Parameters:
- key: The key to store
- value: The value to store
Returns an error if the operation fails.
Example:
err := db.Put([]byte("mykey"), []byte("myvalue"))
if err != nil {
log.Fatal(err)
}
type Iterator ¶
type Iterator struct {
// contains filtered or unexported fields
}
Iterator provides ordered iteration over the database. Iterators must be closed when no longer needed to release resources.
func (*Iterator) Close ¶
Close releases resources associated with the iterator. This must be called when the iterator is no longer needed.
Returns an error if closing fails.
func (*Iterator) Key ¶
Key returns the current key. The returned slice is only valid until the next iterator operation.
func (*Iterator) Next ¶
Next advances the iterator to the next key.
Returns true if a valid key was found, false if the iterator is exhausted.
func (*Iterator) Prev ¶
Prev moves the iterator to the previous key.
Returns true if a valid key was found, false if the iterator is exhausted.
func (*Iterator) SeekGE ¶
SeekGE positions the iterator at the first key that is greater than or equal to the given key.
Parameters:
- key: The key to seek to
Returns true if a valid key was found, false otherwise.
func (*Iterator) SeekLT ¶
SeekLT positions the iterator at the last key that is less than the given key.
Parameters:
- key: The key to seek to
Returns true if a valid key was found, false otherwise.
type LabelCoder ¶ added in v0.2.3
type LabelCoder struct {
// contains filtered or unexported fields
}
LabelCoder manages short codes for label names to reduce index key size. It assigns sequential single-character codes (a-z, A-Z, 0-9, then multi-char) to labels as they are first encountered.
func NewLabelCoder ¶ added in v0.2.3
func NewLabelCoder() *LabelCoder
NewLabelCoder creates a new LabelCoder with optional pre-defined mappings.
func (*LabelCoder) Decode ¶ added in v0.2.3
func (lc *LabelCoder) Decode(code string) (string, bool)
Decode returns the original label for a short code.
func (*LabelCoder) Encode ¶ added in v0.2.3
func (lc *LabelCoder) Encode(label string) string
Encode returns the short code for a label, creating one if necessary.
type MemoryBatch ¶ added in v0.2.1
type MemoryBatch struct {
// contains filtered or unexported fields
}
MemoryBatch represents a batch of operations for MemoryDB.
func (*MemoryBatch) Close ¶ added in v0.2.1
func (b *MemoryBatch) Close() error
Close closes the batch.
func (*MemoryBatch) Commit ¶ added in v0.2.1
func (b *MemoryBatch) Commit() error
Commit applies the batch operations to the MemoryDB.
func (*MemoryBatch) Delete ¶ added in v0.2.1
func (b *MemoryBatch) Delete(key []byte) error
Delete adds a delete operation to the batch.
func (*MemoryBatch) Put ¶ added in v0.2.1
func (b *MemoryBatch) Put(key, value []byte) error
Put adds a put operation to the batch.
type MemoryDB ¶ added in v0.2.1
type MemoryDB struct {
// contains filtered or unexported fields
}
MemoryDB is an in-memory key-value store.
func (*MemoryDB) NewMemoryBatch ¶ added in v0.2.1
func (db *MemoryDB) NewMemoryBatch() *MemoryBatch
NewMemoryBatch creates a new MemoryBatch.
Source Files