Documentation
¶
Index ¶
- func MetadataHash(metadata map[string]string) string
- func MetadataMerge(a map[string]string, b map[string]string) map[string]string
- type IndexDB
- func (ot *IndexDB) Close()
- func (ot *IndexDB) Commit(f fs.AtomicFileWriter, hsh string, shard int, timestamp int64, deletion bool, ...) error
- func (ot *IndexDB) List(ringPart int) ([]*IndexDBItem, error)
- func (ot *IndexDB) Lookup(hsh string, shard int) (timestamp int64, deletion bool, metahash string, metadata []byte, pth string, ...)
- func (ot *IndexDB) TempFile(hsh string, shard int, timestamp int64, sizeHint int64) (fs.AtomicFileWriter, error)
- type IndexDBItem
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func MetadataHash ¶
MetadataHash returns a hash of the contents of the metadata.
Types ¶
type IndexDB ¶
type IndexDB struct {
RingPartPower uint // GLH: Temp exported for fakelist
// contains filtered or unexported fields
}
IndexDB will track a set of objects.
This is the "index.db" per disk. Right now it just handles whole objects, but eventually we'd like to add either slab support or direct database embedding for small objects. But, those details should be transparent from users of a IndexDB.
This is different from the standard Swift full replica object tracking in that the directory structure is much shallower, there are a configurable number of databases per drive instead of a ton of hashes.pkl files, and the version tracking / consolidation is much simpler.
The IndexDB stores the newest object contents it knows about and discards any older ones, like the standard Swift's .data files. It does not have .meta files at all, and certainly not stacked to infinity .meta files. Instead the metadata is stored in the database as JSON.
A given IndexDB may not even store any metadata, such as in an EC system, with just "key" IndexDBs storing the metadata.
func NewIndexDB ¶
func NewIndexDB(dbpath, filepath, temppath string, ringPartPower, dbPartPower, subdirs int, logger *zap.Logger) (*IndexDB, error)
NewIndexDB creates a IndexDB to manage a set of objects.
The ringPartPower is defined by the ring in use, but should be greater than the dbPartPower. The dbPartPower will define how many databases are created (e.g. dbPartPower = 6 gives 64 databases). The subdirs value will define how many subdirectories are created where object content files are placed.
func (*IndexDB) Close ¶
func (ot *IndexDB) Close()
Close closes all the underlying databases for the IndexDB; you should discard the IndexDB instance after this call.
func (*IndexDB) Commit ¶
func (ot *IndexDB) Commit(f fs.AtomicFileWriter, hsh string, shard int, timestamp int64, deletion bool, metahash string, metadata []byte) error
Commit moves the temporary file (from TempFile) into place and records its information in the database. It may actually discard it completely if there is already a newer object information in place for the hash:shard.
Shard is mostly for EC type policies; just use 0 if you're using a full replica policy.
Timestamp is the timestamp for the object contents, not necessarily the metadata.
func (*IndexDB) List ¶
func (ot *IndexDB) List(ringPart int) ([]*IndexDBItem, error)
List returns the items for the ringPart given.
This is for replication, auditing, that sort of thing.
func (*IndexDB) Lookup ¶
func (ot *IndexDB) Lookup(hsh string, shard int) (timestamp int64, deletion bool, metahash string, metadata []byte, pth string, err error)
Lookup returns the stored information for the hsh and shard.
func (*IndexDB) TempFile ¶
func (ot *IndexDB) TempFile(hsh string, shard int, timestamp int64, sizeHint int64) (fs.AtomicFileWriter, error)
TempFile returns a temporary file to write to for eventually adding the hash:shard to the IndexDB with Commit; may return (nil, nil) if there is already a newer or equal timestamp in place for the hash:shard.
Source Files