README
¶
wal
A simple and fast write ahead log for Go.
Features
- High durability
- Fast operations
- Monotonic indexes
- Batch writes
- Log truncation from front or back.
Getting Started
Installing
To start using wal, install Go and run go get:
$ go get -u github.com/tidwall/wal
This will retrieve the library.
Example
// open a new log file
log, err := wal.Open("mylog", nil)
// write some entries
err = log.Write(1, []byte("first entry"))
err = log.Write(2, []byte("second entry"))
err = log.Write(3, []byte("third entry"))
// read an entry
data, err := log.Read(1)
println(string(data)) // output: first entry
// close the log
err = log.Close()
Batch writes:
// write three entries as a batch
batch := new(wal.Batch)
batch.Write(1, []byte("first entry"))
batch.Write(2, []byte("second entry"))
batch.Write(3, []byte("third entry"))
err = log.WriteBatch(batch)
Truncating:
// write some entries
err = log.Write(1, []byte("first entry"))
...
err = log.Write(1000, []byte("thousandth entry"))
// truncate the log from index starting 350 and ending with 950.
err = log.TruncateFront(350)
err = log.TruncateBack(950)
Contact
Josh Baker @tidwall
License
wal source code is available under the MIT License.
Documentation
¶
Index ¶
- Variables
- type Batch
- type Log
- func (l *Log) ClearCache() error
- func (l *Log) Close() error
- func (l *Log) FirstIndex() (index uint64, err error)
- func (l *Log) IsEmpty() (bool, error)
- func (l *Log) LastIndex() (index uint64, err error)
- func (l *Log) Read(index uint64) (data []byte, err error)
- func (l *Log) Sync() error
- func (l *Log) TruncateBack(index uint64) error
- func (l *Log) TruncateFront(index uint64) error
- func (l *Log) Write(index uint64, data []byte) error
- func (l *Log) WriteBatch(b *Batch) error
- type LogFormat
- type Options
Constants ¶
This section is empty.
Variables ¶
var ( // ErrCorrupt is returns when the log is corrupt. ErrCorrupt = errors.New("log corrupt") // ErrClosed is returned when an operation cannot be completed because // the log is closed. ErrClosed = errors.New("log closed") // ErrNotFound is returned when an entry is not found. ErrNotFound = errors.New("not found") // ErrOutOfOrder is returned from Write() when the index is not equal to // LastIndex()+1. It's required that log monotonically grows by one and has // no gaps. Thus, the series 10,11,12,13,14 is valid, but 10,11,13,14 is // not because there's a gap between 11 and 13. Also, 10,12,11,13 is not // valid because 12 and 11 are out of order. ErrOutOfOrder = errors.New("out of order") // ErrOutOfRange is returned from TruncateFront() and TruncateBack() when // the index not in the range of the log's first and last index. Or, this // may be returned when the caller is attempting to remove *all* entries; // The log requires that at least one entry exists following a truncate. ErrOutOfRange = errors.New("out of range") // ErrEmptyLog is returned by Open() when the `AllowEmpty` option was not // provided and log has been emptied due to the use of TruncateFront() or // TruncateBack(). ErrEmptyLog = errors.New("empty log") )
var DefaultOptions = &Options{ NoSync: false, SegmentSize: 20971520, LogFormat: Binary, SegmentCacheSize: 2, NoCopy: false, AllowEmpty: false, DirPerms: 0750, FilePerms: 0640, }
DefaultOptions for Open().
Functions ¶
This section is empty.
Types ¶
type Batch ¶
type Batch struct {
// contains filtered or unexported fields
}
Batch of entries. Used to write multiple entries at once using WriteBatch().
type Log ¶
type Log struct {
// contains filtered or unexported fields
}
Log represents a write ahead log
func (*Log) ClearCache ¶ added in v0.1.0
ClearCache clears the segment cache. This only frees internal buffers and the LRU cache and does not modify the contents of the log.
func (*Log) FirstIndex ¶
FirstIndex returns the index of the first entry in the log. Returns zero when log has no entries. When using the `AllowEmpty` option and when the log is empty, this will return LastIndex+1, which is the next future index.
func (*Log) LastIndex ¶
LastIndex returns the index of the last entry in the log. Returns zero when log has no entries. When using the `AllowEmpty` option and when the log is empty, this will return FirstIndex()-1, which is the last known deleted index.
func (*Log) Sync ¶
Sync performs an fsync on the log. This is not necessary when the NoSync option is set to false.
func (*Log) TruncateBack ¶
TruncateBack truncates the back of the log by removing all entries that are after the provided `index`. In other words the entry at `index` becomes the last entry in the log.
The `AllowEmpty` option may be used to allow for removing all entries in the log by providing `FirstIndex()-1` as the index. Otherwise without `AllowEmpty`, at least one entry must always remain following a truncate.
func (*Log) TruncateFront ¶
TruncateFront truncates the front of the log by removing all entries that are before the provided `index`. In other words the entry at `index` becomes the first entry in the log.
The `AllowEmpty` option may be used to allow for removing all entries in the log by providing `LastIndex+1` as the index. Otherwise without `AllowEmpty`, at least one entry must always remain following a truncate.
func (*Log) WriteBatch ¶
WriteBatch writes the entries in the batch to the log in the order that they were added to the batch. The batch is cleared upon a successful return.
type Options ¶
type Options struct {
// NoSync disables fsync after writes. This is less durable and puts the
// log at risk of data loss when there's a server crash.
NoSync bool
// SegmentSize of each segment. This is just a target value, actual size
// may differ. Default 20 MB
SegmentSize int
// LogFormat is the format of the log files. Default Binary
LogFormat LogFormat
// SegmentCacheSize is the maximum number of segments that will be held in
// memory for caching. Increasing this value may enhance performance for
// concurrent read operations. Default 1
SegmentCacheSize int
// NoCopy allows for the Read() operation to return the raw underlying data
// slice. This is an optimization to help minimize allocations. When this
// option is set, do not modify the returned data because it may affect
// other Read calls. Default false
NoCopy bool
// AllowEmpty allows for a log to have all entries removed through the use
// of TruncateFront() or TruncateBack(). Otherwise without this option,
// at least one entry must always remain following a truncate operation.
// Default false
//
// Warning: using this option changes the behavior of the log in the
// following ways:
// - An empty log will always have the FirstIndex() be equal to
// LastIndex()+1.
// - For a newly created log that has no entries, FirstIndex() and
// LastIndex() return 1 and 0, respectively.
// Without AllowEmpty, both return 0.
AllowEmpty bool
// Perms represents the datafiles modes and permission bits
DirPerms os.FileMode
FilePerms os.FileMode
}
Options for Log
Source Files