README
¶
filerotate
Go package filerotate provides an io.WriteCloser implementation that handles time-based and size-based file rotation, optionally combined with a self-managing, high-performance write buffer.
Features
- Time-Based Rotation: Uses
strftimepatterns (e.g.,%Y-%m-%d) to automatically rotate files based on time. - Size-Based Rotation: Rotates files when they exceed a specified size limit, appending an index (e.g.,
.1,.2). - High-Performance Buffering: An optional, self-tuning write buffer that can automatically release memory when writes stop.
- Symbolic Link Tracking: Optionally maintains a symbolic link that always points to the currently active log file.
Basic Usage
The core functionality is accessed through the OpenFile function, which returns an io.WriteCloser that automatically manages the underlying log file.
package main
import (
"fmt"
"io"
"time"
"github.com/roy2220/filerotate"
)
func main() {
options := filerotate.Options{
// Mandatory: Use strftime format for date/time substitution.
// This pattern creates a new file every day, e.g., "logs/app/2023-10-02.log"
FilePathPattern: "logs/app/%Y-%m-%d.log",
// Optional: Maintain a symbolic link to the currently active log file.
SymbolicLinkPath: "logs/app.log",
// Optional: Rotate the file if it exceeds 100MB
FileSizeLimit: 100 * 1024 * 1024,
// Optional: Ensure every write ends with a newline.
EnsureNewline: true,
// Optional: Enable buffering (default 8MB for 0 value)
// Set to a negative value (e.g., -1) to disable buffering.
BufferSize: 8 * 1024 * 1024,
}
wc, err := filerotate.OpenFile(options)
if err != nil {
panic(err)
}
defer wc.Close()
// Use it like any standard io.Writer
for i := 0; i < 1000; i++ {
_, err := wc.Write([]byte(fmt.Sprintf("Log entry %d at %s", i, time.Now().Format(time.RFC3339))))
if err != nil {
fmt.Println("Write error:", err)
break
}
time.Sleep(10 * time.Millisecond)
}
}
Configuration (Options)
The filerotate.Options struct controls both file rotation and I/O buffering.
File Rotation Settings
| Option | Type | Description |
|---|---|---|
FilePathPattern |
string |
Mandatory. The file naming pattern using strftime format (e.g., "logs/app/%Y-%m-%d.log"). Time-based rotation occurs when the current time generates a different path. |
SymbolicLinkPath |
string |
Optional. The path to a symbolic link that will always point to the most recently active log file. Leave empty to disable. |
FileSizeLimit |
int64 |
The maximum size (in bytes) of a single file. Set to a non-positive value (e.g., 0) to disable size-based rotation. Rotated files are appended with an index, e.g., app-2023-10-02.log.1, app-2023-10-02.log.2, etc. |
EnsureNewline |
bool |
If true, a newline character (\n) is appended to every write operation, unless the written data already ends with one. This is generally most efficient when buffering is enabled (BufferSize >= 1). |
LogInternalError |
func(error) |
A callback function to handle errors that occur during background operations (e.g., auto-flushing failures, file close errors). If nil, a default logger that writes to standard output is used. |
Buffering Settings
| Option | Type | Default | Description |
|---|---|---|---|
BufferSize |
int |
8MB |
The internal buffer size in bytes. Set to a negative value (e.g., -1) to disable buffering. A zero value uses the default 8MB. |
LargeWriteThreshold |
float64 |
$\approx 0.618$ | A ratio (0.0 to 1.0) of BufferSize. If a single write is larger than the calculated threshold, it bypasses the buffer and writes directly to the file to prevent a single large operation from blocking the buffer. |
FlushInterval |
time.Duration |
1s |
How often the background routine automatically flushes the buffer to the file. |
MaxIdleBufferAge |
int |
3 |
The maximum number of consecutive FlushIntervals the buffer can be idle (no new writes) before the auto-flusher stops and releases the buffer's memory to conserve resources. It restarts automatically on the next write. |
Testing Hooks
The package provides hooks for dependency injection, useful for deterministic testing:
| Option | Type | Description |
|---|---|---|
Clock |
clock.Clock |
An interface for time operations. Use to inject a mock clock. |
Fs |
afero.Fs |
An interface for filesystem operations. Use to inject a mock filesystem (e.g., an in-memory FS). |
Go |
func(func()) |
A function to start background goroutines. Use to capture goroutines for deterministic testing. |
Registry |
*sync.Map |
An internal registry for file managers. Used to observe/control the automatic file closing logic in tests. |
License
MIT
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var ErrClosed = errors.New("filerotate: closed")
ErrClosed is returned when an operation is attempted on a WriteCloser that has already been closed.
Functions ¶
Types ¶
type Options ¶
type Options struct {
// FilePathPattern specifies the pattern for file naming.
// It uses strftime format for date/time substitution.
// For example: "logs/app-%Y-%m-%d.log"
FilePathPattern string
// SymbolicLinkPath is the path for a symbolic link that points to the currently
// active log file. An empty string disables the symbolic link feature.
// This link is atomically updated after each file rotation.
SymbolicLinkPath string
// FileSizeLimit is the maximum size (in bytes) a single log file can reach
// before rotation occurs. A non-positive value disables size-based rotation.
FileSizeLimit int64
// EnsureNewline guarantees that every write operation to the file ends with a newline
// character ('\n'), unless the written data already does.
// This option is generally more efficient when used with a buffer (BufferSize > 0),
// as manually appending a newline to the data before calling Write() can lead to
// redundant memory allocation and copying.
EnsureNewline bool
// LogInternalError specifies a callback function for handling internal errors
// that occur during background operations (such as auto-flushing failures or
// file close errors). These errors cannot be returned to the caller directly
// since they happen asynchronously. If nil, a default logger is used that
// writes to the standard log.
LogInternalError func(error)
// BufferSize is the size (in bytes) of the internal buffer.
// A zero value uses a default size of 8MB.
// If a buffered writer is not desired, set it to a negative value (e.g., -1).
BufferSize int
// LargeWriteThreshold is a ratio (0.0 to 1.0) of BufferSize.
// If a single write operation is larger than this threshold, it will bypass the buffer
// and write directly to the underlying file.
// A non-positive value uses a default threshold based on the golden ratio (0.618).
LargeWriteThreshold float64
// FlushInterval specifies how often the buffer should be automatically flushed.
// A non-positive value uses a default interval of 1 second.
FlushInterval time.Duration
// MaxIdleBufferAge specifies the maximum number of consecutive flush intervals
// that the buffer can remain idle (without new writes) before the auto-flusher
// stops and releases the buffer memory.
// This mechanism prevents the buffer from consuming memory indefinitely during
// periods of inactivity. The auto-flusher will restart automatically on the next
// write operation.
// A non-positive value uses a default of 3.
MaxIdleBufferAge int
// Go specifies the function to use for starting background goroutines.
Go func(func())
// Clock specifies the clock interface to use for time-related operations.
// If nil, the real system clock is used.
Clock clock.Clock
// Fs specifies the filesystem interface to use.
// If nil, the local OS filesystem is used.
Fs afero.Fs
// Registry is a pointer to a sync.Map that keeps track of internal instances.
Registry *sync.Map
}
Options defines the configuration for file rotation and buffering.
Source Files
Click to show internal directories.
Click to hide internal directories.