Documentation
¶
Index ¶
- Constants
- func NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error)
- type ReplicaClient
- func (c *ReplicaClient) DeleteAll(ctx context.Context) error
- func (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error
- func (c *ReplicaClient) Init(ctx context.Context) error
- func (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (_ ltx.FileIterator, err error)
- func (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (_ io.ReadCloser, err error)
- func (c *ReplicaClient) SetLogger(logger *slog.Logger)
- func (c *ReplicaClient) Type() string
- func (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, rd io.Reader) (info *ltx.FileInfo, err error)
Constants ¶
const (
DefaultTimeout = 30 * time.Second
)
const ReplicaClientType = "webdav"
Variables ¶
This section is empty.
Functions ¶
func NewReplicaClientFromURL ¶
func NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error)
NewReplicaClientFromURL creates a new ReplicaClient from URL components. This is used by the replica client factory registration. URL format: webdav://[user[:password]@]host[:port]/path or webdavs://... (for HTTPS)
Types ¶
type ReplicaClient ¶
type ReplicaClient struct {
URL string
Username string
Password string
Path string
Timeout time.Duration
// contains filtered or unexported fields
}
func NewReplicaClient ¶
func NewReplicaClient() *ReplicaClient
func (*ReplicaClient) DeleteLTXFiles ¶
func (*ReplicaClient) LTXFiles ¶
func (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (_ ltx.FileIterator, err error)
LTXFiles returns an iterator over all LTX files on the replica for the given level. When useMetadata is true, CreatedAt is read from the LTX header (one ranged GET per file).
func (*ReplicaClient) OpenLTXFile ¶
func (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (_ io.ReadCloser, err error)
func (*ReplicaClient) SetLogger ¶ added in v0.5.10
func (c *ReplicaClient) SetLogger(logger *slog.Logger)
func (*ReplicaClient) Type ¶
func (c *ReplicaClient) Type() string
func (*ReplicaClient) WriteLTXFile ¶
func (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, rd io.Reader) (info *ltx.FileInfo, err error)
WriteLTXFile writes an LTX file to the WebDAV server.
WebDAV Upload Strategy - Temp File Approach:
Unlike other replica backends (S3, SFTP, NATS, ABS) which stream directly using internal.NewReadCounter, WebDAV requires a different approach due to library and protocol constraints:
1. gowebdav Library Limitations:
WriteStream() buffers entire payload in memory for non-seekable readers
WriteStreamWithLength() requires both content-length AND seekable reader
No native support for HTTP chunked transfer encoding
2. Server Compatibility Issues: Research shows HTTP chunked transfer encoding with WebDAV is unreliable:
Nginx + FastCGI: Discards request body → 0-byte files (silent data loss)
Lighttpd: Returns HTTP 411 (Length Required), rejects chunked requests
Apache + FastCGI: Request body never arrives at application
Only Apache + mod_php handles chunked encoding reliably (~30-40% of deployments)
3. LTX Header Requirement:
- Must peek at LTX header to extract timestamp before upload
- Peeking consumes data, making the reader non-seekable
- Cannot calculate content-length without fully reading stream
Solution: Stage to temporary file
To ensure universal compatibility and prevent silent data loss:
- Extract timestamp from LTX header (required for file metadata)
- Stream full contents to temporary file on disk
- Seek back to start of temp file (now seekable + known size)
- Upload using WriteStreamWithLength() with Content-Length header
- Clean up temp file
Trade-offs:
- Universal compatibility with all WebDAV server configurations
- No risk of silent data loss or failed uploads
- Predictable, reliable behavior
- Additional disk I/O overhead
- Requires local disk space proportional to LTX file size
- Diverges from streaming pattern used by other backends
References:
- https://github.com/studio-b12/gowebdav/issues/35 (chunked encoding issues)
- https://github.com/nextcloud/server/issues/7995 (0-byte file bug)
- https://evertpot.com/260/ (WebDAV chunked encoding compatibility)
Source Files