Documentation
¶
Index ¶
- Constants
- func NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error)
- type ReplicaClient
- func (c *ReplicaClient) DeleteAll(ctx context.Context) error
- func (c *ReplicaClient) DeleteLTXFiles(ctx context.Context, a []*ltx.FileInfo) error
- func (c *ReplicaClient) Init(ctx context.Context) error
- func (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, _ bool) (_ ltx.FileIterator, err error)
- func (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (_ io.ReadCloser, err error)
- func (c *ReplicaClient) Type() string
- func (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, rd io.Reader) (info *ltx.FileInfo, err error)
Constants ¶
const (
DefaultTimeout = 30 * time.Second
)
const ReplicaClientType = "webdav"
Variables ¶
This section is empty.
Functions ¶
func NewReplicaClientFromURL ¶
func NewReplicaClientFromURL(scheme, host, urlPath string, query url.Values, userinfo *url.Userinfo) (litestream.ReplicaClient, error)
NewReplicaClientFromURL creates a new ReplicaClient from URL components. This is used by the replica client factory registration. URL format: webdav://[user[:password]@]host[:port]/path or webdavs://... (for HTTPS)
Types ¶
type ReplicaClient ¶
type ReplicaClient struct {
URL string
Username string
Password string
Path string
Timeout time.Duration
// contains filtered or unexported fields
}
func NewReplicaClient ¶
func NewReplicaClient() *ReplicaClient
func (*ReplicaClient) DeleteLTXFiles ¶
func (*ReplicaClient) LTXFiles ¶
func (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, _ bool) (_ ltx.FileIterator, err error)
func (*ReplicaClient) OpenLTXFile ¶
func (c *ReplicaClient) OpenLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, offset, size int64) (_ io.ReadCloser, err error)
func (*ReplicaClient) Type ¶
func (c *ReplicaClient) Type() string
func (*ReplicaClient) WriteLTXFile ¶
func (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, rd io.Reader) (info *ltx.FileInfo, err error)
WriteLTXFile writes an LTX file to the WebDAV server.
WebDAV Upload Strategy - Temp File Approach:
Unlike other replica backends (S3, SFTP, NATS, ABS) which stream directly using internal.NewReadCounter, WebDAV requires a different approach due to library and protocol constraints:
1. gowebdav Library Limitations:
WriteStream() buffers entire payload in memory for non-seekable readers
WriteStreamWithLength() requires both content-length AND seekable reader
No native support for HTTP chunked transfer encoding
2. Server Compatibility Issues: Research shows HTTP chunked transfer encoding with WebDAV is unreliable:
Nginx + FastCGI: Discards request body → 0-byte files (silent data loss)
Lighttpd: Returns HTTP 411 (Length Required), rejects chunked requests
Apache + FastCGI: Request body never arrives at application
Only Apache + mod_php handles chunked encoding reliably (~30-40% of deployments)
3. LTX Header Requirement:
- Must peek at LTX header to extract timestamp before upload
- Peeking consumes data, making the reader non-seekable
- Cannot calculate content-length without fully reading stream
Solution: Stage to temporary file
To ensure universal compatibility and prevent silent data loss:
- Extract timestamp from LTX header (required for file metadata)
- Stream full contents to temporary file on disk
- Seek back to start of temp file (now seekable + known size)
- Upload using WriteStreamWithLength() with Content-Length header
- Clean up temp file
Trade-offs:
- Universal compatibility with all WebDAV server configurations
- No risk of silent data loss or failed uploads
- Predictable, reliable behavior
- Additional disk I/O overhead
- Requires local disk space proportional to LTX file size
- Diverges from streaming pattern used by other backends
References:
- https://github.com/studio-b12/gowebdav/issues/35 (chunked encoding issues)
- https://github.com/nextcloud/server/issues/7995 (0-byte file bug)
- https://evertpot.com/260/ (WebDAV chunked encoding compatibility)
Source Files