Documentation
¶
Overview ¶
Package trash implements retention-based purging of soft-deleted nodes.
Nodes carry a `deleted_at` timestamp; a daily goroutine in server.Start calls PurgeExpired to hard-delete + remove the underlying storage object for nodes whose deleted_at is older than the configured retention window (settings key "trash.retention_days", default 30).
Index ¶
- Constants
- Variables
- func EmptyCutoff(at time.Time, olderThanDays int) time.Time
- func IsTrashPath(rel string) bool
- func NewKey(base string) string
- func OriginalPath(n *model.Node) (orig string, known bool)
- func Reach(ctx context.Context) []int64
- func TakeBack(ctx context.Context, drv storage.Driver, trashKey, origPath string) error
- func Vanished(n *model.Node) bool
- type ConflictError
- type EmptyJob
- type OnPurge
- type Outcome
- type PurgeResult
- type Service
- func (s *Service) EmptyOlderThan(ctx context.Context, olderThanDays int, storageID int64) (PurgeResult, error)
- func (s *Service) List(ctx context.Context, storageID *int64, limit, offset int) ([]TrashEntry, int, error)
- func (s *Service) PurgeExpired(ctx context.Context) (PurgeResult, error)
- func (s *Service) PurgeOne(ctx context.Context, nodeID int64) error
- func (s *Service) Restore(ctx context.Context, nodeID int64) error
- func (s *Service) RestoreFence(ctx context.Context, storageID, nodeID int64) []string
- func (s *Service) RetentionDays(ctx context.Context) int
- func (s *Service) RunDailyLoop(ctx context.Context, interval time.Duration)
- func (s *Service) RunEmpty(ctx context.Context, job EmptyJob, started func() bool, ...) (PurgeResult, error)
- func (s *Service) Tally(ctx context.Context, job EmptyJob) (int, int64, error)
- type StorageResolver
- type TrashEntry
Constants ¶
const DefaultRetentionDays = 30
DefaultRetentionDays is used when the setting is missing or unparseable.
const Prefix = syspath.Trash
Prefix is the in-storage directory soft-deleted objects are renamed into. It is syspath.Trash — the one list of filex's own directories, through which every listing filters it out — and Restore renames back out of it.
const SettingKey = "trash.retention_days"
SettingKey is the settings table row that stores the retention days value.
Variables ¶
var ErrNotInTrash = errors.New("trash: the item is not in the trash")
ErrNotInTrash is PurgeOne's refusal of a row that is not in the trash (any more): nothing was deleted.
var ErrNotStarted = errors.New("trash: the empty was withdrawn before it started")
ErrNotStarted is RunEmpty's answer when `started` declined: the run was cancelled while it waited for its turn, and nothing was purged.
var ErrRowsStillUnder = errors.New("trash: live rows are still under the folder")
ErrRowsStillUnder: a folder was not purged because live rows still name it as their parent, and the cascade would take them with it (purgeOne). The purge counts it as failed and tries again on its next run.
var ErrUnsupported = errors.New("trash: driver supports neither Move nor Copy")
ErrUnsupported is returned when the driver can neither Move nor Copy, so there is no way to preserve the bytes. Callers that choose to hard-delete anyway MUST fire writehook.OnFileDeleted, not OnFileTrashed.
Functions ¶
func EmptyCutoff ¶
EmptyCutoff is the deleted_at bound an "empty the trash" asked for at `at` purges below: `at` itself for 0 days (everything that was in the trash when it was asked for — and nothing deleted after), `at` minus the days otherwise. UTC, for SQLite's TEXT comparison (see PurgeExpired).
func IsTrashPath ¶
IsTrashPath reports whether rel is the trash bucket or lives inside it (syspath.InTrash, which the store asks too).
func NewKey ¶
NewKey returns a fresh storage-relative trash key for base: `.filex-trash/<unix>-<rand>__<base>` — the exact shape vfDelete mints, so every surface (manager, AI, DAV) lands deletions in the same trash layout.
func OriginalPath ¶
OriginalPath resolves the path a trashed row is JUDGED on: where the file came from, which is the only path a grant, a confinement root or a restore is ever written about. `known` is false when the row does not record one.
`storage_key` holds it — SoftDeleteAndRetag writes it there while rewriting `path` to the trash key. When the column is EMPTY (legacy rows; migration 00033 deliberately left those alone) the row's own `path` is the original, because nothing renamed it: the sync tombstone pass's SoftDeleteNode leaves `path` exactly where the file lived.
⚠ The one shape with no answer is a resolved path that is ITSELF inside `.filex-trash/` — the walk used to mint rows for the trash's own bytes, and the tombstone pass soft-deleted them where they stood (see sync.reconcileTrash). A trash key records the basename and nothing else, so the folder the file came from is not recoverable from the row by anyone. Every caller that authorises on the result must DENY when known is false: the alternative asks about `.filex-trash/…`, a path no rule is written about, and hands the answer to whoever holds a grant on the bin. The path is still returned, for display.
func Reach ¶
Reach is what a request may empty: nil when the caller is unscoped or the supertenant (every storage), otherwise a copy of its tenant's storages — non-nil even when that is none, so it can never read as "no restriction".
func TakeBack ¶
TakeBack is the inverse of Put: it returns the bytes parked under a trash key to their original path. Restore uses it so the two directions stay symmetric — a driver that could only trash via Copy+Delete must be able to come back the same way, or trashing would be a one-way trip on that backend.
Like Put it handles both a single rename and the per-object walk an object store needs, and it never deletes anything it has not first copied.
It counts on the context's tally (storage.Tally) the way Put does: the single-call attempt counts as it works and takes back what it only found (countAttempt), and the per-object walk counts each object once itself.
func Vanished ¶ added in v0.48.0
Vanished reports whether a deleted row is one whose bytes were never put in the trash: it was soft-deleted WHERE IT STOOD, its path still the file's own rather than a key inside `.filex-trash/` (ownsBytesAt). The storage sync did that to every file it found gone from the storage up to 0.47, and so did the branches that delete bytes outright.
⚠⚠ Such a row is NOT a trash entry (issue #74). The Trash is where deleted bytes are kept and can be brought back; a vanished row has none, so listing it offered a Restore that could only make a live row for a file that is not there. The trash listing leaves it out, Restore refuses it (ErrNotInTrash), and the storage sync drops it from the catalogue for good (the retention purge would too: purgeOne never touches the backend for it).
A live row is never vanished.
Types ¶
type ConflictError ¶
type ConflictError struct {
// Path is the occupied storage-relative path.
Path string
}
ConflictError reports that a restore's original path is occupied. Nothing was moved and nothing was un-trashed. It unwraps to os.ErrExist, the same error a rename collision maps to 409 with.
func (*ConflictError) Error ¶
func (e *ConflictError) Error() string
func (*ConflictError) Unwrap ¶
func (e *ConflictError) Unwrap() error
type EmptyJob ¶
type EmptyJob struct {
// Before: rows deleted before this instant are purged.
Before time.Time
// StorageID narrows the run to one storage; 0 is every storage in Reach.
StorageID int64
// Reach is the storages the run may touch: nil is every storage (a
// single-tenant install, the supertenant, the platform), otherwise the
// caller's tenant's storages — and an empty, non-nil slice is none.
Reach []int64
}
EmptyJob is one "empty the trash now": what it purges, decided when it was asked for and carried unchanged to the end — through the ops queue, and across a restart that interrupts it.
⚠⚠ Before is fixed at the moment of asking, never "now" at each step. The purge runs in the background for as long as a large trash takes (tens of minutes on S3), and a moving cutoff would purge, for good, a file somebody deleted by mistake WHILE it ran — a file the confirmation never counted and the person believes is still in the trash, waiting to be restored.
type OnPurge ¶
OnPurge is called once per node the instant before its row is destroyed for good, so the caches keyed on that node id can release their bytes.
It exists because a purge is the ONE moment at which "this node will never come back" is true — a soft delete is reversible, and a cache dropped there would be a cache dropped for a file the user can restore. Wiring it as a callback keeps this package free of the thumbnail pipeline and the upload staging area, which both live above it.
Best-effort: an implementation must not fail the purge.
type Outcome ¶
type Outcome struct {
// Key is the storage-relative `.filex-trash/...` location the bytes now
// live under. Empty unless Trashed is true.
Key string
// Trashed is true when the bytes are parked in the trash and restorable.
// It is the caller's signal to fire OnFileTrashed (and only then).
Trashed bool
// Missing is true when there was nothing at `rel` to trash — a stale
// index row or an out-of-band delete. No bytes were touched and no
// content was lost, so this is not an error.
Missing bool
// Files is the number of objects moved. 1 for a plain file; for a folder
// on an object store it is the number of contained objects.
Files int
}
Outcome describes what Put did to the bytes.
func Put ¶
Put moves whatever lives at `rel` into a fresh trash key on drv.
It handles the three shapes a deletion takes in practice:
- a plain file, or a folder the driver can rename in one call (local FS, SFTP, FTP, WebDAV): a single Move;
- a folder on an object store, where no object exists at the prefix and a Move of it 404s: every contained object is moved individually, keeping the sub-structure under the trash key so Restore rebuilds the tree;
- a driver with no Mover at all: the same two shapes, done as Copy then Delete, so the bytes still survive.
Folder-marker objects (`<prefix>` / `<prefix>/`, which filex's Mkdir writes on S3) are cleaned up best-effort once the contents are safe.
type PurgeResult ¶
type PurgeResult struct {
Scanned int `json:"scanned"`
Deleted int `json:"deleted"`
Failed int `json:"failed"`
Bytes int64 `json:"bytes"`
}
PurgeResult is returned by PurgeExpired / EmptyOlderThan to summarise a run.
type Service ¶
type Service struct {
Store db.Store
Resolver StorageResolver
Quota *quota.Service
// Reclaim, when wired, releases the per-node caches (the cached thumbnail
// JPEG, any staging directory still holding this node's bytes). See
// OnPurge. Nil is a no-op, which is what the unit tests use.
Reclaim OnPurge
// contains filtered or unexported fields
}
Service is the retention engine entry point.
func (*Service) EmptyOlderThan ¶
func (s *Service) EmptyOlderThan(ctx context.Context, olderThanDays int, storageID int64) (PurgeResult, error)
EmptyOlderThan ignores the configured retention and purges anything older than the supplied days value (admin "empty trash now" operation). Pass 0 for olderThanDays to wipe every soft-deleted node regardless of age.
storageID limits the purge to one storage; 0 means every storage the caller can reach.
⚠⚠ The parameter exists because it was MISSING while the UI claimed it was there. `POST /api/admin/trash/empty` decoded `storage_id` into a struct field nothing read, and the confirmation dialog said "If you picked a storage, only that one is affected." An admin who narrowed the operation to one storage and confirmed it permanently destroyed the trash of EVERY storage — irreversibly, with the dialog telling them the opposite.
It is RunEmpty in the caller's goroutine, scoped by the caller's context, and it ends with that context. The admin endpoint does not use it for that reason: a large trash outlives any request, so the endpoint queues the same job on the ops queue (ops.SubmitTrashEmpty) and the run belongs to the server.
func (*Service) List ¶
func (s *Service) List(ctx context.Context, storageID *int64, limit, offset int) ([]TrashEntry, int, error)
List returns soft-deleted entries (the trash listing for the admin UI).
Each entry's `Path` is the ORIGINAL path (`storage_key`) so the user sees where the item lived, not the internal `.filex-trash/...` key. `TTLDays` is the days remaining before automatic purge.
func (*Service) PurgeExpired ¶
func (s *Service) PurgeExpired(ctx context.Context) (PurgeResult, error)
PurgeExpired hard-deletes nodes whose deleted_at is older than the configured retention window.
It waits its turn behind a running "empty the trash" rather than sweeping the same rows beside it (see sweepLock).
func (*Service) PurgeOne ¶
PurgeOne immediately hard-deletes a single trashed node (admin / owner).
⚠⚠ Only while it IS trashed — read here, at the moment of the purge, not when the purge was asked for. A purge queued from the Trash page (ops.OpPurge) can wait behind other jobs, and an entry restored in the meantime is a live folder again: purging it hard-deleted its rows and everything that cascades from them (shares, tags, comments, versions), dropped its thumbnails and released its quota — the live bytes stayed, invisible to every listing. A row that is live answers ErrNotInTrash and nothing happens to it.
func (*Service) Restore ¶
Restore lifts the deleted_at flag on a node AND moves the underlying file back from the `.filex-trash/` location to its original path (saved in `storage_key` at delete time).
`path` and `path_hash` flip back to the original; `parent_id` is re-resolved from the original path's parent dir so the listing re-attaches the row in the right tree.
func (*Service) RestoreFence ¶ added in v0.55.0
RestoreFence answers the prefixes a restore of nodeID on storageID fences instead of holding the storage's row gate (rowgate.HoldCtx): the entry's trash key and its original path, when the entry is a folder on a storage that brings a folder back one object at a time (TakeBack's per-object walk on an object store, storage.FenceFor) - minutes or hours for a large one, during which the storage's whole scan would otherwise wait. nil for a file, a storage with real folders, a legacy or vanished entry, or anything it cannot read: the restore then holds the gate as before. It reads one row and asks no storage (the row says whether the entry is a folder).
func (*Service) RetentionDays ¶
RetentionDays reads the configured retention window in days.
func (*Service) RunDailyLoop ¶
RunDailyLoop ticks PurgeExpired every interval until ctx is cancelled. First tick happens after `interval`, not immediately, so a flapping server doesn't hammer the backend on restart.
func (*Service) RunEmpty ¶
func (s *Service) RunEmpty(ctx context.Context, job EmptyJob, started func() bool, progress func(PurgeResult)) (PurgeResult, error)
RunEmpty purges what job names. It waits its turn behind any other sweep (the nightly retention run, another empty), then calls started — which may decline, when the run was withdrawn while it waited (ErrNotStarted) — and reports the running totals to progress after every row.
It ends with ctx: a cancelled run stops at the next row and returns the context's error with what it had done by then.
type StorageResolver ¶
StorageResolver maps storage_id → live driver. Same shape used elsewhere.
type TrashEntry ¶
type TrashEntry struct {
ID int64 `json:"id"`
StorageID int64 `json:"storage_id"`
StorageName string `json:"storage_name,omitempty"`
Path string `json:"path"`
Name string `json:"name"`
Size int64 `json:"size"`
Mime string `json:"mime,omitempty"`
DeletedAt time.Time `json:"deleted_at"`
TTLDays *int `json:"ttl_days,omitempty"`
// Who put it in the trash (migration 00061), in the shape the file
// listing gives an owner: the account id, its display name, and — set by
// the handler, which knows who is asking — whether that was the asker.
// All three are absent when nobody in filex is named on the row: the
// scanner found the object gone, the virus scan quarantined it, or it was
// trashed before filex kept this. The client decides what to call that.
DeletedByID *int64 `json:"deleted_by_id,omitempty"`
DeletedByName string `json:"deleted_by_name,omitempty"`
DeletedBySelf bool `json:"deleted_by_self,omitempty"`
// Draft is set by the handler on a discarded draft of the asker's own
// (issue #71): it came from their Drafts, not from a folder, and Path is
// then just its name — the drafts area is never named to a client.
Draft bool `json:"draft,omitempty"`
// E2eRoot is the end-to-end encrypted folder the item was deleted from
// (a wire path), set by the handler; absent otherwise.
E2eRoot string `json:"e2e_root,omitempty"`
// Vanished: the row was deleted where it stood and nothing of it is in the
// trash (see Vanished). Not a trash entry: the listing leaves it out and a
// restore refuses it. Never sent — a client is never shown one.
Vanished bool `json:"-"`
}
TrashEntry is the projection returned by List — flat shape the admin UI consumes directly.
Source Files