rowgate

Overview ¶

Package rowgate keeps a storage's catalogue judgements apart from the changes filex makes to that storage in two steps.

The two steps ¶

A rename, a move, a delete into the trash and a restore from it change a storage first and the catalogue after: the bytes move (Mover.Move, trash.Put), then the rows follow (protocolsync.MoveRows, SoftDeleteAndRetag, RestoreNodeAt). Between the two, the catalogue says one thing and the storage another, and nothing on either side can tell that apart from a change made outside filex.

The judgements ¶

The storage sync reads exactly that difference. A walk lists a directory and turns what it sees into rows; the tombstone pass takes a row the walk did not see, asks the storage for its object, and drops the row when the answer is "not found". Run inside the gap, both are wrong about a row that is fine:

• a folder listed in its parent just before the rename, then listed itself just after it, looked empty, so everything below it looked gone (issue #192: a queued folder rename opened empty in the explorer, e2e 159 and 172 under load);
• a row whose object had already moved, but which still named the old path, was confirmed gone by a Stat of that path and dropped - with its shares, versions and comments - a moment before the rename re-homed it;
• a listing taken after the bytes moved but before the rows did would find an object with no row and catalogue it as a NEW file, which would then hold the name the rename's own row is moving to.

The gate ¶

So each storage has one gate. A two-step change holds it SHARED (Move) from the first byte it moves to the last row it writes; any number of them run at once. A judgement holds it ALONE (Judge): the walk while it lists one directory and applies that listing, the tombstone pass while it confirms and drops, the lazy catalogue while it reconciles one folder. A judgement therefore sees a storage either before a change or after it, never half way.

⚠ Never hold Move and ask for Judge on the same goroutine (or the other way round), and never take Move twice on one goroutine: a judgement waiting for the gate keeps new holders out, so the second Move waits for a Judge that waits for the first. Take the gate at the outermost step of the change (the queue's job, the request handler), not inside the shared helpers it calls. Take it before opening a database transaction, never inside one: on SQLite a transaction holds the only connection, and the other side may be waiting for that connection while it holds the gate.

The gate is per process. Two filex processes over one database (a replica beside the main instance) do not see each other's gate; what narrows the gap there is that the tombstone pass reads a row again before it judges it and again before it drops it (internal/sync stillAsListed).