Help us shape the future of Go! Take the 2026 Go Developer Survey

coedit

package
v0.56.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 11, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package coedit is filex's relay for editing a file together (task #189): the server half of the app SDK's `coedit.*` bridge methods, for an app granted `files:co-edit` (docs/APP-PLUGINS-API.md → Editing together).

The model

Every member's editor applies the same log entries in the same order. The filex page (AppFrame, packages/core lib/coeditSession) seals each entry an app appends with the session's key (AES-256-GCM, lib/e2eoffice) and hands the sealed bytes here. The relay:

  • gives every entry its place: an entry is written against the head the writer has seen (AppendRequest.Base) and lands right after it or not at all (ConflictError). The writer binds the place into the entry's authenticated data, so the order the relay hands out is the order the writers sealed, and a reader notices a gap, a swap or a replay;
  • keeps the "save lock" promise a Document Server makes with a lease: changes are appended only by the one member holding it, and the lease goes only to a member that has seen every change already in the log;
  • writes the entries everybody has to agree on itself: who joined (with the editor's per-session user index, never given twice), who left (and when a member's page went away: dropped after MemberTTL), and which part of the log a save holds;
  • keeps the session's sealed blobs (the base document a member who joins late opens, the images added while editing) as files under its directory.

The relay never opens an entry, a cursor or a blob. Who a member is and whether it may write are the server's answer - the account signed in on the request and the file's rules, asked by the HTTP layer - never something the joining connection says about itself.

Keys

A session's key is made once, when it starts, and how it is kept depends on the folder (OpenRequest.KeyMode):

  • KeyServer, an ordinary folder: the server makes the key and keeps it sealed with FILEX_SECRET_KEY (Options.Keys); each member's filex page is handed the key when it joins. The server holds the file in the clear anyway, so this is not end-to-end encryption: it is the SAME path as an encrypted folder's, and it keeps the unsaved work in the database and the data directory sealed at rest. Without a secret key an ordinary folder is not edited together (ErrNoServerKey).
  • KeyFolder, an encrypted folder: the members' browsers seal the key with the folder key (SetKey), and the server cannot open it.

State

Everything is in the database (migration 00107): sessions move by compare-and-set on their `rev`, so two filex processes on one database can serve one session. Live delivery is per process: a member's socket waits on this process's notice of a new entry (Watcher.Notify) and reads the entries from the database, so a member connected to another process still gets every entry - on that process's poll (Options.PollEvery). Cursor frames are not kept: they reach the members connected to the same process.

Index

Constants

View Source
const (
	// KeyServer: the server makes the session key and keeps it sealed with
	// its secret key - an ordinary folder.
	KeyServer = "server"
	// KeyFolder: the members' browsers seal the key with the folder key - an
	// encrypted folder. The server cannot open it.
	KeyFolder = "folder"
)

Key modes (OpenRequest.KeyMode).

View Source
const (
	DefaultLeaseTTL         = 60 * time.Second
	DefaultMemberTTL        = 45 * time.Second
	DefaultMaxEntryBytes    = 2 << 20
	DefaultMaxLogBytes      = 64 << 20
	DefaultMaxBlobBytes     = 256 << 20
	DefaultMaxMembers       = 20
	DefaultSavedIdle        = time.Hour
	DefaultUnsavedRetention = 30 * 24 * time.Hour
	DefaultNotReadyTTL      = 10 * time.Minute
	DefaultPollEvery        = 2 * time.Second
	// MaxEphBytes caps one sealed cursor frame.
	MaxEphBytes = 64 << 10
	// MaxSealedKeyBytes caps a sealed key a member hands over (KeyFolder).
	MaxSealedKeyBytes = 4096
)

Defaults.

View Source
const BaseBlob = "base"

BaseBlob is the name of the sealed base document every member opens.

View Source
const CursorEvery = 500 * time.Millisecond

CursorEvery is the least time between two writes of one member's cursor to the database.

View Source
const KeyBytes = 32

KeyBytes is the size of a session key.

View Source
const SweepEvery = 15 * time.Second

SweepEvery is how often Run sweeps.

Variables

View Source
var (
	ErrNoSession   = errors.New("coedit: no such session")
	ErrNotMember   = errors.New("coedit: not a member of this session")
	ErrReadOnly    = errors.New("coedit: this member may not write")
	ErrKind        = errors.New("coedit: a member may not append this kind")
	ErrNoLease     = errors.New("coedit: changes need the lease")
	ErrEmpty       = errors.New("coedit: an entry needs a sealed payload")
	ErrTooLarge    = errors.New("coedit: too large")
	ErrLogFull     = errors.New("coedit: the session log is full; save, and the next opening starts a new session")
	ErrCounter     = errors.New("coedit: the counter must grow")
	ErrBuild       = errors.New("coedit: the session runs another build of the app")
	ErrFileChanged = errors.New("coedit: the file changed outside the session")
	ErrKeySet      = errors.New("coedit: the session key is already set")
	ErrBlobSet     = errors.New("coedit: that blob is already stored")
	ErrBlobName    = errors.New("coedit: bad blob name")
	ErrNoBlob      = errors.New("coedit: no such blob")
	ErrNotReady    = errors.New("coedit: the session has no key or no base yet")
	ErrSaveBehind  = errors.New("coedit: a save of a later point is already recorded")
	ErrSaveAhead   = errors.New("coedit: a save cannot hold entries the log does not have")
	ErrBadRequest  = errors.New("coedit: bad request")
	ErrNoIdentity  = errors.New("coedit: who is joining is not known")
	ErrFull        = errors.New("coedit: the session has as many members as it may")
	ErrNoServerKey = errors.New("coedit: no secret key to keep a session key with (FILEX_SECRET_KEY)")
	ErrBusy        = errors.New("coedit: the session kept changing under this request")
	// ErrNothingToWatch: a person who may only read the file opened it while
	// no session is there to join. A watcher never starts one - its starter
	// stores the base document every member builds on, which a person who
	// may not change the file must not choose.
	ErrNothingToWatch = errors.New("coedit: nobody is editing this file; a watcher starts no session")
)

Errors. The HTTP layer turns each into a code and a sentence of its own.

Functions

func DocKey

func DocKey(storageID int64, path string) string

DocKey is the session key of a file: one session per storage and path.

func MemberKind

func MemberKind(k Kind) bool

MemberKind reports whether a member may append k.

func NewToken

func NewToken() (string, error)

NewToken makes a member's token (handed to its page once, at the join).

func TokenHash

func TokenHash(token string) string

TokenHash is what a member's token is kept as.

func ValidBlobName

func ValidBlobName(name string) bool

ValidBlobName reports whether name may name a blob.

func ValidSID

func ValidSID(s string) bool

ValidSID reports whether s is a session id (32 lower-case hex digits). Nothing that is not one ever reaches the filesystem.

Types

type AppendRequest

type AppendRequest struct {
	Kind Kind
	// Base is the head the writer has seen; the entry lands at Base+1 or not
	// at all.
	Base uint64
	Ctr  uint64
	CT   []byte
}

AppendRequest is a member's sealed entry.

type ConflictError

type ConflictError struct {
	Head uint64
}

ConflictError says the entry was written against another head than the log's. The writer catches up to Head, seals again and retries.

func (*ConflictError) Error

func (e *ConflictError) Error() string

type Departure

type Departure struct {
	SID       string
	StorageID int64
	Path      string
	App       string
	Member    model.CoEditMember
	// Reason: "left" (it said so), "timeout" (its page went away), "closed"
	// (the session ended).
	Reason string
}

Departure is a member that left, for the audit trail.

type Entry

type Entry struct {
	Seq    uint64 `json:"seq"`
	Kind   Kind   `json:"kind"`
	Client string `json:"client"`
	// At is the relay's clock when the entry landed, Unix milliseconds - the
	// same for every member, where a Document Server would use its own.
	At int64 `json:"at"`
	// Ctr is the writer's counter, bound into the sealed entry.
	Ctr uint64 `json:"ctr,omitempty"`
	// CT is a member's sealed payload (base64 on the wire).
	CT []byte `json:"ct,omitempty"`
	// Member is who joined (KindJoin).
	Member *Member `json:"member,omitempty"`
	// Through is the last entry a save holds (KindSaved).
	Through uint64 `json:"through,omitempty"`
}

Entry is one place in a session's log, as the members receive it.

type Eph

type Eph struct {
	Client string `json:"client"`
	Ctr    uint64 `json:"ctr"`
	CT     []byte `json:"ct"`
	// At is when the relay took it (Unix ms) - for the sockets of other
	// processes, which read the latest cursors from the database
	// (cursors.go).
	At int64 `json:"-"`
}

Eph is a frame that is not kept: a cursor, sealed, for the other members.

type Hello

type Hello struct {
	SID string
	// Created: this member started the session and must store the base
	// document (and, KeyFolder, the sealed key) before anybody else joins.
	Created      bool
	Me           Member
	Head         uint64
	ChangesHead  uint64
	SavedThrough uint64
	KeyMode      string
	// Key is the raw session key (KeyServer).
	Key []byte
	// SealedKey is the sealed session key (KeyFolder), "" until set.
	SealedKey string
}

Hello is what a new member needs to start.

type Identity

type Identity struct {
	User int64
	// Person is the person's presence identity (the account, and the end
	// user a shared proxy token names): two of them behind one proxy token
	// are two people. Empty = the account alone.
	Person  string
	Name    string
	CanEdit bool
	// TokenHash is hex SHA-256 of the token the member is handed: the
	// requests that name the member carry it (MemberToken checks it).
	TokenHash string
}

Identity is who a joining member is, as the SERVER knows it: the account signed in on the request and the file's rules - never anything the caller sent.

type KeyBox

type KeyBox interface {
	Enabled() bool
	SealFor(plaintext, aad string) (string, error)
	OpenFor(stored, aad string) (string, error)
}

KeyBox seals and opens the keys the server makes (internal/secretbox.Box).

type Kind

type Kind string

Kind names what an entry is. The first three are written by members and carry a sealed payload the relay cannot read; the last three are the relay's own and carry plain metadata.

const (
	// KindChanges is one chunk of an editor's changes.
	KindChanges Kind = "changes"
	// KindLock is an editor's request for locks.
	KindLock Kind = "lock"
	// KindRelease releases an editor's locks.
	KindRelease Kind = "release"
	// KindJoin records a member joining, with the indexUser it was given.
	KindJoin Kind = "join"
	// KindLeave records a member leaving (or dropped).
	KindLeave Kind = "leave"
	// KindSaved records a save: the file now holds the log up to Through.
	KindSaved Kind = "saved"
)

type LeaseOp

type LeaseOp string

LeaseOp is what a member does with the changes lease.

const (
	LeaseAcquire LeaseOp = "acquire"
	LeaseRelease LeaseOp = "release"
)

type Member

type Member struct {
	Client string `json:"client"`
	// User is an opaque id of the person in THIS session (the same for each
	// of their connections in it), never an account id.
	User string `json:"user"`
	Name string `json:"name"`
	// IndexUser is the editor's per-session user index: given once, never
	// twice in a session, not even after its member left.
	IndexUser int  `json:"index_user"`
	CanEdit   bool `json:"can_edit"`
}

Member is one connection in a session, as the members are told.

type OpenRequest

type OpenRequest struct {
	// Doc is the file's key (DocKey: a digest of the storage and the path).
	Doc       string
	StorageID int64
	Path      string
	// App and Build: the app and the build its members run. Members of one
	// session run one build: another build may write entries this one cannot
	// read.
	App   string
	Build string
	// FileSig is the file as the opener's server read it (size:mtime, etag).
	// A session goes on from the file it started from or from its last save;
	// a file that changed outside it is not the document its log applies to.
	FileSig string
	// KeyMode is KeyServer (an ordinary folder) or KeyFolder (an encrypted
	// one).
	KeyMode string
}

OpenRequest asks for the session of one file.

type Options

type Options struct {
	Store Store
	// BlobDir is where the sealed blobs are kept, one folder per session
	// (<data_dir>/coedit). Several processes on one database must share it,
	// as they share the upload staging directory.
	BlobDir string
	// Keys seals the keys of KeyServer sessions. nil or not enabled: an
	// ordinary folder is not edited together (ErrNoServerKey).
	Keys KeyBox
	// LeaseTTL is how long the changes lease lasts without a renewal (the
	// Document Server's save lock expires after 60 s).
	LeaseTTL time.Duration
	// MemberTTL is how long a member whose page gives no sign of life stays.
	MemberTTL time.Duration
	// MaxEntryBytes caps one sealed payload.
	MaxEntryBytes int
	// MaxLogBytes caps a session's log: past it a member has to save, and
	// the next opening starts a new session.
	MaxLogBytes int64
	// MaxBlobBytes caps a session's blobs together.
	MaxBlobBytes int64
	// MaxMembers caps the members of one session.
	MaxMembers int
	// SavedIdle is how long an empty session whose log is all saved is kept.
	SavedIdle time.Duration
	// UnsavedRetention is how long an empty session with unsaved changes is
	// kept, for the next person to open the file to see and save them.
	UnsavedRetention time.Duration
	// NotReadyTTL is how long a session may wait for its key and its base.
	NotReadyTTL time.Duration
	// PollEvery is how often a member's socket reads the log without a
	// notice from this process (another process wrote the entry).
	PollEvery time.Duration
	// OnDeparture is told about every member that left, after it is recorded.
	OnDeparture func(ctx context.Context, d Departure)
	// Now is the clock (tests).
	Now func() time.Time
}

Options tune a Relay. The zero value of each duration and size takes its default.

type Relay

type Relay struct {
	// contains filtered or unexported fields
}

Relay is the editing-together relay. One per process; safe for concurrent use, and for several processes on one database.

func New

func New(opts Options) *Relay

New builds a relay over opts.Store.

func (*Relay) Append

func (r *Relay) Append(ctx context.Context, sid, client string, req AppendRequest) (uint64, error)

Append puts a member's entry in the log and returns its seq.

func (*Relay) Authorise

func (r *Relay) Authorise(ctx context.Context, sid, client, token string, userID int64) (*model.CoEditMember, error)

Authorise returns the member a request names, when the request is that member's: the same account, and the member's token. Anything else is ErrNotMember - a request that names somebody else's connection learns nothing about it.

func (*Relay) Blob

func (r *Relay) Blob(ctx context.Context, sid, client, name string) (io.ReadCloser, int64, error)

Blob opens a stored sealed blob for a member, with its size.

func (*Relay) Cursors

func (r *Relay) Cursors(ctx context.Context, sid string, sinceMs int64) ([]Eph, error)

Cursors returns the latest cursors of a session's members written after sinceMs (Unix ms), oldest first: what a socket reads on its poll. The caller checked the reader is a member, and skips its own.

func (*Relay) Entries

func (r *Relay) Entries(ctx context.Context, sid string, after uint64, limit int) ([]Entry, error)

Entries returns up to limit entries after `after`, in order. The caller checked the reader is a member.

func (*Relay) Lease

func (r *Relay) Lease(ctx context.Context, sid, client string, op LeaseOp, changesSeen uint64) (bool, error)

Lease acquires (or renews) the changes lease, or releases it. It is given only to a writer that has seen every change in the log (changesSeen is the seq of the last changes entry it applied) while nobody else holds it: the Document Server's isSaveLock, which refuses an editor that is behind and one whose turn it is not.

func (*Relay) Leave

func (r *Relay) Leave(ctx context.Context, sid, client, reason string) error

Leave removes a member: its lease goes, its socket is told, and the log records it (every bridge then drops its locks). reason is "left" or "timeout".

func (*Relay) Member

func (r *Relay) Member(ctx context.Context, sid, client string) (*model.CoEditMember, error)

Member returns a member of a session (ErrNotMember when it is not one). The HTTP layer checks with it that a request naming a member comes from that member's account.

func (*Relay) MemberTTL

func (r *Relay) MemberTTL() time.Duration

MemberTTL is how long a member's page may be silent.

func (*Relay) Open

func (r *Relay) Open(ctx context.Context, req OpenRequest, who Identity) (Hello, error)

Open joins the caller to the file's session, starting it when there is none (or the one there makes way: nobody is in it and everything in it is saved). A session nobody may join yet - its creator is still storing the base document - answers ErrNotReady; ask again in a moment.

func (*Relay) Options

func (r *Relay) Options() Options

Options returns the relay's options with their defaults (the HTTP layer reads the limits).

func (*Relay) PollEvery

func (r *Relay) PollEvery() time.Duration

PollEvery is how often a socket reads the log without a notice.

func (*Relay) PutBlob

func (r *Relay) PutBlob(ctx context.Context, sid, client, name string, body io.Reader) (int64, error)

PutBlob stores a sealed blob of a session, once per name, and returns its size. The base document may be stored by any member while the session has none (its creator, in practice: nobody else joins before); any other blob only by a member that may write. The session's blobs together stay under Options.MaxBlobBytes.

func (*Relay) Run

func (r *Relay) Run(ctx context.Context)

Run sweeps every SweepEvery until ctx ends.

func (*Relay) Saved

func (r *Relay) Saved(ctx context.Context, sid, client string, through uint64, sig string) (uint64, error)

Saved records that a member saved the file with the log up to `through` in it, and the file's signature after the write - the server's own reading of the file, never the member's word. Saves only move forward.

func (*Relay) SendEph

func (r *Relay) SendEph(sid string, e Eph) error

SendEph passes a sealed cursor frame to the other members connected to this process. It is not kept. The caller checked the member may write.

func (*Relay) ServerKeys

func (r *Relay) ServerKeys() bool

ServerKeys reports whether an ordinary folder can be edited together here.

func (*Relay) Session

func (r *Relay) Session(ctx context.Context, sid string) (*model.CoEditSession, error)

Session returns a session's row (ErrNoSession when there is none).

func (*Relay) SetKey

func (r *Relay) SetKey(ctx context.Context, sid, client, sealed string) error

SetKey stores the sealed session key of a KeyFolder session, once - by a member, before anybody else may join.

func (*Relay) Sweep

func (r *Relay) Sweep(ctx context.Context) []string

Sweep drops the members whose page went away (unseen for MemberTTL: a leave entry, the lease goes, every bridge drops their locks) and forgets the sessions nobody is in any more: a saved one after SavedIdle, one with unsaved changes after UnsavedRetention (30 days - for the next person to open the file to see and save them), one that never got its key or base after NotReadyTTL. It returns the ids of the sessions it forgot.

func (*Relay) Touch

func (r *Relay) Touch(ctx context.Context, sid, client string) error

Touch records that a member's page is still there; ErrNotMember once it is no longer a member (it was dropped, or the session ended).

func (*Relay) Watch

func (r *Relay) Watch(sid, client string) *Watcher

Watch registers a member's socket in this process: it is told when an entry lands here (Notify), gets the cursors of the others (Eph) and is told when its membership ends (Ended). Close it when the socket goes.

type Store

type Store interface {
	WithTx(ctx context.Context, fn func(ctx context.Context) error) error
	db.CoEditStore
}

Store is what the relay keeps its state in: db.Store's editing-together half (db.CoEditStore) and its transactions.

type Watcher

type Watcher struct {
	SID    string
	Client string
	// Notify gets a signal (never more than one waiting) when an entry lands
	// in this process: read the log from where the socket is.
	Notify chan struct{}
	// Eph gets the other members' cursors. A full buffer drops them: a
	// cursor is worth nothing late.
	Eph chan Eph
	// Ended gets why the membership ended ("left", "timeout", "closed").
	Ended chan string
	// contains filtered or unexported fields
}

Watcher is one member's open socket in this process.

func (*Watcher) Close

func (w *Watcher) Close()

Close takes the watcher out of the hub. Safe to call twice.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL