Documentation
¶
Overview ¶
Package plugin lets a storage driver live OUTSIDE the filex binary.
Why this exists ¶
Every storage filex speaks — local disk, S3, SFTP, FTP, WebDAV, SMB — is a Go package compiled into the binary and registered from an init(). That is the right shape for the drivers filex ships, and the wrong shape for the driver somebody writes for their own system: they would have to fork filex, learn its build, and rebuild on every release. A plugin is that same driver as a separate program that filex starts (or connects to) and talks to over a small HTTP/JSON protocol. Any language that can serve HTTP can be a filex storage driver.
What a plugin is ¶
One of two things, and the protocol is the same for both:
- a BINARY filex launches itself: it lives under <data-dir>/plugins/<name>/, is started with FILEX_PLUGIN_TOKEN and FILEX_PLUGIN_SOCKET_DIR in its environment, listens on a unix socket in that directory (or on a loopback TCP port), and prints ONE handshake line to stdout — `FILEX-PLUGIN/1 unix:/path/to.sock` or `FILEX-PLUGIN/1 tcp:127.0.0.1:PORT`. filex supervises it: health-checks it, restarts it with backoff when it dies, kills it on shutdown.
- a REMOTE service the operator runs (a sidecar container, a daemon on another host): the admin registers its base URL and a bearer token and filex just connects. Nothing is spawned.
The protocol, in one screen ¶
Every request carries `Authorization: Bearer <token>`. Every error body is {"error": <code>, "message": <text>}. Codes filex maps back to storage errors: not_found, read_only, unsupported, no_instance; anything else is a plain failure with the message attached.
GET /v1/describe → DescribeResponse (name, version, fields, capabilities)
POST /v1/instances {config} → {"instance": id} one per storage row; Init
DELETE /v1/instances/{id} release it
GET /v1/instances/{id}/list?path= → {"objects": [Object]}
GET /v1/instances/{id}/stat?path= → Object
GET /v1/instances/{id}/read?path= → body stream (Range: bytes=a-b honoured when capabilities.range)
PUT /v1/instances/{id}/write?path= ← body stream (X-Filex-Size: n, or -1 when unknown)
POST /v1/instances/{id}/move {src,dst}
POST /v1/instances/{id}/copy {src,dst}
POST /v1/instances/{id}/delete {path} idempotent
POST /v1/instances/{id}/mkdir {path}
POST /v1/instances/{id}/set-mtime {path,mtime} RFC 3339
GET /v1/instances/{id}/watch → text/event-stream of Event, when capabilities.watch
POST /v1/instances/{id}/presign-upload {path,size} → PresignedUpload (capabilities.presign)
POST /v1/instances/{id}/presign-download {path,ttl_s} → {url,expires_at} (capabilities.presign)
POST /v1/instances/{id}/multipart/init {path,total_size,part_count} → {upload_id,part_urls}
PUT /v1/instances/{id}/multipart/part?upload_id=&part=N&path= ← body → {etag}
POST /v1/instances/{id}/multipart/complete {path,upload_id,parts[]}
POST /v1/instances/{id}/multipart/abort {path,upload_id}
POST /v1/selftest → {"instance": id} a THROWAWAY instance for conformance
Conformance ¶
A plugin that declares a capability it cannot actually perform is worse than one that declares nothing: the UI offers the operation, the operation fails, and the user reads that as filex being broken. So filex PROBES what a plugin claims — at install, and again whenever a storage on it is saved — and refuses a plugin whose own claims do not hold up (internal/plugin/ conformance.go). /v1/selftest is how a plugin offers a scratch area for that: it creates an instance backed by throwaway space and returns its id. A plugin without it can still be installed, but it is marked UNVERIFIED and the probes run against the first real storage instead.
The driver a plugin implements is registered in filex as `plugin:<name>`, so a plugin can never shadow a built-in driver, and its config form on the Connections page renders from the fields it describes — exactly as the built-in drivers' forms do (storage.Descriptor). Full specification with the reasoning behind each choice: docs/PLUGINS.md.
Index ¶
- Constants
- Variables
- func FeedURL(source string) (string, error)
- func NewDriver(h Handle, caps Capabilities) storage.Driver
- func ParsePublicKey(k string) (ed25519.PublicKey, error)
- func ParsePublicKeys(keys []string) ([]ed25519.PublicKey, error)
- func ValidName(s string) bool
- func ValidateDescribe(d *DescribeResponse) error
- func VerifyDetached(trusted []ed25519.PublicKey, sha, signature string) error
- type Address
- type Capabilities
- type Client
- func (c *Client) AbortMultipart(ctx context.Context, id, path, uploadID string) error
- func (c *Client) Close()
- func (c *Client) CompleteMultipart(ctx context.Context, id, path, uploadID string, parts []MultipartPart) error
- func (c *Client) Copy(ctx context.Context, id, src, dst string) error
- func (c *Client) CreateInstance(ctx context.Context, cfg map[string]any) (string, error)
- func (c *Client) Delete(ctx context.Context, id, path string) error
- func (c *Client) DeleteInstance(ctx context.Context, id string) error
- func (c *Client) Describe(ctx context.Context) (*DescribeResponse, error)
- func (c *Client) InitMultipart(ctx context.Context, id, path string, totalSize int64, partCount int) (MultipartInitResponse, error)
- func (c *Client) List(ctx context.Context, id, path string) ([]Object, error)
- func (c *Client) Mkdir(ctx context.Context, id, path string) error
- func (c *Client) Move(ctx context.Context, id, src, dst string) error
- func (c *Client) PresignDownload(ctx context.Context, id, path string, ttl time.Duration) (PresignResponse, error)
- func (c *Client) PresignUpload(ctx context.Context, id, path string, size int64) (PresignResponse, error)
- func (c *Client) Read(ctx context.Context, id, path string, off, length int64) (io.ReadCloser, error)
- func (c *Client) SelfTest(ctx context.Context) (string, error)
- func (c *Client) SetMtime(ctx context.Context, id, path string, t time.Time) error
- func (c *Client) Stat(ctx context.Context, id, path string) (Object, error)
- func (c *Client) UploadPart(ctx context.Context, id, path, uploadID string, part int, r io.Reader, ...) (string, error)
- func (c *Client) Watch(ctx context.Context, id string) (<-chan Event, error)
- func (c *Client) Write(ctx context.Context, id, path string, r io.Reader, size int64) error
- type DescribeResponse
- type ErrorResponse
- type Event
- type Feed
- type FeedBinary
- type Handle
- type InstanceRequest
- type InstanceResponse
- type ListResponse
- type Manager
- func (m *Manager) BackgroundUpdates() bool
- func (m *Manager) CapabilitiesFor(driver string) (Capabilities, bool)
- func (m *Manager) CheckUpdates(ctx context.Context) (*UpdateReport, error)
- func (m *Manager) ConformanceMode() string
- func (m *Manager) Dir() string
- func (m *Manager) Get(ctx context.Context, id int64) (*Status, error)
- func (m *Manager) InstallBinary(ctx context.Context, name, filename string, r io.Reader, signature string) (*Status, error)
- func (m *Manager) InstallFromSource(ctx context.Context, name, source string) (*Status, error)
- func (m *Manager) InstallFromURL(ctx context.Context, name, rawURL, sha, signature string) (*Status, error)
- func (m *Manager) InstallRemote(ctx context.Context, name, address, token string) (*Status, error)
- func (m *Manager) LastUpdateCheck(ctx context.Context) time.Time
- func (m *Manager) List(ctx context.Context) ([]*Status, error)
- func (m *Manager) Load(ctx context.Context) error
- func (m *Manager) Remove(ctx context.Context, id int64) error
- func (m *Manager) RequiresSignature() bool
- func (m *Manager) Restart(ctx context.Context, id int64) (*Status, error)
- func (m *Manager) SetEnabled(ctx context.Context, id int64, on bool) (*Status, error)
- func (m *Manager) SetSource(ctx context.Context, id int64, source string) (*Status, error)
- func (m *Manager) SetUpdateHooks(h UpdateHooks)
- func (m *Manager) Shutdown()
- func (m *Manager) StartUpdater(ctx context.Context, enabled bool)
- func (m *Manager) Upgrade(ctx context.Context, id int64, filename string, r io.Reader, signature string) (*Status, error)
- func (m *Manager) UpgradeFromSource(ctx context.Context, id int64) (*Status, error)
- func (m *Manager) WaitReady(d time.Duration)
- type MoveRequest
- type MtimeRequest
- type MultipartCompleteRequest
- type MultipartInitRequest
- type MultipartInitResponse
- type MultipartPart
- type MultipartPartResponse
- type Object
- type Options
- type PathRequest
- type PresignRequest
- type PresignResponse
- type ProbeResult
- type Process
- type RejectedError
- type Report
- type Stats
- type Status
- type UpdateHooks
- type UpdateInfo
- type UpdateReport
Constants ¶
const ( ProbePass = "pass" ProbeFail = "fail" ProbeSkip = "skip" )
Probe statuses.
const ( // DefaultMaxInFlight is how many operations may be inside one plugin at // once. Ten is comfortably above what a browsing user generates and low // enough that a stuck plugin cannot consume the server. DefaultMaxInFlight = 10 // DefaultOpTimeout bounds one operation. Streaming reads and writes are // exempt — a 20 GB upload is legitimately slow — so this applies to the // metadata calls, where slowness means trouble rather than size. DefaultOpTimeout = 60 * time.Second )
Defaults chosen to be generous for a healthy plugin and firm for a sick one. Both are per PLUGIN, not per storage: the process is the resource.
const ( StateDisabled = "disabled" // row says off; nothing runs StateStarting = "starting" // launched, no handshake/describe yet StateRunning = "running" // described, driver registered StateFailed = "failed" // last attempt failed; a binary is backing off, a remote is re-checked StateRefused = "refused" // describe or conformance rejected it; not retried until restart )
Runtime states a plugin can be in, as shown to the admin.
const ( // ConformanceEnforce is the default: a plugin that fails its own claims // is refused, so nobody can build a storage on half a driver. ConformanceEnforce = "enforce" // ConformanceWarn registers the plugin anyway and keeps the report. For // an operator debugging a plugin they are writing — never the default, // because the cost of a broken claim is paid by the user, who reads it as // filex being broken. ConformanceWarn = "warn" // ConformanceOff skips the probes entirely. ConformanceOff = "off" )
Conformance modes (FILEX_PLUGIN_CONFORMANCE).
const ( ErrCodeNotFound = "not_found" ErrCodeReadOnly = "read_only" ErrCodeUnsupported = "unsupported" // ErrCodeNoInstance says the instance id is unknown — the plugin was // restarted and lost its state. The host re-creates the instance from the // saved config and retries once, so a plugin restart is invisible. ErrCodeNoInstance = "no_instance" ErrCodeInvalid = "invalid" )
Error codes the host understands. Anything else is reported verbatim.
const ( UpdateCurrent = "current" UpdateAvailable = "available" UpdateIncompatible = "incompatible" UpdateCheckFailed = "check_failed" )
Update check outcomes (UpdateInfo.Status).
const ConformanceTimeout = 90 * time.Second
ConformanceTimeout bounds a whole run. A plugin that cannot answer a handful of small operations in this long is not one a storage should be built on.
const DriverPrefix = "plugin:"
DriverPrefix namespaces plugin drivers in the storage registry.
const (
// FeedFileName is what a GitHub release attaches.
FeedFileName = "filex-storage.json"
)
const HandshakePrefix = "FILEX-PLUGIN/1"
HandshakePrefix is the first token of the single line a launched plugin prints to stdout, followed by a space and the address it listens on.
const ProtocolVersion = 1
ProtocolVersion is the only version this host speaks. A plugin whose describe reports another one is refused with a clear message rather than half-working.
const SizeHeader = "X-Filex-Size"
SizeHeader carries the write size on PUT /write, because a streamed body may be chunked and Content-Length absent. -1 means unknown.
Variables ¶
var ( ErrHandshake = errors.New("plugin did not print its handshake line") ErrRefused = errors.New("plugin refused by host") ErrStopped = errors.New("plugin stopped") // ErrGaveUp is reported once the supervisor has seen maxConsecutiveFailures // starts in a row that never came up. It stops restarting: a binary that // fails ten times will fail an eleventh, and a loop that keeps trying // forever is a loop that keeps a broken plugin looking "starting". The // admin's Restart button begins a fresh count. ErrGaveUp = errors.New("plugin failed to start 10 times in a row; not restarting until Restart") )
Errors reported through OnDown / Err.
var ( ErrSignatureRequired = errors.New("signature required") ErrSignatureInvalid = errors.New("signature invalid") )
ErrSignatureRequired / ErrSignatureInvalid are what VerifyDetached returns (wrapped in RejectedError by the storage-plugin manager); callers switch on them to pick an error code rather than reading the message.
var ErrBadName = errors.New("plugin name must match [a-z0-9][a-z0-9_-]{0,31}")
var ErrBusy = errors.New("plugin is at its concurrency limit")
ErrBusy is returned when a plugin has no free slot. It is deliberately distinct from a timeout: this one says the plugin is saturated, which is an operator's problem to size, not a bug to chase.
Functions ¶
func FeedURL ¶ added in v0.48.0
FeedURL is where a source's feed is read: `owner/name` (or its GitHub address) → the latest release's filex-storage.json; an https address → itself. "" for an empty source.
func NewDriver ¶
func NewDriver(h Handle, caps Capabilities) storage.Driver
NewDriver builds a fresh storage.Driver for one storage row on plugin h. This is the storage.Factory the manager registers.
The concrete type is chosen by newShape (driver_shapes.go, generated): a plugin gets a value whose METHOD SET is exactly what it declared, because filex reads capabilities by type assertion at forty-odd call sites.
func ParsePublicKey ¶
ParsePublicKey accepts an ed25519 key as hex or standard base64.
func ParsePublicKeys ¶
ParsePublicKeys parses every key, refusing the whole list on the first bad one so a typo cannot silently shrink the trust set.
func ValidateDescribe ¶
func ValidateDescribe(d *DescribeResponse) error
ValidateDescribe applies the rules a plugin must meet before the host registers it. Every message names the rule, because the person reading it is the plugin's author.
func VerifyDetached ¶
VerifyDetached checks signature (hex or base64) over the lower-hex sha256 against the trusted keys. No trusted keys → nothing is required and nil is returned; trusted keys and no signature → ErrSignatureRequired; otherwise ErrSignatureInvalid unless one key verifies.
Types ¶
type Address ¶
type Address struct {
Network string // "unix" | "tcp"
Target string // socket path, or host:port, or full URL for remote
URL string // base URL to put in front of /v1
}
Address is a parsed plugin address: "unix:/path" or "tcp:host:port" from the handshake line, or a full http(s):// URL for a remote plugin.
func ParseAddress ¶
ParseAddress accepts the three spellings the host meets.
type Capabilities ¶
type Capabilities struct {
Range bool `json:"range"`
Write bool `json:"write"`
Delete bool `json:"delete"`
Move bool `json:"move"`
Copy bool `json:"copy"`
Mkdir bool `json:"mkdir"`
SetMtime bool `json:"set_mtime"`
Watch bool `json:"watch"`
Presign bool `json:"presign"`
Multipart bool `json:"multipart"`
}
Capabilities is what the plugin says it can do. It is deliberately NOT storage.Capabilities: presigned URLs and multipart are out of scope for the protocol's first version, and set_mtime is a capability here where the host expresses it as an optional interface (storage.Toucher).
The rules the host applies (documented, enforced by the SDK's own checks):
- read is implied — a driver that cannot list/stat/read is not a driver.
- range: `Range` on /read is honoured. When false, the host emulates ranged reads by discarding the prefix — correct bytes, just not cheap.
- write + delete together make the driver writable in filex's eyes; write without delete is refused at describe time (filex could create files it can never remove, which trash and versioning both need).
- move / copy: emulated by the host (copy = read→write, move = copy→delete) when a writable plugin lacks them, so a plugin author only writes them when the backend can do better than a byte copy.
- mkdir: no-op when a writable plugin lacks it (object stores have no directories; the local driver's semantics do not apply).
- set_mtime: only offered to filex when true — a mtime that is accepted and silently dropped is worse than one that is refused (storage.Toucher).
- watch: /watch is an SSE stream of Events. Consumed by the sync worker when a storage's sync mode is `fsnotify` and the driver is not local — see internal/sync/watch.go.
- presign: the plugin can hand out URLs a BROWSER can use directly, so bytes skip filex entirely. Only claim it when the URL is reachable from the client's network — a loopback-only plugin cannot presign anything useful, and conformance will catch a URL filex itself cannot fetch.
- multipart: resumable uploads in parts. Implies write; the host uses it for the staged-upload path (internal/staging), where it also pushes the parts itself, so the plugin must serve /multipart/part as well as the init/complete/abort trio.
func (Capabilities) Writable ¶
func (c Capabilities) Writable() bool
Writable reports whether the host will offer write, delete (and emulated move/copy/mkdir) on this plugin.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client speaks the protocol to ONE plugin endpoint. It is safe for concurrent use; the storage adapters share one per plugin.
func (*Client) AbortMultipart ¶
AbortMultipart throws an unfinished upload away.
func (*Client) CompleteMultipart ¶
func (c *Client) CompleteMultipart(ctx context.Context, id, path, uploadID string, parts []MultipartPart) error
CompleteMultipart finishes an upload.
func (*Client) CreateInstance ¶
CreateInstance is POST /v1/instances.
func (*Client) DeleteInstance ¶
DeleteInstance is DELETE /v1/instances/{id}. Best effort — the plugin may already be gone.
func (*Client) Describe ¶
func (c *Client) Describe(ctx context.Context) (*DescribeResponse, error)
Describe is GET /v1/describe with the answer validated.
func (*Client) InitMultipart ¶
func (c *Client) InitMultipart(ctx context.Context, id, path string, totalSize int64, partCount int) (MultipartInitResponse, error)
InitMultipart starts a resumable upload.
func (*Client) PresignDownload ¶
func (c *Client) PresignDownload(ctx context.Context, id, path string, ttl time.Duration) (PresignResponse, error)
PresignDownload asks for a URL the CLIENT can GET directly.
func (*Client) PresignUpload ¶
func (c *Client) PresignUpload(ctx context.Context, id, path string, size int64) (PresignResponse, error)
PresignUpload asks for a URL the CLIENT can PUT to directly.
func (*Client) Read ¶
func (c *Client) Read(ctx context.Context, id, path string, off, length int64) (io.ReadCloser, error)
Read streams the object; off/length follow storage.RangeReader (length<0 = to end). off==0 && length<0 sends no Range header at all.
func (*Client) SelfTest ¶
SelfTest asks the plugin for a THROWAWAY instance to run conformance probes against (POST /v1/selftest). A plugin that does not offer one answers 404/unsupported, and the host says so rather than pretending it verified something.
func (*Client) UploadPart ¶
func (c *Client) UploadPart(ctx context.Context, id, path, uploadID string, part int, r io.Reader, size int64) (string, error)
UploadPart sends ONE part from the server side (the staged-upload path holds the bytes itself, so it cannot use a presigned part URL).
type DescribeResponse ¶
type DescribeResponse struct {
// Protocol must equal ProtocolVersion.
Protocol int `json:"protocol"`
// Name is the driver's short identifier: [a-z0-9][a-z0-9_-]{0,31}. The
// storage registry knows it as DriverPrefix+Name.
Name string `json:"name"`
// Version is the plugin's own version string, shown in the admin list.
Version string `json:"version,omitempty"`
// Label is the human name for the driver picker (English; the plugin may
// also ship an I18nKey the host's catalogue does not know — the label is
// the fallback surfaces already apply).
Label string `json:"label"`
// Fields is the config form, in exactly the shape storage.Descriptor uses
// so the Connections page can render it without knowing the driver.
Fields []storage.Field `json:"fields"`
Capabilities Capabilities `json:"capabilities"`
}
DescribeResponse is GET /v1/describe. Everything the host needs to register the driver: its identity, its config form and what it can do.
type ErrorResponse ¶
ErrorResponse is every non-2xx body.
type Event ¶
type Event struct {
Op string `json:"op"` // create | modify | delete | move
Path string `json:"path"`
From string `json:"from,omitempty"`
}
Event is one SSE `data:` payload of /watch.
type Feed ¶ added in v0.48.0
type Feed struct {
Name string `json:"name"`
Version string `json:"version"`
Filex string `json:"filex,omitempty"`
Notes string `json:"notes,omitempty"`
Binaries map[string]FeedBinary `json:"binaries"`
}
Feed is a filex-storage.json.
type FeedBinary ¶ added in v0.48.0
type FeedBinary struct {
URL string `json:"url"`
SHA256 string `json:"sha256"`
// Signature is the detached ed25519 signature over the sha256, for an
// instance that only runs signed plugins (FILEX_PLUGIN_TRUSTED_KEYS).
Signature string `json:"signature,omitempty"`
}
FeedBinary is one platform's build.
type Handle ¶
type Handle interface {
// Client returns the live client, or an error naming why the plugin is
// unavailable right now (stopped, crashed and backing off, disabled).
Client() (*Client, error)
// DriverName is DriverPrefix+name.
DriverName() string
}
Handle is what the adapters hold instead of a *Client: the client behind a launched plugin changes every time the process restarts (new socket, new port), and a storage opened before the restart must keep working after it.
type InstanceRequest ¶
InstanceRequest is POST /v1/instances: the config the admin saved for one storage row, keyed by the field keys the plugin described.
type InstanceResponse ¶
type InstanceResponse struct {
Instance string `json:"instance"`
}
InstanceResponse names the instance every later call addresses.
type ListResponse ¶
type ListResponse struct {
Objects []Object `json:"objects"`
}
ListResponse is GET /v1/instances/{id}/list.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager owns every plugin's lifecycle and its place in the storage registry. One per server.
func (*Manager) BackgroundUpdates ¶ added in v0.48.0
BackgroundUpdates reports whether the daily check runs.
func (*Manager) CapabilitiesFor ¶
func (m *Manager) CapabilitiesFor(driver string) (Capabilities, bool)
CapabilitiesFor returns what the plugin behind a driver name declared, and whether that driver is currently provided at all. The second return is the honest answer to "why can I not save a storage on this": a plugin that is stopped or refused provides nothing.
func (*Manager) CheckUpdates ¶ added in v0.48.0
func (m *Manager) CheckUpdates(ctx context.Context) (*UpdateReport, error)
CheckUpdates reads every binary plugin's source once and records what it found. It installs nothing. One check at a time; a second caller waits.
func (*Manager) ConformanceMode ¶
ConformanceMode is enforce | warn | off.
func (*Manager) InstallBinary ¶
func (m *Manager) InstallBinary(ctx context.Context, name, filename string, r io.Reader, signature string) (*Status, error)
InstallBinary stores an uploaded plugin binary and starts it. filename is the name the file will have inside the plugin's directory (its basename is used; an empty one becomes "plugin" or "plugin.exe").
func (*Manager) InstallFromSource ¶ added in v0.48.0
InstallFromSource installs a binary plugin from the build its source publishes for this platform — the feed's version, held to the feed's sha256 — and keeps the source, so the daily check follows it.
func (*Manager) InstallFromURL ¶
func (m *Manager) InstallFromURL(ctx context.Context, name, rawURL, sha, signature string) (*Status, error)
InstallFromURL downloads a plugin binary. sha256 is REQUIRED: a URL is not a promise about bytes, and the operator has to say what they expect to receive — the same rule the self-update follows.
func (*Manager) InstallRemote ¶
InstallRemote registers a plugin filex connects to rather than runs.
func (*Manager) LastUpdateCheck ¶ added in v0.48.0
LastUpdateCheck is when the last check ran; zero when never.
func (*Manager) Load ¶
Load starts every enabled plugin from the database. Start-up is asynchronous; WaitReady bounds how long the caller waits for the first describes so pre-warmed storages find their drivers.
func (*Manager) Remove ¶
Remove stops the plugin, unregisters its driver, deletes its files and its row. Storages on its driver are left in place and simply fail to open until the plugin is back — deleting an admin's storages is not this function's decision to make.
func (*Manager) RequiresSignature ¶
RequiresSignature reports whether this instance will refuse an unsigned plugin. Surfaces show it so nobody discovers the rule from a rejection.
func (*Manager) Restart ¶
Restart stops and starts a plugin — the way out of StateRefused after the operator fixed the binary or the remote.
func (*Manager) SetEnabled ¶
SetEnabled turns a plugin on or off, persistently.
func (*Manager) SetSource ¶ added in v0.48.0
SetSource names (or, with "", clears) where a binary plugin's newer versions are published. Nothing is fetched: the next check reads it.
func (*Manager) SetUpdateHooks ¶ added in v0.48.0
func (m *Manager) SetUpdateHooks(h UpdateHooks)
SetUpdateHooks wires the range judge and the bell.
func (*Manager) Shutdown ¶
func (m *Manager) Shutdown()
Shutdown stops every plugin. Called once, on server shutdown.
func (*Manager) StartUpdater ¶ added in v0.48.0
StartUpdater runs the daily check (FILEX_APP_PLUGIN_UPDATE_CHECK): a day after the last one, remembered across restarts.
func (*Manager) Upgrade ¶
func (m *Manager) Upgrade(ctx context.Context, id int64, filename string, r io.Reader, signature string) (*Status, error)
Upgrade replaces a binary plugin's file in place, keeping the row, the name, the driver and every storage built on it.
Why this is not remove + install ¶
Removing takes the registration with it, and a storage whose driver has gone is a storage that cannot open. An upgrade is the ordinary thing an operator does — a new build of the same plugin — and it must not put the storages through a window where they are broken, nor make the operator re-enter the configuration. So: stop, swap the file, verify it, start. If the new binary fails to start or fails conformance, the plugin is refused and the old file is restored, because a failed upgrade must not also be a lost plugin. filename is accepted for the caller's convenience and not used: the file keeps the name it was installed under (see the note inside).
func (*Manager) UpgradeFromSource ¶ added in v0.48.0
UpgradeFromSource installs the newer version the plugin's source has — an administrator's approval ("Review update"). The same search the check makes, the bytes held to the feed's sha256 before anything is stopped, then the ordinary upgrade with its signature check, conformance probe and roll-back.
type MoveRequest ¶
MoveRequest is the body of move / copy.
type MtimeRequest ¶
MtimeRequest is the body of set-mtime.
type MultipartCompleteRequest ¶
type MultipartCompleteRequest struct {
Path string `json:"path"`
UploadID string `json:"upload_id"`
Parts []MultipartPart `json:"parts,omitempty"`
}
MultipartCompleteRequest finishes (or, without Parts, aborts) an upload.
type MultipartInitRequest ¶
type MultipartInitRequest struct {
Path string `json:"path"`
TotalSize int64 `json:"total_size"`
PartCount int `json:"part_count"`
}
MultipartInitRequest starts a resumable upload.
type MultipartInitResponse ¶
type MultipartInitResponse struct {
UploadID string `json:"upload_id"`
PartURLs []string `json:"part_urls,omitempty"`
}
MultipartInitResponse carries the upload id and, when the plugin can presign them, one URL per part for the browser to PUT into.
type MultipartPart ¶
MultipartPart is one finished part.
type MultipartPartResponse ¶
type MultipartPartResponse struct {
Etag string `json:"etag"`
}
MultipartPartResponse is the answer to a server-side part upload.
type Object ¶
type Object struct {
Path string `json:"path"`
Name string `json:"name"`
Size int64 `json:"size"`
Kind string `json:"kind"` // "file" | "dir" | "symlink"
Mime string `json:"mime,omitempty"`
Etag string `json:"etag,omitempty"`
Mtime time.Time `json:"mtime,omitempty"`
Metadata map[string]string `json:"metadata,omitempty"`
}
Object is one entry of a listing or a stat. Same fields as storage.Object, spelled out here so the wire format is owned by the protocol and does not move when the internal type does.
type Options ¶
type Options struct {
Store db.Store
// Dir is where binary plugins live: <data-dir>/plugins.
Dir string
// SecretKey seals remote plugins' bearer tokens (secretbox). Empty means
// a remote plugin cannot be registered — the token would have to be
// stored in plaintext, and this codebase does not do that.
SecretKey string
Log *slog.Logger
// MaxBinaryBytes caps an uploaded/downloaded plugin. 0 → 512 MiB.
MaxBinaryBytes int64
// HTTP downloads plugins from URLs. Nil → a guarded client with a
// timeout that refuses private, loopback and link-local targets (after
// DNS, and on every redirect hop) — a plugin URL must not become a probe
// of the server's own network. Supplying a client lifts that guard; it
// exists for tests and for an embedder that has its own egress policy.
HTTP *http.Client
// Conformance is enforce (default), warn or off.
Conformance string
// MaxInFlight bounds concurrent operations per plugin (0 → the default).
MaxInFlight int
// TrustedKeys are ed25519 public keys (hex or base64) that may sign a
// plugin binary. When ANY key is configured, an unsigned or badly signed
// binary is refused at install: the sha256 only proves the file has not
// changed since it arrived, never that it came from someone you trust.
TrustedKeys []string
// Platform is the GOOS/GOARCH key a source's build is chosen by
// (updates.go). Empty = the running one; tests set it.
Platform string
}
Options configure a Manager.
type PathRequest ¶
type PathRequest struct {
Path string `json:"path"`
}
PathRequest is the body of delete / mkdir.
type PresignRequest ¶
type PresignRequest struct {
Path string `json:"path"`
// Size is the object size for an upload (-1 when unknown).
Size int64 `json:"size,omitempty"`
// TTLSeconds is how long a download URL should stay valid.
TTLSeconds int64 `json:"ttl_s,omitempty"`
}
PresignRequest is the body of presign-upload / presign-download.
type PresignResponse ¶
type PresignResponse struct {
URL string `json:"url"`
Method string `json:"method,omitempty"`
Headers map[string]string `json:"headers,omitempty"`
ExpiresAt time.Time `json:"expires_at"`
}
PresignResponse is what a plugin answers to either presign call. Only URL and ExpiresAt are required; the rest describe how the client must send an upload.
type ProbeResult ¶
type ProbeResult struct {
// Name is the capability or behaviour probed ("read", "write", "range"…).
Name string `json:"name"`
// Status is pass | fail | skip.
Status string `json:"status"`
// Detail says what happened — for a failure, precisely enough to fix it.
Detail string `json:"detail,omitempty"`
// Took is how long the probe ran. ⚠ NOT serialised: encoding/json has no
// marshaller for time.Duration and writes the raw int64, so a field named
// took_ms carrying a Duration is off by a million — measured by the admin
// page reading nanoseconds as milliseconds. TookMS below is what goes on
// the wire, and it really is milliseconds.
Took time.Duration `json:"-"`
TookMS int64 `json:"took_ms"`
}
ProbeResult is one capability's verdict.
type Process ¶
type Process struct {
Name string
Binary string // absolute path
Token string
SockDir string // private directory the plugin may create its socket in
Log *slog.Logger
// OnUp is called (from the supervisor goroutine) each time the plugin has
// printed its handshake and answered nothing yet — the manager describes
// it here and decides whether it is acceptable. Returning an error stops
// the plugin for good (a plugin that fails validation will fail it again;
// restarting would only churn).
OnUp func(ctx context.Context, c *Client) error
// OnDown is called each time the process is gone (exit, kill, failed
// handshake), before any restart.
OnDown func(err error)
// contains filtered or unexported fields
}
Process supervises ONE launched plugin binary: start it, wait for the handshake line, hand the client to whoever asked, restart it with backoff when it dies, stop it on request. It knows nothing about storage — the manager layers registration on top through the callbacks.
type RejectedError ¶
type RejectedError struct {
// contains filtered or unexported fields
}
RejectedError marks a failure caused by what the CALLER supplied — a missing signature, one that does not verify — rather than by filex failing.
⚠ This exists because the handler used to classify these by SEARCHING THE MESSAGE for words like "required" or "sha256". Under that scheme "this instance only accepts signed plugins …" happened to contain "sha256" and answered 400, while "signature does not verify against any trusted key" matched nothing and answered 500 — the same class of mistake, reported as two different kinds of event, one of them blamed on the server. A rejected upload is not an outage; it should not spend the server's error budget or page anybody.
func (RejectedError) Error ¶
func (e RejectedError) Error() string
func (RejectedError) Unwrap ¶
func (e RejectedError) Unwrap() error
type Report ¶
type Report struct {
// Verified is true when every declared capability passed.
Verified bool `json:"verified"`
// Scratch says where the probes ran: "selftest" (the plugin's own
// throwaway area) or "storage" (a folder inside a real storage).
Scratch string `json:"scratch"`
Results []ProbeResult `json:"results"`
RanAt time.Time `json:"ran_at"`
}
Report is the outcome of a conformance run.
func RunConformance ¶
func RunConformance(ctx context.Context, drv storage.Driver, caps Capabilities, base, scratchKind string) *Report
RunConformance probes every capability caps declares, using drv as the driver under test and base as a directory the probes may write inside.
It never leaves anything behind that it created (best effort — a plugin whose delete is broken is exactly what the delete probe reports).
func VerifyStorage ¶
VerifyStorage probes a driver against a REAL storage configuration, inside a scratch folder it creates and removes.
This is the second gate. The first (a plugin's selftest at install) proves the code works; this one proves it works against the configuration an operator just typed — the credentials, the bucket, the path. A plugin can be perfectly correct and the storage still be unusable, and finding that out when the storage is SAVED is much cheaper than finding out when a user uploads a file into it.
caps must be the plugin's declared capability set. Returns the report; the caller decides what to do with a failure.
func (*Report) FailureError ¶
FailureError turns the failures into one error whose message a person can act on. Nil when nothing failed.
func (*Report) Failures ¶
func (r *Report) Failures() []ProbeResult
Failures returns the probes that failed, most useful first.
type Stats ¶
type Stats struct {
InFlight int64 `json:"in_flight"`
Waited int64 `json:"waited"`
Rejected int64 `json:"rejected"`
Max int `json:"max_in_flight"`
}
Stats is what the admin surface and Prometheus read.
type Status ¶
type Status struct {
*model.Plugin
State string `json:"state"`
StateError string `json:"state_error,omitempty"`
Restarts int `json:"restarts"`
Capabilities *Capabilities `json:"capabilities,omitempty"`
Label string `json:"label,omitempty"`
FieldCount int `json:"field_count"`
// InUse counts storage rows on this plugin's driver — shown before a
// remove so the admin knows what will stop working.
InUse int `json:"in_use"`
// Conformance is the last verification of the plugin's own claims. Nil
// when it has not been probed (an older row, or probing is off).
Conformance *Report `json:"conformance,omitempty"`
// Load is what this plugin is doing right now: in-flight operations, and
// how often callers have had to wait or been refused a slot. A plugin
// that is merely SLOW shows up here long before it shows up as an error.
Load Stats `json:"load"`
// Update is what the last check of the plugin's source found
// (updates.go); absent for a plugin without a source, or before the
// first check.
Update *UpdateInfo `json:"update,omitempty"`
}
Status is one plugin as the admin sees it: the row plus what is happening.
type UpdateHooks ¶ added in v0.48.0
type UpdateHooks struct {
// Range judges a feed's `filex` range against the running filex
// (wasmplugin.JudgeFilexRange). Nil = no range is checked.
Range func(rng string) (ok bool, requires, filex string, err error)
// Announce rings the administrators' bell, once per newer version.
Announce func(ctx context.Context, name, version, notes string)
}
UpdateHooks wire the check to the rest of filex (server.go).
type UpdateInfo ¶ added in v0.48.0
type UpdateInfo struct {
CheckedAt *time.Time `json:"checked_at,omitempty"`
Status string `json:"status"`
// Version: the newer version (available, incompatible).
Version string `json:"version,omitempty"`
// Requires: incompatible — the range that version declares.
Requires string `json:"requires,omitempty"`
// Notes: the feed's notes for Version, plain text.
Notes string `json:"notes,omitempty"`
// SHA256: the build for this platform (available).
SHA256 string `json:"sha256,omitempty"`
// Platform the build was chosen for ("linux/amd64").
Platform string `json:"platform,omitempty"`
// Error: check_failed — why.
Error string `json:"error,omitempty"`
// Announced is "<status>@<version>" once the administrators were told.
Announced string `json:"announced,omitempty"`
}
UpdateInfo is what the last check of a plugin's source found.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Command gen writes driver_shapes.go: one concrete type per capability combination a plugin can declare.
|
Command gen writes driver_shapes.go: one concrete type per capability combination a plugin can declare. |
|
Package testplugin runs a real filex storage plugin — the HTTP/JSON protocol of internal/plugin, served over httptest — so any package that consumes a storage.Driver can be exercised against a PLUGIN-backed storage without spawning a second process.
|
Package testplugin runs a real filex storage plugin — the HTTP/JSON protocol of internal/plugin, served over httptest — so any package that consumes a storage.Driver can be exercised against a PLUGIN-backed storage without spawning a second process. |