Help us shape the future of Go! Take the 2026 Go Developer Survey

plugin

package
v0.46.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 26, 2026 License: MIT Imports: 34 Imported by: 0

Documentation

Overview

Package plugin lets a storage driver live OUTSIDE the filex binary.

Why this exists

Every storage filex speaks — local disk, S3, SFTP, FTP, WebDAV, SMB — is a Go package compiled into the binary and registered from an init(). That is the right shape for the drivers filex ships, and the wrong shape for the driver somebody writes for their own system: they would have to fork filex, learn its build, and rebuild on every release. A plugin is that same driver as a separate program that filex starts (or connects to) and talks to over a small HTTP/JSON protocol. Any language that can serve HTTP can be a filex storage driver.

What a plugin is

One of two things, and the protocol is the same for both:

  • a BINARY filex launches itself: it lives under <data-dir>/plugins/<name>/, is started with FILEX_PLUGIN_TOKEN and FILEX_PLUGIN_SOCKET_DIR in its environment, listens on a unix socket in that directory (or on a loopback TCP port), and prints ONE handshake line to stdout — `FILEX-PLUGIN/1 unix:/path/to.sock` or `FILEX-PLUGIN/1 tcp:127.0.0.1:PORT`. filex supervises it: health-checks it, restarts it with backoff when it dies, kills it on shutdown.
  • a REMOTE service the operator runs (a sidecar container, a daemon on another host): the admin registers its base URL and a bearer token and filex just connects. Nothing is spawned.

The protocol, in one screen

Every request carries `Authorization: Bearer <token>`. Every error body is {"error": <code>, "message": <text>}. Codes filex maps back to storage errors: not_found, read_only, unsupported, no_instance; anything else is a plain failure with the message attached.

GET    /v1/describe                     → DescribeResponse (name, version, fields, capabilities)
POST   /v1/instances       {config}     → {"instance": id}     one per storage row; Init
DELETE /v1/instances/{id}                                       release it
GET    /v1/instances/{id}/list?path=    → {"objects": [Object]}
GET    /v1/instances/{id}/stat?path=    → Object
GET    /v1/instances/{id}/read?path=    → body stream (Range: bytes=a-b honoured when capabilities.range)
PUT    /v1/instances/{id}/write?path=   ← body stream (X-Filex-Size: n, or -1 when unknown)
POST   /v1/instances/{id}/move   {src,dst}
POST   /v1/instances/{id}/copy   {src,dst}
POST   /v1/instances/{id}/delete {path}     idempotent
POST   /v1/instances/{id}/mkdir  {path}
POST   /v1/instances/{id}/set-mtime {path,mtime}   RFC 3339
GET    /v1/instances/{id}/watch          → text/event-stream of Event, when capabilities.watch
POST   /v1/instances/{id}/presign-upload   {path,size}      → PresignedUpload   (capabilities.presign)
POST   /v1/instances/{id}/presign-download {path,ttl_s}     → {url,expires_at}  (capabilities.presign)
POST   /v1/instances/{id}/multipart/init     {path,total_size,part_count} → {upload_id,part_urls}
PUT    /v1/instances/{id}/multipart/part?upload_id=&part=N&path=  ← body → {etag}
POST   /v1/instances/{id}/multipart/complete {path,upload_id,parts[]}
POST   /v1/instances/{id}/multipart/abort    {path,upload_id}
POST   /v1/selftest                      → {"instance": id}  a THROWAWAY instance for conformance

Conformance

A plugin that declares a capability it cannot actually perform is worse than one that declares nothing: the UI offers the operation, the operation fails, and the user reads that as filex being broken. So filex PROBES what a plugin claims — at install, and again whenever a storage on it is saved — and refuses a plugin whose own claims do not hold up (internal/plugin/ conformance.go). /v1/selftest is how a plugin offers a scratch area for that: it creates an instance backed by throwaway space and returns its id. A plugin without it can still be installed, but it is marked UNVERIFIED and the probes run against the first real storage instead.

The driver a plugin implements is registered in filex as `plugin:<name>`, so a plugin can never shadow a built-in driver, and its config form on the Connections page renders from the fields it describes — exactly as the built-in drivers' forms do (storage.Descriptor). Full specification with the reasoning behind each choice: docs/PLUGINS.md.

Index

Constants

View Source
const (
	ProbePass = "pass"
	ProbeFail = "fail"
	ProbeSkip = "skip"
)

Probe statuses.

View Source
const (
	// DefaultMaxInFlight is how many operations may be inside one plugin at
	// once. Ten is comfortably above what a browsing user generates and low
	// enough that a stuck plugin cannot consume the server.
	DefaultMaxInFlight = 10
	// DefaultOpTimeout bounds one operation. Streaming reads and writes are
	// exempt — a 20 GB upload is legitimately slow — so this applies to the
	// metadata calls, where slowness means trouble rather than size.
	DefaultOpTimeout = 60 * time.Second
)

Defaults chosen to be generous for a healthy plugin and firm for a sick one. Both are per PLUGIN, not per storage: the process is the resource.

View Source
const (
	StateDisabled = "disabled" // row says off; nothing runs
	StateStarting = "starting" // launched, no handshake/describe yet
	StateRunning  = "running"  // described, driver registered
	StateFailed   = "failed"   // last attempt failed; a binary is backing off, a remote is re-checked
	StateRefused  = "refused"  // describe or conformance rejected it; not retried until restart
)

Runtime states a plugin can be in, as shown to the admin.

View Source
const (
	// ConformanceEnforce is the default: a plugin that fails its own claims
	// is refused, so nobody can build a storage on half a driver.
	ConformanceEnforce = "enforce"
	// ConformanceWarn registers the plugin anyway and keeps the report. For
	// an operator debugging a plugin they are writing — never the default,
	// because the cost of a broken claim is paid by the user, who reads it as
	// filex being broken.
	ConformanceWarn = "warn"
	// ConformanceOff skips the probes entirely.
	ConformanceOff = "off"
)

Conformance modes (FILEX_PLUGIN_CONFORMANCE).

View Source
const (
	ErrCodeNotFound    = "not_found"
	ErrCodeReadOnly    = "read_only"
	ErrCodeUnsupported = "unsupported"
	// ErrCodeNoInstance says the instance id is unknown — the plugin was
	// restarted and lost its state. The host re-creates the instance from the
	// saved config and retries once, so a plugin restart is invisible.
	ErrCodeNoInstance = "no_instance"
	ErrCodeInvalid    = "invalid"
)

Error codes the host understands. Anything else is reported verbatim.

View Source
const ConformanceTimeout = 90 * time.Second

ConformanceTimeout bounds a whole run. A plugin that cannot answer a handful of small operations in this long is not one a storage should be built on.

View Source
const DriverPrefix = "plugin:"

DriverPrefix namespaces plugin drivers in the storage registry.

View Source
const HandshakePrefix = "FILEX-PLUGIN/1"

HandshakePrefix is the first token of the single line a launched plugin prints to stdout, followed by a space and the address it listens on.

View Source
const ProtocolVersion = 1

ProtocolVersion is the only version this host speaks. A plugin whose describe reports another one is refused with a clear message rather than half-working.

View Source
const SizeHeader = "X-Filex-Size"

SizeHeader carries the write size on PUT /write, because a streamed body may be chunked and Content-Length absent. -1 means unknown.

Variables

View Source
var (
	ErrHandshake = errors.New("plugin did not print its handshake line")
	ErrRefused   = errors.New("plugin refused by host")
	ErrStopped   = errors.New("plugin stopped")
	// ErrGaveUp is reported once the supervisor has seen maxConsecutiveFailures
	// starts in a row that never came up. It stops restarting: a binary that
	// fails ten times will fail an eleventh, and a loop that keeps trying
	// forever is a loop that keeps a broken plugin looking "starting". The
	// admin's Restart button begins a fresh count.
	ErrGaveUp = errors.New("plugin failed to start 10 times in a row; not restarting until Restart")
)

Errors reported through OnDown / Err.

View Source
var (
	ErrSignatureRequired = errors.New("signature required")
	ErrSignatureInvalid  = errors.New("signature invalid")
)

ErrSignatureRequired / ErrSignatureInvalid are what VerifyDetached returns (wrapped in RejectedError by the storage-plugin manager); callers switch on them to pick an error code rather than reading the message.

View Source
var ErrBadName = errors.New("plugin name must match [a-z0-9][a-z0-9_-]{0,31}")
View Source
var ErrBusy = errors.New("plugin is at its concurrency limit")

ErrBusy is returned when a plugin has no free slot. It is deliberately distinct from a timeout: this one says the plugin is saturated, which is an operator's problem to size, not a bug to chase.

Functions

func NewDriver

func NewDriver(h Handle, caps Capabilities) storage.Driver

NewDriver builds a fresh storage.Driver for one storage row on plugin h. This is the storage.Factory the manager registers.

The concrete type is chosen by newShape (driver_shapes.go, generated): a plugin gets a value whose METHOD SET is exactly what it declared, because filex reads capabilities by type assertion at forty-odd call sites.

func ParsePublicKey

func ParsePublicKey(k string) (ed25519.PublicKey, error)

ParsePublicKey accepts an ed25519 key as hex or standard base64.

func ParsePublicKeys

func ParsePublicKeys(keys []string) ([]ed25519.PublicKey, error)

ParsePublicKeys parses every key, refusing the whole list on the first bad one so a typo cannot silently shrink the trust set.

func ValidName

func ValidName(s string) bool

ValidName is the operator-chosen plugin name rule (same as the driver's).

func ValidateDescribe

func ValidateDescribe(d *DescribeResponse) error

ValidateDescribe applies the rules a plugin must meet before the host registers it. Every message names the rule, because the person reading it is the plugin's author.

func VerifyDetached

func VerifyDetached(trusted []ed25519.PublicKey, sha, signature string) error

VerifyDetached checks signature (hex or base64) over the lower-hex sha256 against the trusted keys. No trusted keys → nothing is required and nil is returned; trusted keys and no signature → ErrSignatureRequired; otherwise ErrSignatureInvalid unless one key verifies.

Types

type Address

type Address struct {
	Network string // "unix" | "tcp"
	Target  string // socket path, or host:port, or full URL for remote
	URL     string // base URL to put in front of /v1
}

Address is a parsed plugin address: "unix:/path" or "tcp:host:port" from the handshake line, or a full http(s):// URL for a remote plugin.

func ParseAddress

func ParseAddress(s string) (Address, error)

ParseAddress accepts the three spellings the host meets.

type Capabilities

type Capabilities struct {
	Range     bool `json:"range"`
	Write     bool `json:"write"`
	Delete    bool `json:"delete"`
	Move      bool `json:"move"`
	Copy      bool `json:"copy"`
	Mkdir     bool `json:"mkdir"`
	SetMtime  bool `json:"set_mtime"`
	Watch     bool `json:"watch"`
	Presign   bool `json:"presign"`
	Multipart bool `json:"multipart"`
}

Capabilities is what the plugin says it can do. It is deliberately NOT storage.Capabilities: presigned URLs and multipart are out of scope for the protocol's first version, and set_mtime is a capability here where the host expresses it as an optional interface (storage.Toucher).

The rules the host applies (documented, enforced by the SDK's own checks):

  • read is implied — a driver that cannot list/stat/read is not a driver.
  • range: `Range` on /read is honoured. When false, the host emulates ranged reads by discarding the prefix — correct bytes, just not cheap.
  • write + delete together make the driver writable in filex's eyes; write without delete is refused at describe time (filex could create files it can never remove, which trash and versioning both need).
  • move / copy: emulated by the host (copy = read→write, move = copy→delete) when a writable plugin lacks them, so a plugin author only writes them when the backend can do better than a byte copy.
  • mkdir: no-op when a writable plugin lacks it (object stores have no directories; the local driver's semantics do not apply).
  • set_mtime: only offered to filex when true — a mtime that is accepted and silently dropped is worse than one that is refused (storage.Toucher).
  • watch: /watch is an SSE stream of Events. Consumed by the sync worker when a storage's sync mode is `fsnotify` and the driver is not local — see internal/sync/watch.go.
  • presign: the plugin can hand out URLs a BROWSER can use directly, so bytes skip filex entirely. Only claim it when the URL is reachable from the client's network — a loopback-only plugin cannot presign anything useful, and conformance will catch a URL filex itself cannot fetch.
  • multipart: resumable uploads in parts. Implies write; the host uses it for the staged-upload path (internal/staging), where it also pushes the parts itself, so the plugin must serve /multipart/part as well as the init/complete/abort trio.

func (Capabilities) Writable

func (c Capabilities) Writable() bool

Writable reports whether the host will offer write, delete (and emulated move/copy/mkdir) on this plugin.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client speaks the protocol to ONE plugin endpoint. It is safe for concurrent use; the storage adapters share one per plugin.

func NewClient

func NewClient(addr Address, token string) *Client

NewClient builds a client for addr authenticating with token.

func (*Client) AbortMultipart

func (c *Client) AbortMultipart(ctx context.Context, id, path, uploadID string) error

AbortMultipart throws an unfinished upload away.

func (*Client) Close

func (c *Client) Close()

Close releases idle connections.

func (*Client) CompleteMultipart

func (c *Client) CompleteMultipart(ctx context.Context, id, path, uploadID string, parts []MultipartPart) error

CompleteMultipart finishes an upload.

func (*Client) Copy

func (c *Client) Copy(ctx context.Context, id, src, dst string) error

func (*Client) CreateInstance

func (c *Client) CreateInstance(ctx context.Context, cfg map[string]any) (string, error)

CreateInstance is POST /v1/instances.

func (*Client) Delete

func (c *Client) Delete(ctx context.Context, id, path string) error

func (*Client) DeleteInstance

func (c *Client) DeleteInstance(ctx context.Context, id string) error

DeleteInstance is DELETE /v1/instances/{id}. Best effort — the plugin may already be gone.

func (*Client) Describe

func (c *Client) Describe(ctx context.Context) (*DescribeResponse, error)

Describe is GET /v1/describe with the answer validated.

func (*Client) InitMultipart

func (c *Client) InitMultipart(ctx context.Context, id, path string, totalSize int64, partCount int) (MultipartInitResponse, error)

InitMultipart starts a resumable upload.

func (*Client) List

func (c *Client) List(ctx context.Context, id, path string) ([]Object, error)

func (*Client) Mkdir

func (c *Client) Mkdir(ctx context.Context, id, path string) error

func (*Client) Move

func (c *Client) Move(ctx context.Context, id, src, dst string) error

func (*Client) PresignDownload

func (c *Client) PresignDownload(ctx context.Context, id, path string, ttl time.Duration) (PresignResponse, error)

PresignDownload asks for a URL the CLIENT can GET directly.

func (*Client) PresignUpload

func (c *Client) PresignUpload(ctx context.Context, id, path string, size int64) (PresignResponse, error)

PresignUpload asks for a URL the CLIENT can PUT to directly.

func (*Client) Read

func (c *Client) Read(ctx context.Context, id, path string, off, length int64) (io.ReadCloser, error)

Read streams the object; off/length follow storage.RangeReader (length<0 = to end). off==0 && length<0 sends no Range header at all.

func (*Client) SelfTest

func (c *Client) SelfTest(ctx context.Context) (string, error)

SelfTest asks the plugin for a THROWAWAY instance to run conformance probes against (POST /v1/selftest). A plugin that does not offer one answers 404/unsupported, and the host says so rather than pretending it verified something.

func (*Client) SetMtime

func (c *Client) SetMtime(ctx context.Context, id, path string, t time.Time) error

func (*Client) Stat

func (c *Client) Stat(ctx context.Context, id, path string) (Object, error)

func (*Client) UploadPart

func (c *Client) UploadPart(ctx context.Context, id, path, uploadID string, part int, r io.Reader, size int64) (string, error)

UploadPart sends ONE part from the server side (the staged-upload path holds the bytes itself, so it cannot use a presigned part URL).

func (*Client) Watch

func (c *Client) Watch(ctx context.Context, id string) (<-chan Event, error)

Watch opens the SSE stream and forwards events until ctx ends or the stream closes. The channel is closed when the stream ends.

func (*Client) Write

func (c *Client) Write(ctx context.Context, id, path string, r io.Reader, size int64) error

Write streams r as the new content of path.

type DescribeResponse

type DescribeResponse struct {
	// Protocol must equal ProtocolVersion.
	Protocol int `json:"protocol"`
	// Name is the driver's short identifier: [a-z0-9][a-z0-9_-]{0,31}. The
	// storage registry knows it as DriverPrefix+Name.
	Name string `json:"name"`
	// Version is the plugin's own version string, shown in the admin list.
	Version string `json:"version,omitempty"`
	// Label is the human name for the driver picker (English; the plugin may
	// also ship an I18nKey the host's catalogue does not know — the label is
	// the fallback surfaces already apply).
	Label string `json:"label"`
	// Fields is the config form, in exactly the shape storage.Descriptor uses
	// so the Connections page can render it without knowing the driver.
	Fields       []storage.Field `json:"fields"`
	Capabilities Capabilities    `json:"capabilities"`
}

DescribeResponse is GET /v1/describe. Everything the host needs to register the driver: its identity, its config form and what it can do.

type ErrorResponse

type ErrorResponse struct {
	Error   string `json:"error"`
	Message string `json:"message,omitempty"`
}

ErrorResponse is every non-2xx body.

type Event

type Event struct {
	Op   string `json:"op"` // create | modify | delete | move
	Path string `json:"path"`
	From string `json:"from,omitempty"`
}

Event is one SSE `data:` payload of /watch.

type Handle

type Handle interface {
	// Client returns the live client, or an error naming why the plugin is
	// unavailable right now (stopped, crashed and backing off, disabled).
	Client() (*Client, error)
	// DriverName is DriverPrefix+name.
	DriverName() string
}

Handle is what the adapters hold instead of a *Client: the client behind a launched plugin changes every time the process restarts (new socket, new port), and a storage opened before the restart must keep working after it.

type InstanceRequest

type InstanceRequest struct {
	Config map[string]any `json:"config"`
}

InstanceRequest is POST /v1/instances: the config the admin saved for one storage row, keyed by the field keys the plugin described.

type InstanceResponse

type InstanceResponse struct {
	Instance string `json:"instance"`
}

InstanceResponse names the instance every later call addresses.

type ListResponse

type ListResponse struct {
	Objects []Object `json:"objects"`
}

ListResponse is GET /v1/instances/{id}/list.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager owns every plugin's lifecycle and its place in the storage registry. One per server.

func New

func New(o Options) (*Manager, error)

New builds a Manager. Call Load to start what the database says.

func (*Manager) CapabilitiesFor

func (m *Manager) CapabilitiesFor(driver string) (Capabilities, bool)

CapabilitiesFor returns what the plugin behind a driver name declared, and whether that driver is currently provided at all. The second return is the honest answer to "why can I not save a storage on this": a plugin that is stopped or refused provides nothing.

func (*Manager) ConformanceMode

func (m *Manager) ConformanceMode() string

ConformanceMode is enforce | warn | off.

func (*Manager) Dir

func (m *Manager) Dir() string

Dir is the plugins directory.

func (*Manager) Get

func (m *Manager) Get(ctx context.Context, id int64) (*Status, error)

Get returns one plugin's status.

func (*Manager) InstallBinary

func (m *Manager) InstallBinary(ctx context.Context, name, filename string, r io.Reader, signature string) (*Status, error)

InstallBinary stores an uploaded plugin binary and starts it. filename is the name the file will have inside the plugin's directory (its basename is used; an empty one becomes "plugin" or "plugin.exe").

func (*Manager) InstallFromURL

func (m *Manager) InstallFromURL(ctx context.Context, name, rawURL, sha, signature string) (*Status, error)

InstallFromURL downloads a plugin binary. sha256 is REQUIRED: a URL is not a promise about bytes, and the operator has to say what they expect to receive — the same rule the self-update follows.

func (*Manager) InstallRemote

func (m *Manager) InstallRemote(ctx context.Context, name, address, token string) (*Status, error)

InstallRemote registers a plugin filex connects to rather than runs.

func (*Manager) List

func (m *Manager) List(ctx context.Context) ([]*Status, error)

List returns every plugin with its runtime state, name-sorted.

func (*Manager) Load

func (m *Manager) Load(ctx context.Context) error

Load starts every enabled plugin from the database. Start-up is asynchronous; WaitReady bounds how long the caller waits for the first describes so pre-warmed storages find their drivers.

func (*Manager) Remove

func (m *Manager) Remove(ctx context.Context, id int64) error

Remove stops the plugin, unregisters its driver, deletes its files and its row. Storages on its driver are left in place and simply fail to open until the plugin is back — deleting an admin's storages is not this function's decision to make.

func (*Manager) RequiresSignature

func (m *Manager) RequiresSignature() bool

RequiresSignature reports whether this instance will refuse an unsigned plugin. Surfaces show it so nobody discovers the rule from a rejection.

func (*Manager) Restart

func (m *Manager) Restart(ctx context.Context, id int64) (*Status, error)

Restart stops and starts a plugin — the way out of StateRefused after the operator fixed the binary or the remote.

func (*Manager) SetEnabled

func (m *Manager) SetEnabled(ctx context.Context, id int64, on bool) (*Status, error)

SetEnabled turns a plugin on or off, persistently.

func (*Manager) Shutdown

func (m *Manager) Shutdown()

Shutdown stops every plugin. Called once, on server shutdown.

func (*Manager) Upgrade

func (m *Manager) Upgrade(ctx context.Context, id int64, filename string, r io.Reader, signature string) (*Status, error)

Upgrade replaces a binary plugin's file in place, keeping the row, the name, the driver and every storage built on it.

Why this is not remove + install

Removing takes the registration with it, and a storage whose driver has gone is a storage that cannot open. An upgrade is the ordinary thing an operator does — a new build of the same plugin — and it must not put the storages through a window where they are broken, nor make the operator re-enter the configuration. So: stop, swap the file, verify it, start. If the new binary fails to start or fails conformance, the plugin is refused and the old file is restored, because a failed upgrade must not also be a lost plugin. filename is accepted for the caller's convenience and not used: the file keeps the name it was installed under (see the note inside).

func (*Manager) WaitReady

func (m *Manager) WaitReady(d time.Duration)

WaitReady blocks until every enabled plugin has left StateStarting or d elapsed. It exists for the server's start-up: storages on a plugin driver are pre-warmed right after, and would otherwise all log "unknown driver".

type MoveRequest

type MoveRequest struct {
	Src string `json:"src"`
	Dst string `json:"dst"`
}

MoveRequest is the body of move / copy.

type MtimeRequest

type MtimeRequest struct {
	Path  string    `json:"path"`
	Mtime time.Time `json:"mtime"`
}

MtimeRequest is the body of set-mtime.

type MultipartCompleteRequest

type MultipartCompleteRequest struct {
	Path     string          `json:"path"`
	UploadID string          `json:"upload_id"`
	Parts    []MultipartPart `json:"parts,omitempty"`
}

MultipartCompleteRequest finishes (or, without Parts, aborts) an upload.

type MultipartInitRequest

type MultipartInitRequest struct {
	Path      string `json:"path"`
	TotalSize int64  `json:"total_size"`
	PartCount int    `json:"part_count"`
}

MultipartInitRequest starts a resumable upload.

type MultipartInitResponse

type MultipartInitResponse struct {
	UploadID string   `json:"upload_id"`
	PartURLs []string `json:"part_urls,omitempty"`
}

MultipartInitResponse carries the upload id and, when the plugin can presign them, one URL per part for the browser to PUT into.

type MultipartPart

type MultipartPart struct {
	PartNumber int    `json:"part_number"`
	Etag       string `json:"etag"`
}

MultipartPart is one finished part.

type MultipartPartResponse

type MultipartPartResponse struct {
	Etag string `json:"etag"`
}

MultipartPartResponse is the answer to a server-side part upload.

type Object

type Object struct {
	Path     string            `json:"path"`
	Name     string            `json:"name"`
	Size     int64             `json:"size"`
	Kind     string            `json:"kind"` // "file" | "dir" | "symlink"
	Mime     string            `json:"mime,omitempty"`
	Etag     string            `json:"etag,omitempty"`
	Mtime    time.Time         `json:"mtime,omitempty"`
	Metadata map[string]string `json:"metadata,omitempty"`
}

Object is one entry of a listing or a stat. Same fields as storage.Object, spelled out here so the wire format is owned by the protocol and does not move when the internal type does.

type Options

type Options struct {
	Store db.Store
	// Dir is where binary plugins live: <data-dir>/plugins.
	Dir string
	// SecretKey seals remote plugins' bearer tokens (secretbox). Empty means
	// a remote plugin cannot be registered — the token would have to be
	// stored in plaintext, and this codebase does not do that.
	SecretKey string
	Log       *slog.Logger
	// MaxBinaryBytes caps an uploaded/downloaded plugin. 0 → 512 MiB.
	MaxBinaryBytes int64
	// HTTP downloads plugins from URLs. Nil → a guarded client with a
	// timeout that refuses private, loopback and link-local targets (after
	// DNS, and on every redirect hop) — a plugin URL must not become a probe
	// of the server's own network. Supplying a client lifts that guard; it
	// exists for tests and for an embedder that has its own egress policy.
	HTTP *http.Client
	// Conformance is enforce (default), warn or off.
	Conformance string
	// MaxInFlight bounds concurrent operations per plugin (0 → the default).
	MaxInFlight int
	// TrustedKeys are ed25519 public keys (hex or base64) that may sign a
	// plugin binary. When ANY key is configured, an unsigned or badly signed
	// binary is refused at install: the sha256 only proves the file has not
	// changed since it arrived, never that it came from someone you trust.
	TrustedKeys []string
}

Options configure a Manager.

type PathRequest

type PathRequest struct {
	Path string `json:"path"`
}

PathRequest is the body of delete / mkdir.

type PresignRequest

type PresignRequest struct {
	Path string `json:"path"`
	// Size is the object size for an upload (-1 when unknown).
	Size int64 `json:"size,omitempty"`
	// TTLSeconds is how long a download URL should stay valid.
	TTLSeconds int64 `json:"ttl_s,omitempty"`
}

PresignRequest is the body of presign-upload / presign-download.

type PresignResponse

type PresignResponse struct {
	URL       string            `json:"url"`
	Method    string            `json:"method,omitempty"`
	Headers   map[string]string `json:"headers,omitempty"`
	ExpiresAt time.Time         `json:"expires_at"`
}

PresignResponse is what a plugin answers to either presign call. Only URL and ExpiresAt are required; the rest describe how the client must send an upload.

type ProbeResult

type ProbeResult struct {
	// Name is the capability or behaviour probed ("read", "write", "range"…).
	Name string `json:"name"`
	// Status is pass | fail | skip.
	Status string `json:"status"`
	// Detail says what happened — for a failure, precisely enough to fix it.
	Detail string `json:"detail,omitempty"`
	// Took is how long the probe ran. ⚠ NOT serialised: encoding/json has no
	// marshaller for time.Duration and writes the raw int64, so a field named
	// took_ms carrying a Duration is off by a million — measured by the admin
	// page reading nanoseconds as milliseconds. TookMS below is what goes on
	// the wire, and it really is milliseconds.
	Took   time.Duration `json:"-"`
	TookMS int64         `json:"took_ms"`
}

ProbeResult is one capability's verdict.

type Process

type Process struct {
	Name    string
	Binary  string // absolute path
	Token   string
	SockDir string // private directory the plugin may create its socket in
	Log     *slog.Logger

	// OnUp is called (from the supervisor goroutine) each time the plugin has
	// printed its handshake and answered nothing yet — the manager describes
	// it here and decides whether it is acceptable. Returning an error stops
	// the plugin for good (a plugin that fails validation will fail it again;
	// restarting would only churn).
	OnUp func(ctx context.Context, c *Client) error
	// OnDown is called each time the process is gone (exit, kill, failed
	// handshake), before any restart.
	OnDown func(err error)
	// contains filtered or unexported fields
}

Process supervises ONE launched plugin binary: start it, wait for the handshake line, hand the client to whoever asked, restart it with backoff when it dies, stop it on request. It knows nothing about storage — the manager layers registration on top through the callbacks.

func (*Process) Client

func (p *Process) Client() *Client

Client is the live client, or nil while the plugin is not up.

func (*Process) Err

func (p *Process) Err() error

Err is the last failure the supervisor saw (nil while healthy).

func (*Process) Restarts

func (p *Process) Restarts() int

Restarts counts crashes since Start.

func (*Process) Start

func (p *Process) Start(ctx context.Context)

Start runs the supervise loop until ctx ends or Stop is called. It returns immediately; failures surface through OnDown and Err.

func (*Process) Stop

func (p *Process) Stop()

Stop terminates the plugin and waits (bounded) for the loop to end.

type RejectedError

type RejectedError struct {
	// contains filtered or unexported fields
}

RejectedError marks a failure caused by what the CALLER supplied — a missing signature, one that does not verify — rather than by filex failing.

⚠ This exists because the handler used to classify these by SEARCHING THE MESSAGE for words like "required" or "sha256". Under that scheme "this instance only accepts signed plugins …" happened to contain "sha256" and answered 400, while "signature does not verify against any trusted key" matched nothing and answered 500 — the same class of mistake, reported as two different kinds of event, one of them blamed on the server. A rejected upload is not an outage; it should not spend the server's error budget or page anybody.

func (RejectedError) Error

func (e RejectedError) Error() string

func (RejectedError) Unwrap

func (e RejectedError) Unwrap() error

type Report

type Report struct {
	// Verified is true when every declared capability passed.
	Verified bool `json:"verified"`
	// Scratch says where the probes ran: "selftest" (the plugin's own
	// throwaway area) or "storage" (a folder inside a real storage).
	Scratch string        `json:"scratch"`
	Results []ProbeResult `json:"results"`
	RanAt   time.Time     `json:"ran_at"`
}

Report is the outcome of a conformance run.

func RunConformance

func RunConformance(ctx context.Context, drv storage.Driver, caps Capabilities, base, scratchKind string) *Report

RunConformance probes every capability caps declares, using drv as the driver under test and base as a directory the probes may write inside.

It never leaves anything behind that it created (best effort — a plugin whose delete is broken is exactly what the delete probe reports).

func VerifyStorage

func VerifyStorage(ctx context.Context, drv storage.Driver, caps Capabilities) *Report

VerifyStorage probes a driver against a REAL storage configuration, inside a scratch folder it creates and removes.

This is the second gate. The first (a plugin's selftest at install) proves the code works; this one proves it works against the configuration an operator just typed — the credentials, the bucket, the path. A plugin can be perfectly correct and the storage still be unusable, and finding that out when the storage is SAVED is much cheaper than finding out when a user uploads a file into it.

caps must be the plugin's declared capability set. Returns the report; the caller decides what to do with a failure.

func (*Report) FailureError

func (r *Report) FailureError() error

FailureError turns the failures into one error whose message a person can act on. Nil when nothing failed.

func (*Report) Failures

func (r *Report) Failures() []ProbeResult

Failures returns the probes that failed, most useful first.

func (*Report) Summary

func (r *Report) Summary() string

Summary is the one-line form for a log or an error message.

type Stats

type Stats struct {
	InFlight int64 `json:"in_flight"`
	Waited   int64 `json:"waited"`
	Rejected int64 `json:"rejected"`
	Max      int   `json:"max_in_flight"`
}

Stats is what the admin surface and Prometheus read.

type Status

type Status struct {
	*model.Plugin
	State        string        `json:"state"`
	StateError   string        `json:"state_error,omitempty"`
	Restarts     int           `json:"restarts"`
	Capabilities *Capabilities `json:"capabilities,omitempty"`
	Label        string        `json:"label,omitempty"`
	FieldCount   int           `json:"field_count"`
	// InUse counts storage rows on this plugin's driver — shown before a
	// remove so the admin knows what will stop working.
	InUse int `json:"in_use"`
	// Conformance is the last verification of the plugin's own claims. Nil
	// when it has not been probed (an older row, or probing is off).
	Conformance *Report `json:"conformance,omitempty"`
	// Load is what this plugin is doing right now: in-flight operations, and
	// how often callers have had to wait or been refused a slot. A plugin
	// that is merely SLOW shows up here long before it shows up as an error.
	Load Stats `json:"load"`
}

Status is one plugin as the admin sees it: the row plus what is happening.

Directories

Path Synopsis
Command gen writes driver_shapes.go: one concrete type per capability combination a plugin can declare.
Command gen writes driver_shapes.go: one concrete type per capability combination a plugin can declare.
Package testplugin runs a real filex storage plugin — the HTTP/JSON protocol of internal/plugin, served over httptest — so any package that consumes a storage.Driver can be exercised against a PLUGIN-backed storage without spawning a second process.
Package testplugin runs a real filex storage plugin — the HTTP/JSON protocol of internal/plugin, served over httptest — so any package that consumes a storage.Driver can be exercised against a PLUGIN-backed storage without spawning a second process.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL