Documentation
¶
Overview ¶
Package plugin is the OpenRun Starlark plugin provider SDK.
An OpenRun plugin provider is a standalone executable that serves one or more Starlark plugin modules to the OpenRun server over gRPC (hashicorp/go-plugin). Apps load provider-served modules with the .ex suffix — load("notes.ex", "notes") — and use them exactly like builtin .in plugins: the same permission approvals, the same plugin_response error handling, and the same request-end resource cleanup apply. The SDK has no Starlark dependency; plugin functions are plain Go methods over plain Go values, and the server converts Starlark values at the process boundary with full fidelity (exact big ints, int/float distinction, dict order, tuples, sets, bytes, typed records).
Writing a provider ¶
A provider's main function calls Serve with a ServeConfig declaring its modules. Each ModuleDef names a builder and the module's functions:
func main() {
plugin.Serve(&plugin.ServeConfig{
ProviderVersion: version,
Modules: map[string]plugin.ModuleDef{
"notes": { // served to apps as "notes.ex"
Builder: NewNotesModule,
Functions: []plugin.FuncDef{
{Name: "add", Type: plugin.WRITE, Method: "Add"},
{Name: "list", Type: plugin.READ, Method: "List"},
},
},
},
})
}
The builder returns a Module; one instance is created per (module, account) pair in each provider process. Module.InitModule receives a ModuleInit with the app identity, the per-account settings from openrun.toml (secrets already expanded by the server), and the raw schema.star bytes for schema-aware plugins.
Every declared function must be an exported method with the Func signature; Serve validates this at startup:
func (m *NotesModule) Add(ctx context.Context, call *plugin.Call) (any, error) {
var text string
if err := plugin.UnpackArgs("add", call, "text", &text); err != nil {
return nil, err
}
...
return id, nil
}
UnpackArgs binds positional and keyword arguments with starlark.UnpackArgs semantics. The returned value becomes response.value in the app; a returned error becomes response.error (use ErrorWithCode for an explicit error_code) and feeds OpenRun's automatic error handling.
Sessions, cursors, and resource lifetimes ¶
All calls made during one app request share Call.Session. Session state (Session.Set, Session.Get) and deferred cleanup (Session.Defer, Session.ClearDefer) scope cross-call resources such as transactions to the request; remaining defers run when the request ends.
A function may return a *Cursor to stream results lazily: the app receives an iterable, batches are fetched on demand, and a cursor the app never consumes is closed at request end and reported as a leaked resource, failing the request under the cursor's LeakKey.
The one rule stateful providers must follow: resources that outlive a single call — a sql.Tx, the sql.Rows behind a Cursor — must be created on Session.Context, not the per-call context. The per-call gRPC context is cancelled when the call returns, which would roll back the transaction or close the rows before the next call arrives.
How the server runs providers ¶
At registration time the server launches the provider briefly and calls Describe; the returned module manifests (functions, read/write flags, constants) are what app loading and permission auditing use, so no provider process runs until an app actually calls the plugin. At runtime the server keeps one provider process per app (mutual-TLS gRPC over a unix socket, binary checksum verified at launch), initializes it once via InitApp/InitModule, and stops it when the app is closed or reloaded. All permission checks and secret expansion happen server-side before a call is dispatched.
Application errors travel inside responses; a gRPC transport error means the provider process failed, its sessions are lost, and calls are never retried automatically.
The out-of-process build of the OpenRun store plugin (internal/app/store/storeprovider in the OpenRun repository) is the reference provider implementation, covering settings, schema access, transactions, and cursors.
Index ¶
- Constants
- Variables
- func DecodeValue(v *pb.StarValue) (any, error)
- func DecodeValueMap(m map[string]*pb.StarValue) (map[string]any, error)
- func EmbeddedProviders() map[string]*ServeConfig
- func EncodeValue(v any) (*pb.StarValue, error)
- func EncodeValueMap(m map[string]any) (map[string]*pb.StarValue, error)
- func ErrorWithCode(code int64, err error) error
- func RegisterEmbedded(name string, config *ServeConfig)
- func Serve(config *ServeConfig)
- func UnpackArgs(fnName string, call *Call, pairs ...any) error
- type AppInfo
- type Call
- type Client
- type Cursor
- type CursorInfo
- type Dict
- type DictEntry
- type Download
- type Func
- type FuncDef
- type FuncRef
- type FunctionType
- type Host
- func (h *Host) Call(ctx context.Context, call *HostCall) (*HostResult, error)
- func (h *Host) Close(ctx context.Context)
- func (h *Host) Config() *ServeConfig
- func (h *Host) CursorClose(ctx context.Context, sessionId, cursorId string) error
- func (h *Host) CursorNext(ctx context.Context, sessionId, cursorId string, max int) ([]any, bool, error)
- func (h *Host) DetachCursor(sessionId, cursorId string) (*Cursor, error)
- func (h *Host) EndSession(ctx context.Context, sessionId string) error
- func (h *Host) InitApp(info AppInfo) error
- func (h *Host) InitModule(ctx context.Context, module, account string, settings map[string]any) error
- func (h *Host) Retire()
- func (h *Host) StartSession(sessionId string)
- type HostCall
- type HostResult
- type HostServices
- type Kwarg
- type LaunchConfig
- type Logger
- type Module
- type ModuleDef
- type ModuleInit
- type Provider
- func (p *Provider) Call(ctx context.Context, req *pb.CallRequest) (*pb.CallResponse, error)
- func (p *Provider) CheckHealth(ctx context.Context) error
- func (p *Provider) CursorClose(ctx context.Context, req *pb.CursorCloseRequest) error
- func (p *Provider) CursorNext(ctx context.Context, req *pb.CursorNextRequest) (*pb.CursorNextResponse, error)
- func (p *Provider) Describe(ctx context.Context) (*pb.DescribeResponse, error)
- func (p *Provider) EndSession(ctx context.Context, req *pb.EndSessionRequest) error
- func (p *Provider) Exited() bool
- func (p *Provider) InitApp(ctx context.Context, req *pb.InitAppRequest) error
- func (p *Provider) InitModule(ctx context.Context, req *pb.InitModuleRequest) error
- func (p *Provider) Kill()
- type ProviderError
- type ServeConfig
- type Session
- func (s *Session) ClearDefer(key string)
- func (s *Session) Context() context.Context
- func (s *Session) Defer(key string, strict bool, fn func(ctx context.Context) error)
- func (s *Session) End(ctx context.Context) error
- func (s *Session) Get(key string) any
- func (s *Session) Id() string
- func (s *Session) Set(key string, value any)
- type Set
- type Struct
- type ThreadState
- type Thunk
- type Tuple
Constants ¶
const MaxMessageSize = 64 * 1024 * 1024
MaxMessageSize is the gRPC send/receive message limit on both sides of the provider connection. Larger results must use a Cursor.
const MaxValueDepth = 100
MaxValueDepth bounds StarValue nesting. Starlark collections are mutable and can be made cyclic; the encoder fails cleanly instead of recursing forever.
const PluginName = "starplugin"
PluginName is the go-plugin dispense name for the plugin provider plugin.
const ProtocolVersion = 1
ProtocolVersion is the go-plugin protocol version for the v1 Starlark plugin provider protocol. Incompatible protocol changes bump this and are served side by side through VersionedPlugins during a transition.
Variables ¶
var ( ErrAppAlreadyInited = errors.New("app already initialized in this provider process") ErrAppNotInited = errors.New("app not initialized") ErrUnknownModule = errors.New("unknown module") ErrModuleNotInited = errors.New("module not initialized") ErrUnknownFunction = errors.New("unknown function") )
Typed errors the gRPC shim maps to protocol-level status errors; every other error is an application-level failure carried in response fields.
var Handshake = goplugin.HandshakeConfig{ ProtocolVersion: ProtocolVersion, MagicCookieKey: "OPENRUN_PLUGIN_PROVIDER", MagicCookieValue: "8e41c2d7-openrun-starlark-plugin-provider", }
Handshake is the go-plugin handshake shared by the server and providers. The magic cookie is a sanity check that the launched executable is a Starlark plugin provider (not, for example, a binding provider); it is not a security measure.
Functions ¶
func DecodeValue ¶
DecodeValue converts a wire value into its Go representation: nil, bool, int64, *big.Int, float64, string, []byte, []any, Tuple, Set, map[string]any (string-keyed dicts) or *Dict, *Struct, and time.Time.
func DecodeValueMap ¶
DecodeValueMap decodes a settings-style map.
func EmbeddedProviders ¶
func EmbeddedProviders() map[string]*ServeConfig
EmbeddedProviders returns the registered embedded providers by name.
func EncodeValue ¶
EncodeValue converts a Go value into its wire representation. It accepts the value shapes plugin functions return: nil, bool, all int/uint widths, *big.Int, float32/64, string, []byte, time.Time, Tuple, Set, *Dict/Dict, *Struct/Struct, []any (and common typed slices), and map[string]any (and common typed maps). Cursors are not encoded here; the provider serve loop registers them and encodes a cursor handle.
func EncodeValueMap ¶
EncodeValueMap encodes a settings-style map.
func ErrorWithCode ¶
ErrorWithCode returns an error that carries a plugin error code, surfaced to Starlark as response.error_code.
func RegisterEmbedded ¶
func RegisterEmbedded(name string, config *ServeConfig)
RegisterEmbedded registers a provider config to be served in-process by an OpenRun binary this package is compiled into. Call it from an init function; the config is validated and conflicts (same provider name, or a module already served by another embedded provider) panic, so a bad custom build fails at startup rather than on a user request.
func Serve ¶
func Serve(config *ServeConfig)
Serve runs the provider plugin. It is called from a provider executable's main function and blocks until the server side closes the plugin. The OPENRUN_PROVIDER_LOG_LEVEL environment variable (set by the server from its own log level) controls provider log verbosity.
func UnpackArgs ¶
UnpackArgs binds a call's positional and keyword arguments to Go variables, mirroring starlark.UnpackArgs: pairs alternate a parameter name and a pointer. A name ending in "?" marks an optional parameter. Supported pointer types: *string, *bool, *int64, *[]string, *[]any, *map[string]any, **Struct, and *any.
Types ¶
type AppInfo ¶
type AppInfo struct {
AppId string
AppPath string
IsDev bool
AppSchema []byte // raw schema.star contents, nil if the app has none
}
AppInfo identifies the app a Host serves.
type Call ¶
type Call struct {
Function string
Args []any
Kwargs []Kwarg
Thread ThreadState
Session *Session
// Host is non-nil only when the module runs in-process. See HostServices.
Host HostServices
}
Call carries one plugin function invocation.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is the typed gRPC client for the provider protocol, dispensed by go-plugin. Server code uses Provider, which wraps process lifecycle around it.
type Cursor ¶
type Cursor struct {
// TypeName names the iterable's Starlark type ("<TypeName> iterator").
TypeName string
// Stream marks a cursor that the app returns from its handler as a
// streaming HTTP response (response.is_stream), instead of iterating it
// in Starlark. A stream cursor is detached from its session when the
// call returns — it is consumed after the request's plugin cleanup runs —
// and no leak entry is registered for it. Supported for in-process
// modules only; an external provider returning a stream cursor fails
// the call.
Stream bool
// LeakKey names the strict cleanup entry the server registers for this
// cursor, e.g. "rows_cursor_<table>_0x...". If the app never consumes or
// closes the cursor, the request fails citing this key.
LeakKey string
// Next returns up to max items and whether iteration is complete. The
// provider closes the cursor itself when it reports done.
Next func(ctx context.Context, max int) ([]any, bool, error)
// Close releases the cursor before exhaustion.
Close func(ctx context.Context) error
}
Cursor is a lazy result iterator that stays in the provider process, scoped to the session of the Call that returned it. The server wraps it in a Starlark iterable; an unconsumed cursor fails the request through the strict deferred-cleanup mechanism, under the name LeakKey.
func PushCursor ¶
func PushCursor(typeName, leakKey string, stream bool, seq func(yield func(any, error) bool)) *Cursor
PushCursor adapts a push-style stream function (repeatedly calling yield with values, like a range-over-func iterator) into a pull-based Cursor. The stream function runs in its own goroutine, started lazily on the first Next; it stops when it returns, yields an error, or the cursor is closed. Next returns the first available item promptly (it does not wait to fill a batch), so live streams flush without delay.
type CursorInfo ¶
CursorInfo is the handle for a cursor returned by a call, to be drained with CursorNext / released with CursorClose in the same session (or, for a stream cursor, detached with DetachCursor and consumed directly).
type Dict ¶
type Dict struct {
Entries []DictEntry
}
Dict is an order-preserving Starlark dict. Incoming dicts whose keys are all strings arrive as map[string]any instead; Dict is used for dicts with non-string keys, and may be returned by a function that needs to control the entry order of a returned dict.
type Download ¶
Download is a file download whose content is produced at response-write time: the app passes it to ace.response(content, download=name) and the producer writes the bytes directly to the HTTP response, after the request's plugin cleanup has run. Supported for in-process modules only; an external provider returning a Download fails the call.
type FuncDef ¶
type FuncDef struct {
Name string
Type FunctionType
Method string
}
FuncDef declares one plugin function: its Starlark name, read/write classification, and the Go method (on the Module implementation) that handles it. The method must have the Func signature.
type FuncRef ¶
FuncRef is a value the server materializes as a zero-argument callable in the app: calling it invokes the named plugin function with Args, in the same module, account, and session as the call that returned it. Use it as a Struct field for members whose result is computed lazily against live plugin state, e.g. the http plugin's response.body() reading a response held open in the session. Function names starting with "_" are internal: they are declared in the module's Functions list (so the method binding is validated) but are not exposed as module attributes, and are callable only through a FuncRef.
type FunctionType ¶
type FunctionType int
FunctionType classifies a plugin function as a read or a write operation. Writes are blocked for stage/preview apps without write access.
const ( READ FunctionType = iota WRITE )
type Host ¶
type Host struct {
// contains filtered or unexported fields
}
Host runs a provider's modules for one app, on plain Go values, with no transport involved. It owns the module instances (one per (module, account) pair), the sessions with their deferred cleanups, and the cursors returned by calls. Both plugin transports are thin layers over it: the gRPC providerServer (external providers) decodes wire values at its edges, and the OpenRun server embeds one Host per (app, provider) for internal plugins, converting Starlark values with the direct bridge. Keeping the semantics here — lazy module init, session-scoped state, strict-leak reporting, cursor batching, module Close on shutdown — is what guarantees a plugin behaves identically internal and external.
func NewHost ¶
func NewHost(config *ServeConfig, logger *Logger) (*Host, error)
NewHost validates the config and returns a Host. The logger may be nil.
func (*Host) Call ¶
Call invokes one plugin function. An error returned by the function is returned as-is (possibly a *ProviderError carrying an error code), with the result's StrictKeys still valid; ErrModuleNotInited / ErrUnknownFunction signal caller bugs.
func (*Host) Close ¶
Close calls Close on every initialized module instance, logging failures. Used at provider process shutdown and at app close/reload for in-process hosts.
func (*Host) Config ¶
func (h *Host) Config() *ServeConfig
Config returns the provider config the Host serves.
func (*Host) CursorClose ¶
CursorClose releases a cursor before exhaustion. Closing an unknown cursor or session is not an error.
func (*Host) CursorNext ¶
func (h *Host) CursorNext(ctx context.Context, sessionId, cursorId string, max int) ([]any, bool, error)
CursorNext returns up to max items from a cursor and whether iteration is complete. When done, the cursor and its session cleanup are forgotten (the cursor released its own resources).
func (*Host) DetachCursor ¶
DetachCursor removes a cursor from its session and returns it for direct consumption. Used for stream cursors, which are consumed after the request's session has ended; the caller owns closing the cursor.
func (*Host) EndSession ¶
EndSession runs the session's remaining deferred cleanups and forgets it. On a retired host, the last session ending closes the host's modules.
func (*Host) InitModule ¶
func (h *Host) InitModule(ctx context.Context, module, account string, settings map[string]any) error
InitModule initializes one (module, account) instance, if not already initialized. Settings are the per-account plugin settings with secrets expanded. An error returned by the module's InitModule is returned as-is. Concurrent first calls for the same (module, account) are serialized: one runs the initialization, the others wait for its outcome, so exactly one instance is ever built (a failed init is forgotten, so a later call retries fresh).
func (*Host) Retire ¶
func (h *Host) Retire()
Retire marks a host that is being replaced (app reload): in-flight requests finish on the old module instances, and the modules are closed when the last active session ends (immediately when none are active). New requests go to the replacement host; a straggler call racing the retire still works — InitModule lazily rebuilds the instance and the session-end close runs again.
func (*Host) StartSession ¶
StartSession registers a request session before its first call. Callers register the session at host-acquisition time so a concurrent Retire (app reload) counts the request as active and keeps the host open until the matching EndSession, instead of closing the modules mid-request.
type HostCall ¶
type HostCall struct {
Module string
Account string
Function string
Args []any
Kwargs []Kwarg
Thread ThreadState
SessionId string
// Host provides host-process services to in-process modules; nil for
// external provider processes.
Host HostServices
}
HostCall is one plugin function invocation at the Host level.
type HostResult ¶
type HostResult struct {
Value any
Cursor *CursorInfo // set instead of Value when the function returned a *Cursor
StrictKeys []string
}
HostResult is the outcome of one call. StrictKeys is always populated (even alongside an error) so the caller can mirror strict deferred-cleanup entries after every call.
type HostServices ¶
type HostServices interface {
// Value returns a host-scoped value by key, nil if not set. Keys are
// host-defined (e.g. the container plugin's request container handler).
Value(key string) any
}
HostServices exposes host-process services to a module running in-process (compiled into the OpenRun binary). It is nil when the module runs in an external provider process, so a module using it is host-bound: it works internal-only and must fail gracefully when Host is nil.
type LaunchConfig ¶
type LaunchConfig struct {
// ExecPath is the provider executable.
ExecPath string
// Logger receives go-plugin lifecycle logs and the provider's forwarded
// log lines. Nil uses go-plugin's default (stderr).
Logger hclog.Logger
// LogLevel is passed to the provider via OPENRUN_PROVIDER_LOG_LEVEL.
LogLevel string
// SecureConfig, when set, verifies the executable's checksum before
// launch.
SecureConfig *goplugin.SecureConfig
}
LaunchConfig configures launching a provider process.
type Logger ¶
Logger is the logger passed to provider implementations. It mirrors the OpenRun server's logger type (a zerolog wrapper) so binding code written against the server's internal interface ports without changes.
type Module ¶
type Module interface {
// InitModule initializes the instance before its first call.
InitModule(ctx context.Context, init ModuleInit) error
// Close releases the instance's resources. Called at process shutdown.
Close(ctx context.Context) error
}
Module is a plugin module instance, scoped to one (module, account) pair within one app's provider process.
type ModuleDef ¶
type ModuleDef struct {
// Builder returns a new module instance. One instance is created per
// (module, account) pair in each provider process (one process per app).
Builder func() Module
// Functions declares the module's plugin functions.
Functions []FuncDef
// Constants are exported as module constants, e.g. store.MAX_LIMIT.
Constants map[string]any
}
ModuleDef declares one Starlark module served by a provider. A module named "store" is loaded by apps as load("store.ex", "store").
type ModuleInit ¶
type ModuleInit struct {
AppId string
AppPath string
Account string // "" for the default account
IsDev bool
// AppSchema is the raw contents of the app's schema.star, nil if the app
// has none.
AppSchema []byte
// Settings are the per-account plugin settings from the server config
// (e.g. [plugin."store.ex#myaccount"]), with secrets already expanded.
Settings map[string]any
Logger *Logger
}
ModuleInit carries the app and account context for a module instance.
type Provider ¶
type Provider struct {
// contains filtered or unexported fields
}
Provider is a running plugin provider process, launched by the OpenRun server. Application-level failures are returned as *ProviderError; any other error is a transport failure (the process died or broke protocol) and the server treats the process and its sessions as dead.
func LaunchProvider ¶
func LaunchProvider(config LaunchConfig) (*Provider, error)
LaunchProvider starts the provider executable and completes the go-plugin handshake. The returned Provider must be closed with Kill.
func (*Provider) Call ¶
func (p *Provider) Call(ctx context.Context, req *pb.CallRequest) (*pb.CallResponse, error)
Call invokes a plugin function. An application-level failure is returned as *ProviderError together with the (possibly nil) response; a transport failure is any other error.
func (*Provider) CheckHealth ¶
CheckHealth reports whether the provider process is responsive.
func (*Provider) CursorClose ¶
CursorClose releases a cursor before exhaustion. Closing an unknown cursor or session is not an error.
func (*Provider) CursorNext ¶
func (p *Provider) CursorNext(ctx context.Context, req *pb.CursorNextRequest) (*pb.CursorNextResponse, error)
CursorNext fetches the next batch of items from a cursor returned by a previous Call in the same session.
func (*Provider) Describe ¶
Describe reports the provider's version and module manifests. Valid before InitApp; used at registration time to capture the manifests.
func (*Provider) EndSession ¶
EndSession releases all provider-side state for a session: remaining deferred cleanups run and the session is forgotten. Called by the server when the request handler finishes.
func (*Provider) InitApp ¶
InitApp establishes the app identity for this provider process. Called once per process, immediately after launch.
func (*Provider) InitModule ¶
InitModule initializes one (module, account) instance in the provider process, called lazily before the instance's first Call.
type ProviderError ¶
type ProviderError struct {
Message string
Code int64 // plugin error code, 1 unless the function set one
}
ProviderError is an application-level error reported by a provider: the plugin function ran and returned a failure. Transport-level failures (provider crashed, protocol error) are returned as ordinary gRPC errors instead; the server treats those as fatal for the provider process.
func (*ProviderError) Error ¶
func (e *ProviderError) Error() string
type ServeConfig ¶
type ServeConfig struct {
// Modules maps each module name served by this provider to its
// definition. A module "store" is loaded by apps as "store.in" (or
// "store.ex" to require the external build).
Modules map[string]ModuleDef
// ProviderVersion is the provider's release version, reported via
// Describe.
ProviderVersion string
}
ServeConfig configures a Starlark plugin provider process.
type Session ¶
type Session struct {
// contains filtered or unexported fields
}
Session groups the plugin calls of one app request. Cross-call state (transactions, handles) and deferred cleanup are scoped to it; the server ends the session when the request handler finishes.
func NewSession ¶
NewSession creates a standalone session, for unit tests of plugin modules. In a provider the host creates and ends sessions itself.
func (*Session) ClearDefer ¶
ClearDefer removes a previously registered cleanup, typically after the resource was released explicitly (e.g. a transaction was committed).
func (*Session) Context ¶
Context returns a context that stays alive until the session ends. Use it (not the per-call context) for resources that outlive one call, such as the query backing a returned Cursor: the per-call gRPC context is cancelled when the call returns, which would invalidate the resource before the next CursorNext arrives.
func (*Session) Defer ¶
Defer registers a cleanup function run when the session ends (unless cleared first). Cleanups run in reverse registration order. Strict entries are mirrored to the server after every call: a strict entry still registered when the request ends fails the request as a leaked resource, matching the strict cleanup contract of builtin plugins. Use strict for resources the app is required to consume or release explicitly, and ClearDefer when it does.
func (*Session) End ¶
End runs the session's remaining deferred cleanups, for unit tests of plugin modules. In a provider the host ends sessions itself.
type Struct ¶
Struct is a schema-typed record. The server materializes returned Structs as typed Starlark values with attribute access (row.field), matching what builtin plugins return; typed values passed as arguments arrive as Structs.
type ThreadState ¶
type ThreadState struct {
RequestId string
UserId string
UserSubject string
UserEmail string
Groups []string
AppUrl string
}
ThreadState snapshots the request-scoped state a plugin function may read.
type Thunk ¶
Thunk is a value the server materializes as a zero-argument callable in the app: calling it returns Value, or fails with Error if set. Use it as a Struct field to return a record with callable members over pre-computed data. The value must be transportable (EncodeValue rules); Name names the callable in error messages.