officeengine

package
v0.60.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 12, 2026 License: MIT Imports: 35 Imported by: 0

Documentation

Overview

Package officeengine runs an app's office engine on this server: the Office editor app's x2t (ONLYOFFICE's converter) and its page layout (sdkjs in QuickJS), both compiled to WebAssembly and shipped in the app's release as engine.zip (docs/ONLYOFFICE.md → The office engine without a Document Server, docs/APP-PLUGINS-API.md → engines.office).

It is the second back end of the office engine apps convert with (`engines:office`) and of office thumbnails, beside the ONLYOFFICE Document Server (internal/onlyoffice). The server picks one of them (internal/server, the `engine` capability of Default apps); this package only runs the app's.

Two sides, one binary

filex never links the engine's code (AGPL): it runs it as a separate WebAssembly program, files in and files out, the way it would run a Document Server's x2t. The PARENT side (Runner, in the server) starts `filex office-engine` - a hidden command of this same binary, the CHILD side (Serve) - and talks to it over the child's stdin and stdout, one JSON line per job (Request) and one per answer (Answer). The child holds the wazero runtimes: x2t-server.wasm, an instance per job, mounted on a directory of its own; and a warm layout worker per kind of document (layout-host.wasm running the app's worker.js over sdkjs).

Why a process of its own

  • Speed: both modules are compiled WITHOUT wazero's context watch (2-4.6 times faster on x2t, measured), and such a module can only be stopped by ending its process. The parent keeps every job's deadline and kills the process when it passes; the next job starts another.
  • Memory: x2t takes 284 MiB at the least and 2.6 GiB for a 200,000-row csv. In a process of its own that memory, a trap and a loop go back to the system with the process, never into filex's heap.

Memory

Every instance is capped (FILEX_OFFICE_ENGINE_MAX_MB, MaxMBDefault): its memory may not grow past the cap. x2t's memory is reserved up to the cap when the instance starts (memory.go), so growing it never copies it - the 200,000-row csv peaked at 6.2 GB of process with copies, 2.6 GiB without. The reservation is address space outside Go's heap: the system backs only what x2t writes and takes it all back when the instance closes, so job after job the process holds what the job used, not the cap. The parent also watches the process as a whole where the system lets it read a process's resident memory (proc_*.go): about twice the cap ends it. A layout worker is started again after WorkerJobs jobs or WorkerMaxBytes of memory; a process left idle for Options.IdleTimeout ends.

Compiling, and "preparing"

wazero compiles x2t-server.wasm in 3.5-5 minutes the first time on a machine; its cache (in filex's data directory) loads it in about a second in every process after that. So a new engine package - an install, an upgrade, a new filex - is PREPARED in the background by one process that compiles both modules and starts a worker of each kind once (which writes the layout's bytecode); until it is done the engine is not Ready, its Status says "preparing", and a Document Server, if one is connected, takes the jobs meanwhile.

Index

Constants

View Source
const (
	EngineX2T    = "x2t-server.wasm"
	EngineLayout = "layout-host.wasm"
	EngineWorker = "layout/worker.js"
)

What engine.zip holds (docs/APP-PLUGINS-API.md → engines.office).

View Source
const (
	KindWord  = "word"
	KindCell  = "cell"
	KindSlide = "slide"
)

Kinds of document, as the Document Server's documentType names them.

View Source
const (
	WorkerJobs     = 20
	WorkerMaxBytes = 1536 << 20
)

Worker renewal.

View Source
const (
	// OpConvert turns In (From) into To, written at Out.
	OpConvert = "convert"
	// OpThumbnail draws In's first page as a PNG fitted into Size x Size, at
	// Out.
	OpThumbnail = "thumbnail"
)

The operations.

View Source
const (
	// CodeUnknown: the engine process ended without an answer, or a failure
	// that may pass (asked again later).
	CodeUnknown = -1
	// CodeTimeout: the job's deadline passed (the parent ended the process).
	CodeTimeout = -2
	// CodeCorrupt: the document could not be read or converted (x2t's 80, 84,
	// 86, a trap, a layout that failed). The same bytes fail again.
	CodeCorrupt = -3
	// CodePassword: the document is protected by a password (x2t's 90, 91)
	// or by DRM (85).
	CodePassword = -5
	// CodeParams: this format is not made from that one (x2t's 82, 88).
	CodeParams = -7
	// CodeDetect: what the input is could not be told (x2t's 89, 95).
	CodeDetect = -9
	// CodeTooLarge: the memory cap, or x2t's own size limits (93, 94, 96).
	CodeTooLarge = -10
)

The failures, as the Document Server's conversion API numbers them (internal/onlyoffice DS*): the server maps both back ends' failures with one table (onlyoffice.ClassOf) and says them with one set of sentences.

View Source
const (
	MaxMBDefault = 2048
	MaxMBMin     = 512
	MaxMBMax     = 4096
)

The memory cap (FILEX_OFFICE_ENGINE_MAX_MB): every instance's memory, in MiB. x2t starts at 320 MiB and may use 4 GiB, its own maximum.

View Source
const ConfigEnv = "FILEX_OFFICE_ENGINE_CONFIG"

ConfigEnv carries the ChildConfig (JSON) to an engine process: the command line stays short and carries no path a process listing would show.

Variables

View Source
var (
	// ErrNoEngine: no running app brings an office engine.
	ErrNoEngine = errors.New("office engine: no app brings one")
	// ErrPreparing: the engine is being compiled (Status says since when).
	ErrPreparing = errors.New("office engine: being prepared")
	// ErrFailed: the engine could not be prepared (Status says why).
	ErrFailed = errors.New("office engine: could not be prepared")
	// ErrClosed: the runner was closed (filex is stopping).
	ErrClosed = errors.New("office engine: closed")
)

Why a job was not even started.

Functions

func ClampMaxMB

func ClampMaxMB(mb int) int

ClampMaxMB is the cap in force for a configured value (0: the default).

func KindOf

func KindOf(ext string) string

KindOf is the editor a document of this extension opens in, "" for none: the kinds the Document Server lays out (a PDF, a picture), and what x2t reads into each editor's format.

func Main

func Main(stdin io.Reader, stdout io.Writer) int

Main is `filex office-engine`: it reads its ChildConfig from ConfigEnv and serves jobs on stdin and stdout until stdin closes. The exit code is 0 for a process that ended because filex closed it (or a Prepare run that finished), 1 for one that could not start.

func Serve

func Serve(ctx context.Context, cfg ChildConfig, in io.Reader, out io.Writer) error

Serve is the engine process's whole life: open the packages, compile both modules, say Hello, then answer one job per line of in.

Types

type Answer

type Answer struct {
	ID int64 `json:"id"`
	OK bool  `json:"ok"`
	// Bytes is the result's size; Pages the pages the layout drew.
	Bytes int64 `json:"bytes,omitempty"`
	Pages int   `json:"pages,omitempty"`
	// Code is a failure as the Document Server's conversion API numbers it
	// (the Code* constants), so a caller says both back ends' failures the
	// same way; What is its short name for a record ("x2t-86", "trap",
	// "memory"); Error the longer English, for a log.
	Code  int    `json:"code,omitempty"`
	What  string `json:"what,omitempty"`
	Error string `json:"error,omitempty"`
	// Over: the result was larger than MaxBytes, and nothing was written.
	Over bool `json:"over,omitempty"`
	// Ms is where the time went (to_bin, layout, from_bin, convert).
	Ms map[string]int64 `json:"ms,omitempty"`
}

Answer is what came of one Request.

type ChildConfig

type ChildConfig struct {
	// Engine is the app's engine.zip, EngineSHA256 the hash it must have.
	Engine       string `json:"engine"`
	EngineSHA256 string `json:"engine_sha256"`
	// Package is the app's interface bundle (ui.zip), which holds the
	// editor's sdkjs/ and fonts/ under EditorDir; mounted read-only.
	Package       string `json:"package"`
	PackageSHA256 string `json:"package_sha256"`
	EditorDir     string `json:"editor_dir"`
	// Cache is the compilation cache's root (wazero's, and the layout's
	// bytecode), shared by every engine process of this filex.
	Cache string `json:"cache"`
	// Work is the process's own temporary directory; the parent removes it
	// when the process ends.
	Work string `json:"work"`
	// MaxMB caps every instance's memory (MiB).
	MaxMB int `json:"max_mb"`
	// Prepare: compile, start a worker of each kind once, say Prepared and
	// end - no jobs.
	Prepare bool `json:"prepare,omitempty"`
}

ChildConfig is what an engine process is started with.

type Conversion

type Conversion struct {
	// Src is the input, From its format; To the conversion API's output type.
	Src, From, To string
	// Delimiter and CodePage are the conversion API's CSV knobs.
	Delimiter, CodePage int
	// Dst is where the result goes (it must not exist); MaxBytes its ceiling.
	Dst      string
	MaxBytes int64
}

Conversion is one office conversion (the office engine apps use).

type Error

type Error struct {
	// Code is the conversion API's number (Code*), What its short name for a
	// record ("x2t-86", "trap", "memory", "timeout", "crash").
	Code int
	What string
	// Over: the result was larger than the request's MaxBytes.
	Over bool
	Err  error
}

Error is a job the engine did not do.

func AsError

func AsError(err error) (*Error, bool)

AsError returns err as an *Error.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Unwrap

func (e *Error) Unwrap() error

type Hello

type Hello struct {
	// Ready: the modules are compiled, jobs may come.
	Ready bool `json:"ready,omitempty"`
	// Prepared: a Prepare run finished (the process ends after it).
	Prepared bool `json:"prepared,omitempty"`
	// Error: the process could not start (a package that does not hash to
	// its pin, a module that does not compile); it ends after saying so.
	Error     string `json:"error,omitempty"`
	CompileMs int64  `json:"compile_ms,omitempty"`
}

Hello is an engine process's first line.

type Options

type Options struct {
	// Command is the argv that starts an engine process (default: this
	// executable, `office-engine`). Env is added to its environment.
	Command []string
	Env     []string
	// CacheDir is the compilation cache (filex's data directory).
	CacheDir string
	// WorkDir is where each process's temporary directory is made (default:
	// the system's).
	WorkDir string
	// MaxMB is the memory cap in force, read when a process starts.
	MaxMB func() int
	// Slots is how many processes a pool may run at once (thumbs.office_slots,
	// 1-4), read at every job.
	Slots func() int
	// ProcessLimit is the most memory a process may hold (bytes), read at
	// every job; default about twice the cap. 0 from the function: none.
	ProcessLimit func() int64
	// IdleTimeout ends a process nobody used for that long (default 3 min);
	// JobsPerProcess ends one after that many jobs (default 50).
	IdleTimeout    time.Duration
	JobsPerProcess int
	// StartTimeout bounds a process's start (default 2 min: the modules come
	// from the cache); PrepareTimeout a prepare run (default 30 min);
	// RetryPrepare is how long after a failed one it is tried again
	// (default 30 min).
	StartTimeout   time.Duration
	PrepareTimeout time.Duration
	RetryPrepare   time.Duration
	Log            *slog.Logger
}

Options tune a Runner.

type Package

type Package struct {
	App, Version string
	Engine       string
	EngineSHA256 string
	UI           string
	UISHA256     string
	EditorDir    string
	// HostVersion is this filex's version: a new filex (and with it a new
	// wazero) prepares the engine again.
	HostVersion string
}

Package is the engine an app brings: its engine.zip and the interface bundle that holds the editor's scripts and fonts, each with the hash it was installed with.

type Purpose

type Purpose string

Purpose names the pool a job runs in: thumbnails and conversions each have their own processes, so a long conversion never holds the thumbnails up (and the other way round).

const (
	PurposeConvert   Purpose = "convert"
	PurposeThumbnail Purpose = "thumbnail"
)

The pools.

type Request

type Request struct {
	ID int64  `json:"id"`
	Op string `json:"op"`
	// In is the input on this machine, From its format (the extension, lower
	// case, no dot).
	In   string `json:"in"`
	From string `json:"from"`
	// To is the conversion API's output type, as the Document Server is asked
	// for it: an extension, or "pdfa".
	To string `json:"to,omitempty"`
	// Out is where the result goes. It must not exist: the child creates it.
	Out string `json:"out"`
	// MaxBytes is the largest result accepted (0: no limit). A larger one is
	// not written and the answer says Over.
	MaxBytes int64 `json:"max_bytes,omitempty"`
	// Delimiter and CodePage are the conversion API's CSV knobs (0: not said):
	// the delimiter as the Document Server numbers it (1 tab, 2 semicolon,
	// 3 colon, 4 comma, 5 space), the code page as Windows does (65001).
	Delimiter int `json:"delimiter,omitempty"`
	CodePage  int `json:"code_page,omitempty"`
	// Size is a thumbnail's box, in pixels.
	Size int `json:"size,omitempty"`
	// Params are the editor's json parameters (a Document Server's
	// m_sJsonParams): a spreadsheet's page layout for its thumbnail.
	Params json.RawMessage `json:"params,omitempty"`
}

Request is one job.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

Runner runs the office engine of the package it is given, in processes of its own.

func New

func New(o Options) *Runner

New makes a runner with no package (StateAbsent).

func (*Runner) Close

func (r *Runner) Close()

Close ends every process; the runner takes no more jobs.

func (*Runner) Convert

func (r *Runner) Convert(ctx context.Context, c Conversion) error

Convert runs one conversion. A failure is an *Error (Over: the result was over MaxBytes), ErrNoEngine / ErrPreparing / ErrFailed when the engine does not run, or ctx's error when the deadline passed.

func (*Runner) DrawThumbnail

func (r *Runner) DrawThumbnail(ctx context.Context, t Thumbnail) ([]byte, error)

DrawThumbnail draws a PNG of the document's first page.

func (*Runner) Ready

func (r *Runner) Ready() bool

Ready reports whether jobs run now. From memory: thumbnails ask it for every listed office file.

func (*Runner) Release

func (r *Runner) Release(app string)

Release ends every process of an app's engine and forgets the package - before the app's files are moved (an upgrade, a removal): on Windows a file a process holds open cannot be moved. SetPackage brings it back.

func (*Runner) SetPackage

func (r *Runner) SetPackage(p *Package)

SetPackage switches the runner to an engine package (nil: none). The same package again changes nothing; another one ends every process of the previous one and prepares the new one unless this machine already has.

func (*Runner) Status

func (r *Runner) Status() Status

Status is the engine's state now.

type State

type State string

State is where the engine is.

const (
	// StateAbsent: no running app brings an office engine.
	StateAbsent State = "absent"
	// StatePreparing: compiling it, in the background (3.5-5 minutes the
	// first time on a machine).
	StatePreparing State = "preparing"
	// StateReady: jobs run.
	StateReady State = "ready"
	// StateFailed: it could not be prepared (Status.Error says why); tried
	// again after Options.RetryPrepare.
	StateFailed State = "failed"
)

The states.

type Status

type Status struct {
	State   State     `json:"state"`
	App     string    `json:"app,omitempty"`
	Version string    `json:"version,omitempty"`
	Error   string    `json:"error,omitempty"`
	Since   time.Time `json:"since"`
}

Status is the engine as the administrator's screens show it.

type Thumbnail

type Thumbnail struct {
	// Src is the document, From its format.
	Src, From string
	// Size is the box the page is fitted into; Params the editor's json
	// parameters (a sheet's page layout); MaxBytes the largest picture.
	Size     int
	Params   json.RawMessage
	MaxBytes int64
}

Thumbnail is one picture of a document's first page.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL