extl

package
v1.4.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 12, 2026 License: MIT Imports: 18 Imported by: 0

README

Package extl

This package provides the extension engine and manager integrated with js runtime (goja).

Documentation

Index

Constants

View Source 
const (
	DEF_MODULE_ENTRY = "main.js"
	DEF_MODULE_HASH  = 16

	EXTRACT_CALLBACK = "extract"

	EXPORTED_END = "end"
)
View Source 
const FUNCTION_REGEXP = `function\s(\w+)\(.*\)\s{(?:\n?.*)+}`

Variables

View Source 
var (
	// ENGINE_STORE is the base directory for engine configuration files.
	ENGINE_STORE = warplib.ConfigDir
	// MODULE_STORE is the directory where extension modules are stored.
	MODULE_STORE = ENGINE_STORE + "/extstore/"

	// DEBUG_ENGINE_STORE is the base directory for debugger engine configuration.
	DEBUG_ENGINE_STORE = ENGINE_STORE + "/debugger/"
	// DEBUG_MODULE_STORE is the directory where debugger extension modules are stored.
	DEBUG_MODULE_STORE = DEBUG_ENGINE_STORE + "/extstore/"
)

Storage path variables define the locations for engine configuration and module files. These can be overridden using SetEngineStore for custom configurations.

View Source 
var (
	// ErrInvalidExtension is returned when an extension lacks a valid manifest.json.
	ErrInvalidExtension = errors.New("invalid extension")

	// ErrExtractNotDefined is returned when a module does not define an extract function.
	ErrExtractNotDefined = errors.New("extract function not defined")
	// ErrInvalidReturnType is returned when the extract function returns a non-string value.
	ErrInvalidReturnType = errors.New("invalid return type")
	// ErrEntrypointNotFound is returned when the module's entrypoint file does not exist.
	ErrEntrypointNotFound = errors.New("entrypoint not found")

	// ErrInteractionEnded is returned when user interaction with a module fails or is explicitly ended.
	ErrInteractionEnded = errors.New("interaction ended")

	// ErrNoMatchFound is returned when no module matches the given URL pattern.
	ErrNoMatchFound = errors.New("no match found")

	// ErrModuleNotFound is returned when a requested module does not exist in the engine.
	ErrModuleNotFound = errors.New("module not found")
)

Error variables define sentinel errors for extension-related failures.

Functions

func SetEngineStore 

func SetEngineStore(dir string) error

SetEngineStore configures custom storage directories for the extension engine. It creates the necessary directory structure and updates the global storage path variables. This is useful for testing or when using non-default configuration locations.

Types

type Engine 

type Engine struct {

	// LoadedModule is a map of path to moduleId.
	// This is used to store the moduleId in the module_engine.json file
	// and to load the module from the module storage when the engine is started.
	LoadedModule map[string]LoadedModuleState `json:"loaded_modules"`
	// contains filtered or unexported fields
}

Engine manages JavaScript extensions using the Goja runtime. It handles loading, activating, deactivating, and executing extension modules. The engine maintains state in a JSON file and provides URL extraction capabilities through registered extension modules.

func NewEngine 

func NewEngine(l *log.Logger, cookieManager *credman.CookieManager, debugger bool) (*Engine, error)

NewEngine creates and initializes a new extension engine. It loads previously registered modules from the engine state file and activates any modules that were marked as activated. The debugger parameter controls whether to use debug-specific storage paths.

func (*Engine) ActivateModule 

func (e *Engine) ActivateModule(moduleId string) (*Module, error)

ActivateModule enables a previously deactivated extension module. It loads the module into memory and marks it as activated in the engine state. Returns ErrModuleNotFound if the module does not exist.

func (*Engine) AddModule 

func (e *Engine) AddModule(path string) (*Module, error)

AddModule installs a new extension module from the given path. It loads the module, migrates it to the engine's storage directory, activates it, and persists the engine state. Returns the loaded module or an error if installation fails.

func (*Engine) Close 

func (e *Engine) Close() error

Close releases resources held by the engine. It closes the underlying configuration file handle.

func (*Engine) DeactiveModule 

func (e *Engine) DeactiveModule(moduleId string) (string, error)

DeactiveModule disables an active extension module without removing it. The module is unloaded from memory but remains registered in the engine state. Returns the extension name and an error if deactivation fails.

func (*Engine) DeleteModule 

func (e *Engine) DeleteModule(moduleId string) (string, error)

DeleteModule removes an extension module from the engine. It unloads the module from memory, removes it from the engine state, and deletes the module files from disk. Returns the extension name and an error if deletion fails.

func (*Engine) Extract 

func (e *Engine) Extract(url string) (string, error)

Extract attempts to extract a download URL using registered extension modules. It iterates through active modules and returns the extracted URL from the first module whose match pattern matches the input URL. Returns the original URL unchanged if no matching module is found.

func (*Engine) GetModule 

func (e *Engine) GetModule(moduleId string) *Module

GetModule retrieves an active module by its identifier. Returns nil if the module is not found or not currently active.

func (*Engine) ListModules 

func (e *Engine) ListModules(all bool) []common.ExtensionInfoShort

ListModules returns information about registered extension modules. If all is true, returns all modules including deactivated ones. If all is false, returns only activated modules.

func (*Engine) Save 

func (e *Engine) Save() error

Save persists the current engine state to the engine configuration file. It truncates the existing file and writes the current state as JSON. 

type Header struct {
	// contains filtered or unexported fields
}

func (Header) Append 

func (h Header) Append(key, value string)

func (Header) Delete 

func (h Header) Delete(key string)

func (Header) Entries 

func (h Header) Entries() [][]string

func (Header) ForEach 

func (h Header) ForEach(callback any)

func (Header) Get 

func (h Header) Get(key string) string

func (Header) GetSetCookies 

func (h Header) GetSetCookies() []string

func (Header) Has 

func (h Header) Has(key string) bool

func (Header) Keys 

func (h Header) Keys() []string

func (Header) Set 

func (h Header) Set(key, value string)

func (Header) Values 

func (h Header) Values() []string

type LoadedModuleState 

type LoadedModuleState struct {
	ModuleId    string `json:"module_id"`
	Name        string `json:"name"`
	IsActivated bool   `json:"is_activated"`
}

LoadedModuleState represents the persisted state of a module in the engine. It tracks the module identifier, display name, and activation status.

type Module 

type Module struct {
	// ModuleId is the unique identifier for the module, generated automatically.
	ModuleId string `json:"-"`
	// Name is the display name of the module.
	Name string `json:"name"`
	// Version is the semantic version of the module.
	Version string `json:"version"`
	// Description provides a brief explanation of what the module does.
	Description string `json:"description"`
	// Matches is an array of regex patterns that this module can handle.
	Matches []string `json:"matches"`
	// Entrypoint is the main file for the module (default: main.js).
	Entrypoint string `json:"entrypoint,omitempty"`
	// Assets should be filled with all the files that must be loaded with the extension.
	// For example: any extra js files that are imported in main.js.
	Assets []string `json:"assets,omitempty"`
	// contains filtered or unexported fields
}

Module represents a loaded JavaScript extension with its metadata and runtime. Each module contains URL match patterns and an extract function that transforms URLs for downloading. Modules are isolated from each other via separate JavaScript runtimes.

func OpenModule 

func OpenModule(l *log.Logger, path string) (*Module, error)

OpenModule tries to create a module object by reading its manifest. It parses the manifest.json file from the given path and returns a Module with its metadata populated. Returns ErrInvalidExtension if manifest.json does not exist.

func (*Module) Extract 

func (m *Module) Extract(url string) (string, error)

Extract invokes the module's JavaScript extract function with the given URL. It returns the transformed download URL or an error if extraction fails. Returns ErrInteractionEnded if the module explicitly ends the interaction.

func (*Module) Load 

func (m *Module) Load() error

Load loads the module to the engine and activates it. Each module is loaded in a new js runtime, hence isolated from each other.

type Request 

type Request struct {
	Method  string            `json:"method"`
	URL     string            `json:"url"`
	Headers map[string]string `json:"headers"`
	Body    string            `json:"body"`
}

type Response 

type Response struct {
	ContentLength int64   `json:"content_length"`
	Body          string  `json:"body"`
	StatusCode    int     `json:"status_code"`
	Headers       *Header `json:"headers"`
}

type Runtime 

type Runtime struct {
	*requirePkg.RequireModule
	*goja.Runtime
	// contains filtered or unexported fields
}

Runtime wraps a Goja JavaScript runtime with module support. It provides an isolated execution environment for extension modules, including built-in functions for I/O, HTTP requests, and module imports.

func NewRuntime 

func NewRuntime(l *log.Logger, wd string) (*Runtime, error)

NewRuntime creates a new JavaScript runtime for extension execution. It initializes the Goja runtime with built-in functions (print, input, require) and HTTP request capabilities. The wd parameter sets the working directory for module resolution.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.