README
¶
Gost-DOM
A headless browser for Go
The Go-to solution for a TDD workflow.
As a developer
In order to work effeciently with code
I want a fast feedback loop for all of my code.
Gost-DOM was born from the philosophy that the fast feedback loop provided by TDD makes it the most effective way to work with the majority of the code base.
Web UIs are typical exceptions to this rule. They rely on real browsers for testing which introduce excessive overhead and slows down the feedback loop; reducing, or even eliminating, the effectiveness of TDD. In addition, developers often struggle with erratic tests due to unpredictable code execution.
Gost-DOM aims to solve that problem for web projects using Go. Gost-DOM simulates a browser environment, using a JavaScript engine to execute client script, allowing you to write test cases in Go to verify application behaviour, and apply an iterative process; supporting refactoring.[^1]
To learn more, read Why Gost-DOM?
Benefits of Gost-DOM
Compared to browser automation, Gost-DOM provides the benefits:
- Tests run in parallel due to complete complete isolation[^2]
- No erratic behaviour; 100% predictable UI reactions.
- Blazingly fast.[^3] No out-of-process calls, not even thread boundaries for web API calls as web application code runs in the test thread.[^4]
- Dependencies can be replaced in tests.
- Write tests at a higher level of abstraction, expressing the expected behaviour of a system, decoupled from implementation details.
Gost-DOM still uses HTTP request and responses for verification, testing the entire stack, including how middlewares affect the behaviour, verifying, and supporting refactoring of e.g., authentication logic.
Looking for sponsors
This tool has reached a level where it can be used to test some web applications with JavaScript, e.g., simple HTMX applications. But there is still a lot to build to support just the most relevant Web APIs.
I've made good progress because of too much spare time; but that will not last. If I could find enough sponsors, it could mean the difference between continued development, or death 😢
For companies wanting to sponsor, I can send formal invoices too. More information on the project's Sponsor page
Getting started
- Read Getting Started
- Familiarize yourself with the Feature list to know what is implemented.
- Join my discord server to chat with me, and stay up-to-date on progress.
- say hi! on the github discussions page.
- Read the contribution guide to see how you can help.
[!NOTE]
This is 0.x version still, and breaking API changes do occur, but will be announced before release in the Gost-DOM discussions (do say Hi! 👋)
Project status
This is still in an early phase, but it's approaching a design that seems promising for the purpose.
At the moment there's an emphasis on high-risk features that can expose poor design choices, but the "primary API" has been reasonably stable for a good amount of time.
Gost-DOM supports basic HTMX interactions, as well as basic Datastar cases.
Future goals
There is much to do, which includes (but this is not a full list):
- Support web-sockets and server-sent events.
- Implement all standard JavaScript classes that a browser should support; but
not part of the ECMAScript standard itself.
- JavaScript polyfills would be a good starting point; which is how xpath is implemented at the moment.
- Implement default browser behaviour for user interaction, e.g. pressing enter when an input field has focus should submit the form.
Pure Go script engine
V8 is a "feature complete" JavaScript environment, so V8 support will never go away. But it has some overhead, and it depends on Cgo. Also, the current V8 layer leaks memory in the scope of a browsing context.
Goja is a pure Go JavaScript engine, and is alsmost fully supported as an alternative. JavaScript bindings in code target a layer of abstraction, allowing the script engine to be replaced.
CSS Parsing
Parsing CSS woule be nice, allowing test code to verify the style properties of HTML elements; in particular whether an element is visible; but having a working DOM with a JavaScript engine is higher priority.
Mock external sites
The system may depend on external sites in the browser, most notably identity providers (IDP), where your app redirects to the IDP, which redirects on successful login; but could be other services such as map providers, etc.
For testing purposes, replacing this with a dummy replacement would have some benefits:
- The verification of your system doesn't depend on the availability of an external service; when working offline
- Avoid tests breaking due to a new UI in your external dependency.
- For an identity provider
- Avoid pollution of dummy accounts to run your test suite.
- Avoid locking out test accounts due to "suspiscious activity".
- The IDP may use a Captcha or 2FA that can be impossible; or difficult to control from tests, and would cause a significant slowdown to the test suite.
- For applications like map providers
- Avoid being billed for API use during testing.
Out of scope.
Full Spec Compliance
A goal is not always meant to be reached, it often serves simply as something to aim at.
- Bruce Lee
While it is a goal to reach whatwg spec compliance, the primary goal is to have a useful tool for testing modern web applications.
Some specs don't really have any usage in modern web applications, like
document.write or depending on quirks mode.
Accessibility tree
It is not currently planned that this library should maintain the accessibility tree.
The Shaman module provides capabilities of querying the DOM at a higher level of abstraction, e.g., find an element with a specific label / accessibility name, allowing tests to be more expressive. This is inspired by the capabilities that Testing Library provides for JavaScript.
Visual Rendering
It is not a goal to be able to provide a visual rendering of the DOM.
But just like the accessibility tree, this could be implemented in a new library depending only on the interface from here.
Terminology
Some words inherntly have multiple meanings.
- Interface. The IDL Specification defines interfaces; which are exposed
in certain scopes, implemented by "classes" in JavaScript.
- The interfaces can be composed of partial or mixin interfaces.
- IDL Interfaces and mixin interfaces are represented in Go, and typically
exposed as Go
interfacetypes.
Attribution / 3rd party included code.
Some web APIs are implemented by embedding polyfills from other open-source JavaScript libraries.
- XPath evaluator uses code from the jsdom project distributed under the MIT license.
- FastestSmallestTextEncoderDecoder distributed under the Creative Commons Zero v1.0 Universal license.
In addition, to verify compatibility with 3rd party JavaScript libraries, the test code in repository contains compiled versions of:
Star History
[^1]: Gost-DOM, by default, embeds a V8 engine - the same JavaScript engine that powers Chrome. [^2]: Complete isolation depends on your code, e.g., if you don't replace database dependencies, tests are not isolated. [^3]: Cliché, I know! But it is! [^4]: This depends on how you configure Gost-DOM.
Documentation
¶
Overview ¶
Package browser is the main entry point for Gost, helping create a window initialized with a script enging, connected to a server.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Browser ¶
type Browser struct {
Client http.Client
ScriptHost ScriptHost
Logger log.Logger
// contains filtered or unexported fields
}
Browser contains an initialized browser with a script engine. Create new windows by calling Browser.Open.
Browser values should be closed by calling Browser.Close, or passing a context:
func TestBrowserWithClose(t *testing.T) {
handler := NewRootHttpHandler()
b := browser.New(browser.WithHandler(handler))
t.Cleanup(func() { b.Close() })
win, err := b.Open("http://example.com")
// ...
}
Passing a context:
func TestBrowserWithContext(t *testing.T) {
handler := NewRootHttpHandler()
b := browser.New(
browser.WithHandler(handler),
browser.WithContext(t.Context(),
)
win, err := b.Open("http://example.com")
// ...
}
func New ¶
func New(options ...BrowserOption) *Browser
New initialises a new Browser. Options can be one of
Script engine defaults to V8. This will change in the future, but a migration path is not ready.
func (*Browser) Close ¶
func (b *Browser) Close()
Close "closes" a browser, releasing resources. This will close any initialized script hosts and contexts. Has two purposes.
- Reuse a template engine, reducing engine initialization overhead.
- Release memory for non-Go engines, e.g., V8
The relevance depends mostly on the script engine. For a pure Go engine, resources would be garbage collections. And the ability to reuse a preconfigured engine depends on engine capabilities.
Note: If a browser is initialized by passing a context.Context to the WithContext option, it will be closed if the context is cancelled.
func (*Browser) NewWindow ¶ added in v0.5.1
func (b *Browser) NewWindow() Window
NewWindow creates a new window. Panics if the browser has been closed
func (*Browser) Open ¶
Open will open a new html.Window, loading the specified location. If the server does not respons with a 200 status code, an error is returned.
See html.NewWindowReader about the return value, and when the window returns.
type BrowserOption ¶ added in v0.5.1
type BrowserOption func(*browserConfig)
func WithContext ¶ added in v0.8.0
func WithContext(ctx context.Context) BrowserOption
WithContext passes a context.Context than can trigger cancellation, e.g.:
- Close any open HTTP connections and disconnect from the server.
- Release resources, and reuse script hosts.
See also: Browser.Close
func WithHandler ¶ added in v0.5.1
func WithHandler(h http.Handler) BrowserOption
WithHandler configures the browser's http.Client to use an http.Roundtripper that bypasses the TCP stack and calls directly into the specified handler as a normal function call.
Note: There is a current limitation that NO requests from the browser will be sent when using this. So sites will not work if they
- Depend on content from CDN
- Depend on an external service, e.g., an identity provider.
That is a limitation that was the result of prioritising more important, and higher risk features.
func WithLogger ¶ added in v0.5.1
func WithLogger(l *slog.Logger) BrowserOption
func WithScriptEngine ¶ added in v0.9.2
func WithScriptEngine(engine html.ScriptEngine) BrowserOption
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
browser
module
|
|
|
Package dom provides the fundamental DOM implementation for Gost-DOM.
|
Package dom provides the fundamental DOM implementation for Gost-DOM. |
|
event
Package event contains core browser event behavior
|
Package event contains core browser event behavior |
|
Package html works on top of the DOM to implement specific HTML elements.
|
Package html works on top of the DOM to implement specific HTML elements. |
|
input
|
|
|
controller
Package controller can simulate user interaction
|
Package controller can simulate user interaction |
|
key
Package key represents keys on the keyboard that the user might type.
|
Package key represents keys on the keyboard that the user might type. |
|
internal
|
|
|
clock
Package clock provides a simulated time for Gost-DOM.
|
Package clock provides a simulated time for Gost-DOM. |
|
constants
Package constants is a collection of values that are used many times in the implementation, but has no relevance to users of the library, e.g., a link to where you can file an issue when you encounter a not-implemented feature; or a feature that is not fully implemented, e.g.
|
Package constants is a collection of values that are used many times in the implementation, but has no relevance to users of the library, e.g., a link to where you can file an issue when you encounter a not-implemented feature; or a feature that is not fully implemented, e.g. |
|
gosthttp
Package gosthttp provides functionality to bypass the TCP stack.
|
Package gosthttp provides functionality to bypass the TCP stack. |
|
interfaces
Package interfaces contains go interfaces generated from IDL specs
|
Package interfaces contains go interfaces generated from IDL specs |
|
interfaces/html-interfaces
Package htmlinterfaces contains internal code representing the IDL interfaces in the HTML DOM API
[HTML DOM API]: https://developer.mozilla.org/en-US/docs/Web/API/HTML_DOM_API
|
Package htmlinterfaces contains internal code representing the IDL interfaces in the HTML DOM API [HTML DOM API]: https://developer.mozilla.org/en-US/docs/Web/API/HTML_DOM_API |
|
interfaces/url-interfaces
Package urlinterfaces contains internal code representing the IDL interfaces in the URL API
[URL API]: https://developer.mozilla.org/en-US/docs/Web/API/URL_API
|
Package urlinterfaces contains internal code representing the IDL interfaces in the URL API [URL API]: https://developer.mozilla.org/en-US/docs/Web/API/URL_API |
|
log
Package log contains functions used internally for logging to a default logger implementing slog.Logger.
|
Package log contains functions used internally for logging to a default logger implementing slog.Logger. |
|
monads/result
package result provides monadic result binding
|
package result provides monadic result binding |
|
promise
Package promise provides a Go way to represent the concept of a Promise.
|
Package promise provides a Go way to represent the concept of a Promise. |
|
test/integration/test-app-main
command
|
|
|
test/scripttests
Package scripttests contains a specification of the behaviour of client-side scripting.
|
Package scripttests contains a specification of the behaviour of client-side scripting. |
|
testing/gomega-matchers
Package gomegamatchers just exposes gomega matchers for easier importing.
|
Package gomegamatchers just exposes gomega matchers for easier importing. |
|
testing/htmltest
Package htmltest contains test helpers when working with the html package
|
Package htmltest contains test helpers when working with the html package |
|
uievents
Package uievents creates and dispatches [UI Events].
|
Package uievents creates and dispatches [UI Events]. |
|
Package logger provides the basic functionality of supplying a custom logger.
|
Package logger provides the basic functionality of supplying a custom logger. |
|
scripting
|
|
|
internal/js
Package js provides an abstraction on top of V8.
|
Package js provides an abstraction on top of V8. |
|
internal/mathml
Package mathml implements (part of) the MathML-Core web API
|
Package mathml implements (part of) the MathML-Core web API |
|
sobekengine
The sobekhost package provides functionality to execute client-scripts in gost-dom.
|
The sobekhost package provides functionality to execute client-scripts in gost-dom. |
|
v8engine
The v8host packages provides functionality to execute client-side scripts in gost-dom.
|
The v8host packages provides functionality to execute client-side scripts in gost-dom. |
|
testing
|
|
|
gomega-matchers
Package matchers contains custom matches for use with the [Gomega] assertion library.
|
Package matchers contains custom matches for use with the [Gomega] assertion library. |
|
Package url contains types defined in the url web IDL spec.
|
Package url contains types defined in the url web IDL spec. |