browser

package module
v0.5.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 11, 2025 License: MIT Imports: 7 Imported by: 0

README

Gost-DOM - A headless browser for Go

The Go-to headless browser for TDD workflows.

Gost-DOM is a headless browser written in Go intended to write tests of web application in Go that relies on JavaScript. Properties of Gost-DOM-based tests:

  • Tests run in parallel due to complete complete isolation[^1]
  • No erratic behaviour due to 100% predictable UI reactions.
  • "Blazingly fast". No out-of-process calls, not even thread boundaries. Web application code runs in the test thread, so a panic in your code keeps a full stack trace to the test case. [^2]
  • Dependencies can be replaced while testing.

Yet Gost-DOM still uses HTTP request and responses for verification, testing the entire stack, as well as middlewares.

Read this to get started, and see a quick example of usage.

[!NOTE]

This is 0.x version still, and breaking API changes do occur, but will be announced before release in the Gost-DOM discussions (do say Hi! 👋)

Looking for sponsors

This tool has reached a level where it can be used to test some web applications with JavaScript, e.g., simple HTMX applications. But there is still a lot to build to support just the most relevant Web APIs.

I've made good progress because of too much spare time; but that will not last. If I could find enough sponsors, it could mean the difference between continued development, or death 😢

For companies wanting to sponsor, I can send formal invoices too. More information on the project's Sponsor page

Join the "community"

Versioning and breaking changes

While still versioned as 0.x, this library will generally follow the convention, that a minor version increment indicates a breaking change.

Breaking changes will be announced up-front in the anouncements discussions.

When feasible withing a reasonable amount of work, an alternate solution will be made available as part of a deprecation before the the breaking change is released.

Adding support for library X

The goal is to support all modern JS frameworks, but I can't make tests for all. If you can contribute a "test project" with specific JS frameworks that will be a great help to detect compatibility issues, and potential extension to the test suites.

See the guidelines in the contributor documentation

Project status

This still early pre-release, and only the core web APIs are supported, and not 100%. Check the Feature list for a list.

The 0.1 focus was to support a common session based login-flow using HTMX, meaning to support content swapping, XHR, forms, and cookies; in order to identify risks and architectural flaws.

Current focus

Lately, little progress has been made, as I have been focusing on using this for an application; to help discover issues; and help prioritise.

But I want to see this support Datastar, another hypermedia framework. This brings some larger changes that have been underway for some time; but this also requires significant additions to v8go:

In addition, some existing features are incorrectly implemented, such as live collections.

A side project worked on in parallel is to support Goja as an alternate script engine; though it has had little attention for some time.

ESM support

Datastar is distrubuted ECMAScript modules, not scripts. So for Gost-DOM to support Datastar, ESM support is needed.

However, v8go, which is the link to V8 doesn't expose Module compilation and execution. I am working on extending v8go to support ESM. This isn't trivial.

ECMAScript object property handlers

"Handlers" is a feature in V8, where access to properties on an object can be intercepted by native code. Gost-DOM uses a modified version of v8go that has where support for indexed getters was hastily added to progresss and explore the problem space.

But full hander support will be needed for e.g., HTMLElement.dataset support.

MutationObserver

DataStar uses the MutationObserver API

Currently, a Go version exists as an internal module. The API is not yet exposed to JavaScript.

The mutation observer API is also intended to serve as support for test code describing behaviour at a higher and more accessibility-friendly manner.

E.g., rather than checking that a role="Alert" was not present before form submit, but present after, you could verify that submitting an invalid form causes an announcement to be made.

Fix already implemented features

Many of the already implemented features or APIs are not completely implemented, a few examples.

  • Assigning to the history doesn't navigate
  • Live collections are static.
  • Submit buttons cannot override form method and action.

To give users a better chance of predicting what works, and what doesn't, it is an aim to make sure that existing features work as they would in a real browser.

Goja support

V8 depends on Cgo, but Goja is a pure Go JavaScript engine. While it may not be as complete as V8, it could be a usable alternative for many projects providing a pure Go option.

V8 support will not go away, so there's always a fallback, if important JS features are lacking from Goja.

Memory Leaks

The current implementation is leaking memory for the scope of a browsing context. all DOM nodes created and deleted for the lifetime of the context will stay in memory until navigating, or the browser is actively disposed.

This is not a problem for the intended use case. Web scraping SPAs might become a problem.

Actions that will reset the browsing context are:

  • Clicking a link (default behaviour not prevented by JavaScript)
  • Submitting a form (default behaviour not prevented by JavaScript)
  • Refresh or navigating history entries generated by these operations.

Examples that will not reset the browsing context are (not necessarily a complete list):

  • Navigating an SPA.
  • Navigating an HTML app with boosted links.
  • Navigating history entries generated by these types of actions.
Why memory leaks

This codebase is a marriage between two garbage collected runtimes, and what is conceptually one object is split into two, a Go object and a JavaScript wrapper. As long of them is reachable; so must the other be.

I could join them into one; but that would result in an undesired coupling; the DOM implementation being coupled to the JavaScript execution engine.

Since this project started, weak references has been introduced to Go 1.24 which I beleve provides a solution to the problem, but this hasn't been prioritised

For that reason; and because it's not a problem for the intended use case, I have postponed dealing with that issue.

Future goals

There is much to do, which includes (but this is not a full list):

  • Support web-sockets and server events.
  • Implement all standard JavaScript classes that a browser should support; but not part of the ECMAScript standard itself.
    • JavaScript polyfills would be a good starting point; which is how xpath is implemented at the moment.
      • Conversion to native go implementations would be prioritized on usage, e.g. fetch would be high in the list of priorities.
  • Implement default browser behaviour for user interaction, e.g. pressing enter when an input field has focus should submit the form.
Long Term Goals
CSS Parsing

Parsing CSS woule be nice, allowing test code to verify the resulting styles of an element; but having a working DOM with a JavaScript engine is higher priority.

Mock external sites

The system may depend on external sites in the browser, most notably identity providers (IDP), where your app redirects to the IDP, which redirects on successful login; but could be other services such as map providers, etc.

For testing purposes, replacing this with a dummy replacement would have some benefits:

  • The verification of your system doesn't depend on the availability of an external service; when working offline
  • Avoid tests breaking due to a new UI in your external dependency.
  • For an identity provider
    • Avoid pollution of dummy accounts to run your test suite.
    • Avoid locking out test accounts due to "suspiscious activity".
    • The IDP may use a Captcha or 2FA that can be impossible; or difficult to control from tests, and would cause a significant slowdown to the test suite.
  • For applications like map providers
    • Avoid being billed for API use during testing.

Out of scope.

Full Spec Compliance

A goal is not always meant to be reached, it often serves simply as something to aim at.

  • Bruce Lee

While it is a goal to reach whatwg spec compliance, the primary goal is to have a useful tool for testing modern web applications.

Some specs don't really have any usage in modern web applications. For example, you generally wouldn't write an application that depends on quirks mode.

Another example is document.write. I've yet to work on any application that depends on this. However, implementing support for this feature require a complete rewrite of the HTML parser. You would need a really good case (or sponsorship level) to have that prioritised.

Accessibility tree

It is not currently planned that this library should maintain the accessibility tree; nor provide higher level testing capabilities like what Testing Library provides for JavaScript.

These problems should eventually be solved, but could easily be implemented in a different library with dependency to the DOM alone.

Visual Rendering

It is not a goal to be able to provide a visual rendering of the DOM.

But just like the accessibility tree, this could be implemented in a new library depending only on the interface from here.

Terminology

Some words inherntly have multiple meanings.

  • Interface. The IDL Specification defines interfaces; which are exposed in certain scopes, implemented by "classes" in JavaScript.
    • The interfaces can be composed of partial or mixin interfaces.
    • IDL Interfaces and mixin interfaces are represented in Go, and typically exposed as Go interface types.

Attribution / 3rd party included code.

This library contains code derived from the jsdom project distributed under the MIT license.

[^1]: Complete isolation depends on your code, e.g., if you don't replace database dependencies, they are not isolated. [^2]: This depends on how you configure Gost-DOM.

Documentation

Overview

Package browser is the main entry point for Gost, helping create a window initialized with a script enging, connected to a server.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Browser 

type Browser struct {
	Client     http.Client
	ScriptHost ScriptHost
	Logger     log.Logger
	// contains filtered or unexported fields
}

Pretty stupid right now, but should _probably_ allow handling multiple windows/tabs. This used to be the case for _some_ identity providers, but I'm not sure if that even work anymore because of browser security.

func New 

func New(options ...BrowserOption) *Browser

New initialises a new Browser with the default script engine.

func NewBrowser deprecated 

func NewBrowser() *Browser

Deprecated: NewBrowser should not be called. Call New instead.

This method will selfdestruct in 10 commits

func NewBrowserFromHandler deprecated 

func NewBrowserFromHandler(handler http.Handler) *Browser

Deprecated: NewBrowserFromHandler should not be called, call, NewFromHandler instead.

This method will selfdestruct in 10 commits

func NewFromHandler deprecated 

func NewFromHandler(handler http.Handler) *Browser

NewFromHandler initialises a new Browser with with an http.Handler

Deprecated: Prefer browser.New(browser.WithHandler(...)) instead.

func (*Browser) Close 

func (b *Browser) Close()

func (*Browser) NewWindow added in v0.5.1 

func (b *Browser) NewWindow() Window

func (*Browser) Open 

func (b *Browser) Open(location string) (window Window, err error)

Open will open a new html.Window, loading the specified location. If the server does not respons with a 200 status code, an error is returned.

See html.NewWindowReader about the return value, and when the window returns.

type BrowserOption added in v0.5.1 

type BrowserOption func(*browserConfig)

func WithHandler added in v0.5.1 

func WithHandler(h http.Handler) BrowserOption

WithHandler configures the browser's http.Client to use an http.Roundtripper that bypasses the TCP stack and calls directly into the specified handler as a normal function call.

Note: There is a current limitation that NO requests from the browser will be sent when using this. So sites will not work if they

  • Depend on content from CDN
  • Depend on an external service, e.g., an identity provider.

That is a limitation that was the result of prioritising more important, and higher risk features.

func WithLogger added in v0.5.1 

func WithLogger(l *slog.Logger) BrowserOption

Source Files

View all Source files

Directories

Path Synopsis
browser module
dom
Package dom provides the fundamental DOM implementation for Gost-DOM.
 Package dom provides the fundamental DOM implementation for Gost-DOM.
event
Package event contains core browser event behavior
 Package event contains core browser event behavior
Package html works on top of the DOM to implement specific HTML elements.
 Package html works on top of the DOM to implement specific HTML elements.
internal
clock
Package clock provides a simulated time for Gost-DOM.
 Package clock provides a simulated time for Gost-DOM.
constants
Package constants is a collection of values that are used many times in the implementation, but has no relevance to users of the library, e.g., a link to where you can file an issue when you encounter a not-implemented feature; or a feature that is not fully implemented, e.g.
 Package constants is a collection of values that are used many times in the implementation, but has no relevance to users of the library, e.g., a link to where you can file an issue when you encounter a not-implemented feature; or a feature that is not fully implemented, e.g.
dom
dom/mutation
domslices
entity
gosterror
gosthttp
html
interfaces
Package interfaces contains go interfaces generated from IDL specs
 Package interfaces contains go interfaces generated from IDL specs
interfaces/dom-interfaces
interfaces/url-interfaces
Package urlinterfaces contains internal code representing the IDL interfaces in the URL API [URL API]: https://developer.mozilla.org/en-US/docs/Web/API/URL_API
 Package urlinterfaces contains internal code representing the IDL interfaces in the URL API [URL API]: https://developer.mozilla.org/en-US/docs/Web/API/URL_API
log
Package log contains functions used internally for logging to a default logger implementing slog.Logger.
 Package log contains functions used internally for logging to a default logger implementing slog.Logger.
test
test/integration/test-app
test/integration/test-app-main command
test/integration/test-app/content
test/scripttests
Package scripttests contains a specification of the behaviour of client-side scripting.
 Package scripttests contains a specification of the behaviour of client-side scripting.
testing/eventtest
testing/fixtures
testing/gomega-matchers
Package gomegamatchers just exposes gomega matchers for easier importing.
 Package gomegamatchers just exposes gomega matchers for easier importing.
testing/gosttest
testing/htmltest
Package htmltest contains test helpers when working with the html package
 Package htmltest contains test helpers when working with the html package
uievents
Package logger provides the basic functionality of supplying a custom logger.
 Package logger provides the basic functionality of supplying a custom logger.
gojahost
The gojahost package provides functionality to execute client-scripts in gost-dom.
 The gojahost package provides functionality to execute client-scripts in gost-dom.
v8host
The v8host packages provides functionality to execute client-side scripts in gost-dom.
 The v8host packages provides functionality to execute client-side scripts in gost-dom.
testing
gomega-matchers
Package matchers contains custom matches for use with the [Gomega] assertion library.
 Package matchers contains custom matches for use with the [Gomega] assertion library.
testservers
Package url contains types defined in the url web IDL spec.
 Package url contains types defined in the url web IDL spec.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.