bundler

package module
v0.0.0-...-d508485 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 17, 2026 License: BSD-3-Clause Imports: 5 Imported by: 0

README

go-ruby-bundler/bundler

bundler — go-ruby-bundler

License Go Coverage CGO

A pure-Go (no cgo) reimplementation of the deterministic, pure-compute core of Ruby's Bundler — the bundler gem's Gemfile DSL reader, its Gemfile.lock codec, the Gem::Version / Gem::Requirement algebra, and the backtracking dependency resolver — without any Ruby runtime.

It is byte-for-byte faithful to MRI's Bundler (lockfile target 2.6.x) on the things that matter for a deterministic, reproducible build: a real Gemfile.lock round-trips identically, version/requirement comparisons match Gem::Version, and the resolver activates the same name → version set Bundler does over the same index. Network gem-fetching is left as a host-side seam (an injected Index); no network, no filesystem installs.

It builds on go-ruby-rubygems for the version algebra, so the whole stack is CGO=0. It is the dependency-resolution backend for go-embedded-ruby (rbgo), a sibling of the other go-ruby-* reimplementations (yaml, regexp, marshal, …).

What's in scope

Area API Notes
Gemfile DSL ParseGemfile(string) (*Gemfile, error) source / ruby / gem "n", "req", opts / group do…end / gemspec / git_source. Options: :group(s), :require, :platform(s), :git, :path, :branch, :ref, :tag, :submodules, :source. Dynamic-Ruby forms are reported as *GemfileError, never dropped.
Lockfile codec ParseLockfile(string) (*Lockfile, error), (*Lockfile).String() / .Bytes() GEM / GIT / PATH sources, nested specs: deps, PLATFORMS, DEPENDENCIES (with ! pins), RUBY VERSION, BUNDLED WITH. Byte-for-byte round-trip.
Version / requirement (re-exported via go-ruby-rubygems) Gem::Version (incl. prerelease ordering), Gem::Requirement (~>, >=, <, =, compound).
Resolver Resolve(deps, Index, *Source) (*Resolution, error) Backtracking + activation + conflict resolution; returns a *VersionConflict Bundler-style on an unsatisfiable graph.
Definition glue (*Definition).Resolve(Index), .Lockfile(*Resolution) Resolve → serialize, grouping specs by source.

What's a host-side seam

  • Fetching the gem index from rubygems.org / a compact-index mirror. Supply an Index (HTTP, a local mirror, or a test fixture — MapIndex is provided).
  • Downloading / unpacking / installing .gem files and the bundle install filesystem writes.
  • Evaluating a dynamic Gemfile (computed names, install_if, scoped source blocks). ParseGemfile reads the static forms; the host evaluates the rest to a []*Dependency.

Usage

gf, err := bundler.ParseGemfile(`
source "https://rubygems.org"
gem "rake", "~> 13.0"
gem "rspec", ">= 3.0", "< 4.0"
`)
if err != nil {
    log.Fatal(err)
}

// idx is your Index implementation (network, mirror, fixture).
src := &bundler.Source{Type: bundler.GemSource, Remotes: []string{"https://rubygems.org/"}}
res, err := bundler.Resolve(gf.Dependencies(), idx, src)
if err != nil {
    log.Fatal(err) // *bundler.VersionConflict on an unsatisfiable graph
}

def := &bundler.Definition{
    Dependencies: gf.Dependencies(),
    Platforms:    []string{"ruby"},
    BundledWith:  "2.6.9",
    Source:       src,
}
os.WriteFile("Gemfile.lock", def.Lockfile(res).Bytes(), 0o644)

Tests & coverage

100% statement coverage, enforced in CI, with a differential MRI oracle: the tests drive the real bundle binary against a local offline gem repo and assert our resolver activates the same set and our serializer reproduces Bundler's Gemfile.lock byte-for-byte; a second oracle drives Bundler::Dsl.evaluate and checks ParseGemfile extracts the same gem model. The oracle tests skip themselves where ruby/bundle are absent (Windows, the cross-arch qemu lanes), so the deterministic ruby-free suite alone holds the 100% gate everywhere.

GOWORK=off go test -race -cover ./...

Validated on the six supported 64-bit Go targets (amd64, arm64, riscv64, loong64, ppc64le, s390x) and three OSes (Linux, macOS, Windows).

License

BSD-3-Clause — see LICENSE. Copyright (c) the go-ruby-bundler/bundler authors.

WebAssembly

Being pure Go (CGO=0), this library also compiles to WebAssembly — both GOOS=js GOARCH=wasm (browser / Node.js) and GOOS=wasip1 GOARCH=wasm (WASI). CI builds both targets on every push, alongside the six 64-bit native/qemu arches.

GOOS=js     GOARCH=wasm go build ./...   # browser / Node
GOOS=wasip1 GOARCH=wasm go build ./...   # WASI (wasmtime, wasmer, wasmedge, …)

Documentation

Overview

Package bundler is a pure-Go (no cgo) reimplementation of the pure-compute core of Ruby's Bundler — the dependency resolver and the Gemfile.lock codec.

It is byte-for-byte faithful to MRI's Bundler (target 2.6.x) on the two things that matter most for a deterministic, reproducible build:

  • LockfileParser / lock serialization: parse a real Gemfile.lock into the spec set, and re-emit the exact same bytes (the canonical full-name ordering, the 2-space spec indent, the 3-space "BUNDLED WITH" line, the descending requirement order, the "!" pinned-dependency markers, and the blank lines between sections). A real lock round-trips identically.

  • The resolver: a backtracking dependency resolver (the same shape of activation + conflict-resolution that Bundler's Molinillo/PubGrub solver produces) over an injected index. Given a set of root dependencies and an Index (name -> available versions, each with its own dependencies), it resolves to a consistent SpecSet, or returns a VersionConflict whose message explains the conflict the way Bundler does.

It builds on github.com/go-ruby-rubygems/rubygems for the version algebra (Gem::Version comparison, Gem::Requirement satisfaction including the pessimistic "~>" bound math, and Gem::Dependency), so the whole stack is CGO=0.

In scope (pure compute)

  • Gemfile DSL reader (ParseGemfile): the canonical, deterministic forms of a Gemfile — source, ruby, gem "name", "req", options (:group/:groups/:require/:platform(s)/:git/:path/:branch/:ref/:tag/ :submodules/:source), group do … end blocks, gemspec, and a custom git_source(:name){ |repo| … } template. Arbitrary-Ruby forms (a computed name, an install_if predicate, a scoped source block) are reported as a *GemfileError naming the line, never silently dropped.
  • Bundler::LockfileParser: parse GEM / PATH / GIT / PLATFORMS / DEPENDENCIES / RUBY VERSION / BUNDLED WITH sections; the remote:/specs: tree; the "gem (ver)" + nested-dependency indentation; the "!" pinned markers. Plus the inverse: serialize a resolution back to lockfile bytes.
  • The resolver: backtracking + activation + conflict resolution + the deterministic sorting that fixes Bundler's output, over an injected Index.
  • Bundler::Dependency: gem name + requirement + groups + platforms + source.
  • Bundler::Definition (pure parts): the glue holding sources, dependencies and a locked SpecSet, and the lock-string production.

Out of scope (host-side seams)

  • Fetching the gem index from a remote source (the rubygems.org API or the compact index). The resolver takes an injected Index instead — supply one however you like (HTTP, a local mirror, a test fixture). No network.
  • Downloading, unpacking and installing .gem files; the `bundle install` filesystem writes; require-time activation.
  • Fully evaluating a *dynamic* Gemfile. ParseGemfile reads the canonical static forms directly (no interpreter); a Gemfile that computes gem names or requirements at runtime, or uses install_if/env/scoped-source blocks, is reported as a *GemfileError so the host (a Ruby evaluator such as rbgo) can evaluate it and hand this library the resulting []*Dependency instead.

Downstream, rbgo (the pure-Go Ruby) drives this library: for a static Gemfile it calls ParseGemfile; for a dynamic one it evaluates the Ruby to a []*Dependency; either way it fetches an index from the network into an Index, calls Resolve, and writes the SpecSet back out with Lockfile.Bytes.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Definition

type Definition struct {
	// Dependencies are the top-level Gemfile dependencies (DEPENDENCIES section).
	Dependencies []*Dependency
	// Platforms is the PLATFORMS section.
	Platforms []string
	// RubyVersion, when set, becomes the "RUBY VERSION" section.
	RubyVersion string
	// BundledWith is the Bundler version to record ("BUNDLED WITH").
	BundledWith string
	// Source is the source the resolved specs are attached to (a GEM remote in
	// the common case).
	Source *Source
}

Definition is the pure part of Bundler::Definition: the glue between the Gemfile's dependencies (supplied by the host after evaluating the Gemfile), the resolution source, the resolved SpecSet, and the lockfile production. The Gemfile evaluation, index fetch and install are all host-side seams.

func (*Definition) Lockfile

func (d *Definition) Lockfile(res *Resolution) *Lockfile

Lockfile turns a Resolution into a serializable Lockfile, grouping the resolved specs under their source. Specs are attached to d.Source (Resolve guarantees this), so a single source section is produced for the common GEM case; mixed-source resolutions group by the spec's own Source pointer.

func (*Definition) Resolve

func (d *Definition) Resolve(index Index) (*Resolution, error)

Resolve runs the resolver over the given index and returns a Resolution.

type Dependency

type Dependency struct {
	Name        string
	Requirement *rubygems.Requirement

	// Groups is the set of bundler groups (default [:default]); empty means the
	// default group.
	Groups []string
	// Platforms is the set of Gemfile platforms the dep applies to (e.g. "ruby",
	// "mri", "jruby"); empty means all.
	Platforms []string
	// Source, when non-nil, pins the dependency to a PATH or GIT source and
	// makes it print with a "!" suffix in DEPENDENCIES.
	Source *Source
}

Dependency is a Bundler::Dependency: a gem name plus a requirement, the bundler groups it belongs to, the platforms it applies to, and the source it is pinned to (if any). The groups/platforms drive `bundle install --without` and platform filtering; resolution itself uses name + requirement + source.

func MustDependency

func MustDependency(name string, constraints ...string) *Dependency

MustDependency is like NewDependency but panics on a malformed constraint.

func NewDependency

func NewDependency(name string, constraints ...string) (*Dependency, error)

NewDependency builds a runtime Dependency from a name and constraint strings, using rubygems to parse the requirement. With no constraints the requirement defaults to ">= 0".

type Gemfile

type Gemfile struct {
	// Sources is the list of `source "url"` URLs in declaration order. The first
	// is Bundler's primary remote.
	Sources []string
	// RubyVersion is the `ruby "x.y.z"` constraint ("" when absent).
	RubyVersion string
	// Gems is the declared dependencies in declaration order, each carrying its
	// requirement, groups, platforms and (git/path) source options.
	Gems []*Dependency
	// Gemspecs holds each `gemspec` directive (development gemspec inclusion).
	Gemspecs []GemspecDirective
	// GitSources maps a custom `git_source(:name){ |repo| url }` name to its URL
	// template (with "%s" where the repo slug is interpolated).
	GitSources map[string]string
}

Gemfile is the parsed model of a Bundler Gemfile (the DSL the host evaluates to drive resolution). It is the structured result of reading the canonical, deterministic DSL forms — the ones a `bundle lock` cares about — without a Ruby interpreter.

A Gemfile is, in full generality, arbitrary Ruby; ParseGemfile reads the common static forms (string literals, symbols, arrays, hash options) and reports anything it cannot statically understand. For Gemfiles that compute gem names or requirements at runtime, the host evaluates the Ruby (e.g. rbgo) and builds the []*Dependency directly.

func ParseGemfile

func ParseGemfile(content string) (*Gemfile, error)

ParseGemfile reads the canonical, deterministic forms of a Bundler Gemfile:

  • source "https://rubygems.org"
  • ruby "3.4.0" (and ruby file: / RUBY_VERSION forms are reported as dynamic)
  • gem "name"[, "req"...][, opt: val...]
  • group :a, :b do ... end (nested gem lines inherit the groups)
  • gemspec [path: "...", name: "...", development_group: "..."]
  • git_source(:name) { |repo| "https://host/#{repo}.git" }

gem option keys understood: :group/:groups, :require, :platform/:platforms, :git, :path, :branch, :ref, :tag, :submodules, :source. Comments (#...) and blank lines are skipped. A form it cannot statically read yields a *GemfileError naming the offending line, rather than silently dropping it.

func (*Gemfile) Dependencies

func (gf *Gemfile) Dependencies() []*Dependency

Dependencies converts the parsed Gemfile gems into the []*Dependency the resolver and lockfile production consume.

type GemfileError

type GemfileError struct {
	Line int
	// contains filtered or unexported fields
}

GemfileError reports a Gemfile DSL line that could not be statically parsed.

func (*GemfileError) Error

func (e *GemfileError) Error() string

type GemspecDirective

type GemspecDirective struct {
	Path             string // :path option (default ".")
	Name             string // :name option (default: the single gemspec found)
	DevelopmentGroup string // :development_group option (default "development")
}

GemspecDirective is a `gemspec` line: it pulls a local .gemspec's runtime dependencies into the Gemfile. The options (path/name/development_group) are captured; resolving the actual gemspec is a host-side seam.

type Index

type Index interface {
	// Versions returns every available spec for name. The resolver sorts them
	// itself, so any order is fine. An unknown gem returns an empty slice.
	Versions(name string) []IndexSpec
}

Index is the resolution seam: it answers "what versions of NAME exist, and what does each depend on?" with NO network and NO filesystem. A test fixture, an in-memory mirror of the compact index, or an HTTP-backed implementation can all satisfy it.

type IndexSpec

type IndexSpec struct {
	Name         string
	Version      *rubygems.Version
	Platform     string
	Dependencies []SpecDependency
}

IndexSpec is one candidate gem version in the resolution index: its name, version, and its own runtime dependencies. It is the unit the resolver activates. Where the candidate comes from (network, mirror, fixture) is a host-side seam.

type LockSource

type LockSource struct {
	Source *Source
	Specs  []*Spec
}

LockSource pairs a Source header with the specs locked under it.

type Lockfile

type Lockfile struct {
	// Sources holds the GEM/PATH/GIT sections in lock order, each carrying the
	// specs that resolved against it.
	Sources []*LockSource
	// Platforms is the PLATFORMS section (sorted on emit).
	Platforms []string
	// Dependencies is the DEPENDENCIES section (the top-level Gemfile deps).
	Dependencies []*Dependency
	// RubyVersion is the "RUBY VERSION" section value ("" when absent).
	RubyVersion string
	// BundledWith is the "BUNDLED WITH" version ("" when absent).
	BundledWith string
}

Lockfile is the parsed/parseable model of a Gemfile.lock: the source sections with their specs, the platforms, the top-level dependencies, an optional locked ruby version, and the bundler version. Serializing it reproduces the original bytes.

func ParseLockfile

func ParseLockfile(content string) (*Lockfile, error)

ParseLockfile parses Gemfile.lock content into a Lockfile. It is byte-for-byte invertible: l.String() of the result reproduces the input for a well-formed Bundler lock.

func (*Lockfile) Bytes

func (l *Lockfile) Bytes() []byte

Bytes is String as a byte slice, for writing to a file (a host-side seam).

func (*Lockfile) String

func (l *Lockfile) String() string

String serializes the lockfile to its canonical bytes. It matches Bundler's LockfileGenerator: sources (each header + its specs sorted by full_name, bundler itself skipped), then PLATFORMS, DEPENDENCIES, RUBY VERSION and BUNDLED WITH, separated by single blank lines.

type MapIndex

type MapIndex map[string][]IndexSpec

MapIndex is a trivial in-memory Index built from a name -> specs map. It is the fixture index used by the tests and a convenient host-side adapter.

func (MapIndex) AddGem

func (m MapIndex) AddGem(name, version string, deps ...SpecDependency) error

AddGem registers a gem version and its dependencies into a MapIndex. deps are alternating name, constraint-string pairs flattened per gem via NewDependency is overkill here; callers pass SpecDependency directly through Add instead.

func (MapIndex) Versions

func (m MapIndex) Versions(name string) []IndexSpec

Versions implements Index.

type ParseError

type ParseError struct {
	// contains filtered or unexported fields
}

ParseError is returned when a Gemfile.lock cannot be parsed.

func (*ParseError) Error

func (e *ParseError) Error() string

type Resolution

type Resolution struct {
	// contains filtered or unexported fields
}

Resolution is the successful result of Resolve: the activated specs keyed by name, in a deterministic order.

func Resolve

func Resolve(deps []*Dependency, index Index, source *Source) (*Resolution, error)

Resolve resolves root dependencies against an index to a consistent set, or returns a *VersionConflict. The produced specs all carry the given source (use a GEM source for the common case). It implements depth-first backtracking with the most-constrained-package-first heuristic and highest-version-first candidate ordering, which yields Bundler's deterministic resolution.

func (*Resolution) Get

func (r *Resolution) Get(name string) *Spec

Get returns the resolved spec for name, or nil.

func (*Resolution) Specs

func (r *Resolution) Specs() []*Spec

Specs returns the resolved specs sorted by name.

type Source

type Source struct {
	Type SourceType

	// Remotes holds the source's remote URLs. For a GEM source there may be
	// several; for PATH/GIT there is exactly one.
	Remotes []string

	// Revision is the locked git commit (GIT only).
	Revision string
	// Ref, Branch, Tag and Submodules are the optional git locators (GIT only),
	// serialized in this fixed order after revision.
	Ref        string
	Branch     string
	Tag        string
	Submodules string
	// Glob is the gemspec glob (PATH and GIT). The empty string means "the
	// default", which is omitted from the lockfile.
	Glob string
}

Source describes one resolution source — a rubygems remote, a local path or a git checkout — exactly as it appears in a Gemfile.lock section header. It is the pure-data model; fetching from it is a host-side seam.

func (*Source) GemRemote

func (s *Source) GemRemote() string

GemRemote is the single remote of a GEM source (or "" if none); a convenience for the common case.

type SourceType

type SourceType string

SourceType is the kind of a lockfile source section.

const (
	// GemSource is a rubygems remote ("GEM") section.
	GemSource SourceType = "GEM"
	// PathSource is a local path ("PATH") section.
	PathSource SourceType = "PATH"
	// GitSource is a git ("GIT") section.
	GitSource SourceType = "GIT"
)

type Spec

type Spec struct {
	Name         string
	Version      *rubygems.Version
	Platform     string // "" means the "ruby" platform (Gem::Platform::RUBY)
	Source       *Source
	Dependencies []SpecDependency
}

Spec is a resolved specification in the lockfile: a name, a version, an optional platform, the source it came from, and its runtime dependencies. It is the pure-data analogue of Bundler::LazySpecification.

type SpecDependency

type SpecDependency struct {
	Name        string
	Requirement *rubygems.Requirement
}

SpecDependency is one runtime dependency edge of a resolved spec: a gem name and its requirement. It mirrors a Gem::Dependency line nested under a spec in the lockfile's specs: tree.

func Dep

func Dep(name string, constraints ...string) SpecDependency

Dep is a helper to build a SpecDependency from a name and constraint strings.

type VersionConflict

type VersionConflict struct {
	// Name is the gem whose requirements conflict.
	Name string
	// Requesters maps each requirement string to the gem(s) that imposed it
	// ("Gemfile" for a root dependency).
	Requesters map[string][]string
	// contains filtered or unexported fields
}

VersionConflict is returned when no consistent set satisfies the requirements. Its message names the gem that could not be resolved and the conflicting requirements, in the shape Bundler reports.

func (*VersionConflict) Error

func (e *VersionConflict) Error() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL