notebook

notebook — go-notebook

Go notebooks with native speed, kept values and a reproducible order.

🌐 Website · 📚 Documentation

A notebook is a Markdown file whose ```go blocks are cells — and ```ruby blocks too, in a pure-Go Ruby with the same numerical libraries, sharing their variables with the Go cells. The kernel:

• compiles it into a real Go program, so cells run at native speed;
• orders the cells by what they use, worked out from the code itself as marimo does for Python, not by the order you ran them in;
• runs only what changed. A cell whose code and inputs did not change keeps its result, and its values are handed over instead of being computed again. That holds across rebuilds and across runs: results and values are kept on disk.

# Spectrum

```go
import "math"

n := 1_000_000
```

The sum below runs once; editing the last cell does not run it again.

```go
sum := 0.0
for i := 1; i <= n; i++ {
	sum += 1 / float64(i*i)
}
```

```go
math.Sqrt(6 * sum)
```

$ notebook run spectrum.md
── cell 0 (line 4, ran in 11µs)
── cell 1 (line 12, ran in 2.065ms)
── cell 2 (line 19, ran in 30µs)
   3.1415916986605086
$ # change the last cell to math.Pi - math.Sqrt(6*sum), then:
$ notebook run spectrum.md
── cell 0 (line 4, kept)
── cell 1 (line 12, kept)
── cell 2 (line 19, ran in 48µs)
   9.54929284535666e-07

The second run is a new process: sum comes back from the kernel's store, it is not summed again.

Use

go install github.com/go-notebook/notebook/cmd/notebook@latest

notebook run analysis.md     # run it, print each cell's results
notebook html analysis.md    # run it, write analysis.html (no scripts, no external resources; $\TeX$ math typeset by go-tex)
notebook watch analysis.md   # run it on every save and keep analysis.html current
notebook serve analysis.md   # the same in your browser at http://127.0.0.1:8888: edit cells there too
notebook irb                 # an interactive Ruby session, as irb, on the same Ruby and bindings

Edit the .md in any editor, or in the page serve shows (double-click a cell, Ctrl/⌘+Enter to save): the file is the only source of truth, and only what changed runs again.

Flags are POSIX, and each verb lists its own with notebook <verb> --help:

For irb, --timeout and --memory apply to each input.

Releases up to v0.3 read flags with one dash (-timeout 30s); that spelling is still accepted, with a note on stderr.

examples/spectrum.md finds two tones in Gaussian noise with go-ndarray (arrays, fft.RFFT, a seeded generator that gives NumPy's numbers for the same seed) and draws the signal and its spectrum with go-plot.

Cells

A cell is Go with statements allowed at the top level:

import "strings"

func words(s string) []string { return strings.Fields(s) }

w := words("to be or not to be")
len(w) // the final expression is shown

• Names defined at the top level are shared with every cell. Each name is defined by one cell, and a cell may not assign to another cell's variable. That is what makes the order a function of the code.
• Names starting with _, and err, stay local to their cell.
• A final call returning (value, error) shows the value; a non-nil error fails the cell.
• A value with a Display() map[string][]byte method shows itself as HTML, SVG or PNG.
• Errors point at the notebook's own lines. A panic stops the cells that read the failing one, and they say why.

Ruby cells

A ```ruby block runs in rbgo, a Ruby written in Go, with the go-scikit bindings: Stats, Special, Optimize, Integrate, DataFrame and Plot over the same Go libraries, and rbgo's NDArray. Ruby cells share one interpreter, so their top-level locals persist from cell to cell. The two languages share their variables:

• A Go variable a Ruby cell reads becomes a Ruby local of the same name. A []float64 becomes an Array, and an *ndarray.Array stays the same NDArray.
• A Ruby local a Go cell reads becomes a Go variable typed from its value: a Float is a float64, an Array of Floats a []float64, an NDArray an *ndarray.Array, a fitted Stats::Gamma the stats.Gamma itself.

Dependencies, caching and re-runs work across languages as for Go alone: a Ruby cell re-runs only when it or a cell it reads changes. The last expression shows its inspect, and a Plot figure shows as SVG. examples/mixed.md draws data in Go and fits and plots it in Ruby; the exact rules are in docs/design.md.

Ruby cells also run in the browser, with no server, in the playground: the same rbgo and bindings compiled to WebAssembly, loaded only when the open notebook has a Ruby cell. Try it on go-notebook's own playground (Go and Ruby notebooks), and the Ruby examples on go-statistics and go-numerics.

Julia cells: parsed and checked, not run yet

A ```julia block is a Julia cell (v0.6.0 and later). It is parsed with go-embedded-julia's parser, a pure-Go port of JuliaSyntax that gives the same trees as julia 1.12 on its whole test corpus, and its names join the notebook's dependency graph under the same rules as the other cells:

• Syntax errors are reported at the notebook's line and column, with julia's own message (5:8: cell 0: syntax error: Expected `)` or `,` ).
• Names follow Julia's scope rules for an interactive context ("Scope of Variables" in the Julia manual): top-level assignments, function, const, struct, abstract type, module, macro, using/import define; let, functions and comprehensions are hard scopes; for/while/try at the top level are soft scopes that assign an existing global, as in the REPL and IJulia. 61 cells are checked against julia 1.12.6 itself, which lowers them.
• Running a Julia cell fails, clearly: Julia cells are parsed and checked, but not run yet: execution arrives with go-embedded-julia's runtime (milestone M3). The cells that read what it defines, in any language, are blocked and say so; the others run; notebook run exits non-zero.

No external julia is started: execution waits for go-embedded-julia's own lowering and runtime.

An interactive session: notebook irb

notebook irb is irb on the same Ruby: rbgo with the go-scikit bindings, line by line, with no notebook file.

$ notebook irb --save session.md
irb(main):001> x = [2.1, 3.4, 1.9, 5.6]
=> [2.1, 3.4, 1.9, 5.6]
irb(main):002> fit = Stats::Normal.fit(x)
=> #<Stats::Normal loc=3.25, scale=1.4739402972983673>
irb(main):003> _.loc * 2
=> 6.5
irb(main):004> def sq(v)
irb(main):005*   v * v
irb(main):006* end
=> :sq
irb(main):007> raise "boom"
boom (RuntimeError)
	from (irb):7:in '<main>'
irb(main):008> exit
notebook irb: wrote session.md

• Editing: on a terminal, the line editor is go-ruby-reline's port of Ruby's Reline (emacs and vi keys, history with Up/Down and Ctrl-R, bracketed paste). An input that is not finished — an open def, bracket or string — continues on the next line with irb's continuation prompt; rbgo's parser decides. Tab completes locals, constants (Stats::Nor → Stats::Normal) and the methods of a variable's value, without calling anything the word contains.
• Results: => inspect after each input, _ the last value, and an exception as message (Class) with its frames, the session going on. Any local may be assigned again: there is no dependency graph here. exit, quit or Ctrl-D ends the session; Ctrl-C abandons the line being typed, or stops the input that is running.
• Limits: each input runs in a worker process, so --timeout and --memory apply to each input as to a cell. Stopping an input (a limit, Ctrl-C) kills its worker: the next input gets a new one, and the session says that its locals, methods and classes are gone.
• Figures: a Plot figure is never printed as SVG; --svg-dir figures saves each one as figures/figure-N.svg.
• --save session.md writes the inputs that ran as ```ruby cells, grouped so that the notebook keeps the rules of cells (an input that assigns a name again joins the cell that defined it; one that reads _ joins the one before), and notebook run session.md runs them again.
• History: irb_history under the user config directory's go-notebook (mode 0600; the inputs only, never their values; the last 1000), in irb's format.

When standard input is not a terminal, lines are read as they come and echoed after their prompt, so notebook irb < script.rb prints a transcript.

A notebook runs with your rights, in either language. Every cell runs in a worker process of its own: --timeout 30s kills a worker whose cell runs longer, and --memory 512 stops one whose cells hold more than 512 MiB of Go heap; the cell fails, and the next run starts a new worker. Both are off by default (what a cell can do, and what stops it).

How it works

format parses the Markdown and analysis builds the dependency graph from the syntax. gen writes the Go program, type-checking cell by cell to learn the types of shared variables. kernel compiles it into a worker, drives it through runtime, and keeps results and values under a hash of each cell's code and inputs. export writes the HTML page. The reasons for each choice, and what is not done yet, are in docs/design.md; the documentation covers the cell rules, the command and the kernel in more detail.

It is part of an ecosystem built for it: go-ndarray (N-dimensional arrays in thirteen dtypes, with FFT, linear algebra, and random numbers bit-identical to NumPy's for the same seed), go-plot (figures, with TeX labels), and, in progress, go-numerics (special functions, optimisation, integration), go-statistics and go-dataframe.

License

BSD-3-Clause. See LICENSE.