Documentation
¶
Overview ¶
Package analysis finds what each code cell of a notebook defines and uses, and orders the cells by those dependencies.
A cell is Go with statements allowed at the top level, as in a REPL:
import "math"
func sq(x float64) float64 { return x * x }
y := sq(math.Pi)
y
Its top-level func, type and import declarations are declarations of the notebook; everything else is a statement of the cell, run in order. The names a cell defines at its top level (by :=, var, const, func or type) are the notebook's variables: every cell can read them, and exactly one cell may define each — the rule that makes the execution order a function of the code, not of the order cells were run in. Names starting with an underscore, and err, stay local to their cell (see Private). A cell may not assign to a variable another cell defines, for the same reason: the change would be invisible to the dependency graph. (Mutation through a method or a slice element cannot be seen statically and is not refused, as in marimo.)
The analysis is syntactic: it needs no type information and works on code that does not compile yet. A cell's final expression statement is its display value.
Index ¶
- Constants
- Variables
- func GoName(name string) bool
- func JuliaBlocked(julia int) string
- func Languages() []string
- func LearnsTypes(name string) bool
- func Private(name string) bool
- func RBlocked(r int) string
- func Register(l Language)
- func Shared(owner, user *Cell, name string) bool
- func Unregister(name string)
- type Cell
- type CellAnalyzer
- type Chunk
- type Diagnostic
- type Error
- type Graph
- type Highlighter
- type Language
- type Limits
- type Module
- type NameChecker
- type Names
- type Run
- type Runtime
- type Source
- type TypeLearner
- type Use
Constants ¶
const ( RubyPath = "github.com/go-scikit/ruby/cells" RubyModule = "github.com/go-scikit/ruby" DefaultRubyVersion = "v0.2.1" )
The go-scikit Ruby bindings that run Ruby cells over go-embedded-ruby.
const ( Go = "go" Ruby = "ruby" )
Languages of code cells.
const Julia = "julia"
Julia is the language of ```julia cells.
const JuliaNotRun = "Julia cells are parsed and checked, but not run yet: execution arrives with go-embedded-julia's runtime (milestone M3)"
JuliaNotRun is the failure of every Julia cell, in the kernel and in the playground: go-embedded-julia parses Julia exactly as julia 1.12 does, but has no interpreter yet, and go-notebook runs no external julia.
const R = "r"
R is the language of ```r cells.
const RNotRun = "R cells are parsed and checked, but not run yet: execution arrives with go-embedded-r's evaluator (milestones M2 and M3)"
RNotRun is the failure of every R cell: go-embedded-r parses R exactly as R 4.6 does, but has no evaluator yet.
Variables ¶
var DefaultLimits = Limits{MaxBytes: 16 << 20, MaxDepth: 10_000}
DefaultLimits are the limits the graph passes to Analyze.
Functions ¶
func GoName ¶ added in v0.3.0
GoName reports whether a Ruby local can be a Go variable of the same name: an identifier that is neither a Go keyword nor predeclared.
func JuliaBlocked ¶ added in v0.6.0
JuliaBlocked is the failure of a cell that reads, directly or through other cells, what Julia cell julia defines.
func Languages ¶ added in v0.7.0
func Languages() []string
Languages returns the registered language names, sorted.
func LearnsTypes ¶ added in v0.7.0
LearnsTypes reports whether the language of the given name is a TypeLearner.
func Private ¶
Private reports whether a name defined at a cell's top level stays local to the cell instead of becoming a notebook variable: a name starting with an underscore (marimo's convention), and err, which nearly every cell declares and none means to share.
func RBlocked ¶ added in v0.8.0
RBlocked is the failure of a cell that reads, directly or through other cells, what R cell r defines.
func Register ¶ added in v0.7.0
func Register(l Language)
Register adds a language. It panics on an empty or duplicate name, as database/sql does: registering is done once, at init.
func Shared ¶ added in v0.3.0
Shared reports whether the cell user sees the name that cell owner defines. Within a language every name is shared. Across languages only variables are:
The rule is the owner's language's (Language.Crosses):
- a Go cell sees a Ruby cell's top-level local variables, but not its methods, constants or classes, nor a local named like a Go keyword or a predeclared identifier (len, int, type, …), which stays Ruby's;
- a Ruby cell sees a Go cell's variables, but not its funcs or types;
- a Julia cell's variables, not its functions, macros, constants or types, are seen by the other languages as a Ruby cell's are, and a Julia cell sees their variables. Julia cells do not run yet, so a cell that reads one of their variables is blocked (see gen), but the edge is there.
func Unregister ¶ added in v0.7.0
func Unregister(name string)
Unregister removes a language, for tests that register a fake one.
Types ¶
type Cell ¶
type Cell struct {
Index int // position among the notebook's code cells
Source string // the cell's source
Lang string // Go, or Ruby (see AnalyzeRubyCell)
// Dynamic marks a cell whose names are not statically resolvable (see
// Names).
Dynamic bool
Chunks []Chunk
// Defines lists the notebook variables the cell defines: those of its
// declarations, then those of its statements, each in order of
// appearance. Funcs and Types are the subsets declared with func and
// type. Imports maps a package name to its import path.
Defines []string
Funcs []string
Types []string
Imports map[string]string
// Consts are the constants, classes and modules a Ruby cell defines.
Consts []string
// RubyDisplay reports that a Ruby cell ends in an expression, whose value
// it shows. (A Go cell's is Display.)
RubyDisplay bool
// Uses lists the free identifiers the cell reads, sorted: notebook
// variables, package names other cells import, builtins, and names
// that are not defined anywhere (the compiler reports those).
Uses []string
// OwnPackages lists, sorted, the names of the packages the cell imports
// itself and refers to.
OwnPackages []string
// Display is the cell's final expression statement, if any, and its
// byte range in Source.
Display ast.Expr
DisplayStart, DisplayEnd int
// Assigns lists the free identifiers the cell assigns to (=, op=, ++,
// --), with the offset of each assignment.
Assigns []Use
// Errors are syntax errors, with offsets in Source.
Errors []Error
}
Cell is the analysis of one code cell.
func AnalyzeCell ¶
AnalyzeCell analyses the source of code cell number index.
func AnalyzeJuliaCell ¶ added in v0.6.0
AnalyzeJuliaCell analyses the source of Julia code cell number index, with go-embedded-julia's parser (a port of JuliaSyntax, the parser of Julia 1.12, which gives the Expr trees julia itself gives).
The names follow Julia's scope rules ("Scope of Variables" in the Julia manual, 1.12) as they apply to the top level of a notebook, which Julia treats as an interactive context:
- the top level, and if, begin and other blocks in it, are the global scope: an assignment there (x = 1, a, b = …, x::Int = 1, global x = 1) defines a global, a notebook variable;
- function, macro, struct, let, do, -> and comprehension or generator bodies are hard local scopes: an assignment there makes a local, and the names read there that no enclosing scope binds are globals the cell uses;
- for, while and try bodies at the top level are soft scopes: an assignment there to a name that is a global already (defined earlier in the cell, or by another cell) assigns that global, as in the REPL and IJulia; otherwise it makes a local of the loop.
Functions (function f, f(x) = …) and macros (@m) are in Funcs; constants, structs, abstract and primitive types, modules and the names using and import bind are in Consts; all are in Defines. Names starting with an underscore stay private to the cell, as in the other languages.
func AnalyzeRCell ¶ added in v0.8.0
AnalyzeRCell analyses the source of R code cell number index, with go-embedded-r's parser (a port of R 4.6's gram.y, which gives the trees parse() gives) and its TopLevelNames rules: what a cell assigns at the top level (x <- v, v -> x, x = v, assign("x", v), the variable of a top-level for) it defines; the free variables of its code and of its functions' bodies it uses. A function assigned at the top level (f <- function(x) …) is in Funcs: like a Ruby method or a Julia function it stays R's. A cell whose names cannot be worked out statically (<<-, eval, assign or get with a computed name, attach, library…) is Dynamic.
func AnalyzeRubyCell ¶ added in v0.3.0
AnalyzeRubyCell analyses the source of Ruby code cell number index, with go-ruby-parser, the front end of the Ruby that runs it.
A Ruby cell's top level is a program evaluated in one binding shared by every Ruby cell, so the names it defines are of two kinds:
- the local variables it assigns at its top level (x = 1, a, b = …): notebook variables, which Go cells can read too;
- the methods it defines at its top level with def (Funcs), and its constants, classes and modules (Consts): visible to later Ruby cells only, but defined by one cell, as any name is.
Both are in Defines. The cell uses every name it reads that it has not assigned before: a local of the top level or of a block, a method called without a receiver, a constant. Names starting with an underscore stay private to the cell. The display is the value of the final statement when that statement is an expression rather than an assignment or a definition.
type CellAnalyzer ¶ added in v0.7.0
CellAnalyzer is implemented by a language whose analysis fills more of a Cell than Names (chunks, funcs, types, the display…). The graph uses it in place of Analyze; the generator needs that detail for the languages it compiles or splices.
type Chunk ¶
Chunk is a run of a cell's text that is either one top-level declaration (func, type or import) or a sequence of statements.
type Diagnostic ¶ added in v0.7.0
type Diagnostic = Error
Diagnostic is a problem found in a cell, at a byte offset of its source. Analyze leaves Cell zero; the graph sets it.
type Graph ¶
type Graph struct {
Cells []*Cell
// Owner maps each notebook variable to the cell that defines it.
Owner map[string]int
// Deps[i] lists the cells cell i reads from, sorted; Dependents is the
// reverse relation.
Deps, Dependents [][]int
// Order is an execution order: every cell after the cells it reads, and
// otherwise in document order. It holds every cell when Errors is empty.
Order []int
// Errors lists syntax errors, names defined twice, assignments to
// another cell's variable, and dependency cycles.
Errors []Error
}
Graph is the dependency graph of a notebook's code cells.
func Analyze ¶
Analyze analyses the sources of a notebook's Go code cells, in document order, and builds their dependency graph.
func AnalyzeSources ¶ added in v0.3.0
AnalyzeSources is Analyze for cells in Go and in Ruby. The two languages share their variables (see Shared), so a Go cell can read what a Ruby cell defines and the reverse, and the graph orders them together.
func (*Graph) Downstream ¶
Downstream returns the cells that must re-run when the given cells change: those cells and everything that depends on them, in execution order.
type Highlighter ¶ added in v0.7.0
type Highlighter interface {
Lexer() string
}
Highlighter is implemented by a language go-rouge has a lexer for; the others are shown escaped, uncoloured.
type Language ¶ added in v0.7.0
type Language interface {
// Name is the fence word: "go", "ruby", "julia"…
Name() string
// Analyze finds the names a cell defines and uses, and its syntax errors.
// It returns an error, not a panic, when src exceeds lim.
Analyze(src string, lim Limits) (Names, []Diagnostic, error)
// Crosses reports whether cell user, of another language, sees name,
// which cell owner, of this language, defines. Within a language every
// name is shared and Crosses is not asked.
Crosses(owner, user *Cell, name string) bool
// Run is the run capability of the language.
Run() Run
}
Language is a cell language, keyed by the first word of its fence's info string.
type Limits ¶ added in v0.7.0
type Limits struct {
MaxBytes int // the longest source, in bytes
MaxDepth int // the deepest nesting of brackets and blocks
}
Limits bound the work Analyze may do on one cell. They protect the notebook from a hostile or accidental huge cell; an implementation returns an error when a limit is exceeded.
type Module ¶ added in v0.7.0
type Module struct {
Path string // the import path of the package the generated program imports
Module string // the module that holds Path
Version string // the version the work module requires by default
}
Module is a Go module the worker needs to run a language's cells.
type NameChecker ¶ added in v0.7.0
NameChecker is implemented by a language that cannot hold every name: when a cell of another language defines a name, a cell of this language sees it only if ValidName holds (a Go variable cannot be named like a keyword).
type Names ¶ added in v0.7.0
type Names struct {
Defines, Uses []string
// Dynamic marks a cell whose names cannot be resolved statically: it
// assigns or reads names computed at run time (R's <<-, assign, get,
// eval(parse()), attach, library()). The graph makes such a cell depend on
// every earlier cell, and every later cell that reads any name depend on
// it.
Dynamic bool
}
Names are the notebook variables a cell defines and reads.
type Run ¶ added in v0.7.0
type Run struct {
// Runs is true when the language's cells execute.
Runs bool
// The rest is set when Runs is false: a cell is parsed and checked only.
// NotRun is the cell's own failure. Blocked(c) is the failure of a cell
// that reads, through the graph, what cell c defines; Reads(c) and
// ReadsBlocked(c) say the same when the generator defers a Go cell that
// reads cell c directly, or reads a cell blocked by cell c.
NotRun string
Blocked func(cell int) string
Reads func(cell int) string
ReadsBlocked func(cell int) string
}
Run says whether the cells of a language run, and what to tell when they do not.
type Runtime ¶ added in v0.7.0
type Runtime interface {
Runtime() Module
}
Runtime is implemented by a language that runs on a Go package the worker links.
type TypeLearner ¶ added in v0.7.0
type TypeLearner interface {
LearnsTypes() bool
}
TypeLearner is implemented by a language whose variables have no static Go type: the kernel learns it when the cell runs and compiles the Go cells that read the variable afterwards.