initializer

package
v0.0.0-...-a45e4f9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 11, 2025 License: MIT Imports: 26 Imported by: 0

README

README for the initializer

The initializer package supplies an Initializer struct which contains the data and supplies the methods necessary to initialize a Pipefish script, by creating and directing a parser and a compiler to operate on a VM. The Initializer returns a Compiler capable of dealing with runtime compilation of requests from the REPL: the Initializer can then be discarded together with its data.

There is one Initializer per module, as with compilers and parsers: an Initializer can spawn further Intitializers recursively to initialize modules and external services.

The initializer package consists of:

  • deserializer.go, which is used to deserialize the APIs of external services.
  • initializer.go, the main file directing initialization.
  • getters.go, which supplies some miscellaneous utility functions for getting and transforming data.
  • gohandler.go which does housekeeping for the Go interop.
  • gogen.go which generates Golang source files.

Fields of note in the Initializer struct are its compiler (naturally); its parser (a shortcut to the parser of the compiler); Common, a bindle of data that all the initializers of all the modules need to share; and GoBucket, which is used to accumulate the miscellaneous data swept up during parsing that we need to generate Go source files.

Documentation

Index

Constants

View Source 
const DUMMY = 4294967295

The maximum value of a `uint32`. Used as a dummy/sentinel value when `0` is not appropriate.

Variables

View Source 
var BUILTIN_FUNCTION_CONVERTER = map[string](func(t uint32, v any) any){
	"bool":   func(t uint32, v any) any { return v.(bool) },
	"float":  func(t uint32, v any) any { return v.(float64) },
	"int":    func(t uint32, v any) any { return v.(int) },
	"rune":   func(t uint32, v any) any { return v.(rune) },
	"string": func(t uint32, v any) any { return v.(string) },
}

But list, set, pair, and map can't go in here because of the recursion.

View Source 
var BUILTIN_VALUE_CONVERTER = map[string]any{
	"bool":   (*bool)(nil),
	"float":  (*float64)(nil),
	"int":    (*int)(nil),
	"rune":   (*rune)(nil),
	"string": (*string)(nil),
}
View Source 
var INTEROP_TOKEN = &token.Token{Source: "golang interop"}
View Source 
var LINKING_TOKEN = &token.Token{Source: "Pipefish linker"}

Tokens to return when no token is available.

View Source 
var PARSEABLE = []declarationType{cloneDeclaration, structDeclaration, constantDeclaration,
	variableDeclaration, functionDeclaration, commandDeclaration}

Functions

func SerializedAPIToDeclarations 

func SerializedAPIToDeclarations(serializedAPI string, xserve uint32) string

TODO --- malformed data would crash the deserializer with e.g. indexing errors. We need to make this a method of the initializer so it can Throw.

func StartCompiler 

func StartCompiler(scriptFilepath, sourcecode string, hubServices map[string]*compiler.Compiler, store *values.Map) *compiler.Compiler

We begin by manufacturing a blank VM, a `CommonParserBindle` for all the parsers to share, and a `CommonInitializerBindle` for the initializers to share. These Common bindles are then passed down to the "children" of the intitializer and the parser when new modules are created.

func StartCompilerFromFilepath 

func StartCompilerFromFilepath(filepath string, svs map[string]*compiler.Compiler, store *values.Map) (*compiler.Compiler, error)

func SummaryString 

func SummaryString(dec tokenizedCode) string

Gives a summary of the contents of a tokenizedCode object in which the tokens in the body of a function/interface/constrained type are just counted.

Types

type CommonInitializerBindle 

type CommonInitializerBindle struct {
	Functions      map[FuncSource]*parsedFunction // This is to ensure that the same function (i.e. from the same place in source code) isn't parsed more than once.
	DeclarationMap map[decKey]any                 // This prevents redeclaration of types in the same sort of way.
	HubCompilers   map[string]*compiler.Compiler  // This is a map of the compilers of all the (potential) external services on the same hub.
	HubStore       *values.Map
}

The CommonInitializerBindle contains information that all the initializers need to share.

func NewCommonInitializerBindle 

func NewCommonInitializerBindle(store *values.Map) *CommonInitializerBindle

Initializes the `CommonInitializerBindle`

type ExternalCallToHubHandler 

type ExternalCallToHubHandler struct {
	Evaluator    func(line string) values.Value
	ProblemFn    func() bool
	SerializeApi func() string
}

func (ExternalCallToHubHandler) Evaluate 

func (ex ExternalCallToHubHandler) Evaluate(line string) values.Value

There is a somewhat faster way of doing this when the services are on the same hub, since we would just need to change the type numbers. TODO. Until then, this serves as a good test bed for the external services on other hubs.

func (ExternalCallToHubHandler) GetAPI 

func (es ExternalCallToHubHandler) GetAPI() string

func (ExternalCallToHubHandler) Problem 

func (es ExternalCallToHubHandler) Problem() *err.Error

type ExternalHttpCallHandler 

type ExternalHttpCallHandler struct {
	Host         string
	Service      string
	Username     string
	Password     string
	Deserializer func(valAsString string) values.Value
}

func (ExternalHttpCallHandler) Evaluate 

func (es ExternalHttpCallHandler) Evaluate(line string) values.Value

func (ExternalHttpCallHandler) GetAPI 

func (es ExternalHttpCallHandler) GetAPI() string

func (ExternalHttpCallHandler) Problem 

func (es ExternalHttpCallHandler) Problem() *err.Error

type FuncSource 

type FuncSource struct {
	Filename     string
	LineNo       int
	FunctionName string
	Pos          uint32 // Exists to distinguish '-' as a prefix from '-' as an infix when defining clone types
}

For indexing the functions in the common function map, to prevent duplication.

type GoBucket 

type GoBucket struct {
	// contains filtered or unexported fields
}

This struct type is used to accumulate the various data encountered during parsing that we need to build a `.go` file or files. There is one per compiler. The "sources" are as given in the `Source` field of any GOCODE token encountered. The sources may be plural because of NULL-imports. The `imports`, `functions`, and `pureGo` maps are indexed by these sources.

type Initializer 

type Initializer struct {
	P *parser.Parser // The parser for the module being initialized.

	Snippets []string                 // Names of snippet types visible to the module.
	Common   *CommonInitializerBindle // The information all the initializers have in Common.
	// contains filtered or unexported fields
}

Definition of the Initializer type.

func NewInitializer 

func NewInitializer() *Initializer

Makes a new initializer.

func (*Initializer) Add 

func (iz *Initializer) Add(functionName string, f *parsedFunction) *parsedFunction

func (*Initializer) AddFieldsToStructsAndCheckForConsistency 

func (iz *Initializer) AddFieldsToStructsAndCheckForConsistency()

We assign abstract types to the fields of the structs, and chek for consistency of private types, i.e. a struct type declared public can't have field types declared private.

func (*Initializer) AddInOrder 

func (iz *Initializer) AddInOrder(S []*parsedFunction, f *parsedFunction) ([]*parsedFunction, *parsedFunction)

func (*Initializer) AddToNameSpace 

func (iz *Initializer) AddToNameSpace(thingsToImport []string)

Function auxilliary to phase 1. NULL-namespaced imports are first read from file and then tokenized into the same array of `TokenizedodeChunks` as the main file.

func (*Initializer) AddType 

func (iz *Initializer) AddType(name, supertype string, typeNo values.ValueType)

Adds a concrete type to the parser, and to the common types it falls under (at least `any` and `any?`).

func (*Initializer) AddTypeToVm 

func (iz *Initializer) AddTypeToVm(typeInfo values.AbstractTypeInfo)

Adds type information to the VM.

For reasons, it's a good idea to have the type info stored as an ordered list rather than a set or hashmap. So we need to do insertion by hand to avoid duplication.

func (*Initializer) ChunkConstOrVarDeclaration 

func (iz *Initializer) ChunkConstOrVarDeclaration(isConst, private bool) (tokenizedCode, bool)

As with all the chunkers, this assumes that the p.curToken is the first token of the thing we're trying to slurp. It will end with the p.curTok being the EOF/NEWLINE terminating the declaration.

func (*Initializer) ChunkFunction 

func (iz *Initializer) ChunkFunction(cmd, private bool) (*tokenizedFunctionDeclaration, bool)

As with all the chunkers, this assumes that the p.curToken is the first token of the thing we're trying to slurp. It will end with the p.curTok being the EOF/NEWLINE terminating the declaration.

func (*Initializer) ChunkFunctionSignature 

func (iz *Initializer) ChunkFunctionSignature() (*tokenizedFunctionDeclaration, bool)

This wraps around chunkFunctionArguments and extracts the right name.

func (*Initializer) ChunkGolangDeclaration 

func (iz *Initializer) ChunkGolangDeclaration(private bool) (tokenizedCode, bool)

func (*Initializer) ChunkImportOrExternalDeclaration 

func (iz *Initializer) ChunkImportOrExternalDeclaration(isExternal, private bool) (tokenizedCode, bool)

As with all the chunkers, this assumes that the p.curToken is the first token of the thing we're trying to slurp. It will end with the p.curTok being the EOF/NEWLINE terminating the declaration.

func (*Initializer) ChunkTypeDeclaration 

func (iz *Initializer) ChunkTypeDeclaration(private bool) (tokenizedCode, bool)

As with all the chunkers, this assumes that the p.curToken is the first token of the thing we're trying to slurp. It will end with the p.curTok being the EOF/NEWLINE terminating the declaration.

func (*Initializer) CompileEverything 

func (iz *Initializer) CompileEverything() [][]labeledParsedCodeChunk

Phase 4 of compilation. We compile the constants, variables, functions, and commands.

func (*Initializer) ErrorsExist 

func (iz *Initializer) ErrorsExist() bool

Return whether the initializer has encountered errors.

func (*Initializer) FindParameterizedType 

func (iz *Initializer) FindParameterizedType(name string, argsToCheck []values.Value) values.ValueType

TODO --- there should be more performant ways of doing this but for now I'll just settle for it working.

func (*Initializer) IsMandatoryImport 

func (iz *Initializer) IsMandatoryImport(aT values.AbstractTypeInfo) bool

func (*Initializer) MakeFunctionForests 

func (iz *Initializer) MakeFunctionForests()

func (*Initializer) MakeFunctionTables 

func (iz *Initializer) MakeFunctionTables()

Phase 4 of compilation. At this point we have our functions as parsed code chunks in the `uP.Parser.ParsedDeclarations(<function/command>Declaration)` slice. We want to read their signatures and order them according to specificity for the purposes of implementing overloading.

func (*Initializer) MakeFunctionTrees 

func (iz *Initializer) MakeFunctionTrees()

Function auxiliary to the above. Having made the parsers FunctionTable, each function name is associated with a (partially) ordered list of associated functions such that a more specific type signature comes before a less specific one. We will now re-represent this as a tree.

func (*Initializer) ParameterTypesMatch 

func (iz *Initializer) ParameterTypesMatch(paramsToCheck, paramsToMatch []values.ValueType) bool

func (*Initializer) ParseEverythingFromFilePath 

func (iz *Initializer) ParseEverythingFromFilePath(mc *vm.Vm, cpb *parser.CommonParserBindle, ccb *compiler.CommonCompilerBindle, scriptFilepath, namespacePath string) (*compiler.Compiler, error)

func (*Initializer) ParseEverythingFromSourcecode 

func (iz *Initializer) ParseEverythingFromSourcecode(mc *vm.Vm, cpb *parser.CommonParserBindle, ccb *compiler.CommonCompilerBindle, scriptFilepath, sourcecode, namespacePath string) *compiler.Compiler

This initializes the initializer's compiler (which initializes its parser), and extracts the source code from the given file, and then calls the `parseEverything“ method, below.

func (*Initializer) ParseNamespacedImportsAndReturnUnnamespacedImports 

func (iz *Initializer) ParseNamespacedImportsAndReturnUnnamespacedImports() []string

We call ParseEverything on the namespaced imports, returning a list of unnamespaced imports which the main phase 1 function will add to the parser.

func (*Initializer) ResolveInterfaceBacktracks 

func (iz *Initializer) ResolveInterfaceBacktracks()

We left DUMMY values in the code for where we'd call a function which fits the interface but hasn't been defined yet. Now we go back and fill in the gaps. TODO --- why are these stored in the common parser bindle and not the common initializer bindle?

func (*Initializer) SerializeApi 

func (iz *Initializer) SerializeApi() string

For a description of the file format, see README-api-serialization.md

func (*Initializer) Throw 

func (iz *Initializer) Throw(errorID string, tok *token.Token, args ...any)

Like everything else, the initializer sends its errors to the Common parser bindle via the parser.

type ParameterInfo 

type ParameterInfo struct {
	Names      []string
	Types      []values.ValueType
	Operations []token.Token
	Typecheck  *token.TokenizedCodeChunk
	ParentType string     // 'struct' if not a clone
	Sig        ast.AstSig // nil if not a struct (because the sig of a clone is implicit in the parent type).
	IsPrivate  bool
	Supertype  string
	Token      *token.Token
}

type TokenizedCodeChunks 

type TokenizedCodeChunks []*token.TokenizedCodeChunk

Stores pretokenized chunks of code for later parsing.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.