env

package
v2.0.0-dev0.1.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 19, 2026 License: BSD-3-Clause Imports: 11 Imported by: 27

README

Docs: GoDoc

Package env defines the Env interface for environments, which determine the nature and sequence of States as inputs to a model. Action responses from the model can also drive state evolution.

By adhering to this interface, it is then easier to mix-and-match environments with models.

Env / Agent

Multiple different environments will typically be used in a model, e.g., one for training and other(s) for testing. Even if these envs all share a common database of patterns, a different Env should be used for each case where different counters and sequences of events etc are presented, which keeps them from interfering with each other. Also, table.NewView can be used to create new views on a common set of pattenrs, so different Envs can present different indexed views. The basic FixedTable env implementation uses this.

The standard String() string fmt.Stringer method must be defined to return a string description of the current environment state, e.g., as a TrialName. A Label() string method must be defined to return the Name of the environment, which is typically the Mode of usage (Train vs. Test).

There is also an Envs map that provides a basic container for managing multiple Envs, using a string key based on the Label() name.

The Step should update all relevant state elements as appropriate, so these can be queried by the user. Particular paradigms of environments must establish naming conventions for these state elements which then allow the model to use the information appropriately -- the Env interface only provides the most basic framework for establishing these paradigms, and ultimately a given model will only work within a particular paradigm of environments following specific conventions.

See e.g., env.FixedTable for particular implementation of a fixed Table of patterns, for one example of a widely used paradigm.

Typically each specific implementation of this Env interface will have multiple parameters etc that can be modified to control env behavior -- all of this is paradigm-specific and outside the scope of this basic interface.

Documentation

Overview

Package env defines an interface for environments, which determine the nature and sequence of States as inputs to a model. Action responses from the model can also drive state evolution.

State is comprised of one or more Elements, each of which consists of an tensor.Values chunk of values that can be obtained by the model. Likewise, Actions can also have Elements. The Step method is the main interface for advancing the Env state.

The standard String() string fmt.Stringer method must be defined to return a string description of the current environment state, e.g., as a TrialName. A Label() string method must be defined to return the Name of the environment, which is typically the Mode of usage (Train vs. Test).

Typically each specific implementation of this Env interface will have multiple parameters etc that can be modified to control env behavior: all of this is paradigm-specific and outside the scope of this basic interface.

See e.g., env.FixedTable for particular implementation of a fixed Table of patterns, for one example of a widely used paradigm.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ModeDi

func ModeDi(mode enums.Enum, di int) string

ModeDi returns the string of the given mode appended with _di data index with leading zero.

Types

type Counter

type Counter struct {

	// Cur is the current counter value.
	Cur int

	// Prev previous counter value, prior to last Incr() call (init to -1)
	Prev int `display:"-"`

	// Changed reports if it changed on the last Step() call or not.
	Changed bool `display:"-"`

	// Max is the maximum counter value, above which the counter will reset back to 0.
	// Only used if > 0.
	Max int
}

Counter maintains a current and previous counter value, and a Max value with methods to manage.

func (*Counter) Incr

func (ct *Counter) Incr() bool

Incr increments the counter by 1. If Max > 0 then if Incr >= Max the counter is reset to 0 and true is returned. Otherwise false.

func (*Counter) Init

func (ct *Counter) Init()

Init initializes counter: Cur = 0, Prev = -1

func (*Counter) Query

func (ct *Counter) Query() (cur, prev int, chg bool)

Query returns the current, previous and changed values for this counter

func (*Counter) Same

func (ct *Counter) Same()

Same resets Changed = false -- good idea to call this on all counters at start of Step or can put in an else statement, but that is more error-prone.

func (*Counter) Set

func (ct *Counter) Set(cur int) bool

Set sets the Cur value if different from Cur, while preserving previous value and setting Changed appropriately. Returns true if changed. does NOT check Cur vs. Max.

type CurPrev

type CurPrev[T tensor.DataTypes] struct {
	Cur, Prev T
}

CurPrev manages current and previous values for basic data types.

func (*CurPrev[T]) Set

func (cv *CurPrev[T]) Set(cur T)

Set sets the new current value, after saving Cur to Prev.

type CurPrevString

type CurPrevString = CurPrev[string]

type Env

type Env interface {
	fmt.Stringer
	labels.Labeler

	// Init initializes the environment for a given run of the model.
	// It is best if the Env has its own random seed and random sequence
	// generator (e.g., lab/randx), and sets a new random seed for each run.
	// It may also implement different parameterizations for different runs
	// (e.g., between-subject manipulations).
	// See Step() for important info about state of env after Init
	// but prior to first Step() call.
	Init(run int)

	// Step generates the next step of environment state.
	// This is the main API for how the model interacts with the environment.
	// The env should update all other levels of state internally over
	// repeated calls to the Step method.
	// If there are no further inputs available, it returns false (most envs
	// typically only return true and just continue running as long as needed).
	//
	// The Env thus always reflects the _current_ state of things, and this
	// call increments that current state, such that subsequent calls to
	// State() will return this current state.
	//
	// This implies that the state just after Init and prior to first Step
	// call should be an _initialized_ state that then allows the first Step
	// call to establish the proper _first_ state. Typically this means that
	// one or more counters will be set to -1 during Init and then get incremented
	// to 0 on the first Step call.
	Step() bool

	// State returns the given element's worth of tensor data from the environment
	// based on the current state of the env, as a function of having called Step().
	// If no output is available on that element, then nil is returned.
	// The returned tensor must be treated as read-only as it likely points to original
	// source data: please make a copy before modifying (e.g., Clone() methdod).
	State(element string) tensor.Values
}

Env defines an interface for environments, which determine the nature and sequence of States as inputs to a model. Action responses from the model can also drive state evolution.

State is comprised of one or more Elements, each of which consists of an tensor.Values chunk of values that can be obtained by the model. The Step method advances the Env state.

The standard String() string fmt.Stringer method must be defined to return a string description of the current environment state, e.g., as a TrialName. A Label() string method must be defined to return the Name of the environment, which is typically the Mode of usage (Train vs. Test).

Typically each specific implementation of this Env interface will have multiple parameters etc that can be modified to control env behavior: all of this is paradigm-specific and outside the scope of this basic interface.

type Envs

type Envs map[string]Env

Envs is a map of environments organized according to the evaluation mode string (recommended key value)

func (*Envs) Add

func (es *Envs) Add(evs ...Env)

Add adds Env(s), using its Label as the key

func (*Envs) ByMode

func (es *Envs) ByMode(mode enums.Enum) Env

ByMode returns env by Modes evaluation mode as the map key. returns nil if not found.

func (*Envs) ByModeDi

func (es *Envs) ByModeDi(mode enums.Enum, di int) Env

ByModeDi returns env by etime.Modes evaluation mode and data parallel index as the map key, using ModeDi function. returns nil if not found.

func (*Envs) Init

func (es *Envs) Init()

Init initializes the map if not yet

type FixedTable

type FixedTable struct {
	// name of this environment, usually Train vs. Test.
	Name string

	// Table has the set of patterns to output.
	// The indexes are used for the *sequential* view so you can easily
	// sort / split / filter the patterns to be presented using this view.
	// This adds the random permuted Order on top of those if !sequential.
	Table *table.Table

	// present items from the table in sequential order (i.e., according to
	// the indexed view on the Table)?  otherwise permuted random order.
	Sequential bool

	// permuted order of items to present if not sequential.
	// updated every time through the list.
	Order []int

	// current ordinal item in Table. if Sequential then = row number in table,
	// otherwise is index in Order list that then gives row number in Table.
	Trial Counter `display:"inline"`

	// if Table has a Name column, this is the contents of that.
	TrialName CurPrevString

	// if Table has a Group column, this is contents of that.
	GroupName CurPrevString

	// name of the Name column -- defaults to 'Name'.
	NameCol string

	// name of the Group column -- defaults to 'Group'.
	GroupCol string
}

FixedTable is a basic Env that manages patterns from a table.Table, with either sequential or permuted random ordering, with a Trial counter to record progress and iterations through the table. Use table.NewView to provide a unique indexed view of a shared table.

func (*FixedTable) Config

func (ft *FixedTable) Config(tbl *table.Table)

Config configures the environment to use given table IndexView and evaluation mode (e.g., etime.Train.String()). If mode is Train then a Run counter is added, otherwise just Epoch and Trial. NameCol and GroupCol are initialized to "Name" and "Group" so set these to something else after this if needed.

func (*FixedTable) Init

func (ft *FixedTable) Init(run int)

func (*FixedTable) Label

func (ft *FixedTable) Label() string

func (*FixedTable) NewOrder

func (ft *FixedTable) NewOrder()

NewOrder sets a new random Order based on number of rows in the table.

func (*FixedTable) PermuteOrder

func (ft *FixedTable) PermuteOrder()

PermuteOrder permutes the existing order table to get a new random sequence of inputs just calls: randx.PermuteInts(ft.Order)

func (*FixedTable) Row

func (ft *FixedTable) Row() int

Row returns the current row number in table, based on Sequential / perumuted Order.

func (*FixedTable) SetGroupName

func (ft *FixedTable) SetGroupName()

func (*FixedTable) SetTrialName

func (ft *FixedTable) SetTrialName()

func (*FixedTable) State

func (ft *FixedTable) State(element string) tensor.Values

func (*FixedTable) Step

func (ft *FixedTable) Step() bool

func (*FixedTable) String

func (ft *FixedTable) String() string

func (*FixedTable) Validate

func (ft *FixedTable) Validate() error

type FreqTable

type FreqTable struct {

	// name of this environment
	Name string

	// Table has the set of patterns to output.
	// The indexes are used for the *sequential* view so you can easily
	// sort / split / filter the patterns to be presented using this view.
	// This adds the random permuted Order on top of those if !sequential.
	Table *table.Table

	// number of samples to use in constructing the list of items to present according to frequency -- number per epoch ~ NSamples * Freq -- see RandSamp option
	NSamples float64

	// if true, use random sampling of items NSamples times according to given Freq probability value -- otherwise just directly add NSamples * Freq items to the list
	RandSamp bool

	// present items from the table in sequential order (i.e., according to the indexed view on the Table)?  otherwise permuted random order.  All repetitions of given item will be sequential if Sequential
	Sequential bool

	// list of items to present, with repetitions -- updated every time through the list
	Order []int

	// current ordinal item in Table -- if Sequential then = row number in table, otherwise is index in Order list that then gives row number in Table
	Trial Counter `display:"inline"`

	// if Table has a Name column, this is the contents of that
	TrialName CurPrevString

	// if Table has a Group column, this is contents of that
	GroupName CurPrevString

	// name of the Name column -- defaults to 'Name'
	NameCol string

	// name of the Group column -- defaults to 'Group'
	GroupCol string

	// name of the Freq column -- defaults to 'Freq'
	FreqCol string
}

FreqTable is an Env that manages patterns from an table.Table with frequency information so that items are presented according to their associated frequencies which are effectively probabilities of presenting any given input -- must have a Freq column with these numbers in the table (actual col name in FreqCol). Either sequential or permuted random ordering is supported, with std Trial / Epoch TimeScale counters to record progress and iterations through the table. It also records the outer loop of Run as provided by the model. It uses an IndexView indexed view of the Table, so a single shared table can be used across different environments, with each having its own unique view.

func (*FreqTable) Init

func (ft *FreqTable) Init(run int)

func (*FreqTable) Label

func (ft *FreqTable) Label() string

func (*FreqTable) Row

func (ft *FreqTable) Row() int

Row returns the current row number in table, based on Sequential / perumuted Order and already de-referenced through the IndexView's indexes to get the actual row in the table.

func (*FreqTable) Sample

func (ft *FreqTable) Sample()

Sample generates a new sample of items

func (*FreqTable) SetGroupName

func (ft *FreqTable) SetGroupName()

func (*FreqTable) SetTrialName

func (ft *FreqTable) SetTrialName()

func (*FreqTable) State

func (ft *FreqTable) State(element string) tensor.Values

func (*FreqTable) Step

func (ft *FreqTable) Step() bool

func (*FreqTable) String

func (ft *FreqTable) String() string

func (*FreqTable) Validate

func (ft *FreqTable) Validate() error

type MPIFixedTable

type MPIFixedTable struct {

	// name of this environment
	Name string

	// Table has the set of patterns to output.
	// The indexes are used for the *sequential* view so you can easily
	// sort / split / filter the patterns to be presented using this view.
	// This adds the random permuted Order on top of those if !sequential.
	Table *table.Table

	// present items from the table in sequential order (i.e., according to the indexed view on the Table)?  otherwise permuted random order
	Sequential bool

	// permuted order of items to present if not sequential -- updated every time through the list
	Order []int

	// current ordinal item in Table -- if Sequential then = row number in table, otherwise is index in Order list that then gives row number in Table
	Trial Counter `display:"inline"`

	// if Table has a Name column, this is the contents of that
	TrialName CurPrevString

	// if Table has a Group column, this is contents of that
	GroupName CurPrevString

	// name of the Name column -- defaults to 'Name'
	NameCol string

	// name of the Group column -- defaults to 'Group'
	GroupCol string

	// for MPI, trial we start each epoch on, as index into Order
	TrialSt int

	// for MPI, trial number we end each epoch before (i.e., when ctr gets to Ed, restarts)
	TrialEd int
}

MPIFixedTable is an MPI-enabled version of the FixedTable, which is a basic Env that manages patterns from a [table.Table[, with either sequential or permuted random ordering, and a Trial counter to record iterations through the table. Use table.NewView to provide a unique indexed view of a shared table. The MPI version distributes trials across MPI procs, in the Order list. It is ESSENTIAL that the number of trials (rows) in Table is evenly divisible by number of MPI procs! If all nodes start with the same seed, it should remain synchronized.

func (*MPIFixedTable) Init

func (ft *MPIFixedTable) Init(run int)

func (*MPIFixedTable) Label

func (ft *MPIFixedTable) Label() string

func (*MPIFixedTable) NewOrder

func (ft *MPIFixedTable) NewOrder()

NewOrder sets a new random Order based on number of rows in the table.

func (*MPIFixedTable) PermuteOrder

func (ft *MPIFixedTable) PermuteOrder()

PermuteOrder permutes the existing order table to get a new random sequence of inputs just calls: randx.PermuteInts(ft.Order)

func (*MPIFixedTable) Row

func (ft *MPIFixedTable) Row() int

Row returns the current row number in table, based on Sequential / perumuted Order.

func (*MPIFixedTable) SetGroupName

func (ft *MPIFixedTable) SetGroupName()

func (*MPIFixedTable) SetTrialName

func (ft *MPIFixedTable) SetTrialName()

func (*MPIFixedTable) State

func (ft *MPIFixedTable) State(element string) tensor.Values

func (*MPIFixedTable) Step

func (ft *MPIFixedTable) Step() bool

func (*MPIFixedTable) String

func (ft *MPIFixedTable) String() string

func (*MPIFixedTable) Validate

func (ft *MPIFixedTable) Validate() error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL