automatic

package
v0.13.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 19, 2026 License: GPL-3.0 Imports: 47 Imported by: 1

Documentation

Overview

Package automatic contains all the logic for the actual gameplay of Crossword Game, which, as we said before, features all sorts of things like wingos and blonks.

Index

Constants

View Source
const InferenceLogColumns = ",inferCount,trueLeave,truePost,truePrior,liftBits,trueRank,inferLeaves,trueMeasured"

InferenceLogColumns are the per-turn log columns describing one inference. They are present on every row of a run that has an inferring bot, and empty on the rows of a bot that does not infer. See inferenceFields.

Variables

View Source
var (
	CVCCounter *expvar.Int
	IsPlaying  *expvar.Int
)
View Source
var MaxTimePerEndgame = 15 * time.Second
View Source
var MaxTimePerTurn = 30 * time.Second

Functions

func AnalyzeLogFile added in v0.4.2

func AnalyzeLogFile(filepath string) (string, error)

AnalyzeLogFile analyzes the given game CSV file and returns formatted stats. Kept for backward compatibility.

func DeriveSeed added in v0.13.8

func DeriveSeed(master uint64, unitIdx int) [32]byte

DeriveSeed produces the seed for one unit of work (a game, or a pair of games) from the run's master seed. Deriving rather than drawing means a worker can be handed unit 900 without having played the first 899, and the same master seed always replays the same experiment.

func ExportGCG added in v0.4.10

func ExportGCG(cfg *config.Config, filename, letterdist, lexicon, boardlayout, gid string,
	half int, out io.Writer) error

ExportGCG writes one game from an autoplay log as GCG. In a -gamepairs run a game ID names a pair rather than a game, so half picks which of its two seatings to write; it is 1 for anything else.

func FormatDivergence added in v0.13.8

func FormatDivergence(r *DivergenceReport) string

FormatDivergence renders a divergence report for the shell.

func FormatInferenceQuality added in v0.13.8

func FormatInferenceQuality(q *InferenceQuality) string

FormatInferenceQuality renders the grading for the shell.

func FormatTable added in v0.13.0

func FormatTable(r *AnalysisResult) string

FormatTable formats analysis results as a side-by-side comparison table.

func GenerateSeeds added in v0.11.8

func GenerateSeeds(n int) ([][32]byte, error)

GenerateSeeds creates n random 32-byte seeds for deterministic game runs

func LoadAutoplayConfig added in v0.13.0

func LoadAutoplayConfig(path string) (*pb.AutoplayConfig, error)

LoadAutoplayConfig reads an AutoplayConfig from a protojson file.

func LoadSeeds added in v0.11.8

func LoadSeeds(path string) ([][32]byte, error)

LoadSeeds reads seeds from a file in base64 format

func RandomMasterSeed added in v0.13.8

func RandomMasterSeed() (uint64, error)

RandomMasterSeed returns a master seed for a run that asked for reproducible games without naming one.

func ResolveExperimentID added in v0.13.0

func ResolveExperimentID(cfg *pb.AutoplayConfig) string

ResolveExperimentID returns the experiment ID from the config, or generates one from the current timestamp and 6 random hex characters.

func SaveSeeds added in v0.11.8

func SaveSeeds(seeds [][32]byte, path string) error

SaveSeeds writes seeds to a file in base64 format (one per line)

func StartAutoplayFromConfig added in v0.13.0

func StartAutoplayFromConfig(ctx context.Context, appCfg *config.Config, expCfg *pb.AutoplayConfig) (string, error)

StartAutoplayFromConfig runs an autoplay experiment defined by a protojson config file. It returns the resolved experiment ID. Output files are written to cfg.OutputDir (or the current directory if empty):

  • {experimentId}.txt — per-turn log
  • games-{experimentId}.txt — per-game summary
  • {experimentId}.config.json — copy of the config for reproducibility

func StartCompVCompStaticGames

func StartCompVCompStaticGames(ctx context.Context, cfg *config.Config,
	numGames int, block bool, threads int,
	outputFilename, lexicon, letterDistribution string,
	players []AutomaticRunnerPlayer, detConfig *DeterministicConfig) error

func StartServer added in v0.13.0

func StartServer(addr string) error

StartServer starts an HTTP server on the given address that exposes the autoanalyze endpoint. Call as:

curl 'http://host:8080/analyze?file=games.txt'
curl 'http://host:8080/analyze?file=games.txt&format=json'

func TurnLogFor added in v0.13.8

func TurnLogFor(gamesFile string) string

TurnLogFor guesses the per-turn log that goes with a per-game log: autoplay writes games-{id}.txt beside {id}.txt.

Types

type AnalysisResult added in v0.13.0

type AnalysisResult struct {
	GamesPlayed   int
	Player1       PlayerStats
	Player2       PlayerStats
	P1Wins        float64
	P1First       float64
	WentFirstWins float64
	WinPValue     float64 // two-sided binomial z-test, H0: win rate = 0.5
	ScorePValue   float64 // two-sided paired z-test on per-game score diff
	ScoreDiff     *stats.Statistic

	// Paired is filled in for a -gamepairs run, where the two games of a pair
	// share a bag and cannot be treated as independent.
	Paired *PairedResult
}

AnalysisResult holds the full results of analyzing a log file.

func AnalyzeLogFileData added in v0.13.0

func AnalyzeLogFileData(filepath string) (*AnalysisResult, error)

AnalyzeLogFileData analyzes the given game CSV file and returns structured results.

type AutomaticRunnerPlayer added in v0.5.0

type AutomaticRunnerPlayer struct {
	LeaveFile            string
	PEGFile              string
	BotCode              pb.BotRequest_BotCode
	MinSimPlies          int
	SimThreads           int
	StochasticStaticEval bool
	// InferenceTau pins the softmax temperature for P(play | leave). Leave it
	// zero for the engine default (SoftmaxTemperature); a value here applies to
	// the whole game.
	InferenceTau                 float64
	InferenceTimeSecs            int
	InferenceSimIters            int
	InferenceMaxEnumeratedLeaves int
	InferenceBudget              int
	OracleInference              bool
}

func PlayersFromConfig added in v0.13.0

func PlayersFromConfig(cfg *pb.AutoplayConfig) []AutomaticRunnerPlayer

PlayersFromConfig converts an AutoplayConfig's player configs to AutomaticRunnerPlayer structs.

type CorpusFilter added in v0.13.8

type CorpusFilter struct {
	// LeaveLen, when non-zero, keeps only inferences of that many tiles. Six
	// is the one-tile-play bucket, where the read goes wrong.
	LeaveLen int
	// MaxBag and MinBag bound the phase; MaxBag 0 means no upper bound.
	MinBag, MaxBag int
	// WorseThan keeps only inferences the run scored below this many bits.
	// Leave at zero and pass AllLifts to take everything.
	WorseThan float64
	AllLifts  bool
	// Limit caps how many positions are returned, 0 for all.
	Limit int
}

CorpusFilter narrows which positions to lift out.

type CorpusPosition added in v0.13.8

type CorpusPosition struct {
	// GameID and Half say which game and which seating it came from; Turn is
	// the play the inference was reading, 1-based as the log numbers them.
	GameID string
	Half   int
	Turn   int

	// Game is replayed up to and including the opponent's play, so the
	// position is exactly what the inferring bot faced -- including the
	// inferring player's own rack, which decides the unseen pool and so the
	// prior every inference is graded against.
	Game *game.Game

	// MyRack is the inferring player's rack at this position, taken from the
	// row where they play next. Replaying alone cannot recover it: the tiles
	// they drew came off a bag this replay never had in the same order.
	MyRack string

	// TrueLeave is what the opponent actually kept -- the answer. OppRack and
	// OppPlay are what produced it.
	TrueLeave string
	OppRack   string
	OppPlay   string
	OppScore  int

	// TilesRemaining is the bag count after the play, and LeaveLen the number
	// of tiles being inferred; both are what the run's own breakdowns cut on.
	TilesRemaining int
	LeaveLen       int

	// LoggedLiftBits is what the original run scored this inference, when it
	// recorded one. A replay that reproduces the run should land near it.
	LoggedLiftBits  float64
	LoggedMeasured  bool
	LoggedRank      int
	LoggedLeaves    int
	HasLoggedResult bool
}

CorpusPosition is one inference problem lifted out of a run.

func LoadCorpus added in v0.13.8

func LoadCorpus(cfg *config.Config, turnFile, lexicon, letterdist, boardlayout string,
	filter CorpusFilter) ([]*CorpusPosition, error)

LoadCorpus replays a run's per-turn log and returns the positions matching the filter, each rebuilt to the moment the inference ran. Games that will not replay are skipped and counted in Skipped; a corpus that comes back empty because every game failed is reported as an error rather than as an empty result, since the two look identical to a caller and mean opposite things.

func LoadCorpusVerbose added in v0.13.8

func LoadCorpusVerbose(cfg *config.Config, turnFile, lexicon, letterdist, boardlayout string,
	filter CorpusFilter) ([]*CorpusPosition, int, error)

LoadCorpusVerbose is LoadCorpus, also returning how many games were skipped because they would not replay.

type DeterministicConfig added in v0.11.8

type DeterministicConfig struct {
	Seeds    [][32]byte // Pre-generated seeds (nil = generate new)
	SeedFile string     // File to read/write seeds
	NumGames int        // Number of games (used when generating seeds)
	// MasterSeed derives a seed per unit of work when there is no seed file.
	MasterSeed uint64
	// GamePairs plays each seed twice with the bots swapping seats.
	GamePairs bool
}

DeterministicConfig holds configuration for deterministic autoplay runs

type DivergenceKind added in v0.13.8

type DivergenceKind int

DivergenceKind says what parted the two halves of a pair.

const (
	// DivergenceNone means the halves played the same game throughout.
	DivergenceNone DivergenceKind = iota
	// DivergenceChoice is the normal case: same rack, different play.
	DivergenceChoice
	// DivergenceTiles means the halves were dealt different racks. The pairing
	// is broken; this should never happen.
	DivergenceTiles
	// DivergenceLength means the halves agreed on every shared turn but one ran
	// longer. Also a bug, since identical play cannot produce different games.
	DivergenceLength
)

func (DivergenceKind) String added in v0.13.8

func (k DivergenceKind) String() string

type DivergenceReport added in v0.13.8

type DivergenceReport struct {
	Bot1, Bot2 string
	Pairs      int
	Divergent  int
	ExactTies  int
	// Broken counts pairs whose halves saw different tiles or ran different
	// lengths -- bugs rather than strategy differences.
	Broken int
	// NoTurnData counts divergent pairs the per-turn log had nothing for.
	NoTurnData int
	// TurnStats is the first-divergence turn across every divergent pair.
	TurnStats *stats.Statistic
	// EarliestTurn and LatestTurn bound the same distribution.
	EarliestTurn, LatestTurn int
	// Details covers the first maxDetails divergent pairs, in pair order.
	Details []PairDivergence
	// Truncated is how many divergent pairs are not in Details.
	Truncated int
}

DivergenceReport summarizes a whole paired run.

func AnalyzeDivergence added in v0.13.8

func AnalyzeDivergence(gamesFile, turnFile string, maxDetails int) (*DivergenceReport, error)

AnalyzeDivergence reads a paired run's per-game log and the per-turn log beside it, and reports where each divergent pair's halves parted ways. maxDetails caps how many pairs are described individually; the counts and the turn statistics cover them all.

type GameRunner

type GameRunner struct {
	// contains filtered or unexported fields
}

GameRunner is the master struct here for the automatic game logic.

func NewGameRunner added in v0.4.2

func NewGameRunner(logchan chan string, cfg *config.Config) *GameRunner

NewGameRunner just instantiates and initializes a game runner.

func (*GameRunner) BotTypeFor added in v0.13.0

func (r *GameRunner) BotTypeFor(playerIdx int) pb.BotRequest_BotCode

BotTypeFor returns the bot code for the given game-level player index. Accounts for player flips done by StartGameWithSeed.

func (*GameRunner) CompVsCompStatic

func (r *GameRunner) CompVsCompStatic(addToHistory bool) error

CompVsCompStatic plays out a game to the end using best static turns, HastyBot against HastyBot on a randomly seeded bag.

func (*GameRunner) CompVsCompStaticBots added in v0.13.7

func (r *GameRunner) CompVsCompStaticBots(addToHistory bool, players []AutomaticRunnerPlayer,
	gidx int, seed [32]byte) error

CompVsCompStaticBots plays out one game to the end with the given bots. It is CompVsCompStatic's parameterized form, for callers that want a different bot pair or a reproducible bag: a non-zero seed fixes the tile order and, through SetUidFromSeed, the game's UID as well. gidx alternates who goes first.

func (*GameRunner) Game added in v0.4.5

func (r *GameRunner) Game() *game.Game

func (*GameRunner) Init

func (r *GameRunner) Init(players []AutomaticRunnerPlayer) error

Init initializes the runner

func (*GameRunner) PlayBestTurn added in v0.5.0

func (r *GameRunner) PlayBestTurn(playerIdx int, addToHistory bool) error

PlayBestTurn generates the best move for the player and plays it on the board.

func (*GameRunner) StartGame

func (r *GameRunner) StartGame(gidx int)

func (*GameRunner) StartGameWithSeed added in v0.11.8

func (r *GameRunner) StartGameWithSeed(gidx int, seed [32]byte)

type InferenceQuality added in v0.13.8

type InferenceQuality struct {
	// Inferences is how many turns produced a graded posterior.
	Inferences int
	// Unscored is how many turns ran inference but had nothing to grade: no
	// history yet, an empty bag, or a bingo that left no tiles to infer.
	Unscored int

	// LiftBits is log2(posterior/prior) per inference. Its mean is the headline.
	LiftBits *stats.Statistic
	// Posterior and Prior are the raw probabilities, kept for context since the
	// lift hides how small both are in a large leave space.
	Posterior, Prior *stats.Statistic
	// RankPct is the true leave's percentile among candidates, 0 being the top.
	// Robust to leave-space size in a way the raw rank is not.
	RankPct *stats.Statistic
	// Leaves is the size of the posterior per inference.
	Leaves *stats.Statistic

	// Better and Worse count inferences that beat and lost to the prior.
	Better, Worse int
	// RuledOut counts inferences that gave the true leave no weight at all --
	// the expensive mistake, since the simmer then never samples the truth.
	RuledOut int
	// Measured counts inferences where the true leave got a real mini-sim rather
	// than an imputed weight.
	Measured int

	// ByLeaveLen breaks the mean lift down by how many tiles were being inferred,
	// which is where inference gets hard.
	ByLeaveLen map[int]*stats.Statistic
	// contains filtered or unexported fields
}

InferenceQuality summarizes how well a run's inferences matched the truth.

func AnalyzeInferenceQuality added in v0.13.8

func AnalyzeInferenceQuality(turnFile string) (*InferenceQuality, error)

AnalyzeInferenceQuality reads a per-turn autoplay log and grades its inferences. It wants the per-turn log, not the per-game one.

func (*InferenceQuality) LiftPercentile added in v0.13.8

func (q *InferenceQuality) LiftPercentile(p float64) float64

LiftPercentile returns the p-th percentile of the per-inference lift, p in [0,100]. NaN when nothing was graded.

type Job

type Job struct {
	// contains filtered or unexported fields
}

type PairDivergence added in v0.13.8

type PairDivergence struct {
	Pair   int
	GameID string
	Kind   DivergenceKind

	// Turn is the first turn the halves disagreed on, 1-based.
	Turn int
	// Rack is what both halves held on that turn. When Kind is
	// DivergenceTiles the two halves held RackA and RackB instead.
	Rack         string
	RackA, RackB string

	// BotA played PlayA in the first half; BotB played PlayB in the second,
	// from the same seat and the same tiles.
	BotA, BotB       string
	PlayA, PlayB     string
	ScoreA, ScoreB   int
	EquityA, EquityB float64

	TurnsA, TurnsB int
	// ScoreDiff is bot 1's score minus bot 2's, summed over both halves. A pair
	// that never diverges scores exactly 0 here.
	ScoreDiff int
}

PairDivergence is where and how one pair's halves parted ways.

type PairedResult added in v0.13.8

type PairedResult struct {
	Pairs int
	// Incomplete counts pairs that did not have both halves in the log -- a run
	// stopped midway, say. They are left out of everything below.
	Incomplete int
	// Divergent counts pairs whose halves ever chose differently, straight from
	// the log column. Pairs that never diverge contribute an exact zero, so this
	// is the count that carries any information.
	Divergent int
	// ExactTies counts pairs whose two margins cancel to zero.
	ExactTies int

	// Swept, Split and Lost count pairs where bot 1 won both games, one each,
	// and neither -- the plainest statement of a paired result, since the
	// pairs that split are the ones tile luck decided. WithDraw counts pairs
	// with a drawn game in them, which fit none of the three.
	Swept, Split, Lost, WithDraw int

	// Margin is bot 1's score margin summed over a pair's two games, one
	// observation per pair. Halve it for a per-game figure.
	Margin *stats.Statistic
	// WinShare is bot 1's share of a pair's two games: 1 for winning both, 0.5
	// for a split, 0 for losing both. The null value is 0.5.
	WinShare *stats.Statistic

	// Correlation is between the two halves' margins. The pairing is working
	// when this is well below zero.
	Correlation float64

	// MarginPValue and WinPValue come from a sign-flip randomization test over
	// the pairs, which assumes only the symmetry the design guarantees.
	MarginPValue float64
	WinPValue    float64
	// ExactP is true when every sign assignment was enumerated rather than
	// sampled.
	ExactP bool
	// MarginTPValue is the same test done by t-test, for comparison.
	MarginTPValue float64

	// NaiveScoreSE is the standard error the per-game analysis reports for the
	// same quantity, kept to show what the pairing bought.
	NaiveScoreSE float64
}

PairedResult analyzes a game-pair run with the pair, rather than the game, as the unit of observation.

The two games of a pair are played from one bag with the seats swapped, so good tiles for one bot in the first game become good tiles for the other bot in the second. The two results are strongly anti-correlated by construction, and treating them as independent games throws away the variance reduction the pairing was run to get. Adding a pair's two margins together cancels the tiles instead, and cancels the first-move advantage with them, since each bot moves first exactly once per pair.

func (*PairedResult) MarginPerGame added in v0.13.8

func (p *PairedResult) MarginPerGame() (float64, float64)

MarginPerGame is the average score margin per game, with its 95% confidence half-width.

func (*PairedResult) PairedSE added in v0.13.8

func (p *PairedResult) PairedSE() float64

PairedSE is the standard error of the per-game margin.

type PlayerStats added in v0.13.0

type PlayerStats struct {
	Name   string
	Score  *stats.Statistic
	Bingos *stats.Statistic
	PPT    *stats.Statistic
}

PlayerStats holds per-player statistics accumulators.

type ShadowAgreementResult added in v0.12.4

type ShadowAgreementResult struct {
	GamesPlayed   int
	TurnsPlayed   int
	Disagreements int
	Details       []string
}

ShadowAgreementResult holds the results of a shadow agreement test.

func RunShadowAgreementTest added in v0.12.4

func RunShadowAgreementTest(cfg *config.Config, numGames int) (*ShadowAgreementResult, error)

RunShadowAgreementTest plays numGames game pairs, comparing moves generated with and without shadow. Both algorithms must produce the same top move by score on every turn. Games use deterministic seeds for reproducibility.

type WMPAgreementResult added in v0.13.0

type WMPAgreementResult struct {
	GamesPlayed   int
	TurnsPlayed   int
	Disagreements int
	Details       []string
}

WMPAgreementResult holds the results of a WMP agreement test.

func RunWMPAgreementTest added in v0.13.0

func RunWMPAgreementTest(cfg *config.Config, w *wmp.WMP, numGames int) (*WMPAgreementResult, error)

RunWMPAgreementTest plays numGames deterministic games, comparing the top play produced by the non-WMP movegen against the top play produced by the WMP movegen on every turn. Both must match by ShortDescription, score, and equity (within 1e-6).

This exercises exactly the path Simmer's rollout loop drives: SetPlayRecorderTopPlay (shadow enabled), addExchange=true, with an equity calculator wired in. It broadens the assertions of montecarlo/wmp_equivalence_test.go across diverse positions drawn from real deterministic play rather than a fixed CGP catalogue.

Games advance along the noWMP generator's top move (the canonical path), so a disagreement is reported at the earliest turn where the two generators diverge. The game then continues along the noWMP choice, which keeps the seed->position mapping reproducible across branches and across WMP-on/off runs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL