Documentation
¶
Overview ¶
Package automatic contains all the logic for the actual gameplay of Crossword Game, which, as we said before, features all sorts of things like wingos and blonks.
Index ¶
- Constants
- Variables
- func AnalyzeLogFile(filepath string) (string, error)
- func DeriveSeed(master uint64, unitIdx int) [32]byte
- func ExportGCG(cfg *config.Config, filename, letterdist, lexicon, boardlayout, gid string, ...) error
- func FormatDivergence(r *DivergenceReport) string
- func FormatInferenceQuality(q *InferenceQuality) string
- func FormatTable(r *AnalysisResult) string
- func GenerateSeeds(n int) ([][32]byte, error)
- func LoadAutoplayConfig(path string) (*pb.AutoplayConfig, error)
- func LoadSeeds(path string) ([][32]byte, error)
- func RandomMasterSeed() (uint64, error)
- func ResolveExperimentID(cfg *pb.AutoplayConfig) string
- func SaveSeeds(seeds [][32]byte, path string) error
- func StartAutoplayFromConfig(ctx context.Context, appCfg *config.Config, expCfg *pb.AutoplayConfig) (string, error)
- func StartCompVCompStaticGames(ctx context.Context, cfg *config.Config, numGames int, block bool, threads int, ...) error
- func StartServer(addr string) error
- func TurnLogFor(gamesFile string) string
- type AnalysisResult
- type AutomaticRunnerPlayer
- type CorpusFilter
- type CorpusPosition
- type DeterministicConfig
- type DivergenceKind
- type DivergenceReport
- type GameRunner
- func (r *GameRunner) BotTypeFor(playerIdx int) pb.BotRequest_BotCode
- func (r *GameRunner) CompVsCompStatic(addToHistory bool) error
- func (r *GameRunner) CompVsCompStaticBots(addToHistory bool, players []AutomaticRunnerPlayer, gidx int, seed [32]byte) error
- func (r *GameRunner) Game() *game.Game
- func (r *GameRunner) Init(players []AutomaticRunnerPlayer) error
- func (r *GameRunner) PlayBestTurn(playerIdx int, addToHistory bool) error
- func (r *GameRunner) StartGame(gidx int)
- func (r *GameRunner) StartGameWithSeed(gidx int, seed [32]byte)
- type InferenceQuality
- type Job
- type PairDivergence
- type PairedResult
- type PlayerStats
- type ShadowAgreementResult
- type WMPAgreementResult
Constants ¶
const InferenceLogColumns = ",inferCount,trueLeave,truePost,truePrior,liftBits,trueRank,inferLeaves,trueMeasured"
InferenceLogColumns are the per-turn log columns describing one inference. They are present on every row of a run that has an inferring bot, and empty on the rows of a bot that does not infer. See inferenceFields.
Variables ¶
var ( CVCCounter *expvar.Int IsPlaying *expvar.Int )
var MaxTimePerEndgame = 15 * time.Second
var MaxTimePerTurn = 30 * time.Second
Functions ¶
func AnalyzeLogFile ¶ added in v0.4.2
AnalyzeLogFile analyzes the given game CSV file and returns formatted stats. Kept for backward compatibility.
func DeriveSeed ¶ added in v0.13.8
DeriveSeed produces the seed for one unit of work (a game, or a pair of games) from the run's master seed. Deriving rather than drawing means a worker can be handed unit 900 without having played the first 899, and the same master seed always replays the same experiment.
func ExportGCG ¶ added in v0.4.10
func ExportGCG(cfg *config.Config, filename, letterdist, lexicon, boardlayout, gid string, half int, out io.Writer) error
ExportGCG writes one game from an autoplay log as GCG. In a -gamepairs run a game ID names a pair rather than a game, so half picks which of its two seatings to write; it is 1 for anything else.
func FormatDivergence ¶ added in v0.13.8
func FormatDivergence(r *DivergenceReport) string
FormatDivergence renders a divergence report for the shell.
func FormatInferenceQuality ¶ added in v0.13.8
func FormatInferenceQuality(q *InferenceQuality) string
FormatInferenceQuality renders the grading for the shell.
func FormatTable ¶ added in v0.13.0
func FormatTable(r *AnalysisResult) string
FormatTable formats analysis results as a side-by-side comparison table.
func GenerateSeeds ¶ added in v0.11.8
GenerateSeeds creates n random 32-byte seeds for deterministic game runs
func LoadAutoplayConfig ¶ added in v0.13.0
func LoadAutoplayConfig(path string) (*pb.AutoplayConfig, error)
LoadAutoplayConfig reads an AutoplayConfig from a protojson file.
func RandomMasterSeed ¶ added in v0.13.8
RandomMasterSeed returns a master seed for a run that asked for reproducible games without naming one.
func ResolveExperimentID ¶ added in v0.13.0
func ResolveExperimentID(cfg *pb.AutoplayConfig) string
ResolveExperimentID returns the experiment ID from the config, or generates one from the current timestamp and 6 random hex characters.
func StartAutoplayFromConfig ¶ added in v0.13.0
func StartAutoplayFromConfig(ctx context.Context, appCfg *config.Config, expCfg *pb.AutoplayConfig) (string, error)
StartAutoplayFromConfig runs an autoplay experiment defined by a protojson config file. It returns the resolved experiment ID. Output files are written to cfg.OutputDir (or the current directory if empty):
- {experimentId}.txt — per-turn log
- games-{experimentId}.txt — per-game summary
- {experimentId}.config.json — copy of the config for reproducibility
func StartServer ¶ added in v0.13.0
StartServer starts an HTTP server on the given address that exposes the autoanalyze endpoint. Call as:
curl 'http://host:8080/analyze?file=games.txt' curl 'http://host:8080/analyze?file=games.txt&format=json'
func TurnLogFor ¶ added in v0.13.8
TurnLogFor guesses the per-turn log that goes with a per-game log: autoplay writes games-{id}.txt beside {id}.txt.
Types ¶
type AnalysisResult ¶ added in v0.13.0
type AnalysisResult struct {
GamesPlayed int
Player1 PlayerStats
Player2 PlayerStats
P1Wins float64
P1First float64
WentFirstWins float64
WinPValue float64 // two-sided binomial z-test, H0: win rate = 0.5
ScorePValue float64 // two-sided paired z-test on per-game score diff
ScoreDiff *stats.Statistic
// Paired is filled in for a -gamepairs run, where the two games of a pair
// share a bag and cannot be treated as independent.
Paired *PairedResult
}
AnalysisResult holds the full results of analyzing a log file.
func AnalyzeLogFileData ¶ added in v0.13.0
func AnalyzeLogFileData(filepath string) (*AnalysisResult, error)
AnalyzeLogFileData analyzes the given game CSV file and returns structured results.
type AutomaticRunnerPlayer ¶ added in v0.5.0
type AutomaticRunnerPlayer struct {
LeaveFile string
PEGFile string
BotCode pb.BotRequest_BotCode
MinSimPlies int
SimThreads int
StochasticStaticEval bool
// InferenceTau pins the softmax temperature for P(play | leave). Leave it
// zero for the engine default (SoftmaxTemperature); a value here applies to
// the whole game.
InferenceTau float64
InferenceTimeSecs int
InferenceSimIters int
InferenceMaxEnumeratedLeaves int
InferenceBudget int
OracleInference bool
}
func PlayersFromConfig ¶ added in v0.13.0
func PlayersFromConfig(cfg *pb.AutoplayConfig) []AutomaticRunnerPlayer
PlayersFromConfig converts an AutoplayConfig's player configs to AutomaticRunnerPlayer structs.
type CorpusFilter ¶ added in v0.13.8
type CorpusFilter struct {
// LeaveLen, when non-zero, keeps only inferences of that many tiles. Six
// is the one-tile-play bucket, where the read goes wrong.
LeaveLen int
// MaxBag and MinBag bound the phase; MaxBag 0 means no upper bound.
MinBag, MaxBag int
// WorseThan keeps only inferences the run scored below this many bits.
// Leave at zero and pass AllLifts to take everything.
WorseThan float64
AllLifts bool
// Limit caps how many positions are returned, 0 for all.
Limit int
}
CorpusFilter narrows which positions to lift out.
type CorpusPosition ¶ added in v0.13.8
type CorpusPosition struct {
// GameID and Half say which game and which seating it came from; Turn is
// the play the inference was reading, 1-based as the log numbers them.
GameID string
Half int
Turn int
// Game is replayed up to and including the opponent's play, so the
// position is exactly what the inferring bot faced -- including the
// inferring player's own rack, which decides the unseen pool and so the
// prior every inference is graded against.
Game *game.Game
// MyRack is the inferring player's rack at this position, taken from the
// row where they play next. Replaying alone cannot recover it: the tiles
// they drew came off a bag this replay never had in the same order.
MyRack string
// TrueLeave is what the opponent actually kept -- the answer. OppRack and
// OppPlay are what produced it.
TrueLeave string
OppRack string
OppPlay string
OppScore int
// TilesRemaining is the bag count after the play, and LeaveLen the number
// of tiles being inferred; both are what the run's own breakdowns cut on.
TilesRemaining int
LeaveLen int
// LoggedLiftBits is what the original run scored this inference, when it
// recorded one. A replay that reproduces the run should land near it.
LoggedLiftBits float64
LoggedMeasured bool
LoggedRank int
LoggedLeaves int
HasLoggedResult bool
}
CorpusPosition is one inference problem lifted out of a run.
func LoadCorpus ¶ added in v0.13.8
func LoadCorpus(cfg *config.Config, turnFile, lexicon, letterdist, boardlayout string, filter CorpusFilter) ([]*CorpusPosition, error)
LoadCorpus replays a run's per-turn log and returns the positions matching the filter, each rebuilt to the moment the inference ran. Games that will not replay are skipped and counted in Skipped; a corpus that comes back empty because every game failed is reported as an error rather than as an empty result, since the two look identical to a caller and mean opposite things.
func LoadCorpusVerbose ¶ added in v0.13.8
func LoadCorpusVerbose(cfg *config.Config, turnFile, lexicon, letterdist, boardlayout string, filter CorpusFilter) ([]*CorpusPosition, int, error)
LoadCorpusVerbose is LoadCorpus, also returning how many games were skipped because they would not replay.
type DeterministicConfig ¶ added in v0.11.8
type DeterministicConfig struct {
Seeds [][32]byte // Pre-generated seeds (nil = generate new)
SeedFile string // File to read/write seeds
NumGames int // Number of games (used when generating seeds)
// MasterSeed derives a seed per unit of work when there is no seed file.
MasterSeed uint64
// GamePairs plays each seed twice with the bots swapping seats.
GamePairs bool
}
DeterministicConfig holds configuration for deterministic autoplay runs
type DivergenceKind ¶ added in v0.13.8
type DivergenceKind int
DivergenceKind says what parted the two halves of a pair.
const ( // DivergenceNone means the halves played the same game throughout. DivergenceNone DivergenceKind = iota // DivergenceChoice is the normal case: same rack, different play. DivergenceChoice // DivergenceTiles means the halves were dealt different racks. The pairing // is broken; this should never happen. DivergenceTiles // DivergenceLength means the halves agreed on every shared turn but one ran // longer. Also a bug, since identical play cannot produce different games. DivergenceLength )
func (DivergenceKind) String ¶ added in v0.13.8
func (k DivergenceKind) String() string
type DivergenceReport ¶ added in v0.13.8
type DivergenceReport struct {
Bot1, Bot2 string
Pairs int
Divergent int
ExactTies int
// Broken counts pairs whose halves saw different tiles or ran different
// lengths -- bugs rather than strategy differences.
Broken int
// NoTurnData counts divergent pairs the per-turn log had nothing for.
NoTurnData int
// TurnStats is the first-divergence turn across every divergent pair.
TurnStats *stats.Statistic
// EarliestTurn and LatestTurn bound the same distribution.
EarliestTurn, LatestTurn int
// Details covers the first maxDetails divergent pairs, in pair order.
Details []PairDivergence
// Truncated is how many divergent pairs are not in Details.
Truncated int
}
DivergenceReport summarizes a whole paired run.
func AnalyzeDivergence ¶ added in v0.13.8
func AnalyzeDivergence(gamesFile, turnFile string, maxDetails int) (*DivergenceReport, error)
AnalyzeDivergence reads a paired run's per-game log and the per-turn log beside it, and reports where each divergent pair's halves parted ways. maxDetails caps how many pairs are described individually; the counts and the turn statistics cover them all.
type GameRunner ¶
type GameRunner struct {
// contains filtered or unexported fields
}
GameRunner is the master struct here for the automatic game logic.
func NewGameRunner ¶ added in v0.4.2
func NewGameRunner(logchan chan string, cfg *config.Config) *GameRunner
NewGameRunner just instantiates and initializes a game runner.
func (*GameRunner) BotTypeFor ¶ added in v0.13.0
func (r *GameRunner) BotTypeFor(playerIdx int) pb.BotRequest_BotCode
BotTypeFor returns the bot code for the given game-level player index. Accounts for player flips done by StartGameWithSeed.
func (*GameRunner) CompVsCompStatic ¶
func (r *GameRunner) CompVsCompStatic(addToHistory bool) error
CompVsCompStatic plays out a game to the end using best static turns, HastyBot against HastyBot on a randomly seeded bag.
func (*GameRunner) CompVsCompStaticBots ¶ added in v0.13.7
func (r *GameRunner) CompVsCompStaticBots(addToHistory bool, players []AutomaticRunnerPlayer, gidx int, seed [32]byte) error
CompVsCompStaticBots plays out one game to the end with the given bots. It is CompVsCompStatic's parameterized form, for callers that want a different bot pair or a reproducible bag: a non-zero seed fixes the tile order and, through SetUidFromSeed, the game's UID as well. gidx alternates who goes first.
func (*GameRunner) Game ¶ added in v0.4.5
func (r *GameRunner) Game() *game.Game
func (*GameRunner) Init ¶
func (r *GameRunner) Init(players []AutomaticRunnerPlayer) error
Init initializes the runner
func (*GameRunner) PlayBestTurn ¶ added in v0.5.0
func (r *GameRunner) PlayBestTurn(playerIdx int, addToHistory bool) error
PlayBestTurn generates the best move for the player and plays it on the board.
func (*GameRunner) StartGame ¶
func (r *GameRunner) StartGame(gidx int)
func (*GameRunner) StartGameWithSeed ¶ added in v0.11.8
func (r *GameRunner) StartGameWithSeed(gidx int, seed [32]byte)
type InferenceQuality ¶ added in v0.13.8
type InferenceQuality struct {
// Inferences is how many turns produced a graded posterior.
Inferences int
// Unscored is how many turns ran inference but had nothing to grade: no
// history yet, an empty bag, or a bingo that left no tiles to infer.
Unscored int
// LiftBits is log2(posterior/prior) per inference. Its mean is the headline.
LiftBits *stats.Statistic
// Posterior and Prior are the raw probabilities, kept for context since the
// lift hides how small both are in a large leave space.
Posterior, Prior *stats.Statistic
// RankPct is the true leave's percentile among candidates, 0 being the top.
// Robust to leave-space size in a way the raw rank is not.
RankPct *stats.Statistic
// Leaves is the size of the posterior per inference.
Leaves *stats.Statistic
// Better and Worse count inferences that beat and lost to the prior.
Better, Worse int
// RuledOut counts inferences that gave the true leave no weight at all --
// the expensive mistake, since the simmer then never samples the truth.
RuledOut int
// Measured counts inferences where the true leave got a real mini-sim rather
// than an imputed weight.
Measured int
// ByLeaveLen breaks the mean lift down by how many tiles were being inferred,
// which is where inference gets hard.
ByLeaveLen map[int]*stats.Statistic
// contains filtered or unexported fields
}
InferenceQuality summarizes how well a run's inferences matched the truth.
func AnalyzeInferenceQuality ¶ added in v0.13.8
func AnalyzeInferenceQuality(turnFile string) (*InferenceQuality, error)
AnalyzeInferenceQuality reads a per-turn autoplay log and grades its inferences. It wants the per-turn log, not the per-game one.
func (*InferenceQuality) LiftPercentile ¶ added in v0.13.8
func (q *InferenceQuality) LiftPercentile(p float64) float64
LiftPercentile returns the p-th percentile of the per-inference lift, p in [0,100]. NaN when nothing was graded.
type PairDivergence ¶ added in v0.13.8
type PairDivergence struct {
Pair int
GameID string
Kind DivergenceKind
// Turn is the first turn the halves disagreed on, 1-based.
Turn int
// Rack is what both halves held on that turn. When Kind is
// DivergenceTiles the two halves held RackA and RackB instead.
Rack string
RackA, RackB string
// BotA played PlayA in the first half; BotB played PlayB in the second,
// from the same seat and the same tiles.
BotA, BotB string
PlayA, PlayB string
ScoreA, ScoreB int
EquityA, EquityB float64
TurnsA, TurnsB int
// ScoreDiff is bot 1's score minus bot 2's, summed over both halves. A pair
// that never diverges scores exactly 0 here.
ScoreDiff int
}
PairDivergence is where and how one pair's halves parted ways.
type PairedResult ¶ added in v0.13.8
type PairedResult struct {
Pairs int
// Incomplete counts pairs that did not have both halves in the log -- a run
// stopped midway, say. They are left out of everything below.
Incomplete int
// Divergent counts pairs whose halves ever chose differently, straight from
// the log column. Pairs that never diverge contribute an exact zero, so this
// is the count that carries any information.
Divergent int
// ExactTies counts pairs whose two margins cancel to zero.
ExactTies int
// Swept, Split and Lost count pairs where bot 1 won both games, one each,
// and neither -- the plainest statement of a paired result, since the
// pairs that split are the ones tile luck decided. WithDraw counts pairs
// with a drawn game in them, which fit none of the three.
Swept, Split, Lost, WithDraw int
// Margin is bot 1's score margin summed over a pair's two games, one
// observation per pair. Halve it for a per-game figure.
Margin *stats.Statistic
// for a split, 0 for losing both. The null value is 0.5.
WinShare *stats.Statistic
// Correlation is between the two halves' margins. The pairing is working
// when this is well below zero.
Correlation float64
// MarginPValue and WinPValue come from a sign-flip randomization test over
// the pairs, which assumes only the symmetry the design guarantees.
MarginPValue float64
WinPValue float64
// ExactP is true when every sign assignment was enumerated rather than
// sampled.
ExactP bool
// MarginTPValue is the same test done by t-test, for comparison.
MarginTPValue float64
// NaiveScoreSE is the standard error the per-game analysis reports for the
// same quantity, kept to show what the pairing bought.
NaiveScoreSE float64
}
PairedResult analyzes a game-pair run with the pair, rather than the game, as the unit of observation.
The two games of a pair are played from one bag with the seats swapped, so good tiles for one bot in the first game become good tiles for the other bot in the second. The two results are strongly anti-correlated by construction, and treating them as independent games throws away the variance reduction the pairing was run to get. Adding a pair's two margins together cancels the tiles instead, and cancels the first-move advantage with them, since each bot moves first exactly once per pair.
func (*PairedResult) MarginPerGame ¶ added in v0.13.8
func (p *PairedResult) MarginPerGame() (float64, float64)
MarginPerGame is the average score margin per game, with its 95% confidence half-width.
func (*PairedResult) PairedSE ¶ added in v0.13.8
func (p *PairedResult) PairedSE() float64
PairedSE is the standard error of the per-game margin.
type PlayerStats ¶ added in v0.13.0
type PlayerStats struct {
Name string
Score *stats.Statistic
Bingos *stats.Statistic
PPT *stats.Statistic
}
PlayerStats holds per-player statistics accumulators.
type ShadowAgreementResult ¶ added in v0.12.4
type ShadowAgreementResult struct {
GamesPlayed int
TurnsPlayed int
Disagreements int
Details []string
}
ShadowAgreementResult holds the results of a shadow agreement test.
func RunShadowAgreementTest ¶ added in v0.12.4
func RunShadowAgreementTest(cfg *config.Config, numGames int) (*ShadowAgreementResult, error)
RunShadowAgreementTest plays numGames game pairs, comparing moves generated with and without shadow. Both algorithms must produce the same top move by score on every turn. Games use deterministic seeds for reproducibility.
type WMPAgreementResult ¶ added in v0.13.0
type WMPAgreementResult struct {
GamesPlayed int
TurnsPlayed int
Disagreements int
Details []string
}
WMPAgreementResult holds the results of a WMP agreement test.
func RunWMPAgreementTest ¶ added in v0.13.0
RunWMPAgreementTest plays numGames deterministic games, comparing the top play produced by the non-WMP movegen against the top play produced by the WMP movegen on every turn. Both must match by ShortDescription, score, and equity (within 1e-6).
This exercises exactly the path Simmer's rollout loop drives: SetPlayRecorderTopPlay (shadow enabled), addExchange=true, with an equity calculator wired in. It broadens the assertions of montecarlo/wmp_equivalence_test.go across diverse positions drawn from real deterministic play rather than a fixed CGP catalogue.
Games advance along the noWMP generator's top move (the canonical path), so a disagreement is reported at the earliest turn where the two generators diverge. The game then continues along the noWMP choice, which keeps the seed->position mapping reproducible across branches and across WMP-on/off runs.