runtime

module
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 27, 2026 License: Apache-2.0

README

chatwright.dev/runtime

The Chatwright engine: platform emulation and the testing runtime for conversational applications.

Its sibling repository, runtime-ts, is the browser runtime — published to npm as @chatwright/runtime, just as this repository (runtime-go) publishes the Go module chatwright.dev/runtime. It is the orchestrator behind the Studio Playground, currently a scaffold per decision 0012. The two runtimes share language-independent contracts — the run-bundle v1 format and the black-box bot protocol — never code; conformance is proven by shared fixtures. Parity is the shipping rule (decision 0015): every runtime feature ships in both runtimes with identical semantics; deviations exist only under documented technical limitation — see the runtime parity register.

This module is where a Chatwright run actually happens. It emulates a chat platform's API server (Telegram first; the WhatsApp surface is present), delivers updates to the bot-under-test over real HTTP, captures everything the bot sends back into an append-only per-chat journal, and drives both deterministic scenarios and AI-goal exploration over that shared journal:

  • cw — the scenario API: platform-neutral verbs (SendText, ExpectBotMessage, ExpectAction, …) bound to a testing.TB, plus scenario fragments and execution-context provenance.
  • telegram, whatsapp, platform — the emulated platform servers and the neutral contracts they implement.
  • observe, goal, actor, campaign, datastate, branching — the observation engine, goal/task contracts, the actor loop, campaign report assembly, data-state assertions and branch exploration.
  • run — part composition (hybrid runs) and run-bundle assembly: it converts this runtime's internal records into the wire types chatwright.dev/sdk owns.

The bot under test may be written in any language or framework — Chatwright only speaks HTTP (see examples/pybot for a Python bot driven as a real subprocess).

Install

go get chatwright.dev/runtime

Usage

package mybot_test

import (
	"testing"
	"time"

	"chatwright.dev/runtime/cw"
)

func TestGreeting(t *testing.T) {
	w := cw.New(t) // boots an emulated Telegram Bot API server
	w.ServeWebhook(myBot.WebhookHandler())

	chat := w.PrivateChat(cw.User{ID: "alice", FirstName: "Alice"})
	chat.SendText("Hi")
	chat.ExpectBotMessage().Within(time.Second).Text("Howdy stranger")
}

A complete, runnable version of this flow — a real bot on its own TCP listener, webhook delivery, language selection via inline buttons and in-place message edits — lives in examples/greetbot.

Dependency rule

The runtime depends on chatwright.dev/sdk, never the reverse. The sdk owns every run-bundle wire type; this module produces bundles by converting its internal records to those types (see run/wire.go) and never redefines a wire shape of its own.

The standard

Specs, format documentation and design decisions live in the standard repository, github.com/chatwright/chatwright, and at chatwright.dev. The run-bundle wire model is github.com/chatwright/sdk-go.

Licence

Apache-2.0 — see LICENSE and NOTICE.

Spec-first

Chatwright is developed spec-first with SpecScore — product specs live in the standard repository; this repository's own specs live under spec/.

Directories

Path Synopsis
Package actor is Chatwright's AI actor loop: the observe-plan-act-validate cycle that drives a goal.CampaignState through a conversation using a pluggable Provider.
Package actor is Chatwright's AI actor loop: the observe-plan-act-validate cycle that drives a goal.CampaignState through a conversation using a pluggable Provider.
anthropic
Package anthropic is the first real actor/actor.Provider implementation: it calls the Anthropic Messages API to propose the next action for an in-flight campaign task.
Package anthropic is the first real actor/actor.Provider implementation: it calls the Anthropic Messages API to propose the next action for an in-flight campaign task.
openai
Package openai is an actor.Provider that speaks the OpenAI-compatible chat-completions wire format: the same request/response shape Ollama, LM Studio, OpenRouter, vLLM and OpenAI itself expose at POST {BaseURL}/chat/completions.
Package openai is an actor.Provider that speaks the OpenAI-compatible chat-completions wire format: the same request/response shape Ollama, LM Studio, OpenRouter, vLLM and OpenAI itself expose at POST {BaseURL}/chat/completions.
Package arena runs Chatwright's actor-model comparison matrix: the same Scenario (a goal plus a platform environment), the same budgets, across a declared set of provider/model configurations, N repeats each — see spec/ideas/actor-model-arena.md in the chatwright/chatwright standard repository for the arena this package implements (mandatory warm-up with cold-start as its own metric, right-sized context windows, a full retry breakdown, evidence over claims).
Package arena runs Chatwright's actor-model comparison matrix: the same Scenario (a goal plus a platform environment), the same budgets, across a declared set of provider/model configurations, N repeats each — see spec/ideas/actor-model-arena.md in the chatwright/chatwright standard repository for the arena this package implements (mandatory warm-up with cold-start as its own metric, right-sized context windows, a full retry breakdown, evidence over claims).
Package branching coordinates database-only scenario checkpoints and branches.
Package branching coordinates database-only scenario checkpoints and branches.
Package campaign assembles Chatwright's evidence-backed campaign report from a completed (or budget-stopped) actor.Loop run: a Goal, the goal.CampaignState snapshot it produced, and the actor.LoopEvents the loop recorded along the way.
Package campaign assembles Chatwright's evidence-backed campaign report from a completed (or budget-stopped) actor.Loop run: a Goal, the goal.CampaignState snapshot it produced, and the actor.LoopEvents the loop recorded along the way.
Package cw is the scenario API of module chatwright.dev/runtime: a framework- and language-agnostic testing harness for conversational applications.
Package cw is the scenario API of module chatwright.dev/runtime: a framework- and language-agnostic testing harness for conversational applications.
Package datastate is the smallest provider-neutral data-state assertion runtime for the data-state-assertions feature (spec/features/chatwright/deterministic-testing/data-state-assertions/README.md): run a read-only DTQL query against a named application database after a settled message/action, immediately before a checkpoint is published, or at branch/fragment completion, and retain a bounded, redacted recordset as evidence so a scenario proves what the application stored, not only what the bot said.
Package datastate is the smallest provider-neutral data-state assertion runtime for the data-state-assertions feature (spec/features/chatwright/deterministic-testing/data-state-assertions/README.md): run a read-only DTQL query against a named application database after a settled message/action, immediately before a checkpoint is published, or at branch/fragment completion, and retain a bounded, redacted recordset as evidence so a scenario proves what the application stored, not only what the bot said.
examples
greetbot
Package greetbot is a minimal, real Telegram bot used to exercise Chatwright end-to-end.
Package greetbot is a minimal, real Telegram bot used to exercise Chatwright end-to-end.
Package goal is Chatwright's goal/task/budget contract for goal-driven AI testing: the campaign's product-level intent (Goal), its trackable units of work (Task) with dependencies and prose success criteria, the limits that bound an autonomous run (Budgets), and the guarded state machine that tracks progress against them (CampaignState).
Package goal is Chatwright's goal/task/budget contract for goal-driven AI testing: the campaign's product-level intent (Goal), its trackable units of work (Task) with dependencies and prose success criteria, the limits that bound an autonomous run (Budgets), and the guarded state machine that tracks progress against them (CampaignState).
Package observe implements the minimum slice of Chatwright's Observation Model: a platform-neutral projection of a chat's visible conversation and available actions, built from a Platform Emulator's structured journal (platform.JournalEntry) rather than from any platform's own wire types.
Package observe implements the minimum slice of Chatwright's Observation Model: a platform-neutral projection of a chat's visible conversation and available actions, built from a Platform Emulator's structured journal (platform.JournalEntry) rather than from any platform's own wire types.
Package platform defines the neutral contracts that let a scenario be written once and executed against any chat platform.
Package platform defines the neutral contracts that let a scenario be written once and executed against any chat platform.
Package run is Chatwright's part-composition runtime: it executes an ordered sequence of Parts — deterministic scenario fragments and ai-goal actor-loop passages — over one shared Environment (one platform.Emulator, one cast, one continuous journal), exactly the shape spec/ideas/hybrid-runs.md describes: "a run is an ordered sequence of parts ...
Package run is Chatwright's part-composition runtime: it executes an ordered sequence of Parts — deterministic scenario fragments and ai-goal actor-loop passages — over one shared Environment (one platform.Emulator, one cast, one continuous journal), exactly the shape spec/ideas/hybrid-runs.md describes: "a run is an ordered sequence of parts ...
Package scenario is the parser, validator and run.Run mapper for Chatwright's self-contained scenario document format (https://chatwright.dev/formats/scenario-document/v1) — the format spec/features/chatwright/scenario-authoring/portable-scenario-documents/ self-contained-scenario-documents/README.md in the standard repository (chatwright/chatwright) defines.
Package scenario is the parser, validator and run.Run mapper for Chatwright's self-contained scenario document format (https://chatwright.dev/formats/scenario-document/v1) — the format spec/features/chatwright/scenario-authoring/portable-scenario-documents/ self-contained-scenario-documents/README.md in the standard repository (chatwright/chatwright) defines.
Package telegram implements the Telegram Platform for Chatwright: an emulated Telegram Bot API server that delivers updates and captures the bot's outbound calls, normalized to Chatwright's neutral platform types.
Package telegram implements the Telegram Platform for Chatwright: an emulated Telegram Bot API server that delivers updates and captures the bot's outbound calls, normalized to Chatwright's neutral platform types.
Package whatsapp implements the WhatsApp Platform for Chatwright: an emulated WhatsApp Cloud API (Graph) server that delivers inbound webhooks and captures the bot's outbound calls, normalized to Chatwright's neutral platform types.
Package whatsapp implements the WhatsApp Platform for Chatwright: an emulated WhatsApp Cloud API (Graph) server that delivers inbound webhooks and captures the bot's outbound calls, normalized to Chatwright's neutral platform types.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL