agentloop

agentloop

A small Go engine for LLM agents that think by writing JavaScript instead of calling named tools.

Each turn the model emits one fenced ```javascript block defining function run(args) { ... }. agentloop executes it in a sandboxed goja runtime and threads its return value into the next turn as args — full fidelity, server-side, never re-serialised into the prompt. The model sees only its own log() output plus a compact structural digest of what it returned. A run finishes when the script calls answer(result).

This return-threading design keeps the prompt small even for many-step runs: stale code is elided from history (only the most recent turn's script is kept verbatim) and large data never appears in the context window twice.

Packages

• agentloop — the reasoning loop itself: Loop.Run drives the rehydrate → prompt → execute → thread cycle for one user message.
• sandbox — the goja-based JS executor and its built-in packs: require, http, fetch + htmlToMarkdown, markdown (structured parsing), ai/aiJSON, help, and skill discovery. A PolicyChecker seam gates side-effecting primitives.
• llm — a small Client interface plus an OpenAI-wire-compatible implementation that works with OpenAI itself, OpenRouter, EdenAI, Groq, together.ai, a local vLLM/Ollama server, or any other gateway that speaks the same /chat/completions protocol.
• agentloopmem — in-memory SessionStore / StepStore for local dev, one-shot CLIs, and eval suites. Not for production traffic (no durability, no cross-process visibility).
• agentlooptest — StepStoreContract, a reusable conformance test harness. Point it at your own StepStore implementation (Postgres, SQLite, whatever) to hold it to the same behavioural guarantees the loop relies on — see agentloopmem's own contract_test.go for the worked example.
• ext — optional sandbox.Packs that are generically useful but don't belong in the core sandbox package: EmailPack (sendEmail), SecretPack (secret), SearchPack (documentSearch), StoresPack (stores.list/stores.read), and OpenAPIPack, which generates a require()-able skill — one JS function per operation — from an OpenAPI 3 document. Each takes a callback, small interface, or (for OpenAPIPack) a parsed spec, same decoupling the core packs use, so this package stays free of any mail/secrets/search-backend/HTTP-client dependency of its own.
• pool — SandboxPool, a SandboxBuilder that reuses one long-lived sandbox per session across every Run instead of paying goja.New() + pack-registration cost on every message. Wraps any other SandboxBuilder; see Extending for the correctness issue it has to solve to do that safely.
• eval — an LLM-judged eval harness: a Suite of Cases (input + judge criteria), each dispatched through an agentloop.Loop and scored 0–10 by a judge llm.Client. agentloop.Loop and agentloop.RunRequest/RunResult already have exactly the shape the harness needs, so Service takes a Loop directly — no adapter interface required.
• evalmem — in-memory eval.Store for local dev and CI, same role agentloopmem plays for SessionStore/StepStore. Not for production traffic.
• redact — Redactor, a small ordered (value, placeholder) list that strips known secret values out of text or bytes; a nil *Redactor is a safe pass-through everywhere. Config.Redactor and eval.NewService's redactor parameter both take one — see Extending for what it's wired into.

Quickstart

client, err := llm.NewOpenAI(llm.ConfigFromEnv())
if err != nil {
    log.Fatal(err)
}

caps := agentloop.DefaultCapabilities(client, "")
loop := agentloop.New(agentloop.Config{
    LLM:            client,
    Sessions:       mySessionStore, // implements agentloop.SessionStore
    Steps:          myStepStore,    // implements agentloop.StepStore
    SandboxBuilder: &agentloop.DefaultSandboxBuilder{Capabilities: caps},
})

result, err := loop.Run(ctx, agentloop.RunRequest{
    SessionID: "session-1",
    Message:   "What's 12 * 7? Reply with just the number.",
})

See examples/cli for a complete runnable program with in-memory stores:

export OPENAI_API_KEY=sk-...
go run ./examples/cli "What's 12 * 7? Reply with just the number."

To point at a different OpenAI-wire-compatible gateway, set OPENAI_BASE_URL and OPENAI_CHAT_MODEL to match it, e.g.:

# OpenRouter
export OPENAI_BASE_URL=https://openrouter.ai/api/v1
export OPENAI_CHAT_MODEL=openai/gpt-4o-mini

# EdenAI
export OPENAI_BASE_URL=https://api.edenai.run/v3
export OPENAI_CHAT_MODEL=google/gemini-2.5-flash

Extending

• Custom capabilities — a Capability is a name plus a Build func returning []sandbox.Pack. Application-specific dependencies (a database handle, an accumulator, a broadcaster) should be closed over when the Capability is constructed rather than threaded through BuildContext — see DefaultCapabilities in defaults.go for the pattern.

• Custom sandbox composition — implement agentloop.SandboxBuilder yourself (its one method, Build) when your capability set varies per scope or session, e.g. pulled from a database per tenant.

• Custom stores — SessionStore and StepStore are small interfaces; back them with whatever persistence you already have. agentloopmem ships in-memory implementations to start from, and agentlooptest.StepStoreContract is a conformance harness to run against your own implementation as an acceptance gate.

• Policy — implement sandbox.PolicyChecker to gate side-effecting primitives (fetch, ai, or your own) per call. sandbox.DefaultPolicy is a conservative default (deny side effects, block private-network fetches); sandbox.AllowAll is the explicit fail-open escape hatch.

• Optional extension packs — ext.EmailPack, ext.SecretPack, ext.SearchPack, and ext.StoresPack are common but not universal, so they live outside sandbox and aren't in DefaultCapabilities. Wrap one in a Capability that closes over your own backend and add it to the slice passed to DefaultSandboxBuilder. sendEmail and secret are already in DefaultPolicy's side-effect list, so they're denied until granted via DefaultPolicy.AllowTools.

• Generate a skill from an OpenAPI spec — ext.OpenAPIPack(spec, cfg) turns an OpenAPI 3 document into a require()-able skill: one JS function per operation, named after its operationId (or derived from the method + path when it has none), taking a single params object (params.<name> per path/query parameter, params.body for the request body) and returning {status, body, headers} — the same shape fetch()/require('http') already return. Generated functions call the sandbox's own internal HTTP primitive, not a new client, so every call is still policy-gated exactly like fetch() already is, and require(<skill name>) is gated by that name like any other skill. cfg.Headers is the whole auth story (a static header map — bearer token, API key — applied to every call); OpenAPI securitySchemes aren't interpreted. The generated Pack's Prompt is one short line (API name, operation count, a pointer to skillGet/require) rather than the full surface — a spec can have far more operations than are worth inlining into every turn. The full docs, returned by skillGet(<skill name>) on demand, are TypeScript ambient declarations — declare function getPetById(params: { petId: string; verbose?: boolean }): { status: number; body: string; headers: Record<string, string> }; — in the same style the core packs' own Prompt fields already use, so a generated skill's API reads the same way a built-in one does:

  spec, err := openapi3.NewLoader().LoadFromFile("petstore.yaml")
  pack, err := ext.OpenAPIPack(spec, ext.OpenAPIConfig{
      Headers: map[string]string{"Authorization": "Bearer " + apiKey},
  })
  caps = append(caps, agentloop.Capability{
      Name: pack.Name,
      Build: func(agentloop.BuildContext) ([]sandbox.Pack, error) { return []sandbox.Pack{pack}, nil },
  })
  

• Session-scoped sandbox reuse — agentloop.New's default is a fresh sandbox per Run (sandbox.New() builds a whole goja.Runtime and re-registers every pack from scratch every time). Wrap your SandboxBuilder in pool.New to reuse one sandbox per session across every Run instead:

  builder := pool.New(&agentloop.DefaultSandboxBuilder{Capabilities: caps}, pool.Options{
      IdleTimeout: 30 * time.Minute, // evict a session's sandbox after this much idle time
  })
  defer builder.Close()
  
  loop := agentloop.New(agentloop.Config{
      // ...
      SandboxBuilder: builder,
  })
  

  This isn't just a cache wrapper — a Capability's Build closes over BuildContext.Ctx once (see DefaultCapabilities in defaults.go), so a naively cached sandbox would keep using the first Run's context — including its cancellation — forever. pool.SandboxPool gives the delegate builder a swappable context instead, and swaps in each Run's real context before handing the sandbox back. It also serializes concurrent Runs for one session: a goja.Runtime isn't safe for concurrent use, so a second Run for a session already in flight blocks until the first releases the sandbox, rather than racing on it. See the pool package doc comment for both mechanisms in detail.

• Distributed tracing (OpenTelemetry) — Config.TracerProvider takes a trace.TracerProvider. Left unset, Run produces no spans (a no-op tracer, a few allocations and nothing else); set it to an OTel SDK TracerProvider — e.g. configured with an OTLP exporter pointed at a Jaeger collector — and every Run produces a span tree: one root agentloop.run span, with agentloop.sandbox_build and one agentloop.turn per LLM round-trip as children, each turn's own agentloop.llm_call and (when the model emits JS) agentloop.execute_js nested under it. Only go.opentelemetry.io/otel/trace (the stable, SDK-free API package) is an agentloop dependency — the SDK, exporter, and Jaeger wiring are entirely the application's to choose:

  loop := agentloop.New(agentloop.Config{
      // ...
      TracerProvider: myOTelSDKTracerProvider, // e.g. wired to an OTLP/Jaeger exporter
  })
  

A capability whose Build fails is logged and skipped (a warning sandbox event, not an aborted session) — one flaky capability shouldn't deny the user their turn.

Evaluating agent quality

eval.Service runs a Suite of Cases — each an input plus judge criteria — through an agentloop.Loop, has a judge llm.Client score every response 0–10, and persists the run:

store := evalmem.New() // or your own eval.Store
svc := eval.NewService(store, loop, judgeClient, nil) // last arg: optional *redact.Redactor

suite, _ := svc.CreateSuite(ctx, "arithmetic", "" /* judge model, empty = client default */)
svc.AddCase(ctx, suite.ID, "sums", "What's 2+2?", "must say 4", 7, nil)

run, err := svc.RunSuite(ctx, suite.ID)
// run.Summary.Passed / .Failed; run.Results[i].{Response,Score,Rationale,Passed}

agentloop.Loop's Run(ctx, RunRequest) (RunResult, error) already has exactly the shape an eval-harness runner needs (RunResult.FinalText is the response to judge), so Service takes a Loop directly rather than some separate runner interface. Each case gets its own fresh, never reused session ID — agentloop.SessionStore.Get is documented to auto-create a shell for an unknown ID, so RunSuite needs no separate session-provisioning step.

A Case can carry either a single Criteria string + PassThreshold (the legacy path), or a per-criterion CriteriaItems rubric — when set, the judge scores each item separately and the case passes only when every item clears its own MinScore, with the overall Score reported as the average. A case that errors (agent failure or judge failure) records the error inline on that case's result rather than aborting the suite — RunSuite always finishes and returns a Run.

evalmem.InMemoryStore implements eval.Store for local dev and CI; back a real deployment with whatever persistence you already have.

Redacting secrets from observability

redact.Redactor holds a small ordered list of (secret value, placeholder) pairs and strips every occurrence out of text or bytes — build one with redact.FromSecrets(map[string]string{"api_key": key, ...}) over whatever values a session's capabilities can return. A nil *Redactor is a safe pass-through everywhere it's used, so this is entirely opt-in.

Two places take one:

• Config.Redactor — applied to every RunEvent (Content and Args, before OnEvent sees it), RunResult.FinalText, persisted RunStep.Content, and error messages recorded on a span. The defense-in-depth case: a script does log(secret("API_KEY")), or a fetch() response happens to echo a credential back, and that value would otherwise land in whatever OnEvent forwards to, the persisted trace, or a trace backend. One trade-off: setting this suppresses "response_chunk" events (live token-by-token streaming), because a secret can split across two chunk boundaries with neither chunk containing the whole value to redact against — the complete, redacted text still arrives via the terminal "response" event.
• eval.NewService's 4th argument — applied to the agent's response before it reaches the judge's prompt (a third-party LLM call) or gets persisted in CaseResult.Response. An eval case exercises the same capabilities production traffic does, so without this a case that happens to trigger a credential-bearing response would send it on to the judge and store it in the run.

Known limitations

This is a synchronous, single-process design, not a durable workflow engine:

• No durable execution. A Run call lives in one goroutine; a process restart mid-run kills it (steps already persisted are fine, but nothing resumes automatically).
• Everything inside a turn is synchronous, including I/O — no Promise/async/.then() in the sandbox (goja parses them but has no event loop, so continuations silently never run; the system prompt warns the model off this).
• Process-level isolation only. goja is memory-safe and interruptible, but sandboxes share the host process's heap/CPU — no per-run resource quota.
• Unbounded args carry. Nothing caps the size of the server-side state threaded between turns (RunResult.DataBytesCarried gives you the observability to notice, not a limit).
• No built-in cost ceiling. MaxIterations bounds turns and token usage is tracked, but there's no per-run or per-tenant token budget.

None of these are architectural dead ends — they're the natural next layer (suspend/resume, batched I/O primitives, resource quotas, budget enforcement) to add on top if/when you need them.

License

MIT — see LICENSE.