README
ΒΆ
Measurement, not guessing β give AI hands on the live Editor.
One Go CLI Β· one C# UPM package Β· zero runtime dependencies Β· MIT.
Install Β· Quick Start Β· Showcase Β· Mockup β UI Β· Commands Β· Batch Β· Custom Tools Β· Architecture Β· FAQ
English Β· νκ΅μ΄
Why hera-agent-unity
LLMs don't know your project. They remember last year's Unity API and generalized patterns β and you pay that gap every week, in tokens and in time.
hera-agent-unity stands between them.
Before AI guesses your code, run it in the Editor and return the result. Before AI assumes a console error, fetch the actual log filtered by type. Before AI hypothesizes a Play Mode outcome, enter it and wait until it finishes. Before AI invents an API that doesn't exist in your Unity version, reflect on the live assembly.
No middleware. No Python, no WebSocket, no JSON-RPC. One Go binary, localhost HTTP, one C# UPM package. When Unity Editor opens, Hera is already there.
Hera responds to commands β never inferring, never assuming. It returns what your Unity is, right now, exactly as it is.
Guessing is expensive. Measurement is the command.
βββββββββββββββ HTTP ββββββββββββββββββββ
β Terminal β ββββββββββββββββΊ β Unity Editor β
β (1 binary) β localhost:8090 β (auto-starts) β
βββββββββββββββ ββββββββββββββββββββ
A small Go CLI, a single C# UPM package, zero runtime dependencies.
Tests, TUI, batch engine, and asset-config layer sit on top β but the engine that talks to Unity stays lean.
Showcase
A game prototype built primarily with hera-agent-unity β the AI agent drove the live Editor through Hera (scene setup, component wiring, Play Mode iteration), not blind code generation.
NoMoreRolls β solo-developed Unity game (3-tier architecture, 9-soul combat system). Play Mode capture.
π¨ Mockup β Live Unity UI
Hand your agent a reference screenshot or an HTML mockup. Get back a real, pixel-measured uGUI hierarchy β built and verified, not approximated.
Ask any AI to build a Unity UI and watch it flail. It knows HTML and CSS cold β but uGUI is a different planet: RectTransform anchors, pivots, stretch offsets, nested LayoutGroups, 9-sliced sprites. So the model guesses the anchor math, hardcodes absolute positions that shatter at the next resolution, and β the real problem β has no way to check its own work against the reference. You get a layout that's vaguely right and pixel-wrong, then burn an afternoon nudging values by hand.
ui_doc turns UI from the agent's worst Unity skill into a deterministic, self-correcting pipeline. The agent designs in the one language it's genuinely fluent in β a compact, HTML-shaped JSON IR (ui_doc/2) β and Hera performs the exact Unity-side translation. Then it closes the loop almost no AI tooling even attempts: it measures the built result against your reference and fixes the difference, instead of eyeballing it and moving on.
The measure-don't-guess loop
This is what makes a reproduction faithful instead of "close enough" β and it runs with no human in it:
reference image (screenshot / mockup)
β
βΌ
1. sample read the true colors out of the reference image
2. author write the ui_doc/2 IR (HTML-shaped JSON)
3. apply the IR becomes a real uGUI hierarchy
4. capture render exactly what you built β PNG
5. compare diff against the reference, then fix the IR
β
βββββββββββ repeat until capture matches the reference
The agent reads the true colors straight out of your screenshot, builds the UI, renders exactly what it built (overlay canvases and all), diffs it against the target, and corrects β on its own.
Seven endpoints, one contract
| Endpoint | Runs in | What it does |
|---|---|---|
export |
Connector | Serialize a live UI subtree β ui_doc/2 IR (defaults omitted), so the agent grounds its design on the project's real structure instead of hallucinating one. |
apply |
Connector | Build the IR under a --parent. --mode create (default) or upsert (match children by name, edit in place β the full export β edit β apply round-trip). The doc rides via --file, never bloating the agent's context. |
import |
Connector | Bring your own sprite files (absolute paths β a downloaded UI kit, exported art) into the project as Sprite assets (optional 9-slice border / ppu / pivot), so apply can reference them by Assets/ path. GIFs are skipped (no GIFβSprite import). |
gen_sprite |
Connector | Bake a procedural sprite (solid / rounded_rect / gradient / nine_slice) and import it as a Sprite β zero external dependency, no image model, no asset pack. |
capture |
Connector | Render the live UI β including the ScreenSpaceOverlay canvases a normal screenshot composites after the camera and silently drops β to a PNG for visual diffing. |
sample |
CLI | Read measured hex colors out of a reference image (--at "x,y" points / --region "x,y,w,h", normalized [0,1]). Pure stdlib decode β no Unity round-trip, usable before a single object exists. |
catalog |
CLI | Scan a folder of UI sprites (--dir <abs>, recursive) into a manifest β per image: size, alpha, dominant palette, a conservative 9-slice border suggestion, and a filename-derived element guess. The vision-capable agent then reads the listed PNGs to classify them and composes a mockup from your own art. Pure stdlib decode β no Unity round-trip. GIFs are catalogued reference-only. |
Watch it rebuild a game HUD from one screenshot
# 1 β Ground on the project's real hierarchy (don't guess the structure)
hera-agent-unity ui_doc export --path /Canvas/HUD
# 2 β Pull the actual colors out of the reference art
hera-agent-unity ui_doc sample --image hud_ref.png --region "0,0,1,0.2"
# 3 β Author the IR in HTML-shaped JSON, then build it under the Canvas
hera-agent-unity ui_doc apply --file hud.json --parent /Canvas --mode upsert
# 4 β Render exactly what shipped, compare to the reference, fix, repeat
hera-agent-unity ui_doc capture --out hud_built.png
That isn't a contrived demo β it's how the feature was dogfooded into existence, rebuilding a real game HUD from a single screenshot until capture matched the reference.
Mock up from your own UI kit
Point the agent at a folder of sprites and it builds the UI from your art, not procedural placeholders:
# 1 β Scan the kit β manifest (size, alpha, palette, 9-slice + element hints)
hera-agent-unity ui_doc catalog --dir /Users/me/Downloads/SciFiUIKit
# β¦the agent reads the listed PNGs and decides what each sprite is.
# 2 β Bring the chosen sprites into the project as Sprite assets
hera-agent-unity ui_doc import --src .../btn_blue.png --into Assets/UI --border 16,16,16,16
# (or many at once, with per-sprite borders, via --file imports.json)
# 3 β Author the IR referencing them, then build it
hera-agent-unity ui_doc apply --file design.json --parent /Canvas
catalog and import are CLI-side scan + a one-shot project import; classifying "what UI is this" stays with the vision-capable agent that reads the actual pixels.
Why it actually holds up
- Designs in the agent's native tongue. An HTML-shaped node tree, not the raw
m_AnchoredPositionmath the model gets wrong. - A real layout system β not absolute coordinates. The IR (
ui_doc/2) carriesHorizontal/Vertical/GridLayoutGroup,LayoutElement,ContentSizeFitter, Image fill modes (the idiomatic progress / HP bar viafill.amount), 9-slice borders, stretch offsets, and text color / alignment / font. Responsive by construction. - Zero-dependency art.
gen_spritebakes its textures procedurally β nocom.unity.vectorgraphics, no external image generator, nothing to install. - It sees what you see.
capturecomposites the overlay canvases ordinary screenshots miss, so the diff is honest. - Compiles in any project. Every uGUI / TextMeshPro type resolves through
TypeCacheβ the connector keeps zero compile-time dependency oncom.unity.ugui.
Full IR reference: docs/UI_DOC_IR.md Β· every flag: docs/COMMANDS.md.
ui_docbuilds the UI right. UI Juicy Mode below makes it feel alive β when on, everyapplycarries a Game UI/UX Bible juice recipe per element type as anagent_hint.
β¨ UI Juicy Mode β UIs that feel alive, not just functional
The difference between a button that works and a button that feels good to press. Now your AI agent knows it too.
When an AI builds a UI, it gives you a button that works β and feels dead. It sits there. Nothing happens when you hover. It snaps when you click. Scores jump from 0 to 1000 in a single frame. Technically correct, emotionally flat.
Real games are full of tiny, satisfying details you feel more than notice: a button that swells when your cursor touches it, squashes when you press, and springs back with a little bounce. A popup that pops in instead of blinking into existence. A score that rolls upward. A damage number that floats up and fades. Designers call this "juice" (or game feel) β and it's usually the first thing AI-built UI is missing, because the model just doesn't think to add it.
UI Juicy Mode flips that default. Turn it on, and every time your agent creates a UI element, Hera hands it a concrete, numbers-included recipe for making that element feel alive β lifted straight from the Game UI/UX Bible. The agent then wires up the animation, sound, and feedback for you. No design brief, no back-and-forth, no "can you make it feel more polished?" β it just comes out juicy.
Dead vs. Juicy
| Off (default) | On | |
|---|---|---|
| Button | Static rectangle. Click registers, nothing moves. | Swells to 110% on hover, dips to 95% on press, springs back with an overshoot, plays a click, buzzes on mobile. |
| Popup | Appears instantly. | Scales up from nothing with a playful overshoot, dims the screen behind it. |
| Score / HP / gold | Snaps 120 β 999. |
Counts up smoothly over a quarter-second, with a little pop on the final value. |
| Damage number | Plain text appears, then vanishes. | Punches in big, floats up 50px, fades on the way out β crits hit bigger, with a screen shake. |
How it works
It's guidance, not bloat. Hera doesn't bolt heavy runtime components onto your scene. Instead, the recipe rides back to the agent inside the manage_ui create response as an agent_hint β and the agent applies it through the normal manage_components / exec path. The connector stays Editor-only; your build stays clean.
// hera-agent-unity manage_ui create button --name PlayButton (with Juicy Mode on)
{
"instance_id": -8420,
"agent_hint": "[Hera] UI Juicy Mode is on β make this feel alive (Game UI/UX Bible).\n
Button feel (Normal β Hover β Press β Release):\n
- Hover: 100% β 110%, EaseOut 0.15s, +5% brightness\n
- Press: β 95%, EaseOut 0.05s (immediate), β10% color\n
- Release: 95% β 110% β 100% with Back overshoot, 0.2s; click SFX; 10ms haptic on mobile\n
- Disabled: desaturate ~50%, opacity 60β70%, block interaction\n
Tweening: DOTween is enabled β use rt.DOScale(1.1f, 0.15f).SetEase(Ease.OutQuad) β¦"
}
Every recipe is specific β exact scale percentages, easing curves, and timings in seconds β so the agent applies real values instead of guessing. There's a tailored recipe for each element type:
| Element | What it learns to do |
|---|---|
| Button | Hover / press / release state machine with overshoot, click SFX, mobile haptics, disabled styling |
| Panel / popup | Slow exaggerated entrance (pop-in), screen dim, fast quiet exit |
| Image | Pop-in on appear, rarity-scaled reward pulse + glow, flash-on-hit (white + interrupt + knockback), hover lift |
| Bar (filled image) | The signature health/progress juice β instant fill drop with a delayed "chip" bar catching up, low-value danger pulse, segment ticks |
| Text | Staggered line entrances, count-up numbers (rising-pitch combo SFX), floating damage popups |
| Container | Staggered child entrances, animated layout changes instead of snapping |
| Canvas | Stand up the tween + audio infrastructure, hit-pause on big impacts, animate every transition, keep UI footprint lean |
It adapts to your toolchain. If DOTween is enabled in Hera Settings, the recipe tells the agent to use DOScale-style tweens (the project standard) β otherwise it falls back to a coroutine/lerp approach that works in any project. And it always reminds the agent to gate strong motion behind a reduce-motion option, so juice never becomes nausea.
Turn it on
It's opt-in β off by default, one toggle away:
hera-agent-unity asset-config juicy on # enable
hera-agent-unity asset-config juicy off # disable
Or tick UI Juicy Mode in the Hera Settings window inside Unity. That's it β every UI element your agent builds from then on comes out alive.
Maximum output for minimum input. You say "add a play button." Your agent ships one that feels like a real game made it.
Installation
CLI
macOS / Linux
curl -fsSL https://raw.githubusercontent.com/NotNull92/hera-agent-unity/main/install.sh | sh
Windows (PowerShell)
irm https://raw.githubusercontent.com/NotNull92/hera-agent-unity/main/install.ps1 | iex
Other installation methods
go install (any platform)
go install github.com/NotNull92/hera-agent-unity@latest
Manual β download a release binary from Releases, then run it once with install to register it on PATH:
chmod +x ./hera-agent-unity-<platform>
./hera-agent-unity-<platform> install
Unity Connector
Package Manager β Add package from git URL
https://github.com/NotNull92/hera-agent-unity.git?path=AgentConnector
Or add to Packages/manifest.json:
"com.notnull92.hera-agent-unity": "https://github.com/NotNull92/hera-agent-unity.git?path=AgentConnector"
The connector starts automatically. No configuration. Requires Unity 6 (6000.0+).
Compatible Unity versions
| Unity version | Status | Notes |
|---|---|---|
| 6000.0 β 6000.4 | β Supported | Unity 6 (LTS + Tech stream) |
6000.5+ (incl. betas, e.g. 6000.5.0b11) |
β Supported | Needs Connector 0.0.20+. Unity 6000.5 promoted the legacy EditorUtility.InstanceIDToObject / Object.GetInstanceID APIs to hard compile errors (replaced by the new EntityId API); the connector adapts automatically behind a UNITY_6000_5_OR_NEWER gate. |
| 2022.x and earlier | β Not supported | Minimum is Unity 6 (6000.0). |
Symptom of a too-old connector on Unity 6000.5+: the HeraAgent menu is missing and
hera-agent-unity statusreports no instance (the assembly failed to compile, so no menu items or HTTP server register). Fix: update the connector to 0.0.20+ β in Package Manager update the package, or re-resolve the git package (delete its entry fromPackages/packages-lock.jsonand refocus the Editor) β then let Unity recompile.
Quick Start
You only ever type four commands at this CLI. Everything else β exec, editor, console, scene, batch, profiler, describe_type, β¦ β is what your AI agent calls on your behalf once you've handed it the wheel.
# 1. Install the CLI (one-time, see Installation above)
# 2. Open Unity with the AgentConnector UPM package β the heartbeat appears
# Is Unity connected?
hera-agent-unity status
# Later, as maintenance:
hera-agent-unity update # pull the latest CLI release
hera-agent-unity uninstall # remove the CLI from PATH
That's the human loop β install, verify, occasionally update. The next section is where the real work happens: hand the agent a one-line trigger and it drives the rest.
Hand the Wheel to Your AI Agent
Open Claude Code, Codex, Cursor β any agent that can run a shell command. Ask:
"Check if hera-agent-unity is installed and explore its capabilities."
The agent will discover the CLI, run list, and start driving Unity.
Compatibility
hera-agent-unity is a plain CLI returning JSON. Any coding agent that can run shell commands works. The ecosystem is converging on AGENTS.md at the project root as the canonical multi-tool rules file β start there.
| Agent | Canonical path | Template | Notes |
|---|---|---|---|
| OpenAI Codex + AGENTS.md-aware tools | AGENTS.md (project root) |
examples/rules/AGENTS.md |
Cross-tool standard. Lead with this. |
| Claude Code CLI | CLAUDE.md (or AGENTS.md) |
examples/rules/CLAUDE.md |
Reads AGENTS.md natively since late 2025. CLAUDE.md still works for path-scoped rules. |
| Kimi Code CLI | AGENTS.md (project root) |
examples/rules/AGENTS.md |
Reads AGENTS.md natively (/init generates it). Also supports Agent Skills. |
| Cursor | .cursor/rules/hera-agent-unity.mdc |
examples/rules/cursor.mdc |
Per-rule files with YAML frontmatter. .cursorrules is deprecated and ignored by Agent mode. |
| GitHub Copilot | .github/copilot-instructions.md |
examples/rules/copilot-instructions.md |
Optional: .github/instructions/*.instructions.md with applyTo frontmatter for file-pattern-specific guidance. |
| Continue.dev | .continuerules |
examples/rules/continuerules |
Plain markdown. |
| Google Antigravity | GEMINI.md (or AGENTS.md) |
examples/rules/GEMINI.md |
GEMINI.md wins over AGENTS.md. On-demand skill at .agents/skills/hera-agent-unity/SKILL.md. |
For multi-tool projects, the cleanest pattern is AGENTS.md as the single source plus a one-liner stub in tool-specific paths (> See AGENTS.md.). Cursor is the one exception β its .mdc files want the full body inline because the frontmatter is what makes the rule active.
One-time setup per project (strongly recommended)
Static β copy the template that matches your agent:
cp examples/rules/AGENTS.md <your-unity-project>/AGENTS.md
cp examples/rules/cursor.mdc <your-unity-project>/.cursor/rules/hera-agent-unity.mdc
Dynamic β let the CLI emit the lean rule body straight into your rules file:
# AGENTS.md / CLAUDE.md / Copilot / Continue.dev β plain markdown
hera-agent-unity doctor --agent-rules >> AGENTS.md
# Cursor β frontmatter prepended automatically
hera-agent-unity doctor --agent-rules --format cursor > .cursor/rules/hera-agent-unity.mdc
Either path locks in the core instruction and the auto-bootstrap protocol β once installed, saying "find hera-agent-unity" (or the Korean equivalent) makes the agent run doctor + status and report in one line, without asking.
Cursor note β Cursor's
.mdcrule files require YAML frontmatter (description,globs,alwaysApply) or the rule is parsed but never activated. Use the template or the--format cursorflag β a plain markdown paste will silently no-op.
Commands
Grouped by what they touch. Run hera-agent-unity <cmd> --help for the full flag list of any entry, or list to inspect every registered tool (including custom ones) at runtime.
Editor & runtime
| Command | What it does |
|---|---|
editor |
Play / stop / pause / refresh / recompile. |
exec |
Run arbitrary C# inside Unity β full editor & runtime access. |
log |
Write to Unity console without the csc compile cost. |
Scene & GameObjects
| Command | What it does |
|---|---|
scene |
Info / load / save / list / close. |
manage_gameobject |
Create / destroy / move / re-parent / set_active / rename / get_transform. |
manage_components |
Component CRUD on a GameObject: add / remove / list / get / set. Property paths are raw SerializedProperty paths. |
find_gameobjects |
Filter scene GameObjects (name / tag / layer / component / path glob) with pagination. |
manage_prefab |
Prefab asset ops: create (GameObject β prefab) / instantiate / headless add_component / remove_component. |
manage_ui |
uGUI authoring: create (UI element + auto Canvas/EventSystem) / get_rect / set_anchor (named preset grid) / set_rect. RectTransform anchor/pivot math without raw m_ paths. With UI Juicy Mode on, create returns DOTween-aware Game UI/UX Bible juice recipes as an agent_hint. |
ui_doc |
HTMLβUnity UI pipeline (uGUI): export (live subtree β compact ui_doc/2 JSON, for grounding) / apply (IR β UI; --mode create or upsert to update existing in place; doc via --file) / gen_sprite (Tier-1 procedural sprite β solid/rounded_rect/gradient/nine_slice β baked + imported, no external dep) / capture (render the live overlay UI β PNG for visual verification) / sample (read measured hex colors from a reference image β point/region, no Unity round-trip). Juice recipes ride apply's agent_hint (deduped per element type) when Juicy Mode is on. |
html-to-uidoc |
Convert an inline-style HTML mockup to ui_doc/2 JSON. Set --width/--height to the HTML design canvas size so the generated IR's canvas.reference_resolution gives 1 HTML px = 1 uGUI unit. No Unity round-trip. |
Assets, materials & shaders
| Command | What it does |
|---|---|
manage_material |
Material asset CRUD: create (with a shader) / get / set (one property) / set_shader. |
manage_asset_import |
Read / write an asset's import settings via its AssetImporter, then reimport. |
describe_shader |
Inspect a shader's properties (name / type / range) or search shader names (--list). |
Packages
| Command | What it does |
|---|---|
manage_packages |
list (sync) / add / remove / embed (async β returns a job_id, CLI polls the result file). |
Console, tests, capture
| Command | What it does |
|---|---|
console |
Read, filter, clear logs. |
test |
Run EditMode / PlayMode tests. |
menu |
Execute any menu item by path. |
screenshot |
Capture Scene or Game view. |
profiler |
Read hierarchy, toggle recording. |
reserialize |
Force Unity to re-serialize YAML after text edits. |
Introspection
| Command | What it does |
|---|---|
describe_type |
Reflect a live type β members, signatures, Unity pitfalls + Manual links. |
find_method |
Search method names across loaded assemblies. |
list_assemblies |
List loaded assembly names (skips System.*; --include_version / --include_location to enrich). |
unity_docs |
Offline Unity ScriptReference lookup β minimal title / signature / summary reply (~33 tokens). |
Workflow
| Command | What it does |
|---|---|
batch |
Execute multiple commands atomically in one HTTP round-trip. |
list |
List registered tools β names only (--names), name+description (default), or --tool <name> for the full schema. |
Status & maintenance
| Command | What it does |
|---|---|
status |
Connection & project info. |
ping |
Token-cheap liveness probe (heartbeat read only β no HTTP round-trip). |
doctor |
Self-diagnose: PATH, installs, shell, Unity reachability (--json / --agent-rules for agents). |
asset-config |
Toggle optional asset integrations + UI Juicy Mode (TUI / list / enable / disable / juicy on|off / detect). |
update |
Self-update from GitHub Releases. |
install / uninstall |
Register or remove the CLI on PATH. |
Stuck? Run hera-agent-unity doctor, or open docs/TROUBLESHOOTING.md.
What's New (post-v0.0.6 capability queue)
The five-tool queue locked-in at 2026-05-28 is complete. Each entry filled a gap where exec's csc warm-up cost or the syntactic overhead of wrapping a Unity API call in C# was too high for a clean AI agent workflow.
| Tool | Connector | What it does |
|---|---|---|
manage_gameobject |
v0.0.5 | Create / destroy / move / re-parent / set_active / rename / get_transform. Shallow instance_id return survives renames and reparenting. |
manage_packages |
v0.0.6 | Client.Add / Remove / Embed / List driver. Async jobs (job_id) ride through the resolver's domain reload via an [InitializeOnLoad] watcher + a Client.List post-reload verifier. |
find_gameobjects |
v0.0.7 | Filter scene GameObjects by name / tag / layer / component / path glob, with stable pagination over a hierarchy-sorted result set. |
manage_components |
v0.0.8 | Component CRUD via raw SerializedProperty paths (m_Mass, m_Materials.Array.data[0]). Reference fields accept an InstanceID, an asset path, or a {instance_id|asset_path} envelope. Establishes the property-set pattern every future manage_* will reuse. |
unity_docs |
v0.0.10 β v0.0.12 | Offline Unity 6 ScriptReference lookup. 31,581 entries ship inside the UPM package itself as a 1.2 MiB gzipped JSONL β no docs folder on the user's machine, no network access, no rate limits. |
describe_shader Β· manage_material Β· manage_prefab Β· manage_asset_import |
v0.0.14 | Asset-editing set. describe_shader (inspect/search shaders) pairs with manage_material (material CRUD, reuses SerializedPropertyValue); manage_prefab edits prefab assets headlessly via LoadPrefabContents; manage_asset_import drives import settings through AssetImporter (the manage_components pattern). |
manage_ui |
v0.0.15 | uGUI authoring. create spins up UI elements (canvas / panel / image / button / text / empty) with auto Canvas + EventSystem scaffolding; set_anchor exposes Unity's named anchor-preset grid and keeps the rect visually fixed (or --snap for Alt+Shift fill); get_rect / set_rect round out RectTransform editing. UI/TMP types resolve via TypeCache, so the connector still compiles in projects without com.unity.ugui. Element property edits stay in manage_components. |
TargetResolver + tests + benchmark |
v0.0.16 | TargetResolver extracts shared GameObject/Component resolution from manage_components / manage_gameobject / manage_ui. Go test coverage expanded (doctor, install, poll, assetconfig). C# HierarchyPath editor tests. Smoke test benchmark: 7 calls = 725B (~181T), avg 26T/call β no token cost increase from refactoring. |
| UI Juicy Mode | v0.0.19 | Opt-in toggle that makes manage_ui create return Game UI/UX Bible "juice" recipes so AI-built UIs feel alive instead of dead. See the dedicated β¨ UI Juicy Mode section above. |
| Unity 6000.5 compat | v0.0.20 | Core/EntityIdCompat shim (gated by UNITY_6000_5_OR_NEWER) routes 27 call sites through the new EntityId API on 6000.5+ and the legacy InstanceID API on 6000.0β6000.4. Unity 6000.5 promoted the old APIs to hard compile errors β without the shim the connector silently fails to compile and the HeraAgent menu vanishes. |
ui_doc |
v0.0.21 β v0.0.27 | HTMLβUnity UI pipeline (uGUI) β the agent's weakest area. export / apply (create + upsert) / gen_sprite / capture / sample around a compact JSON IR (ui_doc/2). Grew into a full layout system (Horizontal / Vertical / Grid LayoutGroup, LayoutElement, ContentSizeFitter), Image fill modes + 9-slice, stretch offsets, text color / align / font, and a measure-don't-guess verify loop. Zero external dependency; still no compile-time com.unity.ugui link. The flagship π¨ Mockup β Live Unity UI section walks the whole loop. |
| Performance & reliability | v0.0.28 | Conservative pass: VBCSCompiler pre-warm on startup, refs-meta.json reference cache (skips the full AppDomain scan), in-memory assembly cache 32 β 128, per-type Serialize reflection cache, faster known-path compiler discovery. CLI side: batch nil-pointer fix, compact/quiet batch output, SendBatch reusing the domain-reload retry, and a 50 MB response guard. No user-facing default changed. |
exec on macOS / Linux |
v0.0.31 | The snippet compiler now runs a Windows-PE csc.exe through Unity's bundled Mono host on macOS / Linux (a managed PE can't be exec'd directly) instead of failing with EXEC_LAUNCH_FAILED. Resolves a defaultCscPath β csc.exe config that pointed exec at a PE. Windows path unchanged. |
exec on Windows / Unity 6.5 |
v0.0.32 β v0.0.34 | Fixed exec failing to compile anything on non-Latin (Korean / Japanese / Chinese) Windows under Unity 6.5+ β every snippet returned EXEC_COMPILE_ERROR with a System.Text.Encoding.CodePages load failure. Unity 6.5 relocated the SDK Roslyn to DotNetSdk/sdk/<version>/Roslyn/bincore/csc.dll, and the resolver was short-circuiting to the bundled Mono csc.exe, which crashes on a CP949 console. Now prefers csc.dll (recursive, version-agnostic), forces -utf8output + a UTF-8 BOM, and ignores a stale Mono csc.exe defaultCscPath. Verified no regression on 6.3. |
| Discovery token cuts | v0.0.35 | Leaner agent-discovery surface: list default drops the per-tool JSON schemas (~β73%), list --names returns a bare name array (~β95%), list_assemblies returns bare names by default (--include_version to enrich, ~β49%), and find_method / describe_type drop the redundant is_static field (the signature already encodes static). |
| Tool-error de-dup | CLI v0.0.24 | A failed tool command printed its message twice β once as the compact JSON envelope, once as a human Error: command failed: β¦ line. AI-target commands now emit the JSON only, halving the error's token cost. |
editor refresh --compile bounded |
CLI v0.0.25 | The recompile wait used a hidden 5-minute cap and ignored --timeout, so a wrapping agent (Claude Code's 120 s bash budget) backgrounded the still-running process. It now honors --timeout (60 s default; raise for big projects) and reports a timeout distinctly from a compile that produced errors. |
UI Juicy Mode β deeper coverage + bar |
v0.0.36 | Audited Core/UIJuiceGuide against the Secrets of Game Feel and Juice playbook and enriched the existing recipes with four cross-cutting techniques (randomized click-SFX pitch + rising-pitch count-up combos, canvas hit-pause ~30β80 ms, image flash-on-hit with tween-interrupt + knockback, and an explicit "double down on the screen's purpose; keep precision/input UI calm" golden rule). Added a genuine new element type β bar (health/progress) β with the signature fill juice (instant drop + delayed "chip" bar, low-value danger pulse, segment ticks). It fires through ui_doc apply: a filled image node (image.fill / type=filled) is classified as bar for the hint. manage_ui's --element set stays locked. |
console --lines default + truncation fix |
v0.0.37 | console now defaults --lines to 20 (matching the documented behavior) instead of returning the entire buffer. Fixed an off-by-one in truncation detection (>= β >) that flagged truncated one entry early. Also synced the agent docs: exec --depth defaults to 3 (the docs incorrectly said 1), and the CLI dropped unreachable help / version entries from humanCategories (both are handled before that whitelist is consulted). |
Error-code normalization + html-to-uidoc |
Connector v0.0.39 / CLI v0.0.27 | All tool errors now carry stable code values (branch on the code, not the message). ToolDiscovery.FindHandler legacy alias removed. ui_doc apply now consumes the IR top-level canvas block to configure CanvasScaler (Scale With Screen Size + reference resolution), and a root element:"canvas" is created at the scene root. New CLI command html-to-uidoc converts inline-style HTML mockups to ui_doc/2 JSON so HTML px values map 1:1 to uGUI units. |
unity_docs β design + benchmarks (v0.0.12)
The same generic shape every "RAG-style" lookup tool has β query β keyed retrieval β minimal context back to the agent β except the data set is small enough (~10 MiB of HTML β 1.2 MiB of gzipped JSONL) that it ships inside the connector. No embedding model, no vector DB, no external API. A single Dictionary<string, Entry> carries the whole Unity 6 ScriptReference; a lazy three-layer pre-filter keeps "did you mean" suggestions cheap on miss.
Build path (one-time per Unity version, maintainer-run):
go run ./tools/build-unity-docs \
--in <path-to-Documentation/en> \
--out AgentConnector/Editor/Data/unity_docs_6.0.jsonl.gz.bytes
The Go script applies the same compiled regex set the connector used to evaluate per call (h1.heading inherit, signature-CS sig-block, h3 Description, switch-link anchor, Unity version) and emits sorted JSONL. Gzip is auto-applied when the output path contains .gz. The committed artefact is what every CLI install picks up via the UPM package.
Runtime path β the connector lazy-loads the bundle the first time unity_docs is invoked, then everything is O(1) dict lookup with a Levenshtein fallback for misses:
| Path | Connector total_ms |
|---|---|
| Happy lookup (dict hit) | < 1 ms |
Miss in-bucket β Rigidbod (1-char typo) |
2 ms |
Miss in-bucket β MonoBehavior (2-char typo) |
4 ms |
Miss with bucket fallback β Wigidbody (first-char typo) |
2 ms |
Miss no-match β XyzNonsenseAbc |
5 ms |
Cold first call (gzip decompress + JSONL parse of 31,581 entries + dict + prefix-bucket build) is ~1.7 s and runs exactly once per Unity domain β domain reloads reset the state, the next call rebuilds.
The 220 β 280 ms CLI end-to-end you see in a terminal is Go binary startup + HTTP roundtrip + Unity queue dispatch β shared by every hera tool, not unique to unity_docs.
Response shape is intentionally minimal:
{ "title": "Rigidbody.mass",
"signature": "public float mass;",
"summary": "The mass of the rigidbody." }
~100 β 185 bytes per reply, ~33 input tokens to the agent. Down ~53 % from a verbose shape that echoed query_normalized, scriptreference_url, unity_version, etc. that the caller could already derive.
Miss-path scan uses three layered cheap pre-filters before paying for a Levenshtein DP table:
- Prefix bucket β keys grouped by lowercase first letter (lazy, built once after
EnsureLoaded). A typo missing only an internal character lands in the right bucket and scans ~1/26 of the corpus. - Length filter β
abs(len(a) - len(b)) > maxDistanceis a lower bound on the edit distance, so length-incompatible pairs are rejected without populating a DP table. - Bounded Levenshtein β the DP scan bails the moment a row min exceeds the budget; returns a
maxDistance + 1sentinel so the caller branches on a single comparison.
If the bucket yields zero results (first-character typo) the full key set is rescanned as a fallback β the Wigidbody row above is that path firing, and it still returns Rigidbody, Rigidbody2D within 2 ms.
exec β Runtime C#
The most powerful command. Full editor + runtime access. Zero boilerplate.
# Evaluate
hera-agent-unity exec "return Application.dataPath;"
hera-agent-unity exec "return GameObject.FindObjectsOfType<Camera>().Length;"
# Modify the scene
hera-agent-unity exec "var go = new GameObject(\"Temp\"); return go.name;"
# ECS / custom assemblies
hera-agent-unity exec "return World.All.Count;" --usings Unity.Entities
# Pipe complex code via stdin β no shell escaping
echo '
var scene = EditorSceneManager.GetActiveScene();
return scene.GetRootGameObjects().Length;
' | hera-agent-unity exec
# Or load from file
hera-agent-unity exec --file scripts/probe.cs
| Flag | Purpose |
|---|---|
--usings ns,... |
Add extra using directives |
--file <path> |
Load code from disk (stdin and positional arg take precedence) |
--csc <path> |
Override the C# compiler path |
--dotnet <path> |
Override the dotnet runtime path |
--no-cache |
Skip the compiled-assembly cache (debug only) |
--depth N |
Limit response object graph depth (default 3, max 8) |
--check |
Compile-only dry run. Returns SuccessResponse on a clean compile, EXEC_COMPILE_ERROR otherwise. No Execute() call, no side effects. |
--stacktrace <mode> |
Shape of EXEC_RUNTIME_ERROR stack traces: none (exception_type only), user (default β drop framework frames), full (raw). |
--strict |
Capture Debug.LogError / LogException / LogAssert raised during the snippet and surface them as EXEC_LOGGED_ERROR. |
How it works. Code is wrapped in a static method, compiled with the system's Roslyn (csc) into a temporary DLL, loaded into a collectible AssemblyLoadContext (no leaks), invoked via reflection, and the result is JSON-serialized. Identical source code is served from an in-memory cache β warm calls skip csc entirely.
Default usings include System, System.Collections.Generic, System.IO, System.Linq, System.Reflection, System.Threading.Tasks, UnityEngine, UnityEngine.SceneManagement, UnityEditor, UnityEditor.SceneManagement, UnityEditorInternal.
Introspection β describe_type, find_method, list_assemblies
These three commands read what's actually loaded in your Unity project, not what an LLM half-remembers from its training cutoff.
# What's loaded? (skips System.* by default)
hera-agent-unity list_assemblies
hera-agent-unity list_assemblies --filter Unity.Entities
# Inspect a type β signatures + curated Unity pitfalls
hera-agent-unity describe_type UnityEditor.EditorApplication
hera-agent-unity describe_type AssetDatabase --members methods --limit 50
# Search method names across the live AppDomain
hera-agent-unity find_method Refresh --namespace UnityEditor
describe_type returns a pitfalls array alongside the schema β short, stable notes paired with Unity 6 Manual links. Coverage: Editor API, MonoBehaviour lifecycle, uGUI (Canvas, RectTransform, EventSystem, LayoutGroup, ScrollRect, Selectable, Mask, CanvasGroup, β¦). Each entry is { text, doc_url } so an agent gets signature + gotcha + fetchable doc in a single round-trip.
Pair with exec for a tight dry-run loop:
describe_type β write code β exec β fix β exec
Batch β Scripted Workflows
Run multi-step pipelines in a single CLI invocation. Ideal for CI, scripted automation, or large agentic plans.
hera-agent-unity batch --file workflow.json
{
"commands": [
{ "command": "refresh_unity", "params": { "compile": "request" } },
{ "command": "exec", "params": { "code": "return EditorSceneManager.GetActiveScene().name;" } },
{ "command": "console", "params": { "type": "error", "lines": 10 } }
],
"options": { "fail_fast": true }
}
Pipe JSON via stdin if you'd rather not write a file:
echo '{"commands":[{"command":"refresh_unity","params":{"compile":"request"}}]}' \
| hera-agent-unity batch
--dry-run previews the plan without execution. fail_fast short-circuits on the first error and reports which step failed.
Profiler
Read the live profiler from your terminal β no UI required.
hera-agent-unity profiler enable # start recording
hera-agent-unity profiler hierarchy # top-level samples (last frame)
hera-agent-unity profiler hierarchy --depth 3 # recursive drill-down
hera-agent-unity profiler hierarchy --root PlayerLoop --depth 5
hera-agent-unity profiler hierarchy --frames 30 --min 0.5 --sort self
hera-agent-unity profiler disable # stop recording
hera-agent-unity profiler status
hera-agent-unity profiler clear
| Flag | Purpose |
|---|---|
--depth N |
Recursion depth (0 = unlimited, default: 1) |
--root <name> |
Substring-match root sample |
--frames N |
Average over the last N frames |
--from N --to N |
Average over an explicit frame range |
--parent ID |
Drill into an item by ID |
--min <ms> |
Filter items below threshold |
--sort <col> |
total (default), self, calls |
--thread N |
Thread index (0 = main) |
Custom Tools
Drop a C# class anywhere in your Editor assembly. It is discovered automatically β no registration, no codegen.
using HeraAgent;
using Newtonsoft.Json.Linq;
using UnityEngine;
[HeraTool(Name = "spawn", Group = "gameplay", Description = "Spawn a prefab at a position")]
public static class SpawnEnemy
{
public class Parameters
{
[ToolParameter("X world position", Required = true)] public float X { get; set; }
[ToolParameter("Y world position", Required = true)] public float Y { get; set; }
[ToolParameter("Z world position", Required = true)] public float Z { get; set; }
[ToolParameter("Prefab name", Default = "Enemy")] public string Prefab { get; set; }
}
public static object HandleCommand(JObject args)
{
var p = new ToolParams(args);
var prefab = Resources.Load<GameObject>(p.Get("prefab", "Enemy"));
var inst = Object.Instantiate(prefab,
new Vector3(p.GetFloat("x"), p.GetFloat("y"), p.GetFloat("z")),
Quaternion.identity);
return new SuccessResponse("Spawned", new { name = inst.name });
}
}
Call it:
hera-agent-unity spawn --x 1 --y 0 --z 5 --prefab Goblin
Rules
- Decorate with
[HeraTool] - Expose
public static object HandleCommand(JObject parameters)(instance methods also work) - Return
SuccessResponse(message, data)orErrorResponse(message) - Use
{ get; set; }properties inParametersβ fields are invisible to the schema generator - Class name auto-converts to
snake_case(SpawnEnemyβspawn_enemy); override withName = - Discovered on Editor start and after every script recompile
- Runs on Unity's main thread β every API is safe
- Duplicate tool names are flagged in the console; first registration wins
hera-agent-unity list exposes the parameter schema so agents can discover and call your tools without reading source.
Performance
hera-agent-unity is designed to keep token costs negligible for AI agent workflows. A recent smoke-test benchmark (v0.0.16, 7 representative scenarios) confirms the overhead stays minimal:
| Metric | Value |
|---|---|
| Total calls | 7 |
| Total round-trip bytes | 725 B |
| Estimated tokens (chars Γ· 4) | ~181 T |
| Avg per call | 26 T |
| Responses β€ 5 B | 71% |
What drives cost: 88% of bytes are the C# code the agent writes (input), not the response. A typical exec call like return Time.time; costs 6 tokens total β the response itself is 2β3 tokens.
A full 50-scenario benchmark and the smoke-test report live in docs/benchmarks/.
Architecture
ββββββββββββββββββββ ββββββββββββββββββββββββββββββββββββ
β CLI (Go) β β Unity Editor β
β β HTTP β β
β ββββββββββββββ β POST β ββββββββββββββ β
β β Discover ββββΌββ/commandββΌββΊβ HttpServer β (localhost:8090+)β
β β Instance β β β βββββββ¬βββββββ β
β ββββββββ¬ββββββ β β β ConcurrentQueue β
β β read β β βββββββΌβββββββ β
β ββββββββΌββββββ β β β Command β EditorApplicationβ
β β Heartbeat β β β β Router β .update β
β β files ββββΌβββwriteββββΌβββ β (main thread) β
β β (instance) β β β βββββββ¬βββββββ β
β ββββββββββββββ β β β SemaphoreSlim(1,1) β
β β β βββββββΌβββββββ β
β ββββββββββββββ β β β Tool β [HeraTool] β
β β Backoff β β β β Discovery β reflection β
β β Polling β β β βββββββ¬βββββββ β
β ββββββββββββββ β β βββββββΌβββββββ β
β β β β Handlers β exec, editor, β
β β β β β test, profilerβ¦ β
ββββββββββββββββββββ β ββββββββββββββ β
ββββββββββββββββββββββββββββββββββββ
| Principle | What it means |
|---|---|
| Stateless | Every request is independent. No sessions, no reconnect logic. |
| Auto-discovery | Scans ~/.hera-agent-unity/instances/ heartbeat files. Matches by CWD, project, or port. |
| Domain-reload safe | [InitializeOnLoad] + assembly-reload events survive script recompiles. |
| Main-thread execution | All tool handlers marshal through ConcurrentQueue + EditorApplication.update. |
| Filesystem cross-process bus | Heartbeat + test-result files survive HTTP server tear-down during domain reloads. |
| Atomic writes | Heartbeats are written to .tmp then renamed β readers never see a half-written JSON. |
Engineered for AI agents
- Function-typed DI in Go. Command handlers receive
sendFnandinstanceResolveras injected functions. Theresolveclosure re-discovers the instance on every call, so domain reloads that rebind the HTTP port are absorbed transparently. - Three-phase orchestration. Compile-triggering commands do
waitForAliveβ send βwaitForReady. Polling uses a 1.5Γ backoff (100ms β 2s cap) β finer than the usual 2Γ because Unity often returns to ready faster than that. - Compile grace period.
editor refresh --compilekeepsstate == "compiling"pinned for 3 s so polling doesn't latch onto a stale"ready"before Unity actually starts the compile. - Domain-reload-safe async jobs. PlayMode tests and Package Manager
add/remove/embedall trigger a domain reload that destroys the in-flight HTTP listener and Request handle. Each writes its outcome to~/.hera-agent-unity/status/{test,package}-result-*.jsonand the CLI polls at 500 ms.[InitializeOnLoad]hooks reattach watchers after the reload settles (and, for packages, run a verifyingClient.Listto reconstruct the outcome) so the result file always materialises. - Atomic self-update. GitHub Releases β backup β rename β cleanup with rollback. Windows defers
.bakdeletion to a spawned PowerShell because the running.exeis locked.
Compared to MCP
| MCP integrations | hera-agent-unity | |
|---|---|---|
| Install | Python + uv + FastMCP + config | Single binary |
| Runtime deps | WebSocket relay, persistent proc | None |
| Protocol | JSON-RPC 2.0 over stdio | Direct HTTP POST |
| Setup | Generate config, restart client | Add UPM package, done |
| Domain reload | Complex reconnect logic | Stateless (filesystem bus) |
| Custom tools | [Attribute] pattern |
Same [Attribute] pattern |
| Compatibility | MCP clients only | Any shell, any agent, any script |
| Multi-instance | Manual setup | CWD / project / port auto-discovery |
Global Flags & Environment
--port <N> # Select Unity instance by active heartbeat port
--project <path> # Select Unity instance by project path (substring match)
--timeout <ms> # Request timeout in ms (default: 60000)
--verbose # Per-phase timings + progress messages to stderr
--quiet # Suppress decorative progress messages (errors stay plain)
--debug # Dump HTTP request / response bodies + discovery info to stderr
--compact-json # Emit JSON without indentation β smaller payloads for AI agents
--narrate # Force waitForAlive / waitForReady narration even on tool commands
# (default: narration is on only for human-target commands)
Each global flag has a HERA_AGENT_* environment-variable counterpart that the
CLI reads when the flag is omitted. Setting these once in your shell profile
lets you keep the flag off the command line.
| Environment variable | Equivalent flag |
|---|---|
HERA_AGENT_PORT |
--port <N> |
HERA_AGENT_PROJECT |
--project <path> |
HERA_AGENT_TIMEOUT_MS |
--timeout <ms> |
HERA_AGENT_VERBOSE |
--verbose |
HERA_AGENT_QUIET |
--quiet |
HERA_AGENT_DEBUG |
--debug |
HERA_AGENT_COMPACT_JSON |
--compact-json |
HERA_AGENT_NARRATE |
--narrate |
HERA_AGENT_NO_PATH_CHECK=1 |
Silence the per-command PATH-mismatch warning (no flag equivalent) |
GITHUB_TOKEN |
Auth used by update when the release lives on a private GitHub mirror |
FAQ
Unity says "port 8090 is taken."
The connector probes 8090, 8091, 8092, β¦ up to 10 attempts. If all are occupied, look for zombie Unity processes or another local service holding the port. The CLI reads the real port from the heartbeat file β port numbers are transparent to you.
Commands hang when Unity is minimized.
They shouldn't β the connector calls RepaintAllViews() after every enqueue so the update loop runs even unfocused. If it still happens, check that the UPM package is installed and the Unity console shows [Hera] HTTP server started on port XXXX.
`exec` fails with "Cannot find csc compiler."
The compiler is auto-detected from .NET SDK, Visual Studio, or Unity's bundled Roslyn. If none are visible, install the .NET SDK or point at the compiler explicitly:
hera-agent-unity exec "return 1+1;" --csc "C:\Program Files\dotnet\sdk\8.0.100\Roslyn\bincore\csc.dll"
On macOS / Linux a Windows-PE csc.exe is launched through Unity's bundled Mono host automatically (Connector 0.0.31+), so a .dll Roslyn isn't required β prefer pointing --csc at a csc.dll only when you want the .NET Roslyn path.
`exec` returns `EXEC_COMPILE_ERROR` with a `System.Text.Encoding.CodePages` failure (non-English Windows + Unity 6.5).
Every snippet β even return 1+1; β fails because the resolver picked Unity's bundled Mono csc.exe, which crashes loading System.Text.Encoding.CodePages on a CP949 / Shift-JIS / GBK console. Update the connector to 0.0.34+: it prefers the .NET SDK Roslyn csc.dll (whose path Unity 6.5 moved to DotNetSdk/sdk/<version>/Roslyn/bincore/) and forces UTF-8 compiler I/O. Re-resolve the git package (delete its entry from Packages/packages-lock.json, refocus the Editor) to pick it up.
How does it pick an instance when multiple Unity editors are open?
Priority order:
--portflag (explicit)--projectflag (path substring match)- Current working directory matches a known project path
- Most recent heartbeat timestamp (fallback)
Is it safe to use in CI?
Yes β batch was designed for it. Exit codes propagate per command; fail_fast short-circuits on the first error. The update command and version notice can be silenced for non-interactive runs.
Projects Using Hera
| Project | Description |
|---|---|
| NoMoreRolls | Solo-developed Unity game β 3-tier architecture, 9-soul combat system. Built with hera-agent-unity. |
Want yours listed? Open an issue or PR.
Author
Victor β Unity/C# developer, 6+ years of live-service MMORPG production. Building NoMoreRolls solo with hera-agent-unity Β· IndieAlchemist on YouTube.
Sponsors
hera-agent-unity is free and open source. If it saves you time or tokens, consider sponsoring.
Your support directly funds:
- New engine support (Godot, Unreal)
- Deeper agent integrations (Cursor, Windsurf, β¦)
- Documentation, video tutorials, sample projects
License
MIT β see LICENSE.
Documentation
ΒΆ
There is no documentation for this package.
Directories
ΒΆ
| Path | Synopsis |
|---|---|
|
internal
|
|
|
tools
|
|
|
build-unity-docs
command
build-unity-docs converts the offline Unity ScriptReference HTML tree into a single JSONL file that ships with the connector.
|
build-unity-docs converts the offline Unity ScriptReference HTML tree into a single JSONL file that ships with the connector. |