CYOACatalog

command module
v0.0.0-...-9ce85ca Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 25, 2026 License: Apache-2.0 Imports: 58 Imported by: 0

README

CYOA Cafe — catalog site

Source of cyoa.cafe: a public catalog of CYOA games. Users browse, submit, tag, upvote, comment, chat, and host games on <author>.cyoa.cafe subdomains.

This repository exists so the site can be rebuilt if the original maintainer disappears. It holds everything needed to build and run the server. It does not hold the database: user accounts are not shared, and a catalog dump may be published separately.

Stack: Go + PocketBase (single binary, SQLite) · React/TypeScript (Vite, MUI) embedded into the binary · Cloudflare (CDN, R2 for hosted game files).

Layout

  • *.go — the server: a PocketBase app with custom routes (game hosting, chat, tag voting, moderation, SEO pages, web push, bump roulette). Entry point: main.go.
  • src/ — the React SPA. vite build output (dist/) is embedded via //go:embed.
  • pb_hooks/ — PocketBase JS hooks, read from disk at runtime (NOT embedded).
  • cheat_shim.js — script injected into hosted games for the build saver / cheat companion.
  • public/ — static assets copied into the build.
  • semantic-search/ — embeddings search service (Python, FastAPI), deployed separately.
  • pb_scripts/ — PocketBase schema/data change scripts; seed.go builds a local dev DB.
  • e2e/ — Playwright tests.

Build and run

Requirements: Go 1.25+, Node 20+ with npm, Python 3 (for pb_scripts/ only).

make install   # frontend dependencies (npm ci, from package-lock.json)
make seed      # local DB: schema from pb_schema.json + today's public catalog from cyoa.cafe
make dev       # open http://localhost:8090 — local API + Vite hot reload
make check     # type check + go vet + go test (writes nothing)
make test      # Playwright end-to-end tests against a throwaway copy of the local DB
make build     # tsc + vite build + go build → dist/serve

make seed never copies users. It creates local accounts instead: admin@local.test / localadmin123 (PocketBase admin UI at /_/), user@local.test / localuser123, moder@local.test / localmoder123 (moderator). Details, and what does not work locally, are in AGENTS.md — written for AI coding agents, equally useful for humans.

⚠️ Use port 8090. Vite's own port 8091 proxies /api to the production site.

Semantic search and "similar games" come from the service in semantic-search/ (port 8100 in dev). The rest of the site works without it.

make build first runs update-oauth, which pulls the forked OAuth2 plugin (github.com/DragonsWho/pocketbase-ext-oauth2, forum SSO) and may touch go.mod; make build-app skips that.

Data

pb_schema.json is a sanitized snapshot of the production collections (no system collections, no secrets). The maintainer refreshes it after every deploy and schema change, so make seed stays current. Don't edit it by hand: schema changes are Python scripts in pb_scripts/ (see pb_scripts/README.md), tested locally and applied to production by the maintainer.

Configuration

Copy .env.example to .env. Everything is optional for local development. Production needs:

  • Cloudflare R2 credentials (game hosting);
  • Turnstile keys;
  • a Cloudflare API token (cache purge);
  • VAPID keys (./dist/serve push-keys);
  • a unique SHOUTBOX_ANON_SALT.

Deploy

make ship cross-compiles for linux/amd64, uploads the binary and hooks over SSH, restarts the systemd service and purges the Cloudflare cache. The SSH target is read from an untracked deploy.mk:

SSH_HOST := root@<origin-ip>

Keep the origin address private: the site relies on Cloudflare in front of it.

License

Apache-2.0 — see LICENSE. The name "cyoa.cafe" is not covered by the license.

Documentation

Overview

Email binding for accounts without a password (Discord OAuth users; PB gave them a random one they never saw). Stock confirm-email-change requires the account password → deadlock with password reset. Here proof of right = owning the NEW mailbox; afterwards the user sets a password via stock "forgot password". Token = PB NewEmailChangeToken (signed with record key, expires, dies when email/password changes). Own email template: the stock one links into the PB admin UI, which asks for the password again. Threats covered: hijacked session → notice to the OLD address; spam via our domain → 3 tries/30min/account; grabbing someone's address → uniqueness checked at request AND at confirm.

Player-uploaded images inside builds (ICC `/IMG#` suffix). Native ICC embeds them as base64 data-URLs, which alone blow the 30k-char showcase-comment limit, so builds store a LINK and bytes live in the `build_images` collection (PB/create_build_images_collection.py). Key = sha256 of content: the same avatar is re-sent on every "Update my build"; dedup turns that into one GET. Only this handler writes (createRule=nil) and it enforces size cap, type whitelist and daily quota. Without the quota this is free image hosting on our bandwidth.

Bump roulette: community raffle deciding which game gets bumped to the top. A once-a-day vote = raffle ticket (max 3 per game per account); a minute cron draws one uniformly random ticket when now ≥ bump_settings.next_draw_at; winner gets bumped_at=now, tickets reset, 30d lockout. Server-authoritative like tag votes: bump_votes is backend-write-only, all rules enforced here, frontend reads aggregates only. Collections bump_votes/bump_draws/bump_settings auto-create on boot; games/users schema untouched. Canon: PB/schema_changes_bump_roulette.md

Custom chat emoji pack: `chat_emoji` collection + moderator handlers. The pack is data, not part of the binary: files are a PB file field served from /api/files/chat_emoji/…, covered by the Cloudflare rule "PB media — cache hard 31d". ⚠ Field `file` MUST stay protected=false: protected adds a per-user token to every URL and the CDN is bypassed. Pack is held in memory, rebuilt by record hooks on any change; served as a ready JSON blob with ETag (no DB hit per request). Image conversion is intentionally NOT server-side: a third of the pack is animated webp (impossible via canvas), and img2webp on the droplet expands all frames to PNG in a memory-starved box. Animations are built by the (private) emoji-pack builder; the browser compresses statics; the server accepts ready webp.

Built-in CYOA cheat companion. cheat_shim.js is injected into a hosted game's index.html ONLY when the request has ?__cheat=1 (see serveFromR2); without the flag prod games are byte-for-byte untouched. The shim runs inside the game document (same JS context as the engine), renders its UI in a shadow root and reaches the ICC+2 store via window.debugApp. The game lives on author.cyoa.cafe (other origin), so all cheat UI stays inside the iframe — no cross-origin postMessage bridge. v1 targets ICC+2 only.

Cloudflare cache purge after card edits. The SPA shell is `public, max-age=3600` (main.go) and game records are fetched WITHOUT Authorization on purpose (`gamesCollectionPublic` in pocketbase.ts) so CF caches them. Cost: a moderator removes a tag and the page shows it for minutes ("edit didn't save"). On every meaningful edit (tags, description, cover, pages, iframe_url, title, authors, hidden) purge: games record + catalog lists (`/api/collections/games/records…` by PREFIX — slug lookups are list queries); game page HTML (`/game/<slug>`, `/game/<id>`); home and search HTML (embedded window.__CATALOG__ snapshot). Same token/zone as `make cf-purge`, via purgeCFURLs (main.go). Runs in a goroutine and NEVER fails the user's request (worst case: expires by TTL). Bursts are coalesced into one call (CF rate-limits purges).

Game card edits, bumps and soft-hide — the user-facing write path for `games`. games.updateRule is moderator-only, so these endpoints are the ONLY way an uploader changes their card. Every mutation: 1) appends a revision to `game_revisions` (old values of changed fields only); 2) archives displaced storage files under revisions/<gameId>/<revisionId>/<filename> — PB deletes replaced files and R2 has no versioning, so this copy is what makes file edits reversible. Nothing hard-deletes: "delete" = hidden=true, a revert is another revision. Design: PB/schema_changes_edits_bumps.md

Moderator edits of language/version variants (hybrid model, see wiki multilang-variants-spec). `games` holds the original's content; other languages/versions live in `game_variants` (updateRule admin-only in the schema), so this handler writes via app.Save, bypassing API rules like game_edits.go. UI replacement for PB/add_game_variant.py.

Game hosting on <hostingSlug>.cyoa.cafe: zip upload → R2 (games/{hostingSlug}/{gameSlug}/v{N}/…), serving from R2 with companion-shim injection, reserved/claim flow for archived author pages, moderator block/restore/purge. Version switching/reupload bodies live in hosting_versions.go.

Chunked hosting upload, shared by the superuser endpoint (/api/hosting/admin/upload-part) and moderator uploads from the helper app (modkit.go). A game arrives in PARTS (each a valid zip of a file subset) to get under the edge proxy's ~100MB request limit. All parts write into ONE version prefix; the hosted_games record is created/updated only on the final part (final=true) after auditing the whole assembly (index.html present, size/count within limits). Protocol (clients: tools/big_upload_chunked/chunked_upload.py, tools/cyoa-helper): part 0 — server computes the version and returns it (client puts index.html in part 0); parts 1..N-1 — client sends that version back (form "version"); final=true — server audits the prefix and finalizes. Parts are idempotent (same keys overwritten). Limits: part ≤ maxZipSize, game ≤ maxChunkedGameSize.

Hosted-game versions: reupload a new version and roll back to an old one — one implementation for four entry points (owner and moderator). R2 layout: games/{hostingSlug}/{gameSlug}/v{N}/… Old prefixes are NEVER deleted (except mod/purge), so a rollback is just changing hosted_games.version + cache purge, no re-upload. Moderator path differs from owner path in two ways: the hosting slug comes from the game's OWNER (otherwise files would land in the moderator's prefix and the public URL would break); the owner's daily quota isn't checked (quota=false) — a user's exhausted limit must not block catalog repair. Bytes are still charged to the owner.

Moderator action journal (`mod_actions`): who did what to what, before/after — so any moderator action can be reverted by hand. Rights: wiki/components/moderator-access.md. Properties: append-only (no create/update/delete rules; written via privileged `app`); best-effort — a write failure is logged and NEVER fails the action; no-op if the collection is missing (binary runs fine before PB/create_mod_actions.py); written only when the actor really is a moderator (owners using /comments/delete don't land here).

Moderator permissions: which moderator may do what. `users.isModerator` was all-or-nothing, fine while the site author was the only moderator; with an external moderator some handlers (bump roulette, publication queue, granting rights) stay owner-only. Capability registry + one check point. Storage: separate `mod_permissions` collection (one row per moderator), NOT a users field — so prod schema isn't touched; it self-creates at startup like game_views. Server-only writes; a user reads only their own row. No row = FULL access (legacy) — otherwise deploying the binary would silently lock the author out of her own site. /moderator/access shows them as "full access (legacy)". `_superusers` tokens (our automation) always pass.

Mod Tools: moderators download games on their own computer with the helper app (tools/cyoa-helper) and drive it from /moderator/mod-tools. The helper never listens for the site: it pairs once (device code), then long-polls /api/modkit/helper/next for jobs and reports back, so no browser has to reach localhost. Moderators then upload the build under the AUTHOR's reserved hosting account (author.cyoa.cafe/game/, never the moderator's) and submit a card (title/authors/tags/description/cover) into the publication queue, where it waits, flagged, until another moderator checks it (publication_queue*.go). Collections helper_devices/helper_jobs: pb_scripts/2026-09-24_helper_collections.py; without them every endpoint here answers 503. Permission: mod_upload (mod_perms.go).

In-flight limiter for /api/* — turns overload into a few honest 503s instead of an hour-long collapse. Incident 2026-07-26 05:50–06:50 UTC: nothing leaked, the box died on arithmetic. Memory in flight = requests in flight × cost; requests in flight = rps × latency. When SQLite slowed, requests lived 30s instead of 30ms, so ~7 rps parked ~200 handlers; a 500-row games list costs 15–20 MB while marshalling → >1 GB on a 961 MB droplet → swap → slower → more handlers. Capping concurrency bounds memory at limit × cost. Over the cap: wait briefly for a slot, then 503 + Retry-After (frontend shows "site is busy"). Scope only /api/*: /api/realtime EXCLUDED (long-lived SSE would eat every slot); /api/health EXCLUDED (monitoring must answer under load); hosted-game traffic (<slug>.cyoa.cafe) never hits /api/ (R2 streams).

Heap/goroutine pprof on a LOOPBACK-only port. After the 2026-07-26 outage we knew HOW MUCH memory (744 MB RSS + 653 MB swap) but not WHERE; Go keeps no heap history, so pprof must already be listening when the spike happens. Idle cost: one socket + a parked goroutine (block/mutex profiler stays off). Bound to 127.0.0.1 only: pprof leaks internals and can stall the process; CF Tunnel forwards only :8090. Access: ssh -L 6060:127.0.0.1:6060 <droplet>, then go tool pprof -http=: http://127.0.0.1:6060/debug/pprof/heap (cyoa-watch grabs snapshots automatically). PPROF_ADDR moves it, PPROF_ADDR=off disables.

Drip-feed publishing: a server cron moves one approved row from game_pipeline_state (state=approved) into games on a timer, independent of the laptop. Dedup verdict (publish_mode) is baked at processing time; the cron just executes it plus a cheap exact check. Files copied via core.Filesystem (local disk or R2). Spec: Documentation/publication_pipeline_architecture.md, PB/schema_changes_publication_queue.md (§3,§6,§7).

Moderator-only endpoints for /moderator/queue. game_pipeline_state / publication_settings are admin-only, so we use privileged $app and check isModerator ourselves. Spec: Documentation/publication_queue_spec.md §5. The actual queue→games move is the Go cron (publication_queue.go).

Web Push (VAPID) — the reason for leaving Discord possible: without push the site chat only reaches people when they come back. Subscriptions in push_subscriptions (never readable via API; list/viewRule nil; all via these endpoints with app.Save). One record = one device (endpoint). Feature flag: VAPID_PUBLIC_KEY + VAPID_PRIVATE_KEY in env; none → endpoints 404, frontend hides the toggle. Keys made once with `./dist/serve push-keys` into .env — CHANGING KEYS KILLS ALL EXISTING SUBSCRIPTIONS at once (browsers won't reissue).

Site chat (shoutbox) core: posting, anti-spam, presence, notifications. Spec: wiki/components/shoutbox-spec.md (desk repo). shoutbox_messages create/update/delete are backend-only; reads and realtime are stock PB list/subscribe (public listRule). All anti-spam state is in-memory (reset on restart — accepted). Feature flag SHOUTBOX_ENABLED=1; off → all endpoints 404, frontend hides UI. v2 fields (rooms, images, etc.) in shoutbox_v2.go.

Anonymous mask: the signature readers see instead of a name ("Anon Fox"). Before: name derived client-side from anon_key = hash(IP, salt, week window) — untraceable beyond a week, but names changed by themselves. Author asked for a months-long, self-chosen mask (decision 2026-09-16). Now the BROWSER keeps the mask (localStorage, anonMask.ts) and sends it with each message; the server validates it and stores it in anon_mask. What the mask is NOT: an identity — mutes, rate limits and delete rights still hang on the server-computed anon_key; a copied mask gives only a signature, no rights or history; and it doesn't prove two messages share an author. Old messages have an empty field and the client derives the name from anon_key (anonIdentity.ts), so an empty mask is normal and silently accepted.

Personal block list ("don't show me this person, no DMs from them"). Author decision 2026-08-04: stored SERVER-side (own collection), not localStorage — the request was literally "block them from DMing me", so whoever lets messages through (the server) must know. Effects: no DMs between the pair (BOTH directions); no bell/push for their mentions; frontend hides their lines in the feed. Both directions on purpose: a two-person conversation where one can't hear the other is a trap (the blocked person writes into a void). Moderator mutes (shoutState.mutes) are different: global and temporary; this is personal and permanent.

User-created threads ("thread chat"): anyone can open a topic with tag stripes, message count, likes; the topic list lives apart from the author's pinned rooms. Handlers on /api/custom/shoutbox; frontend routes /chat/threads (index) and /chat/t/<slug> (topic) in App.tsx.

Per-ROOM notification overrides. Push subscription is per device ("wake me for everything" / "mentions only"); a room overrides it, in its own direction only: "" no override (no DB row at all) — follow the device "all" wake on any message, even if device is mentions-only "mentions" mentions/replies only, even if device wants all "mute" never wake, INCLUDING mentions (muted-but-pinged-by-name is the annoying case) Setting is per PERSON, not per device. Cost (author's main requirement): sending a message needs the prefs of everyone who set something in THIS room — one `channel`-indexed query cached shoutPrefsTTL; the common "nobody set anything" case caches an empty map, cost zero. Collection is closed to the outside (Go superuser only), like blocks and push subs.

Message reactions: one toggle handler POST /api/custom/shoutbox/react. Denormalized map in the message record: {"fire": ["u123","u456"], "eyes": ["u123"]}. A separate collection would mean ~50 queries per feed page or an aggregating endpoint; here reactions arrive with the text and update through the same realtime subscription. Names are validated against the chat_emoji pack held in memory (chat_emoji.go); the frontend gets the same list from the same handler, cache rebuilt by a record hook.

DMs and private rooms (chat v2). Author decisions (wiki/decisions.md, 2026-08-04): any logged-in user can DM any other; any logged-in user can create a private room (per-user cap); moderators do NOT see private content through the site UI. No E2E encryption: messages are plaintext in PocketBase; the DB owner can read anything — "moderators can't see" is about site buttons, not crypto. Everything goes through Go handlers: shoutbox_channels writeRule is superuser-only and must stay so, or anyone could PATCH themselves into a private room's members. A DM is just a private channel with is_dm and exactly two members — no separate collection on purpose, so feed/send/push/unread/delete work unchanged.

Public chat rooms, moderator view: create, rename, reorder, hide, make private. These handlers must NOT: touch DMs (is_dm); touch someone else's private room ("make public" would become "show everyone their conversation") — shoutAdminRoom allows only public or the caller's own; delete (a room is all its messages — use enabled=false instead).

Staff room and staff badges. Moderators need a room closed to visitors, but not created by hand per new moderator. Also the site author must be distinguishable from moderators (red nick used to mean "any staff"). The room is a normal private room (`is_private = true`) with reserved slug `staff`; no new schema flag — message read access is enforced by the shoutbox_messages listRule on channel `members`, which can't check roles. So membership is materialized: the server writes every `isModerator = true` user into members. New moderators are added on their first chat visit (shoutStaffJoin from /channels, cheap, moderators only); demoted ones removed by sync from the rooms panel; leaving is impossible (would be undone). "Admin" (red) = moderator allowed to grant permissions (permPerms), i.e. the site owner. Legacy moderators without a mod_permissions row count as full (mod_perms.go).

Personal chat state that must follow the user across devices (author decisions 2026-08-13): notes about people (your alias for them + private text), pinned conversations and their order, "read up to" per room. All three in ONE record per user, three JSON fields. Why not fields on `users`: its viewRule = "" (public) — anyone can read the whole record; the fields whitelist is frontend-only, so private notes would leak. Why not a row per (user, room): dozens of rows and UPDATEs per "read"; and not shoutbox_channel_prefs — its cache pulls ALL rows of a room on every send. Cost: ZERO new HTTP requests — state rides as a `state` block in GET /channels, read marks piggyback on POST /ping?chat=1 (sent once a minute anyway). DB write at most once a minute per user and only if marks changed. Collection closed to the outside (Go superuser only). Notes are visible ONLY to their author; no handler ever reveals "how others labeled me".

Chat v2 delta over live v1 (shoutbox.go). Spec: wiki/components/shoutbox-v2-spec.md. Adds: channels (incl. private), anon toggle for logged-in users, guest delete password, image messages, "who's here". Separate file because v1's POST path is touched in exactly one place; everything else registers into /api/custom/shoutbox via registerShoutboxV2().

Sane usernames on OAuth signup + "change once" rule. 1) Discord/OIDC often give a nick failing the username pattern (dots, unicode, too short) or already taken → PB core generates faceless `usersNNNNNN`. The hook sets a meaningful unique username in CreateData (nick → name → email) and fills empty name. 2) Author names are held by reserved stubs the pipeline creates when hosting their games (s06_upload._ensure_user); a real author signing up hit the NOCASE unique index ("already in use"). On signup and rename we now return a message linking the claim page <hosting_slug>.cyoa.cafe. 3) updateRule `id = @request.auth.id` lets users edit their record and PB rules can't restrict a SINGLE field, so "change username once" is a server guard: first change sets username_locked, second → 403. Requires bool field `users.username_locked` (default false).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL