Documentation
¶
Overview ¶
Package webpreview is the browser twin of the phone preview: an overlay-injecting reverse proxy that `clank preview` puts in front of a KindWeb dev server (see internal/host/preview), plus the dictation service the overlay's push-to-talk streams into.
The proxy owns three concerns on one loopback origin so the injected overlay never fights CORS or mixed-origin auth:
- serve the overlay assets (embedded, /__clank/overlay.js)
- relay /__clank/api/* to the daemon's unix socket behind a per-run bearer token (the web analog of preview_frontdoor.go's LAN pairing token)
- rewrite proxied HTML to inject the overlay <script> + config
Dictation runs in this process rather than the daemon on purpose: it lives exactly as long as `clank preview` does, and keeping it out of the daemon means no host API surface until a second client (TUI, mobile-web) wants to share it.
Index ¶
- Constants
- func DefaultModelsDir() (string, error)
- func EnsureModels(ctx context.Context, dir string, progress ModelDownloadProgress) error
- func FindClankVoice() (string, error)
- func InjectOverlayResponse(resp *http.Response, snippet []byte) error
- func ModelsPresent(dir string) bool
- func OverlaySnippet(config map[string]any) ([]byte, error)
- func ServeOverlayAsset(w http.ResponseWriter, r *http.Request) bool
- func ShouldInjectOverlay(userAgent string) bool
- type DictationEngine
- type Engine
- type ExecEngine
- type ModelDownloadProgress
- type Options
- type Result
- type Server
- type Session
- type SherpaEngine
Constants ¶
const ( OverlayPath = "/__clank/overlay.js" ChatPath = "/__clank/chat.js" MarkdownPath = "/__clank/markdown.js" TranscriptPath = "/__clank/transcript.js" SettingsPath = "/__clank/settings.js" SourceControlPath = "/__clank/sourcecontrol.js" BoxPosPath = "/__clank/boxpos.js" LauncherPath = "/__clank/launcher.js" ResizePath = "/__clank/resize.js" WorkletPath = "/__clank/worklet.js" APIPrefix = "/__clank/api" // NativePreviewUserAgentToken keeps the JS overlay out of clank-mobile's // WebView, where the Kotlin prompt box owns the interaction surface. NativePreviewUserAgentToken = "ClankNativePreview/1" )
const EngineEnvVar = "CLANK_VOICE_ASR_CMD"
EngineEnvVar overrides the dictation engine with a shell command that reads a 16 kHz mono s16le WAV on stdin and prints the transcript to stdout, e.g.
CLANK_VOICE_ASR_CMD='whisper-cli -m ~/models/ggml-base.en.bin -nt -f -'
One process per utterance, final-only (no partials). When unset, the sherpa-onnx engine is used if a clank-voice binary is installed (see sherpa.go); otherwise voice is off.
const LauncherSeenPath = "/__clank/launcher/seen"
const ModelSlug = "parakeet-tdt-0.6b-v3-int8"
ModelSlug names the dictation model set, mirroring clank-mobile's ModelManager (MODEL_SLUG there): NVIDIA Parakeet TDT 0.6b v3 int8 for ASR plus Silero VAD for segmentation, both in sherpa-onnx form.
Variables ¶
This section is empty.
Functions ¶
func DefaultModelsDir ¶
DefaultModelsDir is where EnsureModels materializes the model set: <CLANK_DIR>/models/<slug>. Living under config.Dir keeps isolated stacks (CLANK_DIR overrides) isolated here too.
func EnsureModels ¶
func EnsureModels(ctx context.Context, dir string, progress ModelDownloadProgress) error
EnsureModels downloads any missing model files into dir. Idempotent and interrupt-safe: each file streams to <name>.tmp and is renamed into place only when complete, so a Ctrl+C mid-encoder re-downloads just that file next time.
func FindClankVoice ¶
FindClankVoice locates the clank-voice binary: next to the running executable first (paired installs), then PATH — the same resolution clank uses for clankd and the daemon uses for clank-host.
func InjectOverlayResponse ¶
InjectOverlayResponse rewrites a successful identity-encoded HTML response. Non-HTML, compressed, and oversized bodies pass through unchanged.
func ModelsPresent ¶
ModelsPresent reports whether every model file exists non-empty in dir. Partial downloads never count: files are written to a .tmp name and renamed only when complete.
func OverlaySnippet ¶
OverlaySnippet renders the config and module tags injected into preview HTML.
func ServeOverlayAsset ¶
func ServeOverlayAsset(w http.ResponseWriter, r *http.Request) bool
ServeOverlayAsset serves one embedded overlay module and reports whether the request matched a reserved asset path.
func ShouldInjectOverlay ¶
ShouldInjectOverlay reports whether a preview page needs the browser UI.
Types ¶
type DictationEngine ¶
type DictationEngine string
DictationEngine selects how the overlay transcribes push-to-talk audio. The value is a user preference persisted across preview runs (see Options.PersistDictationEngine); empty means "not chosen yet", which makes the overlay ask on first dictation.
const ( // DictationLocal transcribes on this machine via the configured // Engine (clank-voice or an exec command) — audio never leaves it. DictationLocal DictationEngine = "local" // DictationWebSpeech transcribes in the browser via the Web Speech // API (SpeechRecognition), which typically uploads audio to the // browser vendor's speech service (Google in Chrome, Apple in // Safari). No audio touches the /__clank/voice socket. DictationWebSpeech DictationEngine = "webspeech" )
func ParseDictationEngine ¶
func ParseDictationEngine(s string) (DictationEngine, bool)
ParseDictationEngine validates a stored or client-sent engine string. Empty is NOT valid here — "unchosen" is a caller-level state, not an engine.
type Engine ¶
type Engine interface {
// Open starts a session. The caller owns it and must Close it.
Open(ctx context.Context) (Session, error)
// Describe returns a short human-readable engine label for the
// preview banner and logs.
Describe() string
}
Engine produces dictation sessions. Implementations may serialize sessions (the sherpa engine funnels everything through one model process); Open blocks until the engine is free or ctx is done.
func EngineFromEnv ¶
func EngineFromEnv() Engine
EngineFromEnv returns the exec-command engine configured via EngineEnvVar, or nil when unset.
type ExecEngine ¶
type ExecEngine struct {
Cmdline string
}
ExecEngine shells out to a user-configured command per utterance. WAV on stdin (not raw PCM) so off-the-shelf tools — whisper.cpp's whisper-cli, sox pipelines, a custom script — work without flags describing the sample format. Final-only: partials would mean re-decoding the whole utterance per chunk at O(n²) cost.
func (*ExecEngine) Describe ¶
func (e *ExecEngine) Describe() string
type ModelDownloadProgress ¶
ModelDownloadProgress reports EnsureModels progress. done/total are bytes for the current file; total is -1 when the server sent no Content-Length.
type Options ¶
type Options struct {
// UpstreamURL is the HTTP(S) origin the proxy fronts. Required.
UpstreamURL *url.URL
// DaemonSocketPath is the clank daemon's unix socket; /__clank/api/*
// relays there. Required.
DaemonSocketPath string
// Token gates /__clank/api/* and /__clank/voice. The injected page
// config carries it; nothing else on the machine learns it. This is
// the same trust move as the LAN front door's pairing token, scoped
// down to loopback.
Token string
// OverlayConfig is serialized into window.__CLANK_PREVIEW for the
// overlay (session context: local_path, backend, hostname, name,
// optional session_id). Token and voice availability are added here.
OverlayConfig map[string]any
// Engine powers local dictation; nil marks the local engine
// unavailable in the overlay config and 503s the voice endpoint.
// The overlay may still offer the browser's Web Speech API, which
// it detects client-side.
Engine Engine
// DictationEngine is the persisted engine choice injected into the
// overlay config. Empty means unchosen (the overlay asks on first
// dictation); anything else must parse via ParseDictationEngine.
DictationEngine DictationEngine
// PersistDictationEngine stores a choice made in the overlay's
// engine picker so it survives preview restarts. nil means choices
// only last for this run.
PersistDictationEngine func(DictationEngine) error
// LauncherSeen suppresses the first-use coachmark. PersistLauncherSeen
// stores the acknowledgement across auto-assigned preview ports.
LauncherSeen bool
PersistLauncherSeen func() error
// ListenPort for the proxy on 127.0.0.1; 0 picks a free port.
ListenPort int
Log *log.Logger
}
Options configures Start.
type Result ¶
type Result struct {
Text string
Final bool
// Err reports a failed decode (Final implied). The session may
// still accept the next utterance unless Results was closed.
Err error
}
Result is one transcription update for the current utterance. Partials are cumulative (the text so far, not a delta), matching how clank-mobile's recognizer commits VAD segments monotonically.
type Server ¶
type Server struct {
// URL is the browser-facing address, http://127.0.0.1:<port>.
URL string
// contains filtered or unexported fields
}
Server is a running overlay proxy.
type Session ¶
type Session interface {
// Feed accepts PCM as it arrives from the mic.
Feed(pcm []byte) error
// End marks push-to-talk release: flush and decode; exactly one
// Final Result follows on Results (empty text = heard nothing).
End() error
// Cancel discards audio buffered since the last End.
Cancel() error
// Results streams partial and final updates. Closed when the
// session is Closed or the engine dies.
Results() <-chan Result
// Close releases the session and, for serializing engines, hands
// the engine to the next Open.
Close() error
}
Session is one dictation conversation, typically bound to one overlay WebSocket. PCM is s16le, 16 kHz, mono — the same shape clank-mobile feeds sherpa-onnx.
type SherpaEngine ¶
type SherpaEngine struct {
// Bin is the clank-voice executable path.
Bin string
// Args are passed verbatim (production: --models <dir>). Split out
// so tests can substitute a fake binary with its own flags.
Args []string
// ReadyTimeout caps the model-load wait after spawn. Zero means
// defaultReadyTimeout.
ReadyTimeout time.Duration
Log *log.Logger
// contains filtered or unexported fields
}
SherpaEngine drives a clank-voice subprocess (voice-engine module): sherpa-onnx with Silero VAD + the same Parakeet model clank-mobile runs on-device. The subprocess is spawned once and kept warm — the model load is the expensive part (~670 MB of weights), so paying it per utterance or per WebSocket would wreck push-to-talk latency.
The cgo/onnxruntime dependency lives entirely in the voice-engine module; this driver is pure Go, so clank's CGO_ENABLED=0 cross-builds (e.g. the fly.io clank-host artifact) are unaffected.
Sessions are serialized: one model process, one utterance stream at a time. Open blocks until the previous session Closes.
func NewSherpaEngine ¶
func NewSherpaEngine(bin, modelsDir string, lg *log.Logger) *SherpaEngine
NewSherpaEngine returns an engine driving bin with the standard production arguments.
func (*SherpaEngine) Close ¶
func (e *SherpaEngine) Close() error
Close tears down the warm model process. Implements io.Closer so the CLI can release ~1 GB of RSS on preview shutdown.
func (*SherpaEngine) Describe ¶
func (e *SherpaEngine) Describe() string
func (*SherpaEngine) Prewarm ¶
func (e *SherpaEngine) Prewarm()
Prewarm starts loading the model in the background so the first push-to-talk doesn't pay the multi-second load. A session opened mid-load simply waits on the spawn lock and proceeds when ready.