vimmary

vimmary

YouTube video summary service. Fetches transcripts via YouTube's InnerTube API, generates LLM summaries, and stores everything in Postgres + pgvector for semantic search. Videos can be added manually via the web UI or automatically through Karakeep webhooks.

How it works

Karakeep ──webhook──▶ vimmary ──▶ fetch transcript ──▶ generate summary
Web UI ──manual URL──▶    │                                   │
                          │                     ┌─────────────┼──────────────┐
                          │                     ▼             ▼              ▼
                          │                 pgvector      Karakeep        Web UI
                          │                 + search      writeback       display
                          │                                                 │
                          ◀──── MCP tools ──────────────────────────────────┘

1. A YouTube video is bookmarked in Karakeep (webhook) or submitted manually via the web UI
2. vimmary fetches the transcript via YouTube's InnerTube API
3. An LLM (Claude or Mistral) generates a structured summary
4. The summary is stored with embeddings for semantic search
5. Results are written back to Karakeep (if applicable) and displayed in the web UI

Features

• Manual URL submission — paste any YouTube URL in the web UI to process it immediately
• Automatic summaries — triggered by Karakeep webhooks, no manual action needed
• Bulk import — import all existing YouTube bookmarks from Karakeep via Settings page
• Two detail levels — medium (automatic) and deep (on-demand via MCP or web UI)
• Hybrid search — keyword + semantic search with Reciprocal Rank Fusion
• Adaptive rate limiting — YouTube API delays scale with queue depth (10s–45s) to avoid 429s during bulk operations
• Auto-retry — transcript fetch failures are automatically retried with exponential backoff (2m/5m/10m, max 3 retries)
• Retry all failed — batch-retry all failed videos from the web UI
• MCP server — 6 tools for searching, browsing, and managing video summaries
• Web UI — React frontend embedded in the Go binary (Videos, Stats, Settings pages)
• Tailscale auth — zero-config authentication via tsnet
• Multi-user support — per-user video libraries (same YouTube video can be bookmarked by multiple users independently)
• Per-user Karakeep integration — each user configures their own API key and webhook token via the Settings page
• Bidirectional sync — summaries written back to Karakeep notes; bookmark deletions in Karakeep remove videos from vimmary
• Karakeep writeback — plain-text summary with vimmary detail link, video-summarized tag added (preserves existing Karakeep AI tags)

Architecture

Installation

Docker Compose (recommended)

1. Clone the repository and create your config:

   git clone https://github.com/meltforce/vimmary.git
   cd vimmary
   cp config.example.yaml config.yaml
   

2. Edit config.yaml with your settings:

   • secrets.mistral_api_key — required for embeddings (get one at console.mistral.ai)
   • secrets.claude_api_key or secrets.mistral_api_key — at least one is needed for summaries (set summary.provider to "claude" or "mistral")
   • external_url — the URL where vimmary is reachable (used for links in Karakeep writebacks)
   • youtube.sub_langs — preferred transcript languages (default: en, de)
   • tailscale.enabled — set to true to enable Tailscale authentication (recommended); when false, the app runs without auth in dev mode

3. Start the stack:

   docker compose up -d
   

   This starts both the app (port 8080) and a PostgreSQL database with pgvector. Migrations run automatically on startup.

4. Open http://localhost:8080 (or your Tailscale hostname) and start adding videos.

Local development

# Start only the database
docker compose up db

# Run the backend (requires Go 1.23+)
go run ./cmd/vimmary --config config.yaml

# Run the frontend with hot-reload (separate terminal)
cd web && npm install && npm run dev

Setup Karakeep integration

1. Open vimmary's Settings page (Tailscale auth required)
2. Enter your Karakeep API key (from Karakeep Settings → API Keys)
3. Copy the generated Webhook URL and Bearer Token
4. In Karakeep Settings → Webhooks, create webhooks for created and deleted events
5. If Karakeep runs in Docker and vimmary is on Tailscale, add CRAWLER_ALLOWED_INTERNAL_HOSTNAMES=.your-tailnet.ts.net to Karakeep's env to allow webhook delivery

Build

# Build binary
CGO_ENABLED=0 go build -o vimmary ./cmd/vimmary

# Build Docker image
docker buildx build --platform linux/amd64 -t meltforce/vimmary:edge .

MCP tools

Related projects

• meltkit — shared Go library (db, config, secrets, middleware, MCP)
• totalrecall — personal knowledge system (architectural blueprint)