ingestkit

IngestKit

High-Performance Event Ingestion System

Handle 30k+ events/sec on a $14 server with zero data loss. Schema-first design with auto-generated type-safe SDKs.

Quick Start

Install

pip install ingestkit

Or: npm install -g ingestkit

Run Server

ingestkit init --server
ingestkit server start

That's it! Server running at http://localhost:8080

Send Events

curl -X POST http://localhost:8080/v1/events/user_signup \
  -H "Authorization: Bearer dev_key_1234567890" \
  -H "Content-Type: application/json" \
  -d '{"user_id": "123", "email": "user@example.com"}'

Features

• 30,000+ RPS on a single $14/month server
• Zero Data Loss with Kafka-compatible buffering (Redpanda)
• Schema-First - Define events in YAML, get type-safe SDKs
• Multi-Tenant - Built-in API key management and data isolation
• Observability - Prometheus metrics and structured logging

Define Your Events

Edit schema/events.yaml:

events:
  user_signup:
    fields:
      user_id:
        type: string
        required: true
      email:
        type: string
        required: true
      plan:
        type: string
        values: [free, pro, enterprise]

Apply changes:

ingestkit schema apply

Type-Safe SDKs

Generate clients for your app:

# Python
ingestkit init --python
ingestkit generate --schema-url http://localhost:8080/schema

Use the generated client:

from ingestkit import Client

client = Client()
client.user_signup.send(user_id="123", email="user@example.com")

// TypeScript
import { Client } from './ingestkit';

const client = new Client();
await client.userSignup.send({ userId: "123", email: "user@example.com" });

Server Commands

Architecture

Client App → IngestKit API → Redpanda (Kafka) → Consumer → PostgreSQL

• API: Validates events, publishes to Kafka
• Consumer: Batches events, writes to PostgreSQL with COPY protocol
• PostgreSQL: Partitioned tables per tenant

Configuration

Environment variables (set in .env):

Benchmarks

Tested on Hetzner CAX31 ($14/mo: 8 vCPU, 16GB RAM):

Documentation

• Project Overview
• SDK Generation
• Docker Deployment
• Development Guide

Development

For contributors building from source:

git clone https://github.com/feat7/ingestkit.git
cd ingestkit
make setup && make start

See Development Guide for details.

License

Apache 2.0 - See LICENSE