README
¶
Hapax
Large Language Model Infrastructure, Simplified
Building with Large Language Models is complex. Multiple providers, varying APIs, inconsistent performance, unpredictable costs—these challenges consume more engineering time than the actual innovation.
Hapax offers a different approach.
What if managing LLM infrastructure was as simple as editing a configuration file? What if switching providers, adding endpoints, or implementing fallback strategies could be done with minimal effort?
Imagine a system that:
- Connects to multiple LLM providers seamlessly
- Provides automatic failover between providers
- Offers comprehensive monitoring and metrics
- Allows instant configuration updates without downtime
This is Hapax.
Real-World Flexibility in Action
Imagine you're running a production service using OpenAI's GPT model. Suddenly, you want to:
- Add a new Anthropic Claude model endpoint
- Create a fallback strategy
- Implement detailed monitoring
With Hapax, this becomes simple:
# Simply append to your existing configuration
providers:
anthropic:
type: anthropic
models:
claude-3.5-haiku:
api_key: ${ANTHROPIC_API_KEY}
endpoint: /v1/anthropic/haiku
No downtime. No complex redeployment. Just configuration.
Intelligent Provider Management
Hapax goes beyond simple API routing. It creates a resilient ecosystem for your LLM interactions:
Automatic Failover: When one provider experiences issues, Hapax seamlessly switches to backup providers. Your service continues operating without interruption.
Deduplication: Prevent duplicate requests and unnecessary API calls. Hapax intelligently manages request caching and prevents redundant processing.
Provider Health Monitoring: Continuously track provider performance. Automatically reconnect to primary providers once they're back online, ensuring optimal resource utilization.
Comprehensive Observability
Hapax isn't just a gateway—it's a complete monitoring and alerting system for your LLM infrastructure:
- Detailed Prometheus metrics
- Real-time performance tracking
- Comprehensive error reporting
- Intelligent alerting mechanisms
API Versioning for Scalability
Create multiple API versions effortlessly. Each endpoint can have its own configuration, allowing granular control and smooth evolutionary paths for your services.
routes:
- path: /v1/completions
handler: completion
version: v1
- path: /v2/completions
handler: advanced_completion
version: v2
Getting Started
# Pull Hapax
docker pull ghcr.io/teilomillet/hapax:latest
# Generate default configuration
docker run --rm -v $(pwd):/output \
ghcr.io/teilomillet/hapax:latest \
cp /app/config.example.yaml /output/config.yaml
# Launch Hapax
docker run -p 8080:8080 \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/teilomillet/hapax:latest
What's Next
Hapax is continuously evolving.
Open Source
Licensed under Apache 2.0, Hapax is open for collaboration and customization.
Community & Support
- Discussions: GitHub Discussions
- Documentation: Hapax Wiki
- Issues: GitHub Issues
Our Vision
We believe LLM infrastructure should be simple, reliable, and adaptable. Hapax represents our commitment to making LLM integration accessible and powerful.
Documentation
¶
There is no documentation for this package.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
hapax
command
|
|
|
Package config provides configuration management for the Hapax LLM server.
|
Package config provides configuration management for the Hapax LLM server. |
|
Package errors provides a comprehensive error handling system for the Hapax LLM gateway.
|
Package errors provides a comprehensive error handling system for the Hapax LLM gateway. |
|
Package server implements the HTTP server and routing logic for the Hapax LLM service.
|
Package server implements the HTTP server and routing logic for the Hapax LLM service. |
|
handlers
Package handlers provides HTTP handlers for the Hapax server.
|
Package handlers provides HTTP handlers for the Hapax server. |
|
middleware
Package middleware provides various middleware functions for HTTP handlers.
|
Package middleware provides various middleware functions for HTTP handlers. |
|
processing
Package processing provides request processing and response formatting for LLM interactions.
|
Package processing provides request processing and response formatting for LLM interactions. |
|
routing
Package routing provides dynamic HTTP routing with versioning and health checks.
|
Package routing provides dynamic HTTP routing with versioning and health checks. |