README
ยถ
RediORM
A modern, schema-driven ORM for Go with a JavaScript runtime interface. RediORM provides a clean separation between Go's type-safe database operations and a Prisma-like JavaScript API for dynamic scripting.
๐ Features
- Dual API Design - Native Go API for performance, JavaScript API for flexibility
- Schema-Driven - Define your data model once, use it everywhere
- Multi-Database Support - SQLite, MySQL, PostgreSQL, and MongoDB
- Smart Field Mapping - Automatic conversion between naming conventions
- Migration System - Track and manage database schema changes (SQL databases)
- Relations Support - One-to-one, one-to-many, many-to-one, and many-to-many relations
- Raw Query Support - Execute SQL or native database commands (including MongoDB)
- SQL Parser for MongoDB - Write familiar SQL queries that are automatically translated to MongoDB operations
- Transaction Support - Full ACID compliance with savepoints (SQL databases), basic transaction support for MongoDB
- Type-Safe Queries - Compile-time safety in Go, runtime validation in JS
๐ฆ Installation
Go Library
go get github.com/rediwo/redi-orm
CLI Tool
Pre-built Binaries (Recommended)
Download the latest release for your platform:
Linux (AMD64):
wget https://github.com/rediwo/redi-orm/releases/latest/download/redi-orm-linux-amd64.tar.gz
tar -xzf redi-orm-linux-amd64.tar.gz
sudo mv redi-orm-* /usr/local/bin/redi-orm
macOS (Intel):
wget https://github.com/rediwo/redi-orm/releases/latest/download/redi-orm-darwin-amd64.tar.gz
tar -xzf redi-orm-darwin-amd64.tar.gz
sudo mv redi-orm-* /usr/local/bin/redi-orm
macOS (Apple Silicon):
wget https://github.com/rediwo/redi-orm/releases/latest/download/redi-orm-darwin-arm64.tar.gz
tar -xzf redi-orm-darwin-arm64.tar.gz
sudo mv redi-orm-* /usr/local/bin/redi-orm
Windows (AMD64):
- Download
redi-orm-windows-amd64.zipfrom releases - Extract and add to your PATH
Build from Source
# Install latest version
go install github.com/rediwo/redi-orm/cmd/redi-orm@latest
# Or build from source
git clone https://github.com/rediwo/redi-orm.git
cd redi-orm
make release-build
Verify Installation
redi-orm version
๐ฏ Quick Start
Go ORM API
The Go ORM API provides a simplified, high-level interface for rapid development using JSON-based queries:
package main
import (
"context"
"fmt"
"log"
"github.com/rediwo/redi-orm/database"
"github.com/rediwo/redi-orm/orm"
)
func main() {
// Create database connection
db, err := database.NewFromURI("sqlite://./myapp.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Connect to database
ctx := context.Background()
if err := db.Connect(ctx); err != nil {
log.Fatal(err)
}
// Create ORM client
client := orm.NewClient(db)
// Load schema from Prisma-style definition
schemaContent := `
model User {
id Int @id @default(autoincrement())
email String @unique
name String
posts Post[]
}
model Post {
id Int @id @default(autoincrement())
title String
content String
userId Int
user User @relation(fields: [userId], references: [id])
}
`
if err := db.LoadSchema(ctx, schemaContent); err != nil {
log.Fatal(err)
}
if err := db.SyncSchemas(ctx); err != nil {
log.Fatal(err)
}
// Get model references for cleaner code
User := client.Model("User")
Post := client.Model("Post")
// Create user (returns map[string]any)
user, err := User.Create(`{
"data": {
"email": "alice@example.com",
"name": "Alice"
}
}`)
if err != nil {
log.Fatal(err)
}
// Create post with relation
post, err := Post.Create(fmt.Sprintf(`{
"data": {
"title": "Hello World",
"content": "My first post",
"userId": %v
}
}`, user["id"]))
if err != nil {
log.Fatal(err)
}
// Find with conditions (returns []map[string]any)
users, err := User.FindMany(`{
"where": {
"email": { "contains": "@example.com" }
},
"orderBy": { "name": "asc" },
"include": { "posts": true }
}`)
if err != nil {
log.Fatal(err)
}
// Update data
updatedUser, err := User.Update(fmt.Sprintf(`{
"where": { "id": %v },
"data": { "name": "Alice Smith" }
}`, user["id"]))
if err != nil {
log.Fatal(err)
}
// Count records
count, err := Post.Count(fmt.Sprintf(`{
"where": { "userId": %v }
}`, user["id"]))
if err != nil {
log.Fatal(err)
}
// Delete records
deleted, err := Post.Delete(fmt.Sprintf(`{
"where": { "id": %v }
}`, post["id"]))
if err != nil {
log.Fatal(err)
}
fmt.Printf("Found %d users\n", len(users))
fmt.Printf("Updated user: %v\n", updatedUser)
fmt.Printf("Post count: %v\n", count)
fmt.Printf("Deleted: %v\n", deleted)
}
๐ก Note: The Go ORM API uses JSON strings for queries, similar to the JavaScript API. This provides a consistent experience across both languages. For low-level database operations and advanced features, see the Driver API documentation.
JavaScript API
The JavaScript API provides a Prisma-like interface for dynamic operations:
// Import the ORM module
const { fromUri } = require('redi/orm');
async function main() {
// Create database connection
const db = fromUri('sqlite://./myapp.db');
await db.connect();
// Load schema from Prisma-style definition
await db.loadSchema(`
model User {
id Int @id @default(autoincrement())
email String @unique
name String
posts Post[]
}
model Post {
id Int @id @default(autoincrement())
title String
content String
userId Int
user User @relation(fields: [userId], references: [id])
}
`);
// Or load from file
// await db.loadSchemaFrom('./schema.prisma');
// Sync schemas with database
await db.syncSchemas();
// Get model references for cleaner code
const User = db.models.User;
const Post = db.models.Post;
// Create user with nested post
const user = await User.create({
data: {
email: 'bob@example.com',
name: 'Bob',
posts: {
create: [
{ title: 'Hello World', content: 'My first post' }
]
}
}
});
// Find with relations
const users = await User.findMany({
where: {
email: { contains: '@example.com' }
},
include: {
posts: true
},
orderBy: {
name: 'asc'
}
});
// Update
await User.update({
where: { id: user.id },
data: { name: 'Bob Smith' }
});
// Raw SQL queries (works with all databases including MongoDB)
const results = await db.queryRaw(
'SELECT u.*, COUNT(p.id) as post_count FROM users u ' +
'LEFT JOIN posts p ON u.id = p.user_id ' +
'GROUP BY u.id'
);
// MongoDB: You can also use native MongoDB commands
if (db.getDriverType() === 'mongodb') {
// Native MongoDB find command
const users = await db.queryRaw(`{
"find": "users",
"filter": {"age": {"$gt": 18}},
"sort": {"name": 1}
}`);
// MongoDB aggregation pipeline
const stats = await db.queryRaw(`{
"aggregate": "users",
"pipeline": [
{"$match": {"active": true}},
{"$group": {
"_id": "$role",
"count": {"$sum": 1},
"avgAge": {"$avg": "$age"}
}}
]
}`);
}
// Execute raw SQL
const { rowsAffected } = await db.executeRaw(
'UPDATE users SET updated_at = ? WHERE id = ?',
new Date(), user.id
);
await db.close();
}
main().catch(console.error);
๐จ CLI Tool
RediORM includes a powerful CLI tool for database migrations and running JavaScript files with ORM support.
Running JavaScript Files
Execute JavaScript files with full ORM support:
# Basic usage
redi-orm run script.js
# With timeout for long-running scripts
redi-orm run --timeout=30000 batch-process.js # 30 seconds
redi-orm run --timeout 60000 data-migration.js # Alternative syntax
# Pass arguments to the script
redi-orm run process.js --input data.json --output results.json
The --timeout flag is useful for:
- Batch processing operations
- Data migration scripts
- Long-running synchronization tasks
- Scripts with multiple async operations
Migration Commands
Development Mode (Auto-migration)
# Auto-migrate based on schema file
redi-orm migrate --db=sqlite://./myapp.db --schema=./schema.prisma
# Preview changes without applying (dry run)
redi-orm migrate:dry-run --db=sqlite://./myapp.db --schema=./schema.prisma
# Force destructive changes (drops columns/tables)
redi-orm migrate --db=sqlite://./myapp.db --schema=./schema.prisma --force
Production Mode (File-based migrations)
# Generate a new migration file
redi-orm migrate:generate --db=sqlite://./myapp.db --schema=./schema.prisma --name="add_user_table"
# Apply pending migrations
redi-orm migrate:apply --db=sqlite://./myapp.db --migrations=./migrations
# Rollback last migration
redi-orm migrate:rollback --db=sqlite://./myapp.db --migrations=./migrations
# Check migration status
redi-orm migrate:status --db=sqlite://./myapp.db
# Reset all migrations (dangerous!)
redi-orm migrate:reset --db=sqlite://./myapp.db --force
CLI Examples
# Run a data processing script with 5 minute timeout
redi-orm run --timeout=300000 scripts/process-large-dataset.js
# Migrate development database
redi-orm migrate --db=sqlite://./dev.db --schema=./schema.prisma --mode=auto
# Generate production migration
redi-orm migrate:generate --db=postgresql://user:pass@localhost/prod --schema=./schema.prisma --name="add_indexes"
# Apply migrations in production
redi-orm migrate:apply --db=postgresql://user:pass@localhost/prod --migrations=./migrations
GraphQL Server
RediORM includes an automatic GraphQL API generator that creates a fully-featured GraphQL server from your Prisma schemas:
# Start GraphQL server with default settings
redi-orm server --db=sqlite://./myapp.db --schema=./schema.prisma
# Custom port and options
redi-orm server --db=postgresql://user:pass@localhost/db --port=8080 --playground=true --cors=true
# Configurable logging for debugging and monitoring
redi-orm server --db=sqlite://./myapp.db --schema=./schema.prisma --log-level=debug
Features:
- โ Automatic schema generation from Prisma models
- โ Full CRUD operations (create, read, update, delete)
- โ Batch operations (createMany, updateMany, deleteMany)
- โ Complex where conditions and filtering
- โ Sorting and pagination
- โ
Relation support - Query across model relationships with
includesyntax - โ GraphQL Playground for testing
- โ CORS support for web applications
- โ Type-safe resolvers with automatic field mapping
- โ Count queries and aggregations
- โ HTTP handler compatible with any Go web framework
- โ Configurable logging - Monitor requests, performance, and errors
GraphQL Schema Generation
The GraphQL schema is automatically generated from your Prisma models:
- Each model becomes a GraphQL type
- CRUD operations are generated for each model
- Relations are automatically resolved
- Field types are mapped appropriately
- Non-nullable fields are enforced
Available Operations
For each model (e.g., User), the following operations are generated:
Queries:
findUniqueUser(where: UserWhereInput!): User- Find a single userfindManyUser(where: UserWhereInput, orderBy: UserOrderByInput, limit: Int, offset: Int): [User!]!- Find multiple userscountUser(where: UserWhereInput): Int!- Count users
Mutations:
createUser(data: UserCreateInput!): User!- Create a userupdateUser(where: UserWhereInput!, data: UserUpdateInput!): User!- Update a userdeleteUser(where: UserWhereInput!): User!- Delete a usercreateManyUser(data: [UserCreateInput!]!): BatchPayload!- Create multiple usersupdateManyUser(where: UserWhereInput, data: UserUpdateInput!): BatchPayload!- Update multiple usersdeleteManyUser(where: UserWhereInput): BatchPayload!- Delete multiple users
Where Conditions
The GraphQL API supports rich filtering with operators:
where: {
# Exact match
email: { equals: "alice@example.com" }
# String operators
name: { contains: "John" }
name: { startsWith: "J" }
name: { endsWith: "son" }
# Comparison operators
age: { gt: 18 }
age: { gte: 21 }
age: { lt: 65 }
age: { lte: 100 }
# List operators
role: { in: ["admin", "editor"] }
role: { notIn: ["guest"] }
# Logical operators
AND: [
{ published: { equals: true } }
{ createdAt: { gt: "2024-01-01" } }
]
OR: [
{ title: { contains: "GraphQL" } }
{ title: { contains: "API" } }
]
}
Example Queries
# Find users with posts (relation support)
query {
findManyUser(
where: { email: { contains: "@example.com" } }
orderBy: { createdAt: DESC }
limit: 10
) {
id
name
email
posts {
id
title
published
author {
name
email
}
}
_count {
posts
comments
}
}
}
# Create user
mutation {
createUser(data: {
name: "Alice"
email: "alice@example.com"
}) {
id
name
email
}
}
# Complex filtering with relations
query {
findManyPost(
where: {
AND: [
{ published: { equals: true } }
{ author: { role: { equals: "admin" } } }
{ createdAt: { gte: "2024-01-01" } }
]
}
orderBy: { views: DESC }
limit: 20
) {
title
author {
name
email
}
_count {
comments
}
}
}
# Batch operations
mutation {
createManyTag(data: [
{ name: "GraphQL" }
{ name: "API" }
{ name: "Tutorial" }
]) {
count
}
}
Logging Configuration
The GraphQL server includes comprehensive logging for monitoring and debugging:
# Production (minimal logging)
redi-orm server --db=sqlite://./app.db --schema=./schema.prisma --log-level=info
# Development (full debugging)
redi-orm server --db=sqlite://./app.db --schema=./schema.prisma --log-level=debug
# Silent (no logging)
redi-orm server --db=sqlite://./app.db --schema=./schema.prisma --log-level=none
Log Levels:
debug- Shows full requests, queries, variables, and responsesinfo- Shows operation types, execution times, and success/failurewarn- Shows warnings and errorserror- Shows only errorsnone- Disables all logging
Sample Output (INFO level):
๐ GraphQL server ready at http://localhost:4000/graphql
๐ฎ GraphQL Playground available at http://localhost:4000/graphql
[GraphQL] 14:31:14 INFO: mutation createUser
[GraphQL] 14:31:14 INFO: Success in 774ยตs
[GraphQL] 14:31:14 INFO: query findManyUser
[GraphQL] 14:31:14 INFO: Success in 237ยตs
Programmatic Usage
You can also use the GraphQL handler programmatically in your Go applications:
import (
"github.com/rediwo/redi-orm/graphql"
"github.com/rediwo/redi-orm/database"
)
// Create GraphQL handler
db, _ := database.NewFromURI("sqlite://./app.db")
schemas := loadYourSchemas()
generator := graphql.NewSchemaGenerator(db, schemas)
schema, _ := generator.Generate()
handler := graphql.NewHandler(schema)
// Use with any HTTP framework
http.Handle("/graphql", handler)
See the examples directory for complete working examples.
๐๏ธ Architecture
Directory Structure
redi-orm/
โโโ database/ # Database abstraction layer
โโโ drivers/ # Database driver implementations
โ โโโ base/ # Shared driver functionality
โ โโโ sqlite/ # SQLite driver
โ โโโ mysql/ # MySQL driver
โ โโโ postgresql/ # PostgreSQL driver
โ โโโ mongodb/ # MongoDB driver
โโโ graphql/ # GraphQL API generator with relation support and logging
โ โโโ handler.go # HTTP handler with configurable logging (framework-agnostic)
โ โโโ schema_generator.go # GraphQL schema generation with relation fields
โ โโโ resolver.go # Auto-generated resolvers with field mapping
โ โโโ server.go # GraphQL server implementation
โโโ schema/ # Schema definition and parsing
โโโ migration/ # Migration system
โโโ modules/ # Feature modules
โ โโโ orm/ # JavaScript ORM interface
โ โโโ tests/ # JavaScript test suite
โโโ utils/ # Common utilities
โโโ types/ # Shared interfaces
โโโ registry/ # Driver registration
Key Components
- Database Layer - Provides connection management and query execution
- Schema System - Handles model definitions and field mappings
- Query Builders - Type-safe query construction for all operations
- Migration Engine - Tracks and applies database schema changes
- JavaScript Runtime - Goja-based JS engine for dynamic operations
- GraphQL Server - Automatic API generation from schemas with full CRUD support, relation fields, and configurable logging
๐ Database Support
| Feature | SQLite | MySQL | PostgreSQL | MongoDB |
|---|---|---|---|---|
| Basic CRUD | โ | โ | โ | โ |
| Transactions | โ | โ | โ | โ * |
| Migrations | โ | โ | โ | โ |
| Raw Queries | โ | โ | โ | โ |
| Field Mapping | โ | โ | โ | โ |
| Relations | โ | โ | โ | โ |
| Savepoints | โ | โ | โ | โ |
| Nested Documents | โ | โ | โ | โ |
| Array Fields | ๐ง | ๐ง | โ | โ |
| Aggregation Pipeline | โ | โ | โ | โ |
| GroupBy/Having | โ | โ | โ | โ |
๐ง = Partial support, โ = Not supported, โ * = Supported with limitations
๐ง Advanced Features
Schema Loading
Both Go and JavaScript APIs support the same schema loading methods:
Go:
// From string
err := db.LoadSchema(ctx, `
model User {
id Int @id @default(autoincrement())
email String @unique @map("email_address")
name String
}
`)
// From file
err := db.LoadSchemaFrom(ctx, "./schema.prisma")
// Multiple schemas
db.LoadSchema(ctx, userSchema)
db.LoadSchema(ctx, postSchema)
db.SyncSchemas(ctx) // Apply all at once
JavaScript:
// From string
await db.loadSchema(`
model User {
id Int @id @default(autoincrement())
email String @unique @map("email_address")
name String
}
`);
// From file
await db.loadSchemaFrom('./prisma/schema.prisma');
// Multiple schemas
await db.loadSchema(userSchema);
await db.loadSchema(postSchema);
await db.syncSchemas(); // Apply all at once
Field Mapping
RediORM automatically handles field name conversions:
- Schema:
firstNameโ Database:first_name - Database:
created_atโ Schema:createdAt - Custom mapping:
@map("custom_name")
Relations
Define relations in your schema:
model User {
id Int @id @default(autoincrement())
email String @unique
posts Post[] // One-to-many relation
}
model Post {
id Int @id @default(autoincrement())
title String
userId Int
user User @relation(fields: [userId], references: [id])
comments Comment[]
}
model Comment {
id Int @id @default(autoincrement())
text String
postId Int
post Post @relation(fields: [postId], references: [id])
}
Query with relations:
// Find users with their posts
const users = await db.models.User.findMany({
include: {
posts: true
}
});
// Find posts with user and comments
const posts = await db.models.Post.findMany({
include: {
user: true,
comments: true
}
});
// Filter by relation fields
const userPosts = await db.models.Post.findMany({
where: {
userId: 1
}
});
// Count related records
const postCount = await db.models.Post.count({
where: {
userId: 1
}
});
Smart Scanning
// Scan into structs
var users []User
err := db.Model("User").FindMany(ctx, &users)
// Scan into maps (automatic)
var results []map[string]any
err := db.Model("User").FindMany(ctx, &results)
// Raw query scanning
rows, _ := db.Query("SELECT * FROM users")
results, err := utils.ScanRowsToMaps(rows)
Transactions
// JavaScript transaction (coming soon)
await db.$transaction(async (tx) => {
const user = await tx.user.create({ data: { name: 'Alice' } });
const post = await tx.post.create({
data: { title: 'Hello', userId: user.id }
});
return { user, post };
});
// Go transaction
err := db.Transaction(ctx, func(tx types.Transaction) error {
// All operations in transaction
_, err := tx.Model("User").Insert(userData).Exec(ctx)
if err != nil {
return err // Automatic rollback
}
_, err = tx.Model("Post").Insert(postData).Exec(ctx)
return err
})
๐งช Testing
The project includes comprehensive test coverage with a unified conformance test suite ensuring consistent behavior across all database drivers.
# Run all tests
make test
# Run specific test suites
make test-orm # JavaScript ORM tests
make test-sqlite # SQLite driver tests
make test-mysql # MySQL driver tests
make test-postgresql # PostgreSQL driver tests
# Additional options
make test-verbose # Verbose output
make test-cover # Coverage report
make test-race # Race detection
make test-short # Skip long-running tests
Type Conversion Utilities
RediORM provides safe type conversion utilities to handle driver-specific differences:
import "github.com/rediwo/redi-orm/utils"
// Handle different driver representations
boolValue := utils.ToBool(result["active"]) // SQLite returns int64, MySQL returns bool
intValue := utils.ToInt64(result["count"]) // MySQL may return string for aggregates
floatValue := utils.ToFloat64(result["average"]) // Handle various numeric types
MongoDB Support
RediORM includes full MongoDB support with special features for document databases:
Connection
// MongoDB connection
const db = fromUri('mongodb://localhost:27017/myapp');
// MongoDB SRV connection
const db = fromUri('mongodb+srv://user:pass@cluster.mongodb.net/myapp');
Document Schema
await db.loadSchema(`
model User {
id String @id @default(auto()) @map("_id") @db.ObjectId
email String @unique
profile Json? // Nested document
tags String[] // Array field
}
`);
Nested Queries
// Query nested fields
const users = await db.models.User.findMany({
where: {
'profile.location.city': 'San Francisco',
tags: { in: ['developer'] }
}
});
// Update nested fields
await db.models.User.update({
where: { id: userId },
data: {
'profile.bio': 'Updated bio',
tags: { push: 'mongodb' }
}
});
Aggregation Pipeline
// Raw aggregation pipeline
const results = await db.queryRaw(`{
"aggregate": "users",
"pipeline": [
{ "$match": { "age": { "$gte": 18 } } },
{ "$group": {
"_id": "$location",
"count": { "$sum": 1 },
"avgAge": { "$avg": "$age" }
}}
]
}`);
MongoDB-Specific Features and Limitations
Supported Features
- โ Full CRUD operations with automatic ID generation
- โ SQL to MongoDB query translation
- โ Native MongoDB command execution
- โ Basic transactions (requires replica set)
- โ
Field mapping with
_idhandling - โ Distinct queries
- โ Full aggregation operations (COUNT, SUM, AVG, MIN, MAX)
- โ GroupBy and Having clauses with aggregation pipeline
- โ Index management
- โ String operators (startsWith, endsWith, contains) with regex
- โ Schema evolution support with nullable field handling
- โ Collection existence validation
Limitations
- โ Savepoints not supported
- โ Migrations not supported (schemaless database)
Best Practices
- Use SQL syntax for simple queries
- Use native MongoDB commands for complex aggregations
- Ensure MongoDB replica set is configured for transactions
- Test string matching carefully due to regex differences
๐ ๏ธ Development
Local Development
# Format code
make fmt
# Run linter
make lint
# Run vet
make vet
# Full development cycle
make dev
# CI workflow
make ci
Docker Development
The project includes a docker-compose setup for testing with real databases:
# Start all databases (MySQL, PostgreSQL, MongoDB)
make docker-up
# Wait for databases to be ready
make docker-wait
# Run tests with Docker databases
make test-docker
# Stop databases
make docker-down
Database credentials (all databases use the same):
- User:
testuser - Password:
testpass - Database:
testdb
Connection strings:
# MySQL
mysql://testuser:testpass@localhost:3306/testdb
# PostgreSQL
postgresql://testuser:testpass@localhost:5432/testdb
# MongoDB (with replica set for transactions)
mongodb://testuser:testpass@localhost:27017/testdb?authSource=admin
# Build with version injection
make release-build
# Show current version
make version
๐ฆ Roadmap
- Core database operations (CRUD, transactions, raw SQL)
- Schema management (Prisma-style definitions)
- Migration system (auto & file-based)
- JavaScript runtime with ORM interface
- Multi-database support (SQLite, MySQL, PostgreSQL, MongoDB)
- Relations support (one-to-one, one-to-many, many-to-many)
- Advanced relation features (eager loading, nested writes, filtering)
- CLI tool with timeout support
- GitHub Actions CI/CD pipeline
- Multi-platform binary releases
- GraphQL integration with relation support and configurable logging
- Query optimization and caching
- Connection pooling configuration
- Middleware/plugin system
- Database introspection tools
- Performance monitoring and metrics
๐ License
MIT License - see LICENSE file for details.
๐ค Contributing
Contributions are welcome! Please ensure:
- All tests pass (
make test) - Code is formatted (
make fmt) - No linter warnings (
make lint) - New features include tests
Built with โค๏ธ by the RediORM team