README
¶
Journal-migrator
Evolving a database schema is often a scary operation for the database administrators. Many schema changes seem safe at first, but may introduce suttle differences on how the applications interact with the databases, or worse: data loss. In order to safely evolve the schemas, and keep a record of all the changes across your organization databases, use Journal.
This component is a library, meant to be used to execute the schema changes in a synchronous or asynchronous manner (i.e. without downtime).
🧑💻 Usage
This component can be run from the console, and provides the following commands:
- Cleanup: to remove a schema-change leftover table, synchronously.
- Migrate: to execute a schema-change, asynchronously or synchronously.
- Validate: to validate a schema-change against the data, synchronously.
Cleanup command
go run cmd/cleanup/main.go [ARGUMENTS]
List of arguments:
| Name | Env variable | Required | Default | Possible values |
|---|---|---|---|---|
| databaseUsername | DATABASE_USER | True | - | * |
| databasePassword | DATABASE_PASS | True | - | * |
| databaseHost | DATABASE_HOST | True | - | * |
| databasePort | DATABASE_PORT | True | - | * |
| databaseName | DATABASE_NAME | True | - | * |
| databaseType | DATABASE_TYPE | True | - | {mysql, postgres} |
| logLevel | LOG_LEVEL | False | INFO | {DEBUG, INFO, WARN, ERROR} |
| table | - | True | - | * |
Migrate command
go run cmd/migrate/main.go [ARGUMENTS]
List of arguments:
| Name | Env variable | Required | Default | Possible values |
|---|---|---|---|---|
| databaseUsername | DATABASE_USER | True | - | * |
| databasePassword | DATABASE_PASS | True | - | * |
| databaseHost | DATABASE_HOST | True | - | * |
| databasePort | DATABASE_PORT | True | - | * |
| databaseName | DATABASE_NAME | True | - | * |
| databaseType | DATABASE_TYPE | True | - | {mysql, postgres} |
| logLevel | LOG_LEVEL | False | INFO | {DEBUG, INFO, WARN, ERROR} |
| table | - | True | - | * |
| operations | - | True | - | * |
| strategy | - | False | SYNC | {ASYNC, SYNC} |
The ASYNC strategy has two pre-requisites:
- The target table must have a Primary Key with one auto-incremented column.
- The database instance must have Foreign Key enforcement disabled.
Validate command
go run cmd/validate/main.go [ARGUMENTS]
List of arguments:
| Name | Env variable | Required | Default | Possible values |
|---|---|---|---|---|
| databaseUsername | DATABASE_USER | True | - | * |
| databasePassword | DATABASE_PASS | True | - | * |
| databaseHost | DATABASE_HOST | True | - | * |
| databasePort | DATABASE_PORT | True | - | * |
| databaseName | DATABASE_NAME | True | - | * |
| databaseType | DATABASE_TYPE | True | - | {mysql, postgres} |
| logLevel | LOG_LEVEL | False | INFO | {DEBUG, INFO, WARN, ERROR} |
| table | - | True | - | * |
| operations | - | True | - | * |
🔧 Development
Dependencies
make install-dev
Linting
make check
Testing
make test-unit
To test a MySQL database:
export DATABASE_PORT=3306
export DATABASE_TYPE=mysql
export COMPOSE_FILE=mysql-8.0.yaml
make services-up && make test-integration
make services-down
To test a PostgreSQL database:
export DATABASE_PORT=5432
export DATABASE_TYPE=postgres
export COMPOSE_FILE=postgres-16.yaml
make services-up && make test-integration
make services-down
Release
make tag
Acknowledgements
This project is a Golang port of the popular LHM Ruby project, generalizing it to support both MySQL 8.0+ and PostgreSQL 16+ databases.
Directories