v4.0

migrator v4.0 comes with the following changes:

• gin web framework introduced together with new versioned API #67
• IMPORTANT /v1/ API is not backward compatible, see README.md for details
• diskLoader no longer uses dir walking to read all source migrations - this was an issue when the baseDir was a top dir of the whole project which consisted of thousands of files #64
• upgrade alpine and golang images used for migrator docker image builds #65
• ApplyMigrations() and AddTenantAndApplyMigrations() now return a Result struct which contains a summary of all applied migrations and scripts #66
• migrator can return only Result struct JSON representation (response: summary) or Results and all applied migrations (response: full) #71, see README.md for details
• migrator now supports: apply, sync, and dry-run modes. apply loads and applies all source migrations and scripts, sync creates migrator entries but does not apply migrations nor scripts - this mode is useful when switching between legacy DB migration framework to migrator, dry-run - is an apply operation except the fact that at the end of the operation transaction rollback is called #72, see README.md for details

v3.1

migrator v3.1 contains the following changes:

• support for SQL scripts - these are special migrations which are applied everytime migrator runs, they are ideal for things like creating and executing triggers and/or stored procedures. SQL scripts are optional. An sample configuration looks like this:

  singleScripts:
    - config-scripts
  tenantScripts:
    - tenants-scripts

• support for both relative and absolute paths in migrator yaml: baseDir

• IMPORTANT migrator v3.1 contains non-backward compatibile changes to the configuration yaml file. All existing configuration files must be updated as below:

  1 singleSchemas renamed to singleMigrations
  2 tenantSchemas renamed to tenantMigrations

  An example of a valid v3.1 configuration file including newly added support for SQL script is as follows:

  singleMigrations:
    - public
    - ref
    - config
  tenantMigrations:
    - tenants
  singleScripts:
    - config-scripts
  tenantScripts:
    - tenants-scripts

v3.0

v3.0 contains a lot of changes. The most important one is the removal of tool mode. Starting from v3.0 only server mode is supported.

Complete list of new features includes:

• removal of CLI, only server supported
• refactoring of server package
• generic webhooks support - see README.md for an example of how webhooks can now be configured
• more flexible env substitution in config file - multiple env variables can now be used in single config property
• enhanced fine-grained request logging, context propagation so that other package can use values from context
• injecting git branch and git commit sha into production builds
• Dockerfile changes together with TestDockerfile (with injecting build variables support)
• better README.md documentation with new Quick Start Guide

v2.2

This is a maintenance release.

Contains better error handling, db mocks, and changes in internal API.

There are no changes that affect end users.

v2.1

In v2.1 among internal tweaks a new functionality of verifying migrations was implemented.

This functionality is especially useful in detecting modifications to existing migrations. Without being able to spot modifications to once applied migrations in test envs one may risk issues when running in production.

Migrator is now:

• storing migration contents and checksums in DB
• verifying if checksums of disk migrations match those in DB - checksum verification is called for every apply and add tenant action (both in tool and server mode)

In order to upgrade migrator v2.0 to v2.1 the following steps are required:

1. Add contents and checksum columns to migrator.migrator_migrations table

Execute manually or add as a new migration to your existing migrator v2.0:

alter table migrator.migrator_migrations add contents text;
alter table migrator.migrator_migrations add checksum varchar(64);

2. Compute and store sha256 checksums in migrator.migrator_migrations table

The following script when run in base dir with migrations will generate update-checksums.sql containing sha256 SQL update statements.

#!/bin/bash

dirs=$(ls)

echo "" > update-checksums.sql

for dir in $dirs
do
  if [ -d $dir ]; then
    sha256sum ${dir}/* | awk -v OFS='' '{print "update migrator.migrator_migrations set contents='\'''\'', checksum = '\''",$1,"'\'' where filename = '\''",$2,"'\'';"}' | tee -a update-checksums.sql
  fi
done

3. Execute update-checksums.sql

Depending on your DB:

psql -U $user -h $ip -p $port -d $database -f update-checksums.sql
mysql -u $user -h $ip -P $port -D $database < update-checksums.sql
sqlcmd -S "$ip,$port" -U $user -P $password -d $database -i update-checksums.sql

v2.0

There were some limitations of adding more engines (mainly related to go's lack of virtual methods). In v2 this was addressed.

The most important changes are:

• introduced Dialect interface to extract all vendor-specific SQL statements to separate implementations - a major overhaul of the whole project!
• added support fo Microsoft SQL Server
• made tests re-executable (by using dynamic assertions), increased test coverage, with 85%+ code coverage and many modules having 100% code coverage
• fixed all fmt, lint, and vet warnings
• merge migrator-docker image into main migrator project

Due to some changes required for MSSQL and new lint guidelines I had to make 2 breaking changes:

1. Config File

The following 2 optional elements:

tenantSelectSql: XXX
tenantInsertSql: YYY

Now have capitalised SQL in their names. New correct form is:

tenantSelectSQL: XXX
tenantInsertSQL: YYY

2. DB changes

MS SQL Server doesn't like public and file keywords. Thus a small adjustment in the naming convention had to be made.

Please add the following migration to migrator v2.0 or execute it manually before starting v2.0:

create schema if not exists migrator;
create table if not exists migrator.migrator_tenants (
		id serial primary key,
		name varchar(200) not null,
		created timestamp default now()
);
insert into migrator.migrator_tenants (name, created) (select name, created from public.migrator_tenants);
drop table public.migrator_tenants;
create table if not exists migrator.migrator_migrations (
	id serial primary key,
	name varchar(200) not null,
	source_dir varchar(200) not null,
	filename varchar(200) not null,
	type int not null,
	db_schema varchar(200) not null,
	created timestamp default now()
);
insert into migrator.migrator_migrations (name, source_dir, filename, type, db_schema, created) (select name, source_dir, file, type, db_schema, created from public.migrator_migrations);
drop table public.migrator_migrations;

v1.0

First production-ready migrator version.

Production use required:

• Higher flexibility and overrides in configs
• Implementation of adding tenants
• Upgrading MySQL driver supporting bulk commands which yields significant performance boost
• Simple performance framework for comparison with other tools
• Reworking go code and moving defaults from config to specific modules
• Making xcli more generic
• Implementation of proper logging