Migrations

What it is for

gas/migrate applies schema changes that individual services declare. Its *migrate.Service implements gas.MigrationManager, the core interface a service injects to register its migrations, along with gas.Service.

There is no central list of migrations. Each service registers its own while it initializes, so a service carries its schema with it, and removing a service removes its migrations. Once every service has initialized, the app applies whatever is pending, from every service, in one global version order. Applied migrations are recorded in a __gas_migrations table in the same database.

It works on PostgreSQL, SQLite and MySQL, through a gas/database provider or any other gas.DatabaseProvider. On MySQL the DSN needs parseTime=true; see MySQL.

Install

go get github.com/gasmod/gas/migrate

Wiring

main.go

app := gas.NewApp(
  gas.WithServiceInstance[gas.ConfigProvider](cfg),
  gas.WithSingletonService[gas.Logger](gaslog.NewSlogLogger()),
  gas.WithSingletonService[gas.DatabaseProvider](database.New()),

  // Registered under gas.MigrationManager: that is the type the app
  // looks up to run migrations, and the type services inject.
  gas.WithSingletonService[gas.MigrationManager](migrate.New()),

  gas.WithSingletonService[*Notes](NewNotes),
)

migrate.New() takes no options and returns a constructor that the container calls with gas.DatabaseProvider, so a database provider must be registered under that interface. The constructor never returns an error.

Register it under gas.MigrationManager

The app looks the manager up as gas.MigrationManager, and when none is registered it runs no migrations and reports nothing. Registered as *migrate.Service, the manager is never found: no migration runs, and any service that takes a gas.MigrationManager fails to resolve, which fails startup.

See Drivers for the databases it supports.

Usage

Declaring migrations

A service takes gas.MigrationManager as a constructor parameter and registers its migrations in Init. Register before Init returns: pending migrations are applied right after the last service initializes.

Register adds one gas.Migration:

tags/service.go

type Tags struct {
  migrations gas.MigrationManager
}

func NewTags(migrations gas.MigrationManager) *Tags {
  return &Tags{migrations: migrations}
}

func (t *Tags) Name() string { return "tags" }

func (t *Tags) Init(context.Context) error {
  t.migrations.Register(t, gas.Migration{
    Version:     "20260102001",
    Description: "create tags table",
    Up:          "CREATE TABLE tags (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE);",
    Down:        "DROP TABLE tags;",
  })
  return nil
}

func (t *Tags) Close(context.Context) error { return nil }

RegisterSlice adds several at once. RegisterFS reads them from an fs.FS, usually an embed.FS:

notes/service.go

// migrationFiles holds {version}_{description}.up.sql and .down.sql pairs.
//
//go:embed migrations/*.sql
var migrationFiles embed.FS

// Notes owns its schema: it registers its migrations in Init, and migrate
// applies them after every service has initialized.
type Notes struct {
  migrations gas.MigrationManager
}

func NewNotes(migrations gas.MigrationManager) *Notes {
  return &Notes{migrations: migrations}
}

func (n *Notes) Name() string { return "notes" }

func (n *Notes) Init(context.Context) error {
  return n.migrations.RegisterFS(n, migrationFiles)
}

func (n *Notes) Close(context.Context) error { return nil }

RegisterFS walks the whole file system, subdirectories included. Every file ending in .up.sql is a migration, named {version}_{description}.up.sql:

• the version is the file name up to the first underscore, so a version must not contain one: 20260101_001_create_notes.up.sql has version 20260101;
• the description is the rest of the name with underscores turned into spaces (create notes);
• a matching .down.sql file is required, and a missing one fails RegisterFS with an error naming the .up.sql file. Nothing from that file system is registered then.

Every registration records the service it was given as the migration’s owner, and the owner’s Name() is stored in the tracking table.

Versions

Versions are compared as strings, so give them a fixed width, such as a timestamp. Every pending migration from every service runs in one ascending version order, and a version must be unique across the app. See Version rules.

When migrations run

gas.App and gas.Worker both run them as part of Start (and so of App.Run): after every service has initialized, before the app’s ready hooks, and before the HTTP server accepts traffic. Start calls RunPending, which applies, in version order, every registered migration whose version is not in the tracking table.

A failed migration stops the run and fails startup. See Transactions and failures.

Fixing a dirty migration

A migration that fails is recorded as dirty, and the app does not start again until you resolve it. To resolve it, fix the cause, then call ClearDirty(version) on the *migrate.Service. It deletes the version’s tracking row, so the next RunPending applies the migration again. For a version that is not dirty, because it applied cleanly or has no tracking row, it changes nothing and returns an error wrapping migrate.ErrNotDirty.

ClearDirty is not part of gas.MigrationManager, and the app cannot start while a version is dirty, so run it from a maintenance command that builds the two services itself:

cmd/migrate-clear/main.go

// clearDirty is a maintenance command, run once the cause of a failed
// migration is fixed. It builds the database and migrate services by hand
// instead of starting the app, because starting the app runs RunPending,
// which refuses to run while a migration is dirty.
func clearDirty(ctx context.Context, cfg gas.ConfigProvider, logger gas.Logger, version string) (err error) {
  db, err := database.New()(cfg, logger)
  if err != nil {
    return err
  }
  if err := db.Init(ctx); err != nil {
    return err
  }
  defer func() { err = errors.Join(err, db.Close(ctx)) }()

  mgr, err := migrate.New()(db)
  if err != nil {
    return err
  }
  if err := mgr.Init(ctx); err != nil {
    return err
  }

  // ClearDirty is on *migrate.Service, not on gas.MigrationManager.
  if err := mgr.ClearDirty(version); err != nil {
    if errors.Is(err, migrate.ErrNotDirty) {
      return fmt.Errorf("version %s is not dirty, nothing to clear: %w", version, err)
    }
    return err
  }
  return nil
}

Rolling back

Down(n) reverts the n highest applied versions, running each one’s down SQL. Nothing in Gas calls Down. Call it from a maintenance command:

// rollback reverts the n most recent migrations. Down is part of
// gas.MigrationManager, so any service can call it; here it runs from a
// maintenance command that resolved the manager from a started Worker.
func rollback(w *gas.Worker, n int) error {
  mgr, err := w.ServiceContainer().Resolve[gas.MigrationManager]()
  if err != nil {
    return err
  }
  return mgr.Down(n)
}

RunPending, Down and ClearDirty take no context and run under context.Background(). See Rollback rules for failures and limits.

Testing

migratetest.MockMigrationManager implements gas.MigrationManager for unit tests. Each method records the call in Calls and delegates to the matching Fn field (RegisterFn, RegisterSliceFn, RegisterFSFn, RunPendingFn, DownFn, and so on) when it is set; otherwise it returns nil, and Name returns gas/migrate. CallCount(method) counts the calls to one method and Reset clears Calls. The mock has no ClearDirty or CheckReady.

notes/service_test.go

func TestNotesRegistersItsMigrations(t *testing.T) {
  mock := &migratetest.MockMigrationManager{}

  notes := NewNotes(mock)
  if err := notes.Init(t.Context()); err != nil {
    t.Fatal(err)
  }

  if got := mock.CallCount("RegisterFS"); got != 1 {
    t.Fatalf("RegisterFS called %d times, want 1", got)
  }
  if svc := mock.Calls[0].Args[0]; svc != notes {
    t.Errorf("migrations registered for %v, want the notes service", svc)
  }
}

The mock shows that a service registers its migrations; it does not run them. To test the SQL itself, start the real gas/migrate on SQLite as shown under Database testing. Every pending migration then runs during Start, as in production.

Errors

Error	Returned by	When
migrate.ErrDirty	RunPending, Down, CheckReady	Any version is marked dirty. The message names the versions
migrate.ErrDuplicateVersion	RunPending, Down, CheckReady	Two different migrations share a version. The message names the version and the services
migrate.ErrNotDirty	ClearDirty	The version is not marked dirty

All three are returned wrapped; match them with errors.Is. Other errors start with gas/migrate:, including the failure of a migration (... failed (marked dirty): <cause>, wrapping the driver’s error). Before Init, RunPending, Down, ClearDirty and CheckReady return gas/migrate: not initialized; after Close, gas/migrate: service is closed. Register, RegisterSlice and RegisterFS work at any time.

Behavior details

The sections above cover everyday use. This section is the full contract, for when an exact edge case matters.

Drivers

Init picks the SQL dialect for the tracking table from the provider’s Driver():

Driver()	Dialect
postgres, pgx	PostgreSQL
mysql	MySQL
sqlite, sqlite3	SQLite

Any other driver name fails Init with gas/migrate: unsupported driver. Init then creates __gas_migrations if it does not exist, under the context Init receives. The module reads no configuration.

MySQL

gas/database accepts postgres, pgx and sqlite as its own drivers. For MySQL, give it a connector with database.WithConnector and set database.driver to mysql, or register your own gas.DatabaseProvider whose Driver() returns mysql; see Database.

The MySQL DSN needs parseTime=true, because the module scans the tracking table’s applied_at column into time.Time. Without it the first RunPending succeeds, but every later read of the tracking table fails: CheckReady, Down, and the next RunPending, so the next start of the app. The error is gas/migrate: failed to query applied migrations: sql: Scan error on column index 5, name "applied_at": unsupported Scan, storing driver.Value type []uint8 into type *time.Time (KI-048, see Known issues).

Version rules

Versions are strings and are compared as strings, so they must have a fixed width to sort as numbers. A timestamp such as 20260101001 works; 9 sorts after 10. Every pending migration, from every service, runs in ascending version order, so a version decides where a migration runs relative to other services’ migrations too.

A version must be unique across the app. When two services register the same version, or one service registers two different migrations under one version, RunPending, Down and CheckReady fail with an error that wraps migrate.ErrDuplicateVersion and names the version and the services, and nothing runs. A service that registers an identical migration (same version, description, up SQL and down SQL) more than once gets it applied once.

Transactions and failures

If RunPending fails, Start closes every service it initialized and returns an error starting with gas: migrations:. The app does not start.

Each migration’s up SQL and its tracking row commit in one transaction. When the SQL or the tracking insert fails, the transaction rolls back, the version is recorded as dirty, and RunPending stops and returns the error. Migrations that committed earlier in the same run stay applied. The up and down SQL run as a single Exec call, so whether a migration can hold several statements depends on the driver.

MySQL DDL is not transactional

On MySQL, DDL such as CREATE TABLE, DROP TABLE and ALTER TABLE commits any open transaction implicitly. A migration containing DDL is therefore not atomic with its tracking row in either direction: a failure after the DDL leaves the schema changed whatever the tracking table says.

Dirty versions

A dirty version blocks the app. While any version is dirty, RunPending, Down and CheckReady return an error wrapping migrate.ErrDirty that names the dirty versions, so every later start fails until you resolve it. A dirty migration’s up SQL was rolled back (except for MySQL DDL), so the schema is as it was before that migration.

Rollback rules

Down(n) reverts the n highest versions in the tracking table, highest first. For each, it runs the down SQL and deletes the tracking row in one transaction. When either fails, that migration’s transaction rolls back, the version is not marked dirty, and Down stops and returns the error; versions it reverted before the failure stay reverted. A version to revert must still be registered, or Down fails with an error naming it. An n larger than the number of applied migrations reverts them all; zero or less reverts nothing. Like RunPending, Down runs nothing while any version is dirty, because a dirty version’s down SQL would act on a schema its up SQL never changed.

The tracking table

__gas_migrations has one row per applied or dirty version:

Column	Holds
version	The migration version, the primary key
service	The owning service’s Name()
description	The migration’s description
migrate_version	The version of gas/migrate that applied it
module_version	The version of the Go module whose path ends in / plus the service name, such as github.com/gasmod/gas/auth for gas/auth; empty when there is none
applied_at	When the row was written
dirty	Whether the migration failed

Both versions are read from the binary’s build information.

Health and readiness

*migrate.Service implements gas.ReadyReporter but not gas.HealthReporter. CheckReady fails when the service is closed or not initialized, while any version is dirty (wrapping migrate.ErrDirty), when two migrations share a version (wrapping migrate.ErrDuplicateVersion), and while any registered migration has not been applied (gas/migrate: <n> pending migration(s)). Once the app has started, the last case means a migration was registered or reverted after RunPending ran.

API reference

pkg.go.dev/github.com/gasmod/gas/migrate