The Migration System — Neat ORM's Killer Feature

The Migration System — Neat ORM's Killer Feature

I want to talk about the most underrated feature in database tooling: migrations.

Not "auto-migrate" where you pass a struct and the ORM adds missing columns. Real migrations. The kind where you write a file, it gets tracked, you can undo it, and you know exactly what happened to your schema and when.

This is the feature that made me build Neat ORM. Let me show you how it works.

---

The Problem With "Just Run ALTER TABLE"

Here's a scenario that happens all the time:

1. You add an avatar column to the users table on your dev machine.
2. You commit the code that uses avatar.
3. Your colleague pulls the code. Their database doesn't have the column. The app crashes.
4. You say "oh, you need to run ALTER TABLE users ADD COLUMN avatar VARCHAR(255)."
5. They run it. It works.
6. Three months later, you deploy to production. Nobody remembers which ALTER statements need to run. Something breaks.

This is not a rare scenario. This is the default scenario. Every team that doesn't use migrations lives through this.

---

What a Migration System Does

A migration system makes schema changes versioned, tracked, and reversible. Here's what that means:

Versioned: Each schema change is a file with a unique signature. You commit it to version control alongside your code.

Tracked: A migration_tracker table records which migrations have run. The system never runs the same migration twice.

Reversible: Each migration has an Up() method (apply the change) and a Down() method (undo the change). You can roll back.

This means:

• New developers run migrate up and get a database that matches everyone else's.
• Production deployments run migrate up and the schema updates automatically.
• When something breaks, you run migrate rollback and the schema reverts.

---

What Go Has Today

Go has a few options, none of which are great:

GORM's AutoMigrate: Pass a struct, it adds missing columns. No files, no rollback, no tracking. It's a schema diff, not a migration system. Fine for prototyping, risky in production.

ent's auto-migration: Same idea, different implementation. Also has a "versioned migrations" mode — but it requires a third-party tool.

Third-party tools (goose, golang-migrate, dbmate): These give you migration files, tracking, and rollback. They work. But they don't speak the same language as the rest of your database layer. You install a separate binary, configure a separate database connection, write raw SQL files. The ORM doesn't know about them. The migration tool doesn't know about the ORM. They always feel foreign — an afterthought bolted on, never truly part of the stack.

The missing piece in Go is integration. A migration system that's part of the ORM, not a separate tool.

---

How Neat ORM Does Migrations

Neat ORM has a complete migration system built in. Here's what a migration looks like:

type CreateUsersTable struct {
    migrator.BaseMigration
}

func (m *CreateUsersTable) Signature() string {
    return "2024_06_15_120000_create_users_table"
}

func (m *CreateUsersTable) Description() string {
    return "Creates users table"
}

func (m *CreateUsersTable) Up() error {
    return m.GetSchema().Create("users", func(bp neat.Blueprint) {
        bp.ID()
        bp.String("name")
        bp.String("email").Unique()
        bp.Timestamps()
    })
}

func (m *CreateUsersTable) Down() error {
    return m.GetSchema().DropIfExists("users")
}

You embed migrator.BaseMigration, which gives you access to the schema builder. You implement four methods: Signature(), Description(), Up(), and Down().

The Up() method uses the Blueprint pattern — you create tables in Go, not SQL. The Down() method drops the table.

---

Running Migrations

Here's how you run them:

m := migrator.NewMigrator(db)
m.AddMigration(&CreateUsersTable{})
m.AddMigration(&CreatePostsTable{})
m.AddMigration(&CreateCommentsTable{})
m.Up(context.Background())

The migrator does the following:

1. Creates a migration_tracker table if it doesn't exist.
2. Reads the tracker to see which migrations have already run.
3. Runs the pending migrations in order.
4. Records each migration with its batch number, start time, and completion time.

If you run m.Up(ctx) again, nothing happens. The tracker knows everything has already run.

---

Rolling Back

This is the part that most Go ORMs don't have at all:

// Undo the last migration
m.Down(ctx)

// Undo the last 3 migrations
m.RollbackSteps(ctx, 3)

// Undo everything after a specific batch
m.RollbackToBatch(ctx, 20240615)

// Undo everything
m.Reset(ctx)

// Drop all tables and re-run all migrations
m.Fresh(ctx)

Each rollback calls the migration's Down() method. The tracker is updated. The database reverts to a known state.

Fresh() is the "start over" button — it drops all tables and re-runs every migration from scratch. Very useful in development when you want a clean slate.

---

Batch Numbering

Here's a feature that sounds boring but is actually really useful: migrations run in batches.

Each time you call Up(), the pending migrations get a batch number — MAX(batch) + 1. So if you add three migrations and run Up(), they all get batch 1. Later, you add two more and run Up(), they get batch 2.

This means you can undo an entire deployment with one command:

m.RollbackToBatch(ctx, 1)  // Undo everything from batch 2 onward

This is exactly how Laravel does it, and it handles the most common rollback scenario perfectly: "the last deployment broke something, undo it."

---

The Tracker Table

The migrator automatically creates and maintains a migration_tracker table:

Column	Description
ID	Migration signature (e.g., 2024_06_15_120000_create_users_table)
Batch	Batch number
Description	Human-readable description
StartedAt	When the migration started
CompletedAt	When the migration finished

And here's a nice detail: the tracker is self-upgrading. If a future version of Neat adds columns to the tracker schema, ensureMigrationTracker automatically adds them. You don't need to migrate the migration tracker. It migrates itself.

---

Transaction Support

Migrations can run inside database transactions:

m.SetTransactionsEnabled(true)  // default

If a migration fails, the transaction rolls back. The database is unchanged. The tracker isn't updated. Clean.

There's a caveat: some schema operations can't run inside transactions. MySQL's DDL statements (CREATE TABLE, ALTER TABLE) auto-commit and can't be rolled back. For those cases, you can disable transactions:

m.SetTransactionsEnabled(false)

This is what Neat does in production — transactions are disabled globally because MySQL DDL auto-commits. For PostgreSQL, which supports DDL inside transactions, this could be re-enabled.

---

Signature Validation and Ordering

Two more features that sound minor but matter:

Signature validation: You can enforce a naming convention:

m.SetSignatureValidation(true, migrator.SignatureFormatDateTime)

Now every migration must follow YYYY_MM_DD_HHMMSS_description. Bad signatures are rejected before they run.

Lexicographical ordering: Instead of running migrations in the order they were added, sort them by signature:

m.SetLexicographicalOrdering(true)

This ensures 2024_06_15_120000_create_users_table runs before 2024_06_16_120000_create_posts_table regardless of registration order. Very useful when migrations are registered dynamically.

---

Real-World Usage

Neat ORM's migrator runs in production across multiple websites. Each migration calls a store's MigrateUp() method, which creates that store's tables using the schema builder. The migrator tracks them all — every table, every change, every rollback.

This is what database management in Go should look like. Not third-party tools. Not auto-migrate-and-hope. A real migration system, built into the ORM, with tracking, rollback, batch numbering, and transaction support.

It's the feature that made me build Neat ORM. And it's the feature that no other Go ORM has.