How to Manage Apache Superset Migrations: A Complete Guide to Drizzle ORM Workflows

Manage Apache Superset migrations by editing TypeScript schema files in packages/db/src/schema/, generating auto-migrations with bunx drizzle-kit generate, and applying them via bun run db:migrate on Neon PostgreSQL branches.

The superset-sh/superset repository uses a modern, type-safe approach to database schema management. Instead of hand-writing SQL, Apache Superset migrations are auto-generated from TypeScript definitions using Drizzle ORM, ensuring that your database schema always matches your compile-time types.

Understanding the Apache Superset Migration Architecture

Drizzle ORM and PostgreSQL via Neon

Superset’s data layer is built on Drizzle ORM with PostgreSQL hosted on Neon. This stack provides type-safe schema definitions and zero-downtime migration capabilities. According to the source code, all schema definitions live in the packages/db/src/schema/ directory, and migrations are auto-generated from these definitions—you never edit raw .sql files or drizzle/meta snapshots manually.

Schema-First Development in packages/db/src/schema/

The authoritative source of truth for your database structure is the TypeScript code in packages/db/src/schema/. When you need to modify the database—whether adding a column, creating a table, or changing an index—you edit these TypeScript files. The Drizzle Kit CLI then compares your new schema against the previous snapshot stored in packages/db/drizzle/meta/ to generate the precise SQL commands needed to bridge the gap.

Step-by-Step Workflow for Apache Superset Migrations

1. Create a Neon Database Branch

Before touching any schema code, create a temporary branch of your production database using Neon's branching feature. This gives you an isolated environment to test destructive changes without risking production data.

neon branch create <feature-branch-name>

Update your root .env file to point to this new branch's connection string before proceeding.

2. Update TypeScript Schema Definitions

Edit the relevant TypeScript files in packages/db/src/schema/. For example, to add a description column to the workspaces table, modify packages/db/src/schema/schema.ts:

// packages/db/src/schema/schema.ts
import { pgTable, serial, text, timestamp, varchar } from "drizzle-orm/pg-core";

export const workspaces = pgTable("workspaces", {
  id: serial("id").primaryKey(),
  name: varchar("name", { length: 255 }).notNull(),
  // NEW column added here
  description: text("description").default("").notNull(),
  createdAt: timestamp("created_at").defaultNow().notNull(),
});

Critical: Always provide default values for new non-nullable columns (e.g., default("") or defaultNow()), or the migration will fail on existing rows.

3. Generate Migration Files with Drizzle Kit

From the repository root, run the generation command with a descriptive snake_case name:

bunx drizzle-kit generate --name="add_description_to_workspaces"

This command creates two auto-generated files:

• packages/db/drizzle/000X_add_description_to_workspaces.sql – The actual SQL migration commands
• packages/db/drizzle/meta/000X_snapshot.json – Updated schema snapshot for future diffs

These files are also recorded in packages/db/drizzle/meta/_journal.json, which tracks the linear order of migrations. Never edit these files manually, as stated in the repository's AGENTS.md policy document.

4. Apply and Test Migrations Locally

Apply the generated migration to your Neon branch to verify it works correctly:

bun run db:migrate

This script executes the SQL files in packages/db/drizzle/ against your database connection. After running, verify the schema change:

bun run db:studio

# Or query directly via psql or Drizzle Studio

Check that existing data remains intact and that the new column behaves as expected.

5. Commit Changes and Open a Pull Request

Stage both your schema changes and the auto-generated migration files:

git add packages/db/src/schema/schema.ts \
        packages/db/drizzle/000X_add_description_to_workspaces.sql \
        packages/db/drizzle/meta/000X_snapshot.json
git commit -m "feat(db): add description column to workspaces"
git push origin <feature-branch>

When you open a pull request, the CI pipeline will run the migration against a fresh test database to ensure idempotence and correctness. Only merge after CI passes.

Critical File Paths for Apache Superset Migration Management

Path	Purpose
packages/db/src/schema/	Authoritative TypeScript schema definitions (tables, columns, enums).
packages/db/drizzle/	Auto-generated migration .sql files and snapshot metadata. Do not edit manually.
packages/db/drizzle/meta/_journal.json	Ordered list of migration snapshots used by Drizzle to compute diffs.
AGENTS.md	Repository policy mandating auto-generation of migrations and prohibiting manual SQL edits.
.env (root)	Database connection string for the Neon branch you are developing against.

Common Pitfalls When Managing Apache Superset Migrations

• Editing .sql files directly. This breaks the snapshot-driven diff algorithm; future drizzle-kit generate calls may overwrite your changes or produce conflicting migrations. Always modify TypeScript schema files instead.

• Forgetting to create a Neon branch. Running generation or migration commands against production can generate migrations that clash with live data or cause downtime. Always branch first.

• Missing default values for new non-nullable columns. The migration will fail when applied to existing databases because existing rows cannot satisfy the NOT NULL constraint without a default value.

• Not updating related TypeScript models. After schema changes, run bun run typecheck to ensure all generated types (e.g., in packages/db/src/schema/types.ts) remain consistent with the database schema.

• Migration order conflicts after rebasing. If two PRs generate migrations with the same timestamp, you may encounter duplicate IDs. Use descriptive --name flags and re-run drizzle-kit generate after rebasing to ensure correct ordering.

Summary

• Apache Superset migrations are auto-generated from TypeScript schema definitions in packages/db/src/schema/ using Drizzle ORM.
• Never manually edit files in packages/db/drizzle/; always use bunx drizzle-kit generate --name="<descriptive_name>".
• Always create a Neon database branch before developing schema changes to avoid impacting production.
• Include default values for new non-nullable columns to ensure migrations succeed on existing data.
• Commit both schema changes and auto-generated migration files together, then let CI verify idempotence before merging.

Frequently Asked Questions

What is the correct command to generate Apache Superset migrations?

Run bunx drizzle-kit generate --name="<descriptive_snake_case>" from the repository root. This compares the current TypeScript schema in packages/db/src/schema/ against the previous snapshot and generates the necessary SQL migration files in packages/db/drizzle/.

Can I manually edit SQL migration files in Superset?

No. The repository's AGENTS.md explicitly prohibits manual edits to files in packages/db/drizzle/. These files are auto-generated by Drizzle Kit; manual changes break the snapshot-driven diff algorithm and will be overwritten by future generation commands.

How do I apply Apache Superset migrations to a local database?

First, ensure your .env file points to a Neon database branch (never production). Then run bun run db:migrate. This executes all pending SQL files in packages/db/drizzle/ against your database in the correct order defined by _journal.json.

What should I do if my migration fails in CI?

Check that you have provided default values for any new non-nullable columns, as CI tests migrations against fresh databases with existing seed data. Also verify that you committed both the schema TypeScript changes and the auto-generated files in packages/db/drizzle/. If you rebased recently, re-run bunx drizzle-kit generate to resolve any timestamp ordering conflicts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}