[{"content":"ClickHouse ships with no migration tooling. None. The database is extraordinary at what it does, and then you go to change a column and discover you\u0026rsquo;re on your own.\nSo teams do what teams do. They write a migrations/ folder full of .sql files. They wire up a runner. And from that day forward, every MergeTree clause, every codec, every materialized view definition gets typed by hand, reviewed by hand, and kept in sync with the application by hand.\nI built DBWarden partly because I got tired of that. This post is about the ClickHouse side of it: what makes ClickHouse schema management genuinely harder than PostgreSQL, what tools exist today, and what it looks like when your models carry the ClickHouse specifics instead of your SQL files.\nUsual disclaimer: I wrote the tool, so I\u0026rsquo;m biased. I\u0026rsquo;ll be specific about where the alternatives are the better call.\nWhy ClickHouse schema management is its own problem If you\u0026rsquo;ve only done migrations against PostgreSQL or MySQL, ClickHouse breaks assumptions you didn\u0026rsquo;t know you had.\nThe engine is part of the schema. A ClickHouse table isn\u0026rsquo;t just columns. It\u0026rsquo;s an engine, and the engine choice changes the semantics of the data. MergeTree stores rows. ReplacingMergeTree deduplicates on merge. SummingMergeTree collapses numeric columns. AggregatingMergeTree stores aggregate states rather than values. CollapsingMergeTree and VersionedCollapsingMergeTree implement cancel-and-replace semantics with a sign column. Picking one is a data-modeling decision, and it lives in the DDL.\nSorting is structural, not an index. ORDER BY in ClickHouse defines physical layout. It is not a CREATE INDEX you can drop and rebuild. You can extend a sorting key in some cases. You cannot freely change it. Getting it wrong means recreating the table and copying the data.\nHalf the objects aren\u0026rsquo;t tables. A real ClickHouse deployment has materialized views doing continuous aggregation, projections giving alternate sort orders inside a table, data-skipping indexes, dictionaries for fast joins against external sources, and named collections holding credentials. Every one of those is a schema object with its own lifecycle.\nCompression is per column and it matters. CODEC(ZSTD(5)) on a wide string column, DoubleDelta on a monotonic timestamp. These aren\u0026rsquo;t micro-optimizations at analytics scale. They\u0026rsquo;re the difference between an affordable cluster and an expensive one. And because they\u0026rsquo;re per column, a table with thirty columns has thirty small decisions encoded in its DDL, every one of which somebody has to keep straight.\nTTL is schema, not a cleanup job. ClickHouse expires data through TTL expressions declared on the table or on individual columns. Retention policy stops being a cron job somebody wrote and becomes part of the table definition, which is better, but it also means retention changes are schema migrations and need the same review as everything else.\nLots of changes are simply illegal. In PostgreSQL, most things are an ALTER away. In ClickHouse, a surprising number of changes are CREATE-time commitments. Change them and you\u0026rsquo;re rebuilding the table, moving the data, and swapping it in.\nAnd it\u0026rsquo;s usually clustered. DDL needs ON CLUSTER, replicated engines need consistent paths across replicas, and getting that wrong produces a schema that\u0026rsquo;s subtly different on node three.\nNone of that is a complaint about ClickHouse. These are the tradeoffs that make it fast. But they mean \u0026ldquo;just write the SQL by hand\u0026rdquo; is a much worse plan here than it is for Postgres, because there\u0026rsquo;s more to get wrong and the failure mode is a table rebuild rather than a quick ALTER.\nWhat already exists Let me be fair about the landscape, because \u0026ldquo;nothing exists\u0026rdquo; would be false and you\u0026rsquo;d catch me at it.\nSQL migration runners. golang-migrate, goose, dbmate, clickhouse-migrations, PyClickHouseMigrator. These are ordered .sql files plus a runner that tracks which ones ran. They work. They\u0026rsquo;re predictable, they stay close to the SQL, and for a lot of teams that\u0026rsquo;s genuinely the right answer. Their model is simple enough to reason about at 3 AM, which counts for a great deal.\nWhat they don\u0026rsquo;t do is write the SQL. Every MergeTree clause is yours. There\u0026rsquo;s no diffing, no drift detection, no relationship between your application\u0026rsquo;s understanding of the schema and the database\u0026rsquo;s.\nAtlas. Declarative, with real ClickHouse support. You describe the desired schema, Atlas plans the diff. It\u0026rsquo;s a good tool and I\u0026rsquo;ve said so at length elsewhere. The tradeoff for a Python team is that the schema lives in Atlas\u0026rsquo;s own representation, HCL or SQL, as a separate artifact from your application.\nBytebase and similar platforms add governance: who changed what, approval flows, environment tracking. Different layer of the problem, and complementary to any of the above.\nSo the gap isn\u0026rsquo;t \u0026ldquo;no tooling.\u0026rdquo; It\u0026rsquo;s narrower and more specific: nothing lets a Python team declare ClickHouse-specific structure in the models they already maintain. If you\u0026rsquo;re running SQLAlchemy and ClickHouse, your options have been hand-written SQL or a separate schema language. That\u0026rsquo;s the gap I went after.\nThe basic shape Here\u0026rsquo;s a ClickHouse table in DBWarden. It\u0026rsquo;s a normal SQLAlchemy model with a class Meta carrying the ClickHouse specifics.\nfrom datetime import date from sqlalchemy import func from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column from dbwarden.databases.clickhouse import CHTableMeta, ch_table, merge_tree class Base(DeclarativeBase): pass class Event(Base): __tablename__ = \u0026#34;events\u0026#34; id: Mapped[int] = mapped_column(primary_key=True) event_date: Mapped[date] = mapped_column() amount: Mapped[float] = mapped_column() class Meta(CHTableMeta): ch = ch_table( engine=merge_tree(), order_by=[\u0026#34;event_date\u0026#34;, \u0026#34;id\u0026#34;], partition_by=func.toYYYYMM(Event.event_date), ) The engine, sort order, and partition expression are declared, not written as DDL. dbwarden make-migrations diffs that against the live database and emits the CREATE TABLE with the right engine clause. dbwarden migrate applies it.\nThe class Meta block is typed and validated at import time. Misspell an attribute and you get a DBWardenConfigError when the module loads, naming the bad attribute, rather than mysterious DDL three deploys later. That matters more in ClickHouse than elsewhere, because a typo\u0026rsquo;d engine setting can silently produce a table with different merge semantics.\nThe engine factories cover the family: merge_tree, replacing_merge_tree, summing_merge_tree, aggregating_merge_tree, collapsing_merge_tree, versioned_collapsing_merge_tree, graphite_merge_tree, and replicated_merge_tree. Special engines too: distributed, buffer, join_engine, set_engine, memory, null, merge, dictionary_engine, and the Log family. Integration engines are there as well, which I\u0026rsquo;ll come back to.\nCodecs and column TTL Compression settings and per-column TTL live on the column, where they belong:\nfrom datetime import datetime from dbwarden.databases.clickhouse import CHColumnMeta, CHTableMeta, ch, ch_table, merge_tree class SensorReading(Base): __tablename__ = \u0026#34;sensor_readings\u0026#34; sensor_id: Mapped[str] = mapped_column() ts: Mapped[datetime] = mapped_column() temp: Mapped[float] = mapped_column() humidity: Mapped[float] = mapped_column() class Meta(CHTableMeta): ch = ch_table( engine=merge_tree(), order_by=[\u0026#34;sensor_id\u0026#34;, \u0026#34;ts\u0026#34;], ) class temp(CHColumnMeta): ch = ch.field(codec=\u0026#34;ZSTD(5)\u0026#34;) class ts(CHColumnMeta): ch = ch.field(codec=\u0026#34;DoubleDelta\u0026#34;, ttl=\u0026#34;ts + toIntervalDay(90)\u0026#34;) class humidity(CHColumnMeta): ch = ch.field(ttl=\u0026#34;ts + toIntervalDay(30)\u0026#34;) Inner classes named after columns. DoubleDelta on the timestamp because it\u0026rsquo;s monotonic, ZSTD(5) on the value column, different retention per column. When you change a codec, DBWarden generates the MODIFY COLUMN statement. When you change a TTL, same.\nThe thing I like about this: the compression strategy sits next to the column it compresses, in the file your application already imports. Not in a migration from eleven months ago that nobody remembers.\nMaterialized views, which are the actual hard part Materialized views are where ClickHouse schema management gets genuinely difficult, and where hand-written SQL hurts most. A ClickHouse MV is a trigger that fires on insert and writes into a target table. There are two shapes: the MV creates and owns an inner table, or it writes TO a table you defined separately. Changing the target is a different operation from changing the query, and changing the query is sometimes MODIFY QUERY and sometimes a full recreate.\nDBWarden models both shapes. Here\u0026rsquo;s the one where the class is the target and the view is generated for you:\nfrom dbwarden.databases.clickhouse import CHViewMeta, MaterializedView, materialized_view, merge_tree class EventDaily(MaterializedView): __tablename__ = \u0026#34;event_daily\u0026#34; date: Mapped[date] = mapped_column(primary_key=True) total: Mapped[float] = mapped_column() cnt: Mapped[int] = mapped_column() class Meta(CHViewMeta): ch = materialized_view( select=\u0026#34;SELECT event_date AS date, sum(amount) AS total, \u0026#34; \u0026#34;count(*) AS cnt FROM events GROUP BY event_date\u0026#34;, engine=merge_tree(), order_by=[\u0026#34;date\u0026#34;], ) You declare the target\u0026rsquo;s columns and engine, plus the SELECT that feeds it. DBWarden emits both the target table and the CREATE MATERIALIZED VIEW, and keeps them consistent. Refreshable materialized views (ClickHouse 24.3 and up) are supported too.\nFor AggregatingMergeTree pipelines, there\u0026rsquo;s a dedicated construct, because writing aggregate-state SQL by hand is genuinely unpleasant:\nfrom dbwarden.databases.clickhouse import AggregatingView, CHViewMeta, agg, aggregating_view class EventAggregated(AggregatingView): __tablename__ = \u0026#34;event_aggregated\u0026#34; class Meta(CHViewMeta): ch = aggregating_view( source=EventDaily, group_by=[EventDaily.date], aggregates=[ agg.sum(EventDaily.total, \u0026#34;Float64\u0026#34;).as_(\u0026#34;state\u0026#34;), ], order_by=[EventDaily.date], ) The aggregate columns get derived from the source model. You say \u0026ldquo;sum this column, grouped by that one,\u0026rdquo; and the AggregateFunction column types and sumState(...) expressions are generated. If you\u0026rsquo;ve ever hand-maintained a three-level rollup of raw table into daily MV into monthly aggregate, you know the specific pain of keeping the state types aligned across all three by hand.\nProjections, skip indexes, dictionaries Projections give a table an alternate physical sort order, so one table can serve queries that want different orderings. Skip indexes let ClickHouse skip granules that can\u0026rsquo;t match. Both are declared as table metadata, and both support MATERIALIZE as an explicit data operation for existing rows, which is important because adding a projection doesn\u0026rsquo;t retroactively build it.\nDictionaries are ClickHouse\u0026rsquo;s in-memory lookup structures, with a source (another table, a database, an HTTP endpoint), a layout (flat, hashed, complex_key_hashed, and so on), and a lifetime controlling refresh. DBWarden declares source, layout, and lifetime as structured config rather than as a wall of CREATE DICTIONARY SQL.\nIntegration engines get the same treatment. Kafka, S3, S3Queue, RabbitMQ, NATS, Redis, MongoDB, MySQL, PostgreSQL, HDFS, URL, and File all have typed settings objects. A Kafka ingestion table plus the materialized view that drains it into a MergeTree is a very common ClickHouse pattern, and it\u0026rsquo;s the kind of thing that\u0026rsquo;s easy to get subtly wrong when it\u0026rsquo;s a hand-typed settings blob.\nNamed collections hold credentials for those integrations. These are deliberately declare-only: DBWarden manages their existence but never diffs their values, because reading secrets back out to compare them is not a behavior I want in a migration tool. Note that named collections ship in the dbwarden-ch-rbac plugin rather than core, alongside the RBAC objects below.\nRBAC as configuration ClickHouse RBAC covers roles, users, row policies, quotas, settings profiles, and grants. In most projects this lives in a wiki page and a shell script somebody ran once.\nDBWarden declares it in your database config. Be clear on where this lives, though: the ch_* config keys below and the handlers that emit their DDL both belong to the dbwarden-ch-rbac plugin, not core. Install it with dbwarden plugin add dbwarden-ch-rbac. Without it, declaring these keys raises DBWardenConfigError when your config loads, naming the plugin to install, rather than quietly producing nothing.\nfrom dbwarden import database_config from dbwarden.databases.clickhouse import ChGrantSpec, ChRoleSpec, ChUserSpec analytics = database_config( database_name=\u0026#34;analytics\u0026#34;, database_type=\u0026#34;clickhouse\u0026#34;, database_url_sync=\u0026#34;clickhouse://localhost:9000\u0026#34;, ch_roles=[ChRoleSpec(\u0026#34;analyst\u0026#34;), ChRoleSpec(\u0026#34;engineer\u0026#34;)], ch_users=[ ChUserSpec(name=\u0026#34;bob\u0026#34;, default_role=\u0026#34;analyst\u0026#34;), ], ch_grants=[ ChGrantSpec(privileges=[\u0026#34;SELECT\u0026#34;], on=\u0026#34;analytics.*\u0026#34;, to=\u0026#34;analyst\u0026#34;), ], ) Roles and grants become part of the reviewed, versioned migration flow. Adding a role is a diff. Dropping a user is a diff, and a gated one. Access control stops being tribal knowledge.\nThat plugin split is deliberate and worth understanding, because it applies across the project. Core owns tables, columns, engines, materialized views, projections, skip indexes, and dictionaries. Plugins own RBAC on both ClickHouse and PostgreSQL, PostgreSQL types and sequences, PostgreSQL functions and triggers and extensions, seed data, and the Testcontainers sandbox providers. So the ClickHouse table modeling in this post is core, and the access-control section is a plugin away.\nA plugin owns its config keys, not just its handlers, and core validates every keyword you pass to database_config() against the plugins actually installed. A key belonging to a missing plugin fails with an install hint; a key no plugin owns fails as an unknown argument, so a typo like ch_role never silently does nothing. Both happen at config load, which is the moment you can still fix it cheaply.\nThe feature I actually care about: knowing what you can\u0026rsquo;t change Everything above is convenience. This part is the reason I think a ClickHouse-aware tool earns its place.\nClickHouse changes fall into very different risk categories, and the DDL doesn\u0026rsquo;t warn you. DBWarden classifies every generated operation into three levels:\nINFO, applied automatically. ADD COLUMN, ADD INDEX, ADD PROJECTION, settings changes, TTL changes. WARN, applied but logged loudly. DROP COLUMN, DROP TABLE, DROP INDEX, mutations, partition drop and replace. CRITICAL, skipped unless you pass --force. Engine changes, ORDER BY non-extensions, PRIMARY KEY changes, materialized view TO target changes, incompatible type changes, and LowCardinality or Nullable toggles. That last category is the one that ruins weeks. Changing ORDER BY from (a, b) to (c) is not an ALTER. It\u0026rsquo;s a table rebuild. DBWarden refuses it by default and tells you why:\nCRITICAL: Changing ORDER BY from (a, b) to (c) requires --force And when you do force it, you get the whole recreate pipeline written out, so you can see exactly what will happen to your data before it happens:\nDETACH TABLE events CREATE TABLE events_new ... INSERT INTO events_new SELECT * FROM events RENAME TABLE ... Compare that to a hand-written migration. You write ALTER TABLE events MODIFY ORDER BY (c), ClickHouse rejects it, and now you\u0026rsquo;re improvising a data migration at whatever hour you discovered it. The tool knowing which changes are structural is worth more than all the declaration syntax above it.\nThere\u0026rsquo;s a --dry-run to preview and a --sandbox to replay first. On the sandbox flag, be precise about what you get: core ships an in-memory SQLite provider, which for ClickHouse is close to useless. Install the dbwarden-sandbox plugin and you get Testcontainers-backed real ClickHouse replay, which is the version worth trusting.\nA day in the life Abstract feature lists are easy to nod along to. Here are two real changes, done both ways.\nChange one: add a column. The easy case, and the one every tool handles.\nWith a SQL runner you create 0007_add_user_agent.sql, type ALTER TABLE events ADD COLUMN user_agent String, and remember to also update whatever your application believes the schema is. Two places, by hand, kept in sync by discipline.\nWith DBWarden you add user_agent: Mapped[str] = mapped_column() to the model and run make-migrations. You get:\nALTER TABLE events ADD COLUMN user_agent String (INFO) Classified INFO, applied automatically, and the model that generated it is the same model your application imports. One place.\nChange two: change the sorting key. The case that separates the tools.\nWith a SQL runner you write ALTER TABLE events MODIFY ORDER BY (c), it fails, and now you\u0026rsquo;re learning about ClickHouse\u0026rsquo;s structural constraints during a deploy. What follows is an improvised rebuild: create a new table with the right sort order, copy the data across, swap the names, hope nothing wrote to the old table mid-copy.\nWith DBWarden you change order_by in the model, run make-migrations, and get stopped:\nCRITICAL: Changing ORDER BY from (a, b) to (c) requires --force No SQL generated, nothing applied. When you pass --force, the generated migration contains the full rebuild rather than a statement ClickHouse will reject:\nDETACH TABLE events CREATE TABLE events_new ... INSERT INTO events_new SELECT * FROM events RENAME TABLE ... The difference isn\u0026rsquo;t convenience. It\u0026rsquo;s when you find out. One tool tells you at generation time on your laptop, with the rebuild written out for review. The other tells you at apply time, in whatever environment you were deploying to.\nDrift, and knowing your cluster still matches your models There\u0026rsquo;s a failure mode specific to analytics databases: somebody adds a column directly on the cluster to unblock a dashboard, and it\u0026rsquo;s never mentioned again. Six months later nobody knows which of the forty tables match their definitions.\nBecause DBWarden diffs models against live state, that drift surfaces the next time anyone generates a migration. Unexpected entries appear in the diff, on someone\u0026rsquo;s laptop, before any deploy. There\u0026rsquo;s also dbwarden diff as a read-only comparison you can run whenever you\u0026rsquo;re suspicious, and it outputs as a Rich table, JSON, or raw SQL depending on whether a human or a pipeline is reading it.\nSnapshots make that cheap. After every migration a checksummed JSON snapshot of the schema lands in .dbwarden/schemas/, so most comparisons don\u0026rsquo;t require interrogating the cluster at all.\nAdopting it on a database that already exists Nobody starts fresh. You have ClickHouse tables in production right now, defined by SQL files of uncertain provenance.\ndbwarden generate-models --database analytics This reverse-engineers the live database into SQLAlchemy models with the class Meta blocks filled in: engines, sort keys, partition expressions, codecs, TTLs, materialized views, projections. --base points it at your project\u0026rsquo;s existing declarative base rather than generating its own.\nThe round-trip is verified rather than assumed. The project\u0026rsquo;s audit harness runs 39 cases against ClickHouse 24.3 and 26.6 and reports zero drift, with a single canonicalizer code path and no version branching between them. Which is to say: models generated from a live database regenerate that same database. I mention the number because \u0026ldquo;supports ClickHouse\u0026rdquo; is a claim every tool makes, and the useful question is always what was measured.\nFrom there you generate a baseline migration and mark it applied with dbwarden migrate --baseline, and your existing schema is now under management without anything being rebuilt.\nClusters, which are not covered One limitation to state plainly, because it\u0026rsquo;s the kind you don\u0026rsquo;t want to discover on a three-node deployment: cluster-aware DDL is not supported. Generated statements do not carry ON CLUSTER, and there\u0026rsquo;s no configuration that makes them. If your ClickHouse is clustered, keep whatever you use for propagating DDL across nodes today.\nEverything else in this post works against a clustered instance the same as a single node, since the table, view, and projection definitions themselves don\u0026rsquo;t change. It\u0026rsquo;s specifically the cluster-wide propagation of the generated DDL that isn\u0026rsquo;t there.\nWhere the other tools fit better I mean this section.\nYou\u0026rsquo;re not a Python shop. DBWarden requires SQLAlchemy, Python 3.12+. If your ClickHouse is fed by Go services, use golang-migrate or Atlas. This isn\u0026rsquo;t a candidate for you and I\u0026rsquo;d rather say so than waste your afternoon.\nYour ClickHouse schema is small and stable. If it\u0026rsquo;s six tables that change twice a year, a migrations/ folder and dbmate is genuinely less machinery than adopting a schema tool. Simplicity has real operational value. Don\u0026rsquo;t buy a diff engine to manage six tables.\nYou want to hand-tune every statement. Some teams want to write the exact DDL, in the exact order, with the exact settings. That\u0026rsquo;s a legitimate preference, especially with unusual cluster topologies, and a SQL runner respects it. DBWarden generates SQL from models; that\u0026rsquo;s the trade.\nYou need governance more than generation. If your actual problem is approvals and audit trails across environments, Bytebase solves that and DBWarden doesn\u0026rsquo;t try to.\nYou want one tool across many databases and languages. Atlas covers more backends and every ecosystem. DBWarden covers PostgreSQL, MySQL, ClickHouse, MariaDB, and SQLite for local development, from Python only.\nWhere I think it genuinely wins You run ClickHouse next to an OLTP database. This is the common case and the strongest argument. Postgres for the application, ClickHouse for analytics. Normally that\u0026rsquo;s two schema workflows: Alembic on one side, hand-written SQL on the other. DBWarden covers both with the same models, same commands, same migration files, same review process.\nYour ClickHouse schema is genuinely complex. Materialized view chains, aggregate states, projections, dictionaries, Kafka ingestion. The more ClickHouse-specific structure you have, the more the typed declarations pay off against hand-written DDL.\nYou\u0026rsquo;ve been bitten by an illegal change. If you\u0026rsquo;ve ever discovered mid-deploy that a sorting key can\u0026rsquo;t be altered, the safety classification alone is the pitch.\nYou want CI without a ClickHouse container. Export model state once with dbwarden export-models, commit it, and generate migrations with make-migrations --offline on any machine. No ClickHouse service in the pipeline just to plan a schema change. Anyone who has waited on a ClickHouse container to become healthy in CI, only to discover the job was checking whether a column name changed, will recognize why I built this.\nYour ClickHouse knowledge is unevenly distributed across the team. This one is less about features and more about how teams actually work. Usually one or two people genuinely understand MergeTree, and everyone else copies an existing table definition and edits it. Typed declarations plus import-time validation plus a safety classifier turn a lot of that tribal knowledge into something the tooling enforces. The person who doesn\u0026rsquo;t know that sorting keys are structural gets stopped by the tool instead of by an incident.\nCommon questions Does this replace my ClickHouse SQL knowledge? No, and it shouldn\u0026rsquo;t. You still need to know why ReplacingMergeTree deduplicates on merge rather than on insert, and why your sorting key determines query performance. What changes is that you stop typing the DDL and start declaring the decisions. The generated SQL is right there in the migration file, so if anything you end up reading more ClickHouse DDL than before, just not writing it.\nWhat if DBWarden generates SQL I don\u0026rsquo;t want? Read it in the migration file and change the model, or write the migration yourself. dbwarden new creates a manual SQL migration that lives in the same versioned sequence as generated ones. The generator handles the common cases; the escape hatch is always there for the odd one.\nDoes it handle ClickHouse version differences? The canonicalizer has zero version branching: one code path covers 24.3 through 26.6, verified by the same 39 audit cases against both, with zero drift. That\u0026rsquo;s a deliberate design constraint rather than an accident. Version-specific branches in a schema differ are where subtle bugs accumulate.\nCan it manage both my Postgres and my ClickHouse? Yes, and this is the strongest reason to use it. Declare both in dbwarden.py, assign models per database, and run dbwarden migrate --all to apply to each in sequence. Same models, same commands, same reviewable SQL artifacts, same rollback contract. In my experience this is where most of the value lands, because the alternative is genuinely two separate toolchains with two separate review cultures.\nWhat about the analytics-specific stuff like backfills? Data operations are modeled: partition operations, mutations, OPTIMIZE, and POPULATE for materialized views. Materializing a projection on existing data is an explicit operation rather than something you hope happened, which matters because adding a projection does nothing to rows already written.\nIs ClickHouse support actually complete, or is it a checkbox? Fair question, and the honest answer is to look at what\u0026rsquo;s measured rather than what\u0026rsquo;s claimed. The round-trip audit is 39 cases across two ClickHouse versions with zero drift, and the docs list deliberate exclusions explicitly rather than staying quiet about them. Replicated databases are extraction-only, which is documented as a gap. I\u0026rsquo;d rather point you at the gaps than have you find them.\nWhat does adopting it cost? Python 3.12+ and SQLAlchemy 2.0+ as hard floors. A new mental model where models are authoritative and migration files are artifacts. And the honest one: if your ClickHouse schema is small and stable, this is more machinery than your problem needs.\nThe short version ClickHouse gives you no migration story, so the ecosystem filled the gap with SQL runners, and SQL runners mean hand-writing DDL forever. Atlas offers a declarative path if you\u0026rsquo;ll maintain schema files separately from your application.\nWhat didn\u0026rsquo;t exist, as far as I could find, was the option to declare ClickHouse\u0026rsquo;s specifics in the models a Python team already owns: engine families, codecs, TTLs, materialized views, aggregate states, projections, dictionaries, and RBAC. All typed, all validated at import, all diffed into reviewable SQL with the structural changes flagged before they detonate.\nThat\u0026rsquo;s the thing I wanted, so that\u0026rsquo;s the thing I built.\nThe ClickHouse documentation covers all of it in more depth than a blog post can. If you\u0026rsquo;d rather start from the comparison angle, there\u0026rsquo;s DBWarden vs Alembic on declarative versus imperative, DBWarden vs Atlas on two declarative tools, and DBWarden vs Django migrations.\n","permalink":"https://blog.emiliano-go.com/works/clickhouse-schema-management-in-python/","summary":"Every ClickHouse migration tool I found makes you hand-write the DDL or learn a separate schema language. I wanted to declare MergeTree specifics in the models I already had, so I built that. Here is what it looks like, and where the other tools still win.","title":"ClickHouse Schema Management in Python"},{"content":"First, the disclaimer: I built DBWarden. I am biased. But I used Alembic for a long time before writing a single line of DBWarden, and I respect it. It\u0026rsquo;s maintained by the same person behind SQLAlchemy itself, and it earned its place as the default. If this post reads like bashing, I failed. The goal is to show you a real difference in philosophy, so you can pick the right tool for your project.\nOne more thing before we start. You probably know Alembic and have never heard of DBWarden. So this post won\u0026rsquo;t just compare. For every piece of the migration workflow, I\u0026rsquo;ll explain what the piece is, how Alembic handles it, and then how DBWarden handles it. By the end you\u0026rsquo;ll understand both tools well enough to choose. That\u0026rsquo;s the whole point.\nThe two camps: imperative vs declarative Every schema management tool answers one question: how do I get my database from the shape it has to the shape I want? There are exactly two philosophies for answering it.\nImperative tools have you author changes. You write a script that says \u0026ldquo;add this column, rename that table, create this index\u0026rdquo;. Each script moves the schema from version N to version N+1. The scripts chain together. That chain, the full history of every change ever made, becomes the source of truth for what your database should look like. To know the current schema, you replay the history in your head (or trust the ORM models and hope nobody diverged).\nDeclarative tools have you author the desired state. You describe what the schema should be, and the tool figures out the changes needed to get there. The description is the source of truth. History is a byproduct.\nIf you\u0026rsquo;ve used Terraform, you know this split already. Nobody writes \u0026ldquo;create an EC2 instance\u0026rdquo; scripts chained by hand anymore. You declare the infrastructure you want, and the tool plans the diff. Databases are one of the last places where the imperative approach is still the default. I think that\u0026rsquo;s mostly inertia.\nAlembic is imperative. Every change becomes a revision script: a Python file with an upgrade() and a downgrade() function, linked to its parent by a revision id.\nDBWarden is declarative. Your SQLAlchemy models are the schema definition. There is no second representation. You change a model, and DBWarden generates the SQL to reconcile the database with it. Rollback included.\nThat\u0026rsquo;s the core. Now let\u0026rsquo;s walk through the actual workflow, piece by piece.\nSetup: env.py vs dbwarden.py What this piece is: every migration tool needs to know two things. Where your database lives, and where your models live.\nAlembic\u0026rsquo;s answer is alembic init, which scaffolds a directory: an alembic.ini file for configuration, an env.py script that wires your engine and metadata into the migration context, plus a versions/ folder and a script template. The env.py file is real Python that runs on every command. It\u0026rsquo;s flexible, and that flexibility is genuinely useful when you have exotic setups. But most projects copy an env.py from a previous project, tweak the target_metadata line, and never look at it again. It\u0026rsquo;s boilerplate you own and occasionally have to debug.\nDBWarden\u0026rsquo;s answer is a single dbwarden.py file in your project root. It\u0026rsquo;s not a script that runs migrations. It\u0026rsquo;s a declaration of your databases:\nfrom dbwarden import database_config primary = database_config( database_name=\u0026#34;primary\u0026#34;, default=True, database_type=\u0026#34;postgresql\u0026#34;, database_url_sync=\u0026#34;postgresql://user:pass@localhost:5432/myapp\u0026#34;, database_url_async=\u0026#34;postgresql+asyncpg://user:pass@localhost:5432/myapp\u0026#34;, ) That\u0026rsquo;s the entire configuration. Model discovery is automatic (you can pin it down with model_paths if you want control). Then dbwarden init sets up the migrations directory and internal state. No template, no env.py, no ini file. One config object per database. If you have five databases, you declare five of these, and every command takes a --database flag.\nThe migration itself: revision scripts vs models What this piece is: the artifact. The thing that gets written, reviewed, committed, and executed when your schema changes.\nAlembic\u0026rsquo;s artifact is the revision script. A Python file that looks like this:\nrevision = \u0026#34;ae1027a6acf\u0026#34; down_revision = \u0026#34;1975ea83b712\u0026#34; def upgrade(): op.add_column(\u0026#34;users\u0026#34;, sa.Column(\u0026#34;bio\u0026#34;, sa.Text(), nullable=True)) def downgrade(): op.drop_column(\u0026#34;users\u0026#34;, \u0026#34;bio\u0026#34;) You author this file, or generate it and then edit it. It gets committed. From that moment it is part of the chain. The down_revision pointer links it to its parent, and Alembic replays the chain in order to build your schema. The revision history is the schema definition. Your models describe the same schema a second time, and keeping the two in agreement is your job.\nThis is the part I want you to sit with, because it\u0026rsquo;s the entire disagreement between the two tools. You maintain two representations of your schema: the models your application uses, and the migration chain your database uses. Every schema change must be made in both. When they drift apart, nothing tells you until something breaks.\nDBWarden\u0026rsquo;s artifact is different in two ways. First, it\u0026rsquo;s not authored, it\u0026rsquo;s derived. Second, it\u0026rsquo;s not Python, it\u0026rsquo;s SQL.\nYour models are normal SQLAlchemy models. DBWarden adds an optional, fully typed class Meta for database-level metadata that SQLAlchemy models can\u0026rsquo;t naturally express:\nfrom sqlalchemy import Column, Integer, String, Text from sqlalchemy.orm import declarative_base from dbwarden.databases import TableMeta, IndexSpec Base = declarative_base() class User(Base): __tablename__ = \u0026#34;users\u0026#34; id = Column(Integer, primary_key=True) email = Column(String(255), unique=True, nullable=False) bio = Column(Text, nullable=True) class Meta(TableMeta): comment = \u0026#34;Core user accounts\u0026#34; indexes = [ IndexSpec(name=\u0026#34;ix_users_bio\u0026#34;, columns=[\u0026#34;bio\u0026#34;]), ] The class Meta convention is lifted straight from Django, deliberately. It isn\u0026rsquo;t a SQLAlchemy idiom, but Django proved that the stuff about a table which isn\u0026rsquo;t a column deserves a structured home on the model, and there was no reason to invent a worse name for it.\nA detail I care about: class Meta is validated at import time by a metaclass. If you write commet = \u0026quot;...\u0026quot; or my_engin = \u0026quot;InnoDB\u0026quot;, you don\u0026rsquo;t get silently wrong DDL three weeks later. You get a DBWardenConfigError the moment the module loads, naming the unknown attribute. Typos fail loudly and early.\nThen you run:\ndbwarden make-migrations \u0026#34;create users\u0026#34; And the output is a plain .sql file with both directions in it:\n-- upgrade CREATE TABLE IF NOT EXISTS users ( id INTEGER PRIMARY KEY, email VARCHAR(255) NOT NULL UNIQUE, bio TEXT ); COMMENT ON TABLE users IS \u0026#39;Core user accounts\u0026#39;; CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_users_bio ON users (bio); -- rollback DROP TABLE users; You review it, commit it, and apply it with dbwarden migrate. Any tool can execute it. Your DBA can read it without knowing Python. There is no migration runtime on the target machine, because there is nothing to run except SQL.\nNotice the small things. CONCURRENTLY on PostgreSQL index creation is the default, because locking a production table to build an index is a mistake you only make once. IF NOT EXISTS guards are emitted where they\u0026rsquo;re safe. These defaults exist because I got paged for their absence.\nAnd here\u0026rsquo;s the key structural difference: with DBWarden, migration files are artifacts, not authority. The models remain the source of truth forever. The .sql files are a record of how the database got here, useful for review and for sequential deploys, but the schema definition never leaves your models.\nGeneration: autogenerate vs make-migrations What this piece is: the generator. The thing that looks at your models, looks at your database, and writes the change for you.\nAlembic\u0026rsquo;s generator is alembic revision --autogenerate. It compares your model metadata against the live database and emits a candidate revision script. It\u0026rsquo;s genuinely useful, and if you use Alembic without it you\u0026rsquo;re doing unnecessary manual labor. This is also the point where people say \u0026ldquo;so Alembic is declarative too\u0026rdquo;. Not quite, and the difference matters. Three reasons.\nOne: the generated revision is still a Python script you own. You review it, you edit it, you commit it, and from that moment it\u0026rsquo;s part of the chain. The generator ran once; the artifact lives forever.\nTwo: the chain remains the source of truth. Your models are just an input to the generator. If the chain and the models disagree, the chain wins, and you find out at deploy time.\nThree: autogenerate has documented limits. The docs are honest about them, which I appreciate. The big one: renames. Autogenerate can\u0026rsquo;t tell a rename from a drop-and-create. Rename a column and the generated script drops your data unless you catch it in review. Some constraint changes go undetected entirely. The tool expects you to review and fix by hand, every time.\nDBWarden\u0026rsquo;s generator is the tool. There is no non-generated path for schema changes, so the generation had to be trustworthy enough to carry the whole workflow. A few things make that possible.\nSnapshots. After every migration, DBWarden writes a checksummed JSON snapshot of the full schema to .dbwarden/schemas/. Diffs run against snapshots, deterministically. The same models and the same state produce the same SQL, every time, on every machine.\nExplicit renames. Since a diff can\u0026rsquo;t prove a rename, DBWarden doesn\u0026rsquo;t guess. You declare it:\ndbwarden make-migrations \u0026#34;rename name to full_name\u0026#34; --rename users.name:full_name And you get ALTER TABLE users RENAME COLUMN name TO full_name, not a drop and a create. Table renames work the same way with --rename-table. Snapshot comparison also helps detect rename candidates. The point is that data-destroying ambiguity is never resolved silently.\nSafe type changes. --safe-type-change generates the boring-but-correct multi-step version of a column type change: add temp column, backfill, swap, drop old. You\u0026rsquo;ve written this dance by hand. Now it\u0026rsquo;s a flag.\nColumn-level precision. Type, nullability, default, and comment changes generate targeted ALTER COLUMN statements. And before anything is written, --plan shows you the migration plan as JSON, and --sql prints the raw SQL to stdout. You always get to look before anything exists.\nGoing back: downgrade() vs the rollback contract What this piece is: the escape hatch. The thing you run when a migration goes wrong at 2 AM.\nAlembic\u0026rsquo;s answer is the downgrade() function in each revision. The design is right: every change should know how to undo itself. The enforcement is where it gets weak. downgrade() is your responsibility. Nothing checks that it exists, works, or stays correct after you edit the script. And we\u0026rsquo;ve all seen this in real repos:\ndef downgrade(): pass An empty downgrade doesn\u0026rsquo;t fail review, doesn\u0026rsquo;t fail CI, and doesn\u0026rsquo;t fail at deploy. It fails at 2 AM, when you run it and nothing happens. The rollback path is the least-tested code in most codebases, and it\u0026rsquo;s the code you run under the most pressure.\nDBWarden\u0026rsquo;s answer is a contract. Every generated migration carries an executable -- rollback section, generated together with the upgrade. Placeholder rollback is refused by default: if DBWarden cannot emit executable rollback SQL, generation fails. If a change is genuinely irreversible (some ClickHouse engine changes, PostgreSQL enum value additions), you must say so explicitly, in the migration file:\n-- dbwarden: irreversible That line is visible in code review. Your reviewer sees \u0026ldquo;this migration cannot be undone\u0026rdquo; as a declared fact, not as an empty function nobody read. Rollback moves from convention to contract.\nOperationally you get dbwarden rollback and dbwarden downgrade to walk back, with --to-version targeting, and dbwarden make-rollback for generating rollbacks. The rollback SQL sits in the same reviewed, committed file as the upgrade. What you tested is what you run.\nApplying migrations: upgrade head vs migrate What this piece is: the execution step. Taking the pending changes and actually running them against a database, while keeping track of what has already run.\nAlembic\u0026rsquo;s answer is alembic upgrade head. It reads the alembic_version table in your database, finds where you are in the revision chain, and executes every upgrade() function between there and the newest revision. You can target a specific revision instead of head, step relatively with +1, and alembic stamp marks a revision as applied without running it, which is how you onboard a database that already has the schema.\nThis works well. My friction was never with the mechanics. It was with the runtime: applying migrations means running Python, which means the target environment needs your virtualenv, your dependencies, and your env.py to behave. The migration is not a thing, it\u0026rsquo;s a program.\nDBWarden\u0026rsquo;s answer is dbwarden migrate. Same job, different texture. It reads its migration table, finds pending .sql files, and applies them in order. The controls are all flags:\ndbwarden migrate # apply everything pending dbwarden migrate --count 2 # apply the next two dbwarden migrate --to-version 0007 # stop at a specific version dbwarden migrate --baseline --to-version 0005 # mark as applied without executing dbwarden migrate --all # every configured database, sequentially --baseline is the stamp equivalent, and it\u0026rsquo;s how you onboard an existing database. dbwarden history shows what ran and when. dbwarden status shows what\u0026rsquo;s pending. And because the artifacts are plain SQL, you always have the exit: take the .sql file and hand it to psql, to your DBA, or to whatever deployment machinery your company already trusts. The tool is convenient, not required. I consider that a feature. Lock-in through file formats is a tax, and SQL is the one format every database tool on earth can read.\nOne more apply-time detail. DBWarden also supports two special migration types besides versioned ones: runs_always migrations that execute on every migrate run, and runs_on_change migrations that re-execute when their content changes. Grants, refresh routines, and idempotent maintenance SQL finally get a home that isn\u0026rsquo;t a cron job.\nDrift: the silent killer What this piece is: drift is when the database\u0026rsquo;s actual schema no longer matches what your tooling believes. A hotfix applied by hand on a Friday. A migration that half-ran. A colleague\u0026rsquo;s \u0026ldquo;temporary\u0026rdquo; index from eight months ago.\nWith Alembic, the version table (alembic_version) records which revision the database is at. But it records which scripts ran, not what the schema is. If someone alters the database out-of-band, the version table still says everything is fine. Models say one thing, chain says another, database says a third. You typically discover the disagreement when the next migration fails in production, which is the worst possible moment.\nWith DBWarden, drift detection is structural, not optional. Every make-migrations run diffs the models against actual state, so out-of-band changes surface as unexpected diff entries at generation time, on your machine, not at deploy time in production. dbwarden status shows pending migrations, dbwarden check validates the setup, and dbwarden diff is a read-only comparison tool that outputs a Rich table, JSON, or raw SQL, so you can inspect exactly how reality differs from your models any time you get suspicious.\nA day in the life: the same change in both tools Abstract philosophy is nice. Here\u0026rsquo;s the same task, adding a bio column to users, done twice.\nWith Alembic:\nEdit the model: add bio = Column(Text, nullable=True). Run alembic revision --autogenerate -m \u0026quot;add bio\u0026quot;. Open the generated revision. Verify the upgrade() is right. Verify autogenerate didn\u0026rsquo;t misread anything else in your metadata as a change. Write or verify the downgrade(). Commit the model change and the revision script. Two files, both authoritative. Deploy runs alembic upgrade head with your Python environment on the target. With DBWarden:\nEdit the model: add bio = Column(Text, nullable=True). Run dbwarden make-migrations \u0026quot;add bio\u0026quot;. Open the generated .sql file. Read the ALTER TABLE users ADD COLUMN bio TEXT and the rollback below it. Commit the model change and the SQL artifact. One file is authoritative, the other is a receipt. Deploy runs dbwarden migrate, or your DBA runs the file, or your pipeline pipes it to psql. Both flows are short. The difference is in what each step asks of you. Alembic\u0026rsquo;s step 3 and 4 are verification and authorship duties that never go away, on every change, forever. DBWarden\u0026rsquo;s step 3 is reading SQL. On a five-person team shipping schema changes weekly, that delta compounds into real time and, more importantly, into real mistakes that never happen.\nOffline mode: CI without a database What this piece is: generating or planning migrations without a live database connection. This matters more than it sounds. If your migration tool needs a database to think, then every CI job needs a database service, seeded to the right state, just to answer \u0026ldquo;what would change?\u0026rdquo;.\nAlembic has an offline mode: alembic upgrade head --sql renders the SQL of pending revisions to stdout instead of executing them. It\u0026rsquo;s good for generating scripts a DBA will run. But generating a new revision with autogenerate still requires a live database to compare against, so CI pipelines that validate schema changes still need that Postgres service container.\nDBWarden decouples generation from the database entirely. You export the model state once:\ndbwarden export-models --database primary git add .dbwarden/model_state.primary.json Then, on any machine, with no database connection at all:\ndbwarden make-migrations \u0026#34;add bio column\u0026#34; --offline The state file is the reference point, and it updates in place after each migration. Your CI pipeline can generate and validate migrations with zero database services. Faster pipelines, no seeding scripts, no service containers. This is the feature the declarative model makes almost free, and it\u0026rsquo;s one of my favorites.\nBeyond parity: what the declarative choice unlocks Everything above maps DBWarden onto workflow pieces Alembic also has. This section covers the pieces that don\u0026rsquo;t map, because they only make sense when the tool fully understands your target schema.\nImpact analysis. Before a destructive change ships, dbwarden check-impact scans your codebase with AST analysis (grep fallback for templates) and tells you what still references the thing you\u0026rsquo;re about to destroy:\ndrop_column on users.username References: 2 app/routes/users.py:34 attribute_access app/templates/profile.jinja2:12 grep You know what breaks before it breaks. Dropping a column stops being an act of faith.\nSandbox validation. dbwarden migrate --sandbox replays your migrations in a throwaway database before they touch a real one. Be precise about what you get out of the box, though: core ships an in-memory SQLite provider, so on a PostgreSQL project the replay runs through SQL translation. That\u0026rsquo;s a smoke test, not a rehearsal. Installing the dbwarden-sandbox plugin registers a Testcontainers-backed provider instead, and then the replay happens against a real disposable PostgreSQL or ClickHouse, which is the version actually worth trusting. There\u0026rsquo;s also --dry-run to preview what would be applied, and --with-backup to snapshot before applying. I am the paranoid engineer these flags were built for.\nMulti-database, including analytics. One project can declare PostgreSQL, MySQL, ClickHouse, MariaDB, and SQLite databases, fully isolated, each with backend-specific Meta extensions: PGTableMeta for partitioning and RLS, MyTableMeta for engines and charsets, CHTableMeta for MergeTree engines and codecs. If you\u0026rsquo;ve ever managed a ClickHouse schema with shell scripts because your migration tool didn\u0026rsquo;t speak ClickHouse, this one\u0026rsquo;s for you.\nDev mode. Declare a dev_database_type and run SQLite locally against a PostgreSQL production schema, with automatic SQL translation. Local dev without a local Postgres.\nReverse engineering. dbwarden generate-models turns a live database into SQLAlchemy models, class Meta blocks included, with round-trip support. This is also the adoption path for legacy schemas: point it at the database you inherited and get models that regenerate it.\nPlugins. The core stays focused, and a plugin system extends it: seed data management (dbwarden-seeds), FastAPI integration (dbwarden-fastapi), Testcontainers sandboxes (dbwarden-sandbox), PostgreSQL RBAC, custom types and extensions, ClickHouse RBAC.\nWhere Alembic fits better This section matters more than the last one. I mean every word of it.\nPython data migrations. Alembic revisions are Python, and that\u0026rsquo;s a real superpower. You can backfill data with your own code, import your models, call your services, loop with real logic. DBWarden supports manual migration files via dbwarden new, but those are SQL. Plenty of backfills are expressible in SQL, and honestly SQL is often the better tool for them, but if your workflow leans on Python-level transformations inside migrations, Alembic serves you better. Full stop.\nBranching and merging. Alembic\u0026rsquo;s revision graph handles parallel branches and merge points. Two developers create revisions on separate branches, and alembic merge reconciles them. It\u0026rsquo;s a genuinely elegant design for large teams with many concurrent schema changes. DBWarden\u0026rsquo;s linear versioned flow doesn\u0026rsquo;t replicate it.\nEcosystem familiarity. Most Python tutorials, framework templates, and answers on the internet assume Alembic. Every SQLAlchemy developer you hire already knows it. That network effect has real onboarding value, and pretending otherwise would be dishonest.\nCompatibility. DBWarden requires Python 3.12+ and SQLAlchemy 2.0+. Alembic supports much older environments. If you\u0026rsquo;re pinned to Python 3.9, this decision has been made for you.\nManual control. If you want to hand-craft every migration step, an imperative tool is the honest choice. DBWarden derives SQL from your models; that\u0026rsquo;s the deal you\u0026rsquo;re making.\nCommon questions Is DBWarden a wrapper around Alembic? No. Zero shared code, zero shared runtime. It\u0026rsquo;s a from-scratch implementation of a different philosophy: diff engine, snapshot store, SQL generators per backend, safety classifier. The only thing the two tools share is SQLAlchemy models as an input.\nCan I keep using my existing models? Yes, unchanged. DBWarden reads normal SQLAlchemy models. The class Meta blocks are optional, additive metadata. You add them when you want comments, indexes, or backend-specific features that models can\u0026rsquo;t express, not before.\nHow do I move an existing Alembic project to DBWarden? Your database already has the schema, so you generate a baseline and mark it as applied. Point DBWarden at your models, run dbwarden make-migrations to produce the baseline migration, then dbwarden migrate --baseline so it\u0026rsquo;s recorded without executing. From there, new model changes generate new migrations. The full walkthrough lives in the migrating from Alembic guide. Your Alembic history stays in git; it just stops growing.\nCan the two coexist during a transition? They don\u0026rsquo;t fight. Alembic tracks its state in alembic_version, DBWarden in its own migration table. Run both while you gain confidence, generate the same change in each, and compare the SQL. When DBWarden\u0026rsquo;s output has earned your trust, retire the Alembic side. I\u0026rsquo;d keep the transition short, though. Two sources of truth is the exact disease we\u0026rsquo;re treating.\nWhat about databases Alembic doesn\u0026rsquo;t focus on? This is a real differentiator. DBWarden treats ClickHouse as a first-class backend, MergeTree engines and codecs included, next to PostgreSQL and MySQL. If your stack has an analytics database managed by hand-rolled scripts, one tool can now own both schemas.\nWhat does adopting DBWarden actually cost? Honestly: Python 3.12+ and SQLAlchemy 2.0+ as hard requirements, a new mental model for renames (explicit flags instead of editing generated scripts), and giving up Python-level logic inside migrations unless you write manual SQL files with dbwarden new. If any of those are dealbreakers, stay on Alembic. It will keep working fine.\nSo which one? Pick Alembic if you need Python data migrations inside revisions, depend on revision branching and merging, run older Python, or want manual control over every migration step.\nPick DBWarden if you want your models to be the single source of truth, want reviewable plain SQL artifacts, want rollbacks enforced by contract instead of convention, want to know a migration\u0026rsquo;s blast radius before deploying, or want migration generation in CI without a live database.\nAnd if you\u0026rsquo;re torn, here\u0026rsquo;s a practical tiebreaker. Look at your last ten migrations. If most of them are pure schema changes (columns, indexes, constraints, tables), you\u0026rsquo;re doing declarative work with imperative tooling, and DBWarden will remove a whole category of manual labor from your week. If half of them contain data backfills in Python, loops over rows, or calls into your application code, you\u0026rsquo;re doing genuinely imperative work, and Alembic is built for exactly that.\nBoth tools solve the same problem. They just ask you to maintain a different thing. Alembic asks you to maintain a history of changes. DBWarden asks you to maintain a description of the destination. I spent years maintaining histories. I\u0026rsquo;d rather maintain destinations. But that\u0026rsquo;s my bias, and now you know exactly where it comes from.\nIf you want to try the declarative side, the docs include a guide for migrating from Alembic to DBWarden. Your models stay exactly where they are. That\u0026rsquo;s the point.\nThis is the first post in a series comparing DBWarden with other schema tools. Next up: DBWarden vs Atlas and DBWarden vs Django migrations.\n","permalink":"https://blog.emiliano-go.com/works/dbwarden-vs-alembic/","summary":"I built DBWarden, so I\u0026rsquo;m biased. This is my honest comparison with Alembic anyway: every piece of the migration workflow, explained side by side, plus a clear list of cases where Alembic is still the right call.","title":"DBWarden vs Alembic: A Declarative Alembic Alternative for SQLAlchemy"},{"content":"Same disclaimer as the Alembic post: I built DBWarden, so I am biased. And this comparison is a strange one to write, because Atlas and DBWarden are on the same side of the big philosophical divide. Both are declarative. Both believe you should describe the schema you want, not the steps to get there. I have genuine respect for what the Atlas team has built; they\u0026rsquo;ve done more than almost anyone to push declarative schema management into the mainstream.\nSo this post can\u0026rsquo;t be \u0026ldquo;imperative vs declarative\u0026rdquo;. I already wrote that one, against Alembic. This one is about everything that\u0026rsquo;s left once two tools agree on philosophy: where the schema lives, what artifact gets applied, and who the tool is built for. Those differences turn out to be big.\nAs before: you might know one tool and not the other. So I\u0026rsquo;ll explain each piece properly before comparing. No assumed knowledge beyond \u0026ldquo;I have a database and I change its schema sometimes\u0026rdquo;.\nWhat Atlas is Atlas is a schema management tool written in Go, distributed as a single binary, built by Ariga. Its pitch is \u0026ldquo;manage your database schema as code\u0026rdquo;, and it takes that literally. You describe your schema in a dedicated representation, and Atlas owns the diffing, planning, and applying.\nThe schema source can be one of several things:\nHCL files, the same configuration language Terraform uses, with a schema-specific dialect: tables, columns, indexes, and foreign keys as HCL blocks. A table looks like this: table \u0026#34;users\u0026#34; { schema = schema.public column \u0026#34;id\u0026#34; { type = int } column \u0026#34;email\u0026#34; { type = varchar(255) null = false } primary_key { columns = [column.id] } index \u0026#34;idx_email\u0026#34; { columns = [column.email] unique = true } } Plain SQL files containing CREATE TABLE statements that describe the desired end state. An ORM, through integrations that load your ORM\u0026rsquo;s metadata (there are providers for various frameworks, including SQLAlchemy). Atlas then works in one of two modes, and understanding them is understanding Atlas:\nDeclarative mode (atlas schema apply): Atlas inspects the live database, compares it against your desired schema, generates a migration plan, shows it to you, and applies it directly. Terraform for databases, in the most direct sense. There are no migration files; the diff is computed and applied in one motion. Versioned mode (atlas migrate diff and atlas migrate apply): Atlas generates SQL migration files into a directory, and applies them in order. Classic versioned migrations, but with the planning automated from your declared schema. Around that core sits serious tooling: atlas schema inspect to reverse-engineer a live database, a migration linter with dozens of analyzers that flag destructive changes, table locks, and backward-incompatible changes, plus a commercial cloud product, a Terraform provider, and a Kubernetes operator. Atlas is a platform, and a polished one.\nIf you take one thing from this description, take this: Atlas is intentionally language-agnostic. It doesn\u0026rsquo;t care if your application is Go, Java, Node, Python, or a pile of bash scripts. The schema is its own first-class citizen, defined in Atlas\u0026rsquo;s terms, managed by Atlas\u0026rsquo;s workflow. That neutrality is a deliberate design decision, and it\u0026rsquo;s the root of almost every difference in this post.\nWhat DBWarden is DBWarden is a declarative migration and schema management tool for SQLAlchemy specifically. The one-line version: your SQLAlchemy models are your migrations. The desired schema state is not a new file format. It\u0026rsquo;s the models your application already imports:\nfrom sqlalchemy import Column, Integer, String, Text from sqlalchemy.orm import declarative_base from dbwarden.databases import TableMeta, IndexSpec Base = declarative_base() class User(Base): __tablename__ = \u0026#34;users\u0026#34; id = Column(Integer, primary_key=True) email = Column(String(255), unique=True, nullable=False) bio = Column(Text, nullable=True) class Meta(TableMeta): comment = \u0026#34;Core user accounts\u0026#34; indexes = [ IndexSpec(name=\u0026#34;ix_users_bio\u0026#34;, columns=[\u0026#34;bio\u0026#34;]), ] Configuration is one dbwarden.py file declaring your databases. The workflow is four commands:\ndbwarden init dbwarden make-migrations \u0026#34;create users\u0026#34; dbwarden migrate dbwarden status And the artifact is a plain .sql file carrying both the upgrade and an executable rollback:\n-- upgrade CREATE TABLE IF NOT EXISTS users ( id INTEGER PRIMARY KEY, email VARCHAR(255) NOT NULL UNIQUE, bio TEXT ); COMMENT ON TABLE users IS \u0026#39;Core user accounts\u0026#39;; CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_users_bio ON users (bio); -- rollback DROP TABLE users; DBWarden supports PostgreSQL, MySQL, and ClickHouse with full round-trip fidelity, SQLite for local development, and MariaDB at the schema layer. It requires Python 3.12+ and SQLAlchemy 2.0+. It is unapologetically a Python-ecosystem tool. Keep that sentence in mind; it decides most of this comparison. Where Atlas asks \u0026ldquo;what is your desired schema, in any language?\u0026rdquo;, DBWarden asks \u0026ldquo;why should a SQLAlchemy project describe its schema anywhere other than its models?\u0026rdquo;. Both questions are good. They just have different people in mind.\nWhere the schema lives The question: every declarative tool needs a description of the desired state. Where does that description live, and who maintains it?\nAtlas\u0026rsquo;s answer: in a dedicated schema representation. If you use HCL, your schema lives in .hcl files, and you edit those files to change it. If you use SQL as the source, your schema lives in .sql files of CREATE statements. Either way, the schema definition is a separate artifact with one job: being the source of truth. There\u0026rsquo;s real elegance in that separation. The schema stands alone, reviewable on its own, independent of any application.\nBut if you\u0026rsquo;re a Python shop with a SQLAlchemy application, notice what happened: you now have two representations again. Your SQLAlchemy models, which your application actually uses, and the Atlas schema files, which the database actually follows. The thing declarative tools were supposed to eliminate came back through a side door. Atlas knows this, which is why the ORM providers exist: they load your ORM metadata and hand it to Atlas as the desired state, so your models can drive the process. It works. But the ORM sits at the edge of the system, as one possible input among several, translated on the way in. The workflow, the config, and the concepts remain Atlas\u0026rsquo;s own.\nDBWarden\u0026rsquo;s answer: the schema lives in your models, full stop. There is no schema file, no HCL, no second representation at any distance. The class Meta blocks extend the models in place for things SQLAlchemy can\u0026rsquo;t express (comments, advanced indexes, backend-specific storage options), and they\u0026rsquo;re validated at import time: a typo in a Meta attribute raises DBWardenConfigError when the module loads, with the unknown attribute named. Because the models are the native input rather than a translated one, nothing gets lost in translation, and every SQLAlchemy construct your application depends on is exactly what the migration engine sees.\nThe trade is obvious and I\u0026rsquo;ll state it plainly: this only makes sense if you\u0026rsquo;re on SQLAlchemy. Atlas\u0026rsquo;s separation is what makes it polyglot. DBWarden\u0026rsquo;s integration is what makes it deep. Same design fork, opposite choices.\nThere\u0026rsquo;s a second-order effect of the model-native approach worth naming. When the schema is your models, schema review happens in the same diff as code review. The PR that adds the bio column shows the model change, the generated SQL, and the application code using the column, together, in one review. With an external schema representation, the schema change and the application change can land in different PRs, different repos, sometimes different teams, and keeping them synchronized becomes process instead of physics. Some organizations want that separation deliberately. Most small teams just inherit it as friction.\nThe runtime: a Go binary vs a Python package The question: what does the tool itself run on, and what does that mean day to day?\nAtlas ships as a single static Go binary. That\u0026rsquo;s a real strength: nothing to install beyond one file, no interpreter, no dependency conflicts, identical behavior on a laptop and in a scratch container. It\u0026rsquo;s part of why Atlas can serve every language community at once. The flip side for a Python team is that Atlas lives outside your environment. It can\u0026rsquo;t import your code, doesn\u0026rsquo;t participate in your virtualenv, and pins its understanding of your project to whatever the provider integration exports.\nDBWarden is a Python package: uv add dbwarden, and it\u0026rsquo;s in the same environment as your application. That costs you the single-binary neatness, and it means Python 3.12+ is a hard floor. What it buys is everything in this post that requires being inside: importing your models directly, validating class Meta at import time, walking your codebase\u0026rsquo;s AST for impact analysis, and plugins like dbwarden-fastapi that hook straight into your web framework. A tool outside the interpreter can see your schema. A tool inside it can see your project.\nNeither choice is wrong. They\u0026rsquo;re the same trade as the schema-location one, seen from the ops side: Atlas optimizes for universality, DBWarden for depth in one ecosystem.\nWhat gets applied: declarative apply vs versioned artifacts The question: when it\u0026rsquo;s time to change production, what actually happens?\nAtlas\u0026rsquo;s answer, mode one: atlas schema apply computes the diff and applies it, after showing you the plan. This is the purest form of declarative schema management, and for some environments (dev, ephemeral stacks, preview branches) it\u0026rsquo;s genuinely great. No files to manage at all.\nFor production, pure apply asks for a lot of trust. The plan you approve is computed at apply time, against whatever state the database is in at that moment. Many teams want the artifact frozen earlier than that: generated in a PR, reviewed by a human, tested in staging, and then applied to production byte-for-byte identical. Atlas agrees, which is why versioned mode exists and why Atlas\u0026rsquo;s own guidance points production users toward it. In versioned mode, atlas migrate diff writes SQL migration files, a checksummed directory tracks them, and atlas migrate apply executes them in order.\nDBWarden\u0026rsquo;s answer: there is only one mode, and it\u0026rsquo;s the reviewed-artifact one. Every change becomes a versioned .sql file at generation time, on your machine, in your PR. What was reviewed is what runs. I made this choice because in my experience the \u0026ldquo;reviewable frozen artifact\u0026rdquo; property is not a nice-to-have, it\u0026rsquo;s the thing that lets a team trust automation with their production schema at all.\nWithin that single mode you get the operational controls you\u0026rsquo;d expect: --dry-run to preview, --sandbox to replay migrations in a throwaway database first (in-memory SQLite with core, or a real containerized PostgreSQL or ClickHouse once the dbwarden-sandbox plugin is installed), --with-backup to snapshot before applying, --count and --to-version for partial applies, and --baseline to onboard databases that already have the schema. Plus two special migration types, runs_always and runs_on_change, for grants and idempotent maintenance SQL that versioned files don\u0026rsquo;t model well.\nThe honest framing: Atlas gives you two modes and lets you choose per environment. DBWarden gives you the production-safe mode only. If you want pure declarative apply for ephemeral environments, Atlas has a real feature DBWarden doesn\u0026rsquo;t.\nThe rollback story The question: every change should know how to undo itself. Who enforces that, and how?\nAtlas\u0026rsquo;s answer: in declarative mode, rollback is conceptually trivial: declare the old state and apply again; the diff engine works in any direction. In versioned mode, Atlas has migrate down tooling to revert applied migrations, computing or executing the reverse changes.\nDBWarden\u0026rsquo;s answer makes rollback part of the artifact itself. Every generated migration file carries a -- rollback section next to its -- upgrade section, generated at the same time, reviewed in the same PR, frozen together. Placeholder rollback is refused by default: if executable rollback SQL can\u0026rsquo;t be generated, generation fails unless you explicitly declare the migration irreversible with a -- dbwarden: irreversible marker, visible to your reviewer. The rollback you run in an incident is the one that sat in the PR, not one computed under pressure at 2 AM.\nThis is a difference in where correctness is enforced. Atlas leans on its engine being able to reverse states. DBWarden leans on the contract that no migration enters the repo without its tested exit path or an explicit confession that none exists. I trust artifacts more than engines when I\u0026rsquo;m paged. That\u0026rsquo;s a temperament, and I\u0026rsquo;ve built a tool around it.\nA day in the life: the same change in both tools Let\u0026rsquo;s make it concrete. Adding a bio column to users, in each tool\u0026rsquo;s production-oriented workflow.\nWith Atlas (versioned mode, HCL source):\nEdit the HCL: add a column \u0026quot;bio\u0026quot; block to the users table. If your application uses an ORM, edit the model too, so the application knows the column exists. Run atlas migrate diff add_bio with your dev database available. Atlas computes the change and writes a new SQL migration file into the directory. Review the generated SQL. The lint analyzers check the plan in CI. Commit and let atlas migrate apply run it against production. With DBWarden:\nEdit the model: add bio = Column(Text, nullable=True). There is no second place. Run dbwarden make-migrations \u0026quot;add bio\u0026quot;, offline if you want. Review the .sql file: the ALTER TABLE and its rollback, together. Commit and let dbwarden migrate run it against production. Both flows are sane. The structural difference is step 2 in the Atlas flow: if you have an ORM, the schema change happens twice, once for the database\u0026rsquo;s benefit and once for the application\u0026rsquo;s. The ORM providers can eliminate that step by making the models the schema source, and if you go that route the flows converge quite a bit. At which point the remaining question is which tool treats your models as its native language rather than one input dialect among several. That question is the next section.\nDrift: who notices when reality diverges The question: someone hotfixes production by hand on a Friday. When do you find out?\nAtlas\u0026rsquo;s answer: atlas schema diff compares any two schema states on demand, live databases included, so you can audit whenever you choose. Its versioned mode also maintains a checksummed migration directory, so tampering with migration files gets caught. Continuous, automatic drift monitoring is part of its cloud platform.\nDBWarden\u0026rsquo;s answer: drift detection is a side effect of the core loop, not a separate audit. Every make-migrations run diffs the models against actual state, so an out-of-band change shows up as an unexpected entry in the very next diff, on the very next developer\u0026rsquo;s machine. dbwarden diff gives you the on-demand comparison (as a Rich table, JSON, or raw SQL), and schema snapshots in .dbwarden/schemas/ are checksummed, so the recorded history can\u0026rsquo;t quietly rot either.\nNeither tool lets drift hide for long. The difference is posture: Atlas offers drift checking as a capability you invoke or subscribe to. DBWarden makes it a thing that happens to you, whether you thought about it or not. Nobody schedules an audit; the audit is a side effect of doing your normal job. For a small team without a platform engineer whose role includes remembering to audit, I prefer the ambush. For an organization with real platform discipline and dashboards someone actually watches, Atlas\u0026rsquo;s model scales further. Know which one you are before you weigh this section.\nSafety: linting the SQL vs knowing your codebase The question: how does the tool stop you from shipping a destructive change?\nAtlas\u0026rsquo;s answer: migration linting, and it\u0026rsquo;s excellent. Dozens of analyzers inspect planned changes for destructive operations, data-dependent changes that might fail on real data, table locks, and backward-incompatible changes. It runs in CI and blocks bad plans. This is one of Atlas\u0026rsquo;s strongest features, and I\u0026rsquo;ll say clearly: its analyzer coverage of the SQL side is broader than DBWarden\u0026rsquo;s today.\nDBWarden\u0026rsquo;s answer has a safety classifier for destructive operations too, plus generation-time defaults that encode operational scar tissue: CREATE INDEX CONCURRENTLY as the PostgreSQL default, --safe-type-change to expand a risky type change into the add-backfill-swap-drop sequence, explicit --rename flags so a rename can never silently become a drop-and-create.\nBut DBWarden\u0026rsquo;s distinctive safety feature looks in the other direction: at your application. dbwarden check-impact scans your codebase with AST analysis before a destructive migration ships:\ndrop_column on users.username References: 2 app/routes/users.py:34 attribute_access app/templates/profile.jinja2:12 grep A SQL linter can tell you that dropping a column is destructive. It cannot tell you that app/routes/users.py line 34 still reads that column. Because DBWarden lives inside the Python project, it can. Schema safety and application safety are the same problem, and only a tool that can see both sides can check both sides.\nCI and the dev database The question: what does the tool need in order to think? Plan a change, validate a PR, compute a diff.\nAtlas\u0026rsquo;s answer involves a clever concept called the dev database: a temporary, empty database (often spun up by Atlas itself) used as a scratchpad to normalize schemas and validate SQL. It\u0026rsquo;s how Atlas gets the database engine\u0026rsquo;s own opinion on your schema without touching production. It works well, but it means your workflow generally has a database around, even if ephemeral.\nDBWarden\u0026rsquo;s answer is snapshots and exported state. After every migration, a checksummed JSON snapshot of the schema lands in .dbwarden/schemas/. And dbwarden export-models writes a model state file you commit to git:\ndbwarden export-models --database primary git add .dbwarden/model_state.primary.json From then on, any machine can generate migrations with make-migrations --offline and no database at all. No service container in CI, no Docker socket, no scratchpad. The diff runs against committed state, deterministically: same models plus same state equals same SQL, on every machine. For local development there\u0026rsquo;s also dev mode, where you declare a dev_database_type and run SQLite locally against a PostgreSQL production schema with automatic SQL translation.\nEcosystem and the shape of the project The question: what are you buying into, beyond the binary?\nAtlas is a company\u0026rsquo;s flagship product. That brings real benefits: a commercial cloud offering with schema registries and deploy tracking, a Terraform provider, a Kubernetes operator, professional support, and a development pace funded by venture capital. It also brings the usual open-core reality: some capabilities live behind the paid tier, and the project\u0026rsquo;s direction follows the company\u0026rsquo;s.\nDBWarden is MIT-licensed open source with a plugin system. Core stays focused on migrations; official plugins cover seed data management (dbwarden-seeds), FastAPI integration (dbwarden-fastapi), Testcontainers sandboxes (dbwarden-sandbox), PostgreSQL RBAC, custom types, and extensions, and ClickHouse RBAC. The ClickHouse support deserves a highlight: MergeTree engine families, replicated engines, projections, dictionaries, materialized views, and codecs are modeled as first-class Meta metadata, and dbwarden generate-models reverse-engineers live ClickHouse databases into models. If your stack pairs Postgres with ClickHouse for analytics, one tool owns both schemas.\nWhere Atlas fits better I mean this section as sincerely as the equivalent one in the Alembic post.\nYou\u0026rsquo;re not a Python shop. This is the big one, and it\u0026rsquo;s not close. Atlas serves Go, Java, Node, and everything else. DBWarden requires SQLAlchemy. If your services are polyglot and you want one schema tool across all of them, Atlas is the right choice and DBWarden isn\u0026rsquo;t a candidate.\nYou want schema-as-code independent of any application. Some teams deliberately want the schema defined outside every codebase, as its own reviewed artifact with its own lifecycle. Atlas\u0026rsquo;s HCL/SQL schema sources are exactly that. DBWarden\u0026rsquo;s whole premise points the other way.\nYou want declarative apply for ephemeral environments. Preview databases, per-branch stacks, dev sandboxes: atlas schema apply with no migration files is a genuinely better fit there than generating versioned artifacts you\u0026rsquo;ll throw away.\nYou want deeper SQL linting. Atlas\u0026rsquo;s analyzer suite for planned SQL is broader than DBWarden\u0026rsquo;s safety classifier today. If CI-enforced SQL analysis is your top criterion, Atlas leads.\nYou want commercial support and platform integrations. Terraform provider, Kubernetes operator, cloud dashboard, a company answering the phone. DBWarden gives you an MIT license and a maintainer who cares. Those are different products.\nWhere DBWarden fits better You\u0026rsquo;re a SQLAlchemy shop. Then the calculus flips completely. Your models already exist and already describe your schema. DBWarden uses them directly, natively, with import-time validation, rather than through a translation layer into someone else\u0026rsquo;s workflow. No HCL to learn, no second artifact to maintain, no Go binary in a Python toolchain.\nYou want every change frozen as reviewable SQL with a rollback contract. One mode, always versioned, rollback refused unless executable or explicitly declared irreversible, in the file, in the PR.\nYou want safety checks that see your application. Impact analysis knows your routes and templates. A schema-side tool can\u0026rsquo;t.\nYou want CI with zero databases. Offline generation from committed state means no service containers anywhere in the pipeline.\nYou run ClickHouse next to your OLTP database. First-class analytics backend support in the same tool, same models, same workflow.\nCommon questions Is DBWarden just \u0026ldquo;Atlas for Python\u0026rdquo;? Tempting shorthand, but no. Atlas is a schema platform with its own representation and two apply modes. DBWarden is a migration tool that promotes your existing SQLAlchemy models to schema authority and only ever produces versioned SQL artifacts. The overlap is the declarative philosophy. The products are shaped very differently.\nAtlas has a SQLAlchemy provider. Doesn\u0026rsquo;t that make DBWarden redundant? It\u0026rsquo;s a fair challenge, and if the provider fits your team, use it. The difference is depth and direction. For Atlas, SQLAlchemy metadata is one supported input, translated into its own model of the schema, driving its own workflow and config. For DBWarden, SQLAlchemy is the native substrate: typed class Meta extensions validated at import time, backend metadata (PostgreSQL partitioning, MySQL engines, ClickHouse MergeTree families) declared per-model, reverse engineering back into models with generate-models, and application-aware impact analysis that reads your Python code. One tool speaks SQLAlchemy with an accent. The other thinks in it.\nDoes DBWarden have a declarative apply mode? No, and it won\u0026rsquo;t. Every change becomes a reviewed, versioned SQL file before it can touch a database. For ephemeral environments where that ceremony is pure overhead, Atlas\u0026rsquo;s schema apply is genuinely the better tool, and I\u0026rsquo;d rather tell you that than pretend.\nCan I migrate from Atlas to DBWarden? If your desired state is HCL or SQL files, apply them to a database, run dbwarden generate-models against it to get SQLAlchemy models with class Meta blocks, then generate a baseline and mark it applied with dbwarden migrate --baseline. If you were already using the SQLAlchemy provider, skip the first step: your models are ready, and DBWarden reads them as-is.\nWhat does DBWarden cost? Nothing. MIT license, no cloud tier, no feature gates. The trade is that there\u0026rsquo;s also no vendor: no support contract, no managed registry, no Terraform provider. Some teams need those. Atlas sells them, and that\u0026rsquo;s a legitimate reason to choose Atlas.\nSo which one? Here\u0026rsquo;s the honest decision tree. If your organization is polyglot, wants schema-as-code as an independent artifact, or wants a commercial platform: Atlas, and it\u0026rsquo;s not close. If your organization lives in Python on SQLAlchemy and wants the schema defined exactly once, in the models the application already imports, with SQL artifacts and a rollback contract: DBWarden, and it\u0026rsquo;s not close either.\nAnd if you\u0026rsquo;re somewhere in the middle, a mostly-Python team with one Go service, here\u0026rsquo;s my practical advice. Decide who owns each schema. Tools don\u0026rsquo;t need to be exclusive; they need clear jurisdictions. DBWarden owning the SQLAlchemy application\u0026rsquo;s databases while Atlas owns the standalone service\u0026rsquo;s database is a perfectly boring, perfectly functional arrangement. The failure mode isn\u0026rsquo;t using two tools. It\u0026rsquo;s two tools believing they own the same schema.\nThe interesting thing is how rarely the two tools actually compete head-on. Atlas made declarative schema management respectable across every ecosystem. DBWarden makes it native in one. If you\u0026rsquo;re reading this as a SQLAlchemy developer who tried Atlas and felt the impedance of maintaining schema files next to models, DBWarden was built for exactly that feeling. And if you\u0026rsquo;re a platform engineer wrangling six languages, close this tab and go install Atlas. I\u0026rsquo;ll say it so you don\u0026rsquo;t have to.\nThis is the second post in a series. The first, DBWarden vs Alembic, covers the imperative side of the divide. The third covers DBWarden vs Django migrations. And if you\u0026rsquo;re coming from Alembic, the docs have a step-by-step migration guide.\n","permalink":"https://blog.emiliano-go.com/works/dbwarden-vs-atlas/","summary":"Atlas and DBWarden agree on the philosophy: declare the schema you want, let the tool derive the changes. So this comparison is about everything else. Where the schema lives, what gets applied, and who each tool is really for.","title":"DBWarden vs Atlas: Two Declarative Schema Management Tools Compared"},{"content":"Same disclaimer as the rest of this series: I built DBWarden, so I am biased. And this post carries a debt on top of the bias. Django\u0026rsquo;s migration system is, as far as I\u0026rsquo;m concerned, the reason a whole generation of Python developers expects schema changes to flow from models. Django normalized the idea. If you\u0026rsquo;ve ever typed makemigrations and felt that small satisfaction of the framework just handling it, you already believe most of what DBWarden believes.\nSo this is not a takedown. It\u0026rsquo;s a family portrait. Django migrations and DBWarden sit on the same branch of the tree: both are model-driven, both generate changes instead of asking you to author them. The differences are in the artifacts, the coupling, and the guarantees. And as with the other posts in this series, I\u0026rsquo;ll explain each piece properly before comparing, because you probably know one tool much better than the other.\nOne scoping note up front. If you\u0026rsquo;re building a Django application, use Django migrations. Full stop, no asterisks. DBWarden is for SQLAlchemy, which means FastAPI, Flask, Litestar, plain scripts, data pipelines, and every Python service that isn\u0026rsquo;t Django. The comparison matters because so many of us learned migrations inside Django and then moved to SQLAlchemy stacks, looked around, and missed what we left behind.\nWhat Django migrations are For readers coming from the SQLAlchemy side who never used Django, here\u0026rsquo;s the system, properly. A little history helps too. Django didn\u0026rsquo;t always have migrations; for years the community relied on a third-party tool called South, and its ideas were folded into Django itself in version 1.7. That lineage matters because it means Django\u0026rsquo;s system was designed from a decade of real-world lessons about what model-driven migrations need: state reconstruction, dependency ordering, an interactive rename questioner. It\u0026rsquo;s a mature design.\nDjango models are Python classes declaring fields, and they are the canonical description of your schema:\nfrom django.db import models class User(models.Model): email = models.EmailField(unique=True) bio = models.TextField(blank=True, default=\u0026#34;\u0026#34;) class Meta: db_table = \u0026#34;users\u0026#34; When you change a model, you run python manage.py makemigrations. Django compares your models against its idea of the current schema and writes a migration file: a Python module containing operation objects.\nclass Migration(migrations.Migration): dependencies = [(\u0026#34;accounts\u0026#34;, \u0026#34;0003_user_last_login\u0026#34;)] operations = [ migrations.AddField( model_name=\u0026#34;user\u0026#34;, name=\u0026#34;bio\u0026#34;, field=models.TextField(blank=True, default=\u0026#34;\u0026#34;), ), ] Then python manage.py migrate applies pending migrations, recording them in a django_migrations table. You can migrate backwards by naming an earlier migration, view the SQL a migration would run with sqlmigrate, list state with showmigrations, and compress a long chain with squashmigrations.\nTwo design details matter for this comparison, and they\u0026rsquo;re clever, so let\u0026rsquo;s give them their due.\nFirst: the state is rebuilt from the migration files themselves. When makemigrations runs, Django doesn\u0026rsquo;t inspect your database. It replays your entire migration history in memory, reconstructs what the models should look like at the end of it, and diffs your actual models against that reconstruction. Elegant consequence: generating migrations needs no database connection at all. Important consequence: the migration chain, not the database, is what your models get compared against. Hold that thought.\nSecond: the interactive questioner. Rename a field and makemigrations asks you, in the terminal: \u0026ldquo;Did you rename user.name to user.full_name?\u0026rdquo;. Ambiguity gets resolved by a human at generation time. It\u0026rsquo;s a genuinely good piece of design that most tools never copied.\nAnd then there\u0026rsquo;s the feature everyone remembers: RunPython. A migration can carry arbitrary Python that runs inside the migration sequence, with access to historical versions of your models. Backfills, data transformations, splitting a column into two: Django lets schema and data changes interleave in one ordered history. It\u0026rsquo;s the system\u0026rsquo;s superpower, and I\u0026rsquo;ll come back to it in the fairness section.\nWhat DBWarden is DBWarden gives the same model-driven workflow to SQLAlchemy. Your models are the schema definition; the tool derives the rest: migration SQL, rollbacks, snapshots, and safety checks.\nConfiguration is one dbwarden.py file declaring your databases:\nfrom dbwarden import database_config primary = database_config( database_name=\u0026#34;primary\u0026#34;, default=True, database_type=\u0026#34;postgresql\u0026#34;, database_url_sync=\u0026#34;postgresql://user:pass@localhost:5432/myapp\u0026#34;, database_url_async=\u0026#34;postgresql+asyncpg://user:pass@localhost:5432/myapp\u0026#34;, ) Models are normal SQLAlchemy models, extended by an optional, typed class Meta that will feel immediately familiar to Django hands:\nfrom sqlalchemy import Column, Integer, String, Text from sqlalchemy.orm import declarative_base from dbwarden.databases import TableMeta, IndexSpec Base = declarative_base() class User(Base): __tablename__ = \u0026#34;users\u0026#34; id = Column(Integer, primary_key=True) email = Column(String(255), unique=True, nullable=False) bio = Column(Text, nullable=True) class Meta(TableMeta): comment = \u0026#34;Core user accounts\u0026#34; indexes = [ IndexSpec(name=\u0026#34;ix_users_bio\u0026#34;, columns=[\u0026#34;bio\u0026#34;]), ] Yes, the resemblance to Django\u0026rsquo;s class Meta is intentional. It\u0026rsquo;s a good idea and I stole it openly. Django proved that \u0026ldquo;the stuff about the table that isn\u0026rsquo;t a column\u0026rdquo; deserves a structured home on the model, instead of being scattered across config files and raw SQL. DBWarden\u0026rsquo;s version is typed and validated at import time: a metaclass checks every attribute, so misspelling one gets you a DBWardenConfigError naming it the moment the module loads, not silently wrong DDL three deploys later. And it goes deeper than Django\u0026rsquo;s, because it has to cover more ground: backend-specific subclasses like PGTableMeta expose PostgreSQL partitioning, row-level security, fillfactor, and tablespaces; MyTableMeta covers MySQL engines, charsets, and row formats; CHTableMeta covers ClickHouse engine specs. Column-level Meta classes exist too, so a comment or a storage option lives next to the column it describes. The index story alone covers partial indexes with WHERE clauses, covering indexes with INCLUDE columns, USING access methods, NULLS NOT DISTINCT, per-column sort order, and storage parameters, all as typed IndexSpec fields rather than strings you hope are spelled right.\nThe workflow rhymes with Django\u0026rsquo;s on purpose:\ndbwarden init dbwarden make-migrations \u0026#34;add bio to users\u0026#34; dbwarden migrate dbwarden status The lineage here is specific, so let me be precise about it. make-migrations is Django\u0026rsquo;s makemigrations with a hyphen, and migrate is the same word doing the same job. Those two are deliberate. The rest of the CLI is not Django\u0026rsquo;s: init, status, history, and rollback follow the conventions you\u0026rsquo;d recognize from Alembic and from general command-line tooling, not from startproject and showmigrations. So the workflow rhymes at the point where you generate and apply, which is the part you type every day, and diverges everywhere else.\nAnd the artifact that comes out is where the two systems really part ways, so let\u0026rsquo;s get into the actual differences.\nThe artifact: Python operations vs plain SQL What this piece is: the file that gets generated, reviewed, committed, and executed.\nDjango\u0026rsquo;s artifact is Python: a list of operation objects (AddField, AlterField, RunPython) that Django\u0026rsquo;s engine translates into SQL for your backend at execution time. You can preview the SQL with sqlmigrate, but the SQL is a rendering, produced on demand. The thing in your repo, the thing your reviewer reads, is the operations. Executing a migration means running Django.\nDBWarden\u0026rsquo;s artifact is the SQL itself:\n-- upgrade ALTER TABLE users ADD COLUMN bio TEXT; -- rollback ALTER TABLE users DROP COLUMN bio; What you review is what will run, byte for byte. Your DBA can read it without knowing Python. Your deploy pipeline can execute it with psql if it wants to; the dbwarden migrate runner is convenient, not required. And on PostgreSQL the generated SQL carries operational defaults like CREATE INDEX CONCURRENTLY, because index builds that lock production tables shouldn\u0026rsquo;t require remembering a keyword.\nDjango\u0026rsquo;s abstraction has real benefits: backend portability of the operation objects, and the ability to embed Python (again: RunPython). The cost is a layer between review and reality. You approve an AlterField and trust the rendering. Most of the time that trust is fine. The times it isn\u0026rsquo;t are the times you learn to read sqlmigrate output very carefully.\nSource of truth and drift: what gets compared against what What this piece is: the reference point. When the tool generates a change, what does it diff your models against? The answer decides when you find out about drift.\nDjango diffs models against the state reconstructed from the migration chain. The database is not consulted at generation time. This is what makes makemigrations work offline, and it\u0026rsquo;s elegant. But it means the migration history is the effective source of truth for what the schema \u0026ldquo;is\u0026rdquo;, and the actual database is trusted to match it. If someone alters production by hand, Django\u0026rsquo;s tooling has no moment where it would notice. Models agree with the chain, the chain believes it was applied, and the database quietly disagrees with both until a migration fails or a query breaks.\nDBWarden diffs models against actual state: the live database, or the checksummed schema snapshots it writes to .dbwarden/schemas/ after every migration, or an exported model-state file. The models are the authority, and reality is the thing being measured. Out-of-band changes surface as unexpected diff entries the very next time anyone runs make-migrations. There\u0026rsquo;s also dbwarden diff as a dedicated read-only comparison (Rich table, JSON, or raw SQL output) and dbwarden check-db for connectivity and schema validation when you\u0026rsquo;re suspicious.\nNeither design is careless; they optimize for different failure modes. Django optimizes for a world where all changes flow through migrations, and in a disciplined Django team that\u0026rsquo;s largely true. DBWarden assumes the world where hotfixes happen, because I\u0026rsquo;ve lived in that world, and I wanted the tool that notices.\nRollback: reverse operations vs a contract What this piece is: going backwards, on purpose, under pressure.\nDjango\u0026rsquo;s answer: most schema operations know their own reverse, so migrate accounts 0003 walks back automatically. Good design. The gaps appear at the edges: RunPython needs an explicitly provided reverse function or the migration is irreversible, irreversibility surfaces as an IrreversibleError when you attempt the rollback, and nothing in code review shows you the rollback path, because it doesn\u0026rsquo;t exist as an artifact. It\u0026rsquo;s computed when needed.\nDBWarden\u0026rsquo;s answer: rollback is part of the generated file. Every migration carries an executable -- rollback section, produced together with the upgrade, reviewed in the same PR. Placeholder rollback is refused by default: if executable rollback SQL can\u0026rsquo;t be generated, generation fails unless the migration explicitly declares itself irreversible with a -- dbwarden: irreversible marker. Your reviewer sees the escape route, or sees the declared absence of one, before merge. At incident time, dbwarden rollback and dbwarden downgrade execute exactly the SQL that sat in the repo.\nThe distinction is when you learn a change can\u0026rsquo;t be undone. Django tells you when you try. DBWarden tells you when you generate. I\u0026rsquo;ve been on the wrong end of the first timing, and it\u0026rsquo;s why the second one exists.\nRenames: the questioner vs explicit flags Renames deserve their own section because they\u0026rsquo;re where diff-based tools destroy data.\nDjango detects a likely rename and asks you interactively at generation time. Human answers, correct migration gets written. The limitation is the interactivity itself: in scripts and CI there\u0026rsquo;s no one to answer, and the heuristic needs a plausible before-and-after to even ask.\nDBWarden makes renames a declaration instead of a dialogue:\ndbwarden make-migrations \u0026#34;rename name\u0026#34; --rename users.name:full_name dbwarden make-migrations \u0026#34;rename orders\u0026#34; --rename-table order:orders You get RENAME DDL, never a drop-and-create, and the intent is recorded in your shell history and your migration description rather than in an answered prompt nobody can audit later. Snapshot comparison helps flag rename candidates too. Same goal as Django\u0026rsquo;s questioner, different medium: explicit flags work in automation and leave a trace.\nApplying and tracking: migrate vs migrate What this piece is: execution and bookkeeping. Running what\u0026rsquo;s pending, knowing what ran.\nDjango records applied migrations in the django_migrations table. python manage.py migrate applies everything pending across all apps, resolving cross-app dependencies into a valid order. showmigrations displays the checklist. Targeting an earlier migration walks backwards. --fake marks migrations as applied without running them, which is how you adopt the system on a database that already has the schema. --plan previews what would run. It\u0026rsquo;s a complete, polished toolkit, refined over a decade of releases.\nDBWarden keeps the same vocabulary where it can, because familiarity is a feature. dbwarden migrate applies pending migrations in version order and records them in its migration table. dbwarden status is your checklist, dbwarden history the audit trail. --baseline is the --fake equivalent for onboarding existing databases, --dry-run is the preview, and --count or --to-version control how far to go. --all runs every configured database sequentially, which has no Django equivalent because Django migrates one database\u0026rsquo;s worth of apps at a time.\nDBWarden adds one concept Django doesn\u0026rsquo;t have: migration types. Besides normal versioned migrations, runs_always migrations execute on every migrate run, and runs_on_change migrations re-execute whenever their file content changes. Grants, permission refreshes, and idempotent maintenance SQL usually end up in a Django RunPython that checks its own state, or in a cron job. Here they\u0026rsquo;re just files with a different prefix, tracked like everything else.\nCoupling: a framework feature vs a standalone tool What this piece is: what you must adopt to use each system.\nDjango migrations require Django. Not just the ORM: the app registry, the settings module, the management commands. That\u0026rsquo;s not a flaw; it\u0026rsquo;s the point. Django is an integrated framework and its migration system is one of the rewards for buying in. But it also means the system is unavailable to everyone outside. There\u0026rsquo;s no practical way to bring makemigrations to a FastAPI service on SQLAlchemy.\nDBWarden requires SQLAlchemy, and nothing else. No framework, no app structure, no settings module. FastAPI, Flask, Litestar, a queue worker, a cron script: if it has SQLAlchemy models, it can have this workflow. There\u0026rsquo;s even an official dbwarden-fastapi plugin providing session dependencies and health endpoints for the most common pairing, and the config declares both sync and async database URLs because modern SQLAlchemy stacks are usually async at runtime even when their tooling isn\u0026rsquo;t. This is the gap DBWarden exists to fill. The number of people who left Django for FastAPI and then discovered that \u0026ldquo;migrations\u0026rdquo; now meant maintaining revision scripts by hand is large. I was one of them. The workflow I missed wasn\u0026rsquo;t complicated. Change the model, generate, review, apply. It just didn\u0026rsquo;t exist outside the framework.\nStandalone also changes the local development story. Django assumes you run the same database engine locally that you run in production, or at least it leaves the mismatch to you. DBWarden has dev mode: declare a dev_database_type of SQLite in your config and develop locally against your PostgreSQL production schema, with automatic SQL translation between the dialects. No local Postgres container just to hack on a side feature. When the translation can\u0026rsquo;t be faithful, it tells you instead of guessing.\nMulti-database is also broader than Django\u0026rsquo;s story. Django can route between several relational databases, but its migration system targets Django-supported backends. DBWarden declares multiple databases in one project with full isolation, and treats ClickHouse as a first-class backend: MergeTree engine families, codecs, projections, and materialized views declared in class Meta (CHTableMeta), with the same migration workflow as your PostgreSQL database. MySQL and MariaDB get their own typed metadata too. Your OLTP schema and your analytics schema, one tool.\nA day in the life: the same change in both worlds Adding that bio column, end to end.\nDjango:\nAdd bio = models.TextField(blank=True, default=\u0026quot;\u0026quot;) to the model. Run python manage.py makemigrations. Skim the generated 0004_user_bio.py, maybe run sqlmigrate to see the SQL. Commit. Deploy runs python manage.py migrate. DBWarden:\nAdd bio = Column(Text, nullable=True) to the model. Run dbwarden make-migrations \u0026quot;add bio\u0026quot;. Read the .sql file: upgrade and rollback, together, exactly as they\u0026rsquo;ll execute. Commit. Deploy runs dbwarden migrate, or anything else that can execute SQL. The flows are nearly identical, and that\u0026rsquo;s the compliment: DBWarden is deliberately the same shape as the workflow Django proved out. The differences live in what you reviewed in step 3 (operations vs final SQL, with the rollback visible), and in what step 4 requires (a Django process vs anything).\nNow stretch the example one step: two weeks later, product wants bio renamed to about. In Django, makemigrations notices the disappearance and the appearance, asks you \u0026ldquo;Did you rename user.bio to user.about?\u0026rdquo;, and writes a RenameField. In DBWarden, you pass --rename users.bio:about and get a RENAME COLUMN statement plus its reverse in the rollback section. Same outcome, different interface: a question answered in a terminal versus a flag recorded in the migration\u0026rsquo;s provenance. And in both systems, the naive path (delete the field, add a new one, don\u0026rsquo;t tell the tool) produces a data-destroying drop-and-create, which is why both systems built an answer here at all.\nWhat DBWarden adds beyond the Django playbook A few capabilities have no Django-side equivalent, because they come from choices Django didn\u0026rsquo;t need to make.\nImpact analysis. Before a destructive migration ships, dbwarden check-impact scans your codebase with AST analysis and reports what still references the doomed column or table, file and line included. Django\u0026rsquo;s rough analog is grepping and hoping. This exists because DBWarden lives inside your Python project and can read it.\nSandbox and paranoia flags. dbwarden migrate --sandbox replays migrations in a throwaway database first, using an in-memory SQLite provider in core and a real containerized database once the dbwarden-sandbox plugin is installed; --dry-run previews; --with-backup snapshots before applying.\nOffline state for CI. Like Django, DBWarden can generate without a live database, via dbwarden export-models and make-migrations --offline against a committed state file. Unlike Django, the reference state is checksummed and explicitly managed, with recover-model-state for repair.\nReverse engineering. dbwarden generate-models converts an existing live database (PostgreSQL, MySQL, ClickHouse, SQLite) into SQLAlchemy models with class Meta metadata filled in. Django\u0026rsquo;s inspectdb does something similar for Django models; DBWarden\u0026rsquo;s version round-trips, meaning the generated models regenerate the same schema.\nSafe type changes. --safe-type-change expands a column type change into the add-backfill-swap-drop sequence instead of a naive ALTER.\nThe state question: implicit chain vs explicit files One more structural difference, subtle but worth understanding before you choose.\nDjango\u0026rsquo;s schema state is implicit. It exists only as the sum of the migration chain, recomputed in memory every time. There\u0026rsquo;s no file you can point at and say \u0026ldquo;this is what Django thinks the schema is\u0026rdquo;. That\u0026rsquo;s tidy, but it has a consequence: the migration files become load-bearing forever. Delete one and the reconstruction breaks; every file in the chain must remain importable and correct for the lifetime of the project. Squashing exists precisely to manage the weight of that ever-growing chain.\nDBWarden\u0026rsquo;s schema state is explicit. Checksummed snapshot files in .dbwarden/schemas/ record the schema after each migration, and the exported model state file (.dbwarden/model_state.primary.json, named after the database) carries the reference point for offline generation. Because the truth lives in the models and the state files, old migration files are safely deletable. They\u0026rsquo;re receipts, not structure. The flip side is a real operational rule: the model state file must never be deleted carelessly, and the docs are loud about it. If it goes missing, you restore it from git or regenerate with export-models, and dbwarden recover-model-state exists for repair. Explicit state means state you can also mishandle. I\u0026rsquo;ll take that trade, because explicit state is state you can inspect, diff, and back up, but it\u0026rsquo;s a trade, and you should know you\u0026rsquo;re making it.\nWhere Django migrations fit better Sincerely, as always.\nYou\u0026rsquo;re building a Django app. Then this entire post was theoretical. Django\u0026rsquo;s migrations are integrated with the admin, the test runner, the app ecosystem, and thousands of third-party packages. Using anything else inside Django would be self-harm. Don\u0026rsquo;t.\nRunPython data migrations. Python code interleaved into the migration sequence, with access to historical model states, is something DBWarden does not replicate. DBWarden\u0026rsquo;s dbwarden new creates manual migration files, but they\u0026rsquo;re SQL. A lot of backfills express fine in SQL, and SQL backfills are often faster. But \u0026ldquo;run this Python against the old schema shape\u0026rdquo; is a Django capability, full stop.\nThe interactive questioner. For a developer at a terminal, being asked about a rename is friendlier than knowing the flag. DBWarden chose auditability and automation-friendliness; Django chose conversational UX. Both are defensible, and Django\u0026rsquo;s is more welcoming.\nSquashing. squashmigrations compresses years of history into a compact restatement. DBWarden\u0026rsquo;s model makes old migration files safely deletable (the models and snapshots hold the truth), which addresses the same pain differently, but Django\u0026rsquo;s explicit squash tooling is more established.\nApp-scoped migration graphs with dependencies. Django\u0026rsquo;s per-app chains with cross-app dependency declarations handle a large modular monolith\u0026rsquo;s ordering problems elegantly. DBWarden\u0026rsquo;s per-database versioned sequences are simpler, which is a benefit right up until you need that graph.\nCommon questions Can I use DBWarden inside a Django project? Technically nothing stops you from having SQLAlchemy models next to Django, but you shouldn\u0026rsquo;t. Django\u0026rsquo;s migrations are woven into its ORM, its test framework, and its ecosystem. DBWarden is for stacks where SQLAlchemy is the ORM. Using both ORMs in one project is a decision you\u0026rsquo;d have to justify on other grounds entirely, and I won\u0026rsquo;t help you justify it.\nIs DBWarden \u0026ldquo;Django migrations for FastAPI\u0026rdquo;? As an elevator pitch, honestly, yes. That\u0026rsquo;s the itch. FastAPI\u0026rsquo;s own documentation pairs it with SQLAlchemy, and the migration story has historically been \u0026ldquo;set up Alembic\u0026rdquo;. DBWarden replaces that with the generated, model-driven flow Django users expect, and the dbwarden-fastapi plugin adds session dependencies and health endpoints on top. But the pitch undersells the differences: SQL artifacts instead of Python operation files, the rollback contract, impact analysis, and ClickHouse support have no Django equivalent. It\u0026rsquo;s the same genus, not the same species.\nI\u0026rsquo;m migrating a Django app to FastAPI. What happens to my schema? This is a surprisingly common situation and it\u0026rsquo;s well supported. Your database already exists, so let DBWarden read it: dbwarden generate-models produces SQLAlchemy models from the live schema, class Meta blocks included, with a --base flag to use your project\u0026rsquo;s own declarative Base. Then generate a baseline migration and mark it applied with dbwarden migrate --baseline. Your django_migrations table becomes an archaeological artifact. It hurts nothing; it just stops mattering.\nDoes DBWarden understand Django-style app structure? There\u0026rsquo;s no concept of apps, because SQLAlchemy has no concept of apps. Model discovery is automatic, and model_paths in your database_config pins down which modules to scan when you want explicit control. In a project with several databases, model_tables assigns tables to databases, which covers the main thing Django\u0026rsquo;s app-scoping actually buys at migration time.\nWhat about testing migrations? Django\u0026rsquo;s test runner builds the test database from your migrations automatically, which is a quiet, excellent feature. DBWarden\u0026rsquo;s equivalents are explicit: dbwarden migrate --sandbox replays migrations in a temporary database, and the dbwarden-sandbox plugin provides Testcontainers-based sandbox providers so your test suite can spin up a real PostgreSQL or ClickHouse, apply the migration chain, and verify it converges. Different ergonomics, same assurance.\nSeeds and fixtures? Django has fixtures and data migrations. DBWarden splits the concern into a dedicated plugin, dbwarden-seeds, with code-based and file-based SQL or Python seeds, tracked in their own table, applied with dbwarden seed apply or automatically after migrations if you configure auto_apply_seeds. Seed data and schema migrations are related but different problems, and keeping them separate keeps both simple.\nSo which one? If you\u0026rsquo;re in Django: Django migrations. That\u0026rsquo;s the whole answer, and any tool author who tells you otherwise is selling something.\nIf you\u0026rsquo;re on SQLAlchemy: you never actually had the Django option. Your real choices are Alembic\u0026rsquo;s revision scripts, Atlas\u0026rsquo;s schema-as-code, or DBWarden. And if what you miss is specifically the Django feeling, models as the single source of truth, migrations as something generated rather than authored, a typed class Meta for the database details, then DBWarden is the closest thing to makemigrations your stack can get, with two upgrades earned along the way: artifacts you can read as plain SQL, and rollbacks enforced as a contract instead of computed on demand.\nDjango taught us that developers shouldn\u0026rsquo;t hand-write schema changes. It proved the model-driven workflow at a scale nobody can argue with, inside one framework. DBWarden\u0026rsquo;s whole premise is that the lesson was bigger than the framework, and that SQLAlchemy users, which today means most of the Python web world outside Django, deserve it too. If you\u0026rsquo;ve spent years typing makemigrations and recently found yourself hand-editing a revision script at midnight, wondering how the ecosystem went backwards: it didn\u0026rsquo;t. The workflow just hadn\u0026rsquo;t been ported yet.\nThis closes the series. The other posts: DBWarden vs Alembic on the imperative-vs-declarative divide, and DBWarden vs Atlas on two declarative tools with very different shapes. If you\u0026rsquo;re ready to try it, start with the docs, and if you\u0026rsquo;re coming from Alembic, there\u0026rsquo;s a step-by-step migration guide.\n","permalink":"https://blog.emiliano-go.com/works/dbwarden-vs-django-migrations/","summary":"Django migrations and DBWarden agree that models should drive schema changes. This post explains how each one does it, where the artifacts differ, and why a FastAPI developer missing makemigrations is exactly who DBWarden was built for.","title":"DBWarden vs Django Migrations: Bringing makemigrations to SQLAlchemy"},{"content":"Every Docker Compose stack I’ve ever operated has suffered from the same quiet flaw: when a container fails, the rest of the fleet is blind to it. The database crashes, and the API keeps hammering it with writes, piling errors into logs. A background worker runs out of memory, and the frontend still shows a green checkmark. Docker restarts the dead container (if you’ve asked it nicely) but no one tells the neighbours what happened, why, or when the danger has passed.\nWe’ve all built workarounds. A healthcheck endpoint here, a hand‑rolled event watcher there, maybe a webhook to Slack that some intern wrote in an afternoon. But these are brittle, one‑way, and entirely unaware of recovery. They can tell you something is broken; they can’t tell you it’s fixed. The fleet remains deaf, and every service reinvents its own primitive ears.\nI wanted something more than an alarm bell. I wanted a shared, semantic language of distress and recovery, a signalling layer that every container could understand, in any language, without dependencies. And I wanted it to heal itself when things went wrong, the way living systems do. The answer, it turned out, was already inside us.\nBorrowing from biology In your body, when a cell is infected by a virus, it releases small proteins called interferons. These proteins don’t fight the virus directly. They diffuse to neighbouring cells and bind to receptors on their surface, triggering those cells to raise their antiviral defences, slowing protein synthesis, activating immune cells, hardening their membranes. Once the infection is cleared, the signalling stops. The tissue returns quietly to its resting state. That elegant loop - detect a threat, warn the neighbours, prepare a measured response, and stand down when it’s over - is exactly what a container fleet lacks.\nInterferon (the project) borrows this loop whole. A sidecar process watches the Docker daemon and, when a container dies, degrades, or encounters a critical error, it emits a typed signal: db_down, api_degraded, worker_oom. It doesn’t restart the container; it doesn’t manage anything. It just says, clearly and immediately, “this cell is in trouble, and here’s why.” Every other container that cares can listen and adapt: the API switches to read‑only mode, the frontend shows a warning banner, the job queue throttles its workers. No one needed to know about the database directly; they simply respond to the signal they’ve evolved to understand.\nBut the most important half of this loop is the one we usually forget: the all‑clear. Real interferons don’t linger forever. Interferon’s state machine tracks every container from HEALTHY to DOWN, then through RECOVERING and back to HEALTHY. Each transition broadcasts a new signal - \u0026lt;role\u0026gt;_recovering, \u0026lt;role\u0026gt;_healthy - so the fleet can relax when the danger has passed. Signals can even carry a time‑to‑live: if a container emits db_degraded because of replica lag, but then the lag clears and the signal isn’t renewed, Interferon automatically sends a db_degraded_expired signal. The immune response self‑resolves. The organism returns to baseline.\nThe cell that sacrifices itself The design goes deeper. If Interferon is an immune system, it must itself be immune to failure. A signalling layer that can go silent and stay silent is worse than no signalling layer at all. So the specification includes a durability contract that sounds almost biological: the watcher will never die due to its own logic. When it does die from external causes, it comes back seamlessly, in seconds, with correct state. That’s achieved through careful internal supervision, crash‑only design, and a persistent state store that remembers what Docker cannot: things like programmatic degradation signals that no Docker event ever produced.\nBut there is one failure that no process can detect from inside itself: its event loop might hang. The code that would notice is frozen. In the body, a cell that becomes dangerously dysfunctional undergoes apoptosis, programmed cell death. It dismantles itself quietly, packaging its contents for cleanup, without triggering inflammation. The tissue never even notices.\nInterferon borrows this too. A separate, minimal timer thread watches the main event loop, and if it stops ticking, the watcher calls os._exit(): it kills itself on purpose. Docker’s restart policy then brings a brand‑new instance back up, which reconciles state and resumes signalling. From the fleet’s perspective, the immune system blinked and re‑woke, and the danger signals were never lost. No external watchdog is required (though one is available for the truly paranoid). The watcher is designed to treat its own death as a healing strategy. That is pure apoptosis, and it turns what could be a catastrophic hang into a brief, self‑repairing hiccup.\nWhy the metaphor matters I could have called this project docker-event-bus and named the pieces relay, subscriber, and filter. But that would have missed the entire point. A bare event bus tells you that a container stopped. It doesn’t tell you why it matters, what to do about it, or when you can stop worrying. The biological metaphor forces the design to answer those questions. It gives you a vocabulary - receptors, recovery, TTL, apoptosis - that makes the system’s behaviour predictable and its purpose obvious. When I explain to an engineer why the watcher self‑terminates, I say “apoptosis,” and they nod. The metaphor carries the load of justification. It also acts as a design compass: “What should happen when a TTL signal expires? Well, what happens when an interferon degrades?”\nInterferon isn’t just a tool; it’s a philosophy. It says that every container in a Compose stack should be part of a single, self‑healing organism. That the fleet should feel pain and signal it, then heal and announce that too. That infrastructure should be immunologically literate: tolerant enough not to overreact to a 3‑second hiccup, but fast and decisive when a true threat appears.\nWhat Interferon is and isn\u0026rsquo;t Interferon is not an orchestrator. It will never restart your containers, scale them, or replace Kubernetes. It’s not a monitoring tool, a metrics history, or a replacement for Prometheus. It’s a signalling layer: it detects, it classifies, it broadcasts, and it tracks recovery. Your containers remain fully in control of how they respond. All Interferon does is give them the shared language they’ve been missing.\nIt’s also deliberately single‑host. A single Docker daemon, a single watcher, a single immune system per host. If you need cross‑node signalling, you’re in the land of service meshes and multi‑host orchestrators, and that’s fine, but it’s not this. Interferon is for the thousands of us who run production Compose stacks on a beefy VM and wish the pieces could talk to each other without us writing glue scripts.\nA preview of what’s being built Interferon will ship as two artifacts: a small sidecar image that does the watching and broadcasting, and a Python SDK that makes it trivial for any container to subscribe to signals and register handlers. But the transport is plain HTTP SSE, so a non‑Python service can listen with nothing more than curl. The signal schema is stable JSON. The watcher is self‑contained, supervised internally, and backed by a SQLite state store that remembers what Docker cannot. It will have debounce and circuit‑breaker logic to stop a transient blip from cascading into a fleet‑wide panic, because an overreacting immune system is an autoimmune disease, and we’ve all seen a test environment tear itself apart.\nI’m building the prototype now: the watcher, the SSE server, the state machine that walks a container from HEALTHY to DOWN to RECOVERING to HEALTHY, and the apoptosis logic that makes the whole thing self‑healing. The first demo will simulate a database crash, watch the API receive a db_down signal and switch to read‑only, then see the database recover and emit db_recovering and db_healthy, all without a human touching a keyboard.\nIf that demo feels as natural as I think it will, the rest follows. A Redis transport for production‑grade durability. A tiny watchdog for those who want an external executioner. Prometheus metrics. And eventually, custom probes that turn latency thresholds and HTTP healthcheck results into the same typed signals the fleet already speaks.\nInterferon is open source and in its earliest days. If you’ve ever stared at a Compose file and wished your containers could simply tell each other when they were hurting, you understand the gap. If you’ve ever written a bash script that polls docker ps and sends a webhook, you know the pain. I think we can do better. I think we can give our container fleets a real immune system. Not a crude alarm, but a nuanced, self‑limiting, self‑healing signalling layer that borrows from four billion years of evolution.\nThe repository is scaffolding right now (at the time of writing), but the you\u0026rsquo;re free to AMA about the spec, and the ideas are ready for critique. If you’re a systems thinker who gets excited about the marriage of biology and infrastructure, I’d love your eyes on it. Let’s build the immune system your Docker host deserves.\n","permalink":"https://blog.emiliano-go.com/works/interferon_project/","summary":"\u003cp\u003eEvery Docker Compose stack I’ve ever operated has suffered from the same quiet flaw: when a container fails, the rest of the fleet is blind to it. The database crashes, and the API keeps hammering it with writes, piling errors into logs. A background worker runs out of memory, and the frontend still shows a green checkmark. Docker restarts the dead container (if you’ve asked it nicely) but no one tells the neighbours what happened, why, or when the danger has passed.\u003c/p\u003e","title":"The Immune System Your Docker Host Deserves"},{"content":"Studying, the art of learning, and, for me, an amazing experience.\nI\u0026rsquo;ve always considered me a Filomath, as I\u0026rsquo;ve been curious about everything and anything for as long as I can remember.\nWhen I was younger, I was really into Astronomy. My grandfather was (and is) an amateur astronomer, and studied astronomy for years in college, even tough he was a doctor. I\u0026rsquo;ve learned a lot of what I know about astronomy from him, but I\u0026rsquo;ve also studied it on my own. Did you know that, hypothetically, we could all vanish in about 8 minutes if even a tiny spec of rare matter collided with earth? Fun, right?\nAnyway, as I grew up, different topics attracted my interest: immunology, psychology, web development (as a whole), and then, as of this year, data science. Having these interests (and a strict studying regime) allowed me to develop a specific way of studying as to maximize my learning.\nMy learning workflow is separated in two big steps: Absorbing and Applying.\nThere are some differences on how I learn different topics, but I\u0026rsquo;ve got a good enough generalization, which I\u0026rsquo;ll use from now on, to separate topics in three categories:\nMathematics, be it Linear Algebra, Calculus or Statistics, but specifically, the practical aspect, not much the theory Software Engineering, enclosing everything from APIs to Data Structures, both for backend and infrastructure, etc. Theoretical Courses, those being heavy text-based topics, like immunology, psychology, statistical theory, business logic, etc. I\u0026rsquo;ll specify which study method or technique I use each of those groups.\nAbsorbing Absorbing knowledge is easy, it\u0026rsquo;s retaining knowledge what is actually hard. To absorb information, you usually only got to spam the shit out of a topic until you understand it enough as to start applying it by yourself.\nFor Mathematics, I often take a book about the topic I want to learn and do all the exercises available while going through the book. For simple topics I don\u0026rsquo;t really take notes, unless there\u0026rsquo;s some super specific rule or concept I need to continue. On the other hand, for complex topics I typically take notes, write demonstrations and sometimes even add a practical example, if it\u0026rsquo;s difficult enough. Most of the time I just read and do exercises for 1-2hs. If there\u0026rsquo;s something I don\u0026rsquo;t understand, I use DeepSeek or Claude for a quick explanation. Most math problems are methodical after the 20ht exercise, so it\u0026rsquo;s important to mechanize the process while understanding why that mechanism works.\nFor Software Engineering, reading documentations beats almost everything else. For overviews, I watch videos like Luke Barousse\u0026rsquo;s Data Engineer overview or Tech with Tim\u0026rsquo;s Data Structures video, which allow me see the bigger picture and find what I want or need to learn. But to learn something, nothing is better than raw documentation. FastAPI documentation is, by far, one of the best ones I\u0026rsquo;ve read, but Django\u0026rsquo;s and Fastify\u0026rsquo;s are also very good. Documentation is good to understand syntax, code structure, inner workings and patterns, but falls short if not properly implemented. Also, you can\u0026rsquo;t code on the bus, but you can almost always read. Although, for more complex and in-depth implementations, like Data Structures or Patterns, a book always wins. Not because documentation is not good enough, but because documentation is technical, while books are human. What I mean by this is that documentation expects an experienced reader that is revisiting the topic, while books (and their authors) introduce the topic from a more approachable perspective, usually with IRL examples and easier language.\nFor Theoretical Courses, books books books. Even though they\u0026rsquo;re heavy to process, long and extensive books allow for a better understanding of topics like psychology, because they have the length to get through complex topics, like human behavior or innate immune response. Be aware, learning pure theory is a Sisyphean task, but that\u0026rsquo;s why choosing the right book is very, very important. For immunology, as an introduction, I always recommend Philip Dettmer\u0026rsquo;s “Immune” book, and just then start reading proper medical books. The same applies for statistics, psychology, etc. What I\u0026rsquo;ve found especially useful is to join forums and ask for book recommendations (and, as always, wait for people to stop killing themselves before reading), be it through Discord, Reddit, or even local forums.\nNow, as a general note, I always take notes, independently of what I\u0026rsquo;m learning. Notes don\u0026rsquo;t have to be very extensive nor exact, but must instead focus on critical concepts, flows, ideas, that allow to expand on the topic when cleaning the notes. Yes, I clean my notes. I take my bazillion pages of gibberish Samsung Notes and then take my time “translating” everything into something useful to come back to, so I have grounds to come back to if I ever get lost. For that, I use obsidian, on a public vault you can check, though, be aware, it\u0026rsquo;s not meant to be pretty, but useful.\nApplying Applying knowledge doesn\u0026rsquo;t mean doing set exercises or guided demonstrations. Applying knowledge means to take whatever you know and dump it to solve an esoteric problem you made up exclusively to apply said knowledge. This obviously isn\u0026rsquo;t very easy to implement in practice, but it can be done if you\u0026rsquo;re creative enough. Obviously it\u0026rsquo;s easier to apply things related to Software Engineering or Statistics, but I\u0026rsquo;ve found that, through code, almost everything can be represented.\nFucking Around to Find Out To Find Out what problems I can solve, the best way I\u0026rsquo;ve found is fucking around, both IRL and in forums, with people. Why? Social interactions allow you to discover new things to solve or something related that evolves into a well-defined issue to attack. Most of the bigger practical applications I\u0026rsquo;ve made come from solving other people\u0026rsquo;s concerns, be it a complex angle calculator or a full-fledged CRUD application for storing recipes, both of which I used as practical application of concepts.\nEven if you can\u0026rsquo;t get a detailed objective of what you can do, start small: create a small and perfect script that solves the aspect you understood of the problem, and then expand it be both abstract and general, allowing, in the end, the creation of a complete system probably no one will use, but that will demonstrate to yourself (and your possible employer) that you know X or Y things.\nMake, Break, Fix, Expand, Repeat \u0026ldquo;Make, Break, Fix, Expand, Repeat\u0026rdquo; represents the central motto of my studying method, independently of the topic. It can be easily applied to most projects and is probably the best approach I\u0026rsquo;ve found. This approach obviously depends on having a somewhat defined objective with some kind of closed scope (as to avoid big generalization), so this is the step after finding out.\nMake You start writing code. Yes, code. Oh, you studied math? Don\u0026rsquo;t care, code. Same for immunology, statistics, software engineering (duh) and anything else you find. If it exists, it means it can be conceptualized and therefore written as code, be it a math formula, human behavioral patterns or interferon propagation.\nFor mathematics, you need a use case for the formula. Then, write the formula from scratch, no libraries, and use it as is, even if it\u0026rsquo;s not efficient. Same for any algorithm you need to implement for software engineering. For theoretical topics, things become harder: you need to create a full representation of the situation (e.g., an interferon propagation simulator) to correctly represent the topic,\nBreak Because I made things fast and dirty, that means they\u0026rsquo;re gonna break, and I want them to break. The same way falling is part of learning to walk, breaking is part of learning to create. Everything you have created and will create has or will eventually break. This is expected, as nothing lasts forever, and that\u0026rsquo;s why you require things to break during your “training”, because that allows you to understand:\nWhat broke Where it broke How it broke Why it broke. What broke identifies what part of the complex system you made before broke. That can be a math formula giving a wrong result, an edge case causing a memory leak, an over-fitted model, etc. Knowing how to identify the culprit is hard, and that why things need to brake early. The bigger the tower, the bigger the fall.\nWhere it broke is the abstraction layer and specific location (line, function, pipeline stage or data slice) where the failure manifests.\nHow it broke represents the sequence of operations and state changes leading from correct preconditions to the failure itself.\nWhy it broke is, most of the time, the debugging. It is the break point, which usually is pretty much the same everywhere: mishandled edge case, recursive functions, wrong features selected, etc. These require good knowledge of the topic you\u0026rsquo;re dealing with, but also good understanding of what the code does (that\u0026rsquo;s why you don\u0026rsquo;t use AI) so logic debugging is easier.\nFix With all that knowledge about the issue I now know, I start fixing. First, the good old print debugging never fails to get rid of 90% of the errors I found.\nThen, the funny step begins: logical debugging. This is, by far, the best and worse part of fixing. The best because it\u0026rsquo;s the one when you really learn how to properly build a system, as you learn proper debugging and better coding techniques, and the worse because it\u0026rsquo;s the hardest kind of debugging. Logical debugging means finding the error on your own thoughts, translated to code. So you need to translate back the code to thoughts and then think again \u0026ldquo;Why did this fail?\u0026rdquo;. That\u0026rsquo;s why I don\u0026rsquo;t write code with AI, translating code I didn\u0026rsquo;t write into thoughts is very, very hard. Fixing is a very important step in learning, as it allows you to apply new concepts, be it as patches, reworks, refactors or straight up a completely new approach.\nExpand Expanding means to give 105%, to go a step further, to take a risk. \u0026ldquo;Nothing changes if nothing changes\u0026rdquo; kinda vibe. You always want to make that weird feature, to try to optimize that function, to write that blog. This is where you grow. But be aware, \u0026ldquo;Grow for the sake of grow is the philosophy of a cancer cell\u0026rdquo;. What do I mean by that? Think before expanding. Think about scope creep, simplicity over complexity, overhead, etc. Not only think about \u0026ldquo;does this make sense\u0026rdquo; but \u0026ldquo;Is this a good next step\u0026rdquo;. Maybe you don\u0026rsquo;t need to make another feature, but reconcile, write some notes, see where you\u0026rsquo;re at, document, clean up, optimize, tidy code up and write a blog post.\nRepeat Rinse and repeat. The last step, but also the first. This is, by far, the best characteristic of a good learner, and something I like to think I do well enough. Repeating, in this context, means to never stop searching for something new to learn. Repeating is going back to step one. It\u0026rsquo;s making new things, be it entirely new projects or refactoring old ones. This step is where you catch a break before routing back to your next adventure,\nConclusions Make a checkpoint every once in a while, as that allows you to see what you\u0026rsquo;ve done, so you can plan head. Take a step back and see how far you\u0026rsquo;ve come. Set goals for the future, but remember, as Jimmy Carr said, \u0026ldquo;It\u0026rsquo;s not the pursuit of happiness, it\u0026rsquo;s the happiness of the pursuit\u0026rdquo;, so go slow, but steady, and learn everything you can. Learning can manifest in many ways: making new projects, refactoring old ones, talking to new people, listening to old ones, reading a new book or revisiting old classics. What matters is to never stop. Your brain needs to learn every single day to stay healthy, as does any muscle.\nThis ended up being a mix on a summary of how I approach learning and a guide to learn (super biased), as I changed tones mid-way, but still, it\u0026rsquo;s a good recap nonetheless.\n","permalink":"https://blog.emiliano-go.com/studies/howistudy/","summary":"\u003cp\u003eStudying, the art of \u003cstrong\u003elearning\u003c/strong\u003e, and, for me, an amazing experience.\u003c/p\u003e\n\u003cp\u003eI\u0026rsquo;ve always considered me a \u003cstrong\u003eFilomath\u003c/strong\u003e, as I\u0026rsquo;ve been curious about everything and anything for as long as I can remember.\u003c/p\u003e\n\u003cp\u003eWhen I was \u003cstrong\u003eyounger\u003c/strong\u003e, I was really into \u003cstrong\u003eAstronomy\u003c/strong\u003e. My grandfather was (and is) an amateur astronomer, and studied astronomy for years in college, even tough he was a doctor. I\u0026rsquo;ve learned a lot of what I know about astronomy from him, but I\u0026rsquo;ve also studied it on my own. Did you know that, hypothetically, we could all vanish in about 8 minutes if even a \u003cstrong\u003etiny spec\u003c/strong\u003e of rare matter collided with earth? Fun, right?\u003c/p\u003e","title":"The Way I Study"}]