From ed7945c16686e47b8f48c9d4fd3ecad248f61910 Mon Sep 17 00:00:00 2001 From: ditadi Date: Mon, 19 Oct 2026 13:04:01 +0200 Subject: [PATCH 1/3] docs: design 001, database development flow Signed-off-by: ditadi --- designs/001_DatabaseDevelopmentFlow.md | 443 +++++++++++++++++++++++++ 1 file changed, 443 insertions(+) create mode 100644 designs/001_DatabaseDevelopmentFlow.md diff --git a/designs/001_DatabaseDevelopmentFlow.md b/designs/001_DatabaseDevelopmentFlow.md new file mode 100644 index 000000000..184e23d3d --- /dev/null +++ b/designs/001_DatabaseDevelopmentFlow.md @@ -0,0 +1,443 @@ +# 001 - Database development flow + +| | | +|---|---| +| **Status** | Draft | +| **Owner** | Victor Ditadi | + +The [Google Doc](https://docs.google.com/document/d/14jcyp2b0YTuwMg9SHU9RShIlWyJPaP_H7UUr6ONWLNk/edit) is the copy for comments and edits. Changes there are synced to this file. + +## Background and Motivation + +The DatabasePlugin (`@databricks/appkit/beta`, shipped in v0.87 and v0.88) turns `config/database/schema.ts` into +typed server access and generated CRUD routes over Lakebase Postgres. It reads tables. It never creates them. Today a +developer gets from an idea to a deployed table by hand: + +| Step | Today on `main` | +|---|---| +| Pick a database | `databricks apps init` asks for a Lakebase project, branch and database. The plugin is `hidden: true`, so only apps that add it by hand get this far. | +| Develop locally | `.env` points at the branch the app deploys to. Every developer reads and writes production rows. | +| Create a table | Run DDL by hand, then declare the table in `schema.ts`. The plugin's startup check fails until both match. | +| Permissions | The app's service principal has no `CREATE` on `public`. Someone grants it, or creates the tables as another role and grants access. | +| Change a table | Run `ALTER` by hand on every branch that needs it, in the right order, and remember which branches already have it. | +| Review | Schema changes are not in the pull request. Reviewers see `schema.ts` only. | +| Deploy | Nothing ties a code deploy to the schema it needs. A missing column surfaces as the app failing its startup check. | + +Lakebase sets the constraints any design has to live with: + +- A branch fork copies its parent's data, its roles, and its objects with their owners. Role state is per branch. +- `GRANT ... TO databricks_superuser` is rejected. A custom `NOLOGIN` role can own app objects, and its members get + owner rights, but someone with `ADMIN OPTION` on it has to grant it to each new identity. +- On `public`, only `pg_database_owner` and `databricks_superuser` have `CREATE`. A role with `CREATE` on the + database can create and own a schema of its own. +- A role is created and deleted through the Lakebase role API. Deleting a role can reassign what it owns + (`reassign_owned_to`). +- A project allows 10 unarchived branches by default and 20 active computes outside the default branch. +- The pooled endpoint is not available to OAuth roles and runs PgBouncer in transaction mode, which has no advisory + locks. Apps connect to the direct endpoint. + +Apps sets two more: + +- A horizontally scaled app rolls a deploy to one instance first while the others keep serving, up to five instances. +- Removing a database resource from an app hands the service principal's objects to the app manager. Re-adding it + restores database access but not object ownership, so the next deploy fails with `permission denied for schema` + ([XTA-18463](https://databricks.atlassian.net/browse/XTA-18463)). +- Genie App Builder builds on AppKit. Its apps scale to zero after 30 idle minutes, so start runs on every cold start, + and they deploy from the UI with no CI step. + +### Relation to the July design + +The approved [AppKit Database Plugin design](https://docs.google.com/document/d/12PNcx6e8dvzaSyuBODW9wznlLV3em_3QqJ1-FLdxuyE/edit) +had an admin run `appkit db migrate --prod` and `appkit db grant-access --prod`, with the service principal as a +DML grantee. This design moves deploy-branch migrations into the app. The admin step is a human touchpoint on every +schema change, needs a person with DDL rights on the production branch, and has no place to run for App Builder apps. +The costs of the move are in the architecture decisions below. + +The stack below this design was built and verified against a Lakebase project with a throwaway deploy branch and app. +Its PR descriptions record the live results. + +## Goals + +1. From `databricks apps init` to a deployed app with one table, a developer runs no SQL, grants nothing and opens no + console. +2. Each developer works on a Lakebase branch of their own. Only the deployed app changes the branch it deploys to. +3. Schema changes are committed as SQL migrations and reviewed with the code that needs them. +4. A deploy applies exactly the migrations its commit carries, or fails before serving with a message that names the + migration and the fix. +5. Every command that writes refuses a branch it was not built for, before it connects. + +## Out of Scope + +- Down migrations and automatic schema rollback. Fixes go forward. +- Per-user database identities (OBO). The plugin runs as the app's service principal. +- Several apps sharing one Postgres schema. +- drizzle-kit v1's migration folder format. This design ships on drizzle-kit 0.31 (journal v7). +- Seeding or copying data into the deploy branch. +- Migrations that cannot run in a transaction (`CREATE INDEX CONCURRENTLY`, `VACUUM`). + +## Product Decisions + +### The schema lives in TypeScript, and migrations are committed SQL + +`config/database/schema.ts` stays the source of truth. `appkit db migrate` writes drizzle-kit output under +`config/database/migrations/`: one `NNNN_.sql` per change, a `meta/NNNN_snapshot.json` per migration and +`meta/_journal.json`. The SQL is what reviewers read and what the deployed app runs. + +### Every developer gets a Lakebase branch + +`appkit db init` forks the project's default branch into one of two kinds: + +- `dev--`, one per developer, no expiry, the same name on every machine. +- `session--`, one per checkout, with a 7-day TTL. The id lives in `.env` as `APPKIT_DB_SESSION`. + +`` is the first six hex characters of the SHA-256 of the user name, so a branch name says whose it is. + +The developer gets a Postgres role with `DATABRICKS_SUPERUSER` on that branch, and owns the app's objects there. +`.env` then points at the branch. A fork copies production rows. See Security Considerations. + +Branches count against the project's limits, so init keeps them cheap. It creates the branch's endpoint with a +5-minute scale to zero, because an idle compute counts against the 20 active ones. When the project is at its branch +limit, init stops before creating anything and lists your other branches with the command that deletes them. With the +default limit of 10, a project holds the deploy branch and about eight developers. + +### The deployed app applies its migrations when it starts + +The app applies the committed migrations its deploy branch has not recorded, as its service principal, before it +serves. Developers never migrate the deploy branch. This is the only option that keeps one owner per branch with no +extra resource to deploy. See the architecture decision below. + +### Tables default to the Postgres schema `app` + +`defineSchema()` without `schemaName` puts tables in `app`. The service principal creates and owns `app` on its first +start, so the first deploy needs no grant. `{ schemaName: "public" }` keeps working for apps that already use it, but +migrations on start cannot create objects there without a manual `GRANT CREATE ON SCHEMA public`. + +### Existing beta apps keep working after the upgrade + +An app that upgrades from v0.88 does not start migrating, because the deploy flag below is off unless `apps init` set +it. If the startup check finds every missing table in `public`, its error says the default moved to `app` and offers +`{ schemaName: "public" }`. The release notes carry an upgrade section. + +### Seven commands, each with one job + +| Command | Job | Needs a database | +|---|---|---| +| `appkit db init` | Point `.env` at your dev or session branch, creating it when missing. | Yes | +| `appkit db migrate` | Write a migration when `schema.ts` changed, then apply pending migrations to your branch. | Yes | +| `appkit db introspect` | Write `schema.ts` and a baseline from tables that already exist on your branch. | Yes | +| `appkit db clean` | Drop the app schema on your branch. | Yes | +| `appkit db seed` | Run `config/database/seed.sql` on your branch. | Yes | +| `appkit db verify` | Read-only check of your branch, or with `--prod` of the deployed app's binding. | Yes, or API only | +| `appkit db check` | Offline check of the committed history, for CI. | No | + +`appkit db check` is new in this design. It is the only check that runs in CI without Lakebase credentials. + +## Architecture Decisions + +### Migrate on start, behind an explicit flag + +The plugin applies migrations in `setup()`, after the pool opens and before the schema check and `server.start()`. +It runs only when `APPKIT_DB_MIGRATE_ON_START=true` and the process has service principal credentials +(`DATABRICKS_CLIENT_ID` and `DATABRICKS_CLIENT_SECRET`). `apps init` writes the flag into `app.yaml` for apps that +select the plugin. `.env` never carries it, so `npm start` on a laptop never migrates. + +The stack today infers "deployed" from `NODE_ENV`, `DATABRICKS_APP_NAME` and the credentials. The template's `start` +script sets `NODE_ENV=production` and its `.env` sets `DATABRICKS_APP_NAME`, so a local `npm start` with M2M +credentials migrates the dev branch as the service principal and breaks the one-owner rule. + +Rejected alternatives: + +| Option | Why not | +|---|---| +| Infer deployment from the environment | False positive on a local `npm start` with M2M credentials. No way to turn it off. | +| An admin runs `appkit db migrate --prod` (the July design) | A human step on every schema change, by a person with DDL rights on the production branch. App Builder has nobody to run it. | +| A shared owner role, with CI migrating on merge ([tmm Lakebase Agentic CI](https://github.com/databricks/tmm/tree/main/Lakebase-Agentic-CI)) | Works. It needs a one-time bootstrap on the deploy branch by someone with `ADMIN OPTION`, database credentials in CI, and a grant to every new app service principal. Developers skip the fork hand-off only if they are members on the deploy branch, which gives them DDL rights there. App Builder apps have no CI. | +| A one-shot migration job per deploy | Adds a resource to every app, and still needs a shared owner role and its bootstrap so the app can use what the job created. | +| A bundle script hook | Runs as the person or CI identity that deploys, which then owns the objects. | + +The decision costs two rules that the docs and `appkit db migrate` carry: + +- **Each migration works with the previous release.** A horizontally scaled rollout updates one instance first while + the old instances keep serving. A dropped, renamed or retyped column breaks them. Drops go in a later deploy than + the code that stops using the column (expand, then contract). `appkit db migrate` warns when it writes `DROP`, + `RENAME` or `ALTER ... TYPE`. +- **A migration fits in the start window.** A migration that runs longer than the platform's start timeout can never + land. Long backfills run as a separate job against the deploy branch, not as a migration. + +### One transaction per migration + +Each pending migration commits in its own transaction, in journal order, with the journal row drizzle-orm would +write. A deploy then replays the same commits the developer's branch went through one `appkit db migrate` at a time. + +drizzle-orm's migrator runs every pending migration in one transaction. That fails on deploy for changes that applied +cleanly on the dev branch. `ALTER TYPE ... ADD VALUE` in one migration and a use of the new value in the next fails +with SQLSTATE 55P04 when both run together. A failure after some migrations commit leaves the app not started and +the old instances serving. Because every migration works with the previous release, that state is safe, and the next +deploy resumes from it. + +The alternative keeps one transaction and refuses, before `begin`, a batch where an enum value is added before the +last migration. It keeps all-or-nothing, but the deploy still differs from what the developer ran. + +### One lock per schema, with a bounded wait + +A session advisory lock keyed on the schema name serializes instances that start together. The wait polls +`pg_try_advisory_lock` until a 5-minute deadline, and each poll logs the holder from `pg_locks` and +`pg_stat_activity`. The migrating session sets `application_name = 'appkit-migrate'` so an operator can find it, and +`client_connection_check_interval` so Postgres cancels a statement whose instance was killed. Without it a killed +instance's statement keeps running and holds the lock. `lock_timeout` drops from 30s to 5s, with up to three tries per +migration. A waiting `ACCESS EXCLUSIVE` request queues every later query on its table, so the timeout bounds how long +traffic stalls, not only the migration. + +Most starts have nothing to apply, and an App Builder app starts on every cold start. The migrator reads the recorded +rows first without the lock. It takes the lock only when an entry is pending, then reads again under it. A current +branch then costs one connection and one read of the journal table on the start path. + +A transaction-mode pooler would void a session lock. The pooled Lakebase endpoint is closed to OAuth roles, so the +plugin connects direct today. The connector refuses a `-pooler` host so that stays true. + +### Fail closed on a history the migrator would skip + +drizzle-orm applies only journal entries newer than the last recorded `created_at`. Two developers who add migrations +on parallel git branches produce, after the merge, an entry older than one the deploy branch already ran. That entry +is skipped for good. The startup schema check catches a skipped `CREATE TABLE` but not a skipped index, constraint, +backfill or `DROP`. + +The migrator reconciles the committed history with the recorded rows once, under the lock, as a pure function: + +| State | Meaning | On start | +|---|---|---| +| `empty` | No journal table, or no rows. | Apply all, or adopt the baseline. | +| `pending` | Entries newer than the last recorded one. | Apply them. | +| `current` | Every entry recorded with its hash. | Nothing. | +| `stranded` | An entry older than the last recorded one, not recorded. | Fail. Nothing changes. | +| `edited` | A recorded entry whose file hash changed. | Warn. | +| `ahead` | A recorded row with no committed entry, for example after a code rollback. | Warn. | +| `partial-baseline` | `0000_baseline` tables exist only in part. | Fail. Nothing changes. | + +`appkit db check` fails in CI when `when` does not increase with `idx`, when the snapshot `prevId` chain forks, when a +listed file is missing, or when `schema.ts` differs from the latest snapshot. A journal on `main` that only grows can +then never strand an entry on the deploy branch, because the deploy branch applies a prefix of it. To fix a failed +check, the developer deletes their own migration, merges `main`, and runs `appkit db migrate` again. + +Two other fixes lose. Applying by hash instead of by `when` runs DDL in an order nobody reviewed. A custom journal +forks drizzle-kit's format, and drizzle-kit v1 already replaces the journal. + +### One owner per branch + +The identity that runs a branch's migrations owns the app's objects there. On the deploy branch that is the app's +service principal. On a dev or session branch it is the developer. There is no shared owner role and no `SET ROLE`, +because Lakebase rejects `GRANT ... TO databricks_superuser`. + +A fork copies objects with their owners, so `appkit db init` hands the fork's app objects to the developer. It deletes +the fork's copy of the role that owns schema `app` and reassigns its objects (`reassign_owned_to`). It touches only +that role, on the fork, and says so before it does. The stack today deletes every role-API role that owns anything in +any schema of the fork. + +The role delete returns HTTP 500 when Postgres refuses the drop with SQLSTATE 2BP01, for example when the role still +owns objects in another database of the branch +([ES-2265656](https://databricks.atlassian.net/browse/ES-2265656), open). Init lists that role's remaining +dependencies from `pg_shdepend` before the delete, and on a 500 prints them with the fix instead of the API error. + +Single ownership by the service principal breaks when that principal changes. A recreated app has a new one, and +removing and re-adding the database resource hands the objects to the app manager (XTA-18463). Either way the app +can no longer use or alter its schema, and its next start fails. That recovery is an open question, not part of +this stack. The startup error names the current owner of schema `app` and the recovery doc. + +### Branch policy is one resolver and one table + +One function resolves the `.env` branch to a kind, in this order: project default, protected, deploy target, dev +(yours), session (this checkout), other. Every command reads that kind. The rules live in one table: + +| Kind | `init` | `migrate` | `introspect` | `clean`, `seed` | `verify` | +|---|---|---|---|---|---| +| dev (yours), session (this checkout) | Use | Write and apply | Write and record | Run | Read | +| deploy target, project default, protected, other | Refuse | Write only | Refuse | Refuse | Read | + +`init` resolves the kind of the branch it would create or reuse before creating it or writing `.env`. A deploy target +is any `postgres_branch` value in `databricks.yml`, from every target and from the variable default. + +The name allowlist is the safety boundary. Only `dev--` and `session--` are +writable, and the denylist checks come first. Before connecting, the CLI checks that `PGHOST` is a host of +`LAKEBASE_ENDPOINT`. + +### drizzle-kit generates, the plugin applies + +drizzle-kit 0.31.10 generates migrations through `drizzle-kit/api`. It is an exact optional peer dependency, loaded +lazily from the `@databricks/appkit/database-tooling` entry point that the runtime never imports. The loader checks +the installed version and the `drizzle-orm` it resolves, and prints the install command on a mismatch. + +`appkit db init` installs drizzle-kit as a dev dependency when it is missing, since only database apps run it. A +template dev dependency would add about 32 MB, 12 downloads and two deprecation warnings to every app, including apps +that never use it. The template ships one lockfile, so it cannot be dropped per app. + +The shared CLI's view of the tooling API is an interface exported from `shared` that the tooling `satisfies`. Drift +then fails typecheck. The stack today keeps a 131-line hand-written declaration with no check. + +### Introspect renders the AppKit builder, not drizzle's DSL + +`appkit db introspect` reads `pg_catalog` and renders `defineSchema()` code. `drizzle-kit pull` emits drizzle's +`pgTable` DSL, which the plugin does not accept, and most of the renderer is skip and approximation policy that pull +does not have. Every catalog string reaches the output escaped. Nineteen hostile names, including newline, CR, +U+2028, U+2029, `*/`, backtick and `${`, loaded without running code. + +Introspect records its migration as applied only if the branch's last recorded migration is the one just before it, +checked inside the migration lock. A branch ahead of the checkout would otherwise record DDL it never ran. + +### Flag Strategy + +| Flag | Default | Introduced in | Turning it off | +|---|---|---|---| +| `APPKIT_DB_MIGRATE_ON_START` (app env) | Off. `apps init` renders `true` into `app.yaml` for database apps. | #654 | The app never migrates. The startup schema check still runs. | +| `hidden` in the plugin manifest | `true` | Already on `main` | `apps init` does not offer the plugin. Removed in #675. | +| `appkit db` hidden from `--help` | Hidden | #666 | Discoverability only. The commands are local tools and change only branches `init` created. Shown in #675. | + +Not flagged: + +- The `app` default (#655). The plugin is beta, which allows API changes in a minor release, and the startup hint + plus the upgrade note cover existing apps. +- `appkit db check`, `verify` and `introspect` on your own branch. They are read-only or write only local files and + your own branch. + +## Main Design + +### A new app, from scaffold to deploy + +1. `databricks apps init` scaffolds the app with the plugin. It asks for the Lakebase project and the deploy branch + and database, which go into `databricks.yml`. +2. `apps init` runs `appkit db init --yes`. It forks the project's default branch into `dev--`, adds a + read-write endpoint and the developer's role, hands the fork's app objects to the developer, writes `.env`, and + starts the schema (migrate when `schema.ts` exists, introspect otherwise). +3. The developer declares a table in `schema.ts` and runs `appkit db migrate`. It writes `0000_migration.sql`, + regenerates `shared/appkit-types/database.d.ts`, and applies the migration to the dev branch. +4. The developer commits `config/database/` and runs `databricks apps deploy`. +5. The app starts with `APPKIT_DB_MIGRATE_ON_START=true`. Under the lock it reconciles, applies `0000`, creates and + owns schema `app`, checks the declared tables, and listens. + +Until the CLI release that runs `appkit db init` ships, step 2 is a command the README tells the developer to run. + +### A table change + +Edit `schema.ts`, run `appkit db migrate`, commit, deploy. CI runs `appkit db check`. After the deploy, +`appkit db verify --prod` reports whether the app is running the commit's newest migration. + +### A second developer + +Clone, install, `appkit db init`. Init forks the default branch, which already holds the deployed migrations, and +applies any committed migration the fork has not recorded. + +### Existing tables + +On a fork that has tables but no `schema.ts`, `appkit db init` runs introspect. It writes `schema.ts` and +`0000_baseline`, records the baseline on the fork, and the deployed app adopts it on start when every baseline table +and column already exists. A partial match fails start and changes nothing. + +### API Changes + +- CLI: `appkit db init | migrate | introspect | clean | seed | verify | check`. +- `defineSchema()` defaults `schemaName` to `"app"`. +- Package export `@databricks/appkit/database-tooling`. It is internal, with no semver guarantee outside the CLI. +- App env: `APPKIT_DB_MIGRATE_ON_START`. Local env written by init: `LAKEBASE_PROJECT`, `LAKEBASE_ENDPOINT`, + `PGHOST`, `PGDATABASE`, `APPKIT_DB_SESSION`. +- Optional peer dependency: `drizzle-kit` `0.31.10`. + +## Observability + +Migrations run before `TelemetryManager.start()` registers the tracer provider, so a span opened during setup is +dropped. The migrator returns `lockWaitMs`, per-migration `durationMs`, the state, and the tags. The plugin logs them +as one structured event per run. After setup completes it emits a `database.migrate` span with explicit start and end +times and these attributes: `applied_count`, `adopted_count`, `state`, `outcome`, `sqlstate`. It also records a +duration histogram and an outcome counter. Tags and SQL text are never attributes. The start log names the branch, +the schema and the role. + +## Security Considerations + +- **Dev branches copy production data.** A fork of the default branch copies its rows, and the developer gets + `DATABRICKS_SUPERUSER` on it. That role is required, because it is the only role API membership that grants + `CREATE` on the database. The docs say so. `appkit db init --schema clean` starts the fork with an empty `app` + schema, but rows in other schemas stay. +- **Creating branches needs CAN MANAGE on the project.** That also allows deleting the deploy branch. The narrowest + permission is an open question. +- **Writes stay on your branches.** The name allowlist, the denylist and the `PGHOST` check run before any connection. +- **Generated code.** Introspect escapes every catalog string that reaches `schema.ts`. Tests cover each escape + vector, including CR and U+2029. +- **No driver text leaves the server.** Startup errors carry the SQLSTATE and the migration tag. Clients see + `Database setup failed`. +- **`.env`.** Init writes it with mode 0600 through a temporary file and a rename, and refuses a symlink. +- **Session branches are bound to a user.** The stack today lets anyone with a copy of `.env` pass the guard for its + session branch. With the user hash in the name, a copied `.env` fails the guard for everyone but its creator. + +## Platform Dependencies + +| Ask | Why | +|---|---| +| Databricks CLI: `apps init` runs `appkit db init --yes` after install. | Removes the manual init step. Without it the README tells the developer to run init. | +| Databricks CLI: a minimum version recorded in the template. | An older CLI scaffolds without init, and `.env` points at the deploy branch. | +| Lakebase: `client_connection_check_interval` allowed for app roles. | Cancels the statement of a killed instance that still holds the migration lock. | +| Lakebase: role delete returns the 2BP01 dependency error as a 4xx, not a 500 ([ES-2265656](https://databricks.atlassian.net/browse/ES-2265656), open). | Init's hand-off fails on every rerun when it hits this, with no reason given. | +| Lakebase: raise the default of 10 unarchived branches per project ([LKB-12894](https://databricks.atlassian.net/browse/LKB-12894), backlog, proposes 100). | One branch per developer caps a project at about eight developers. | +| Apps: keep object ownership with the app across a database resource remove and re-add ([XTA-18463](https://databricks.atlassian.net/browse/XTA-18463), to do). | The app cannot use its own schema after a re-add, and start fails. | +| Apps: the start timeout and whether a non-zero exit restarts the app. | Sets the lock wait and the longest migration. The template exits 0 on a failed start today. | + +## Rollout Plan + +1. **Design.** Land this doc. Done when the Apps, Lakebase and CLI reviewers have answered the open questions marked + for them. +2. **Runtime, dark.** Land #654 and #655 with the flag, the per-migration transactions, the reconcile states and the + bounded lock. Done when a real-Postgres test suite passes in CI, and the dev-playground deployed with the flag on + applies `0000` and then `0001`. +3. **Commands, hidden.** Land #666 to #674 and `appkit db check`. Done when two people on the dev-playground add + migrations on parallel git branches, `appkit db check` fails the second PR, and the documented fix passes. +4. **Reveal.** Land #675 once the CLI release that runs `appkit db init` is out. Done when `apps init` with the + plugin, one table and a deploy need no step outside the README. +5. **GA of the plugin** is a separate plan. + +## PR Stack + +1. `00-design`: this doc. +2. [#654](https://github.com/databricks/appkit/pull/654) Migrations on deploy start. Changes from this design: the + flag, per-migration transactions, the reconcile states with fail-closed `stranded`, the bounded lock wait, + `application_name`, `client_connection_check_interval`, the 5s lock timeout, the lock-free read on a current + branch, the `-pooler` host refusal, the schema owner in the startup error, structured migration events, and a + real-Postgres suite. +3. [#655](https://github.com/databricks/appkit/pull/655) `app` default schema. Changes: the `public` hint in the + startup check and an upgrade note. +4. [#666](https://github.com/databricks/appkit/pull/666) `appkit db migrate`. Changes: discard the run's migration + when applying it fails, the drizzle-kit version check, the destructive-statement warning, the `satisfies`-checked + contract, and the branch resolver. +5. [#670](https://github.com/databricks/appkit/pull/670) `appkit db introspect`. Changes: compare-and-set recording, + refuse on branches you do not own, a drop refusal that matches the actual targets, and catalog tests that run SQL. +6. [#672](https://github.com/databricks/appkit/pull/672) `appkit db init`, `clean`, `seed`. Changes: the hand-off + scoped to the app schema's owner, the 2BP01 dependency report, resolve before create, session ids with the user + hash, the 5-minute scale to zero, the branch-limit stop, atomic `.env` writes, extension-aware `public` clean, and + drizzle-kit install. +7. [#674](https://github.com/databricks/appkit/pull/674) `appkit db verify`. Changes: report `ahead` rows, + `--target`, and compare the deployed journal with the local one. +8. New: `appkit db check`. +9. [#675](https://github.com/databricks/appkit/pull/675) Offer the plugin in `apps init`. Changes: the flag in + `app.yaml`, no drizzle-kit in the template, a non-zero exit on a failed start, README wording that works before + the CLI release, and docs for existing apps, rollback and CI. + +## Open Questions + +1. Should the app's service principal own the deploy branch's objects at all? XTA-18463 shows Apps moving them to the + app manager on a resource re-add. A shared owner role survives that, but needs a holder of `ADMIN OPTION` that a + zero-touch flow does not have. This is the decision most worth challenging. Reviewer: Lakebase, Apps, AppKit. +2. What is the Apps start timeout, and does a process that exits non-zero get restarted? SUP-36075 suggests about 10 + minutes, unconfirmed. Reviewer: Apps runtime. +3. Is `client_connection_check_interval` allowed for Lakebase roles? Reviewer: Lakebase. +4. What is the narrowest Lakebase permission that lets a developer fork a branch and manage roles on their own fork? + CAN MANAGE also allows deleting the deploy branch. Reviewer: Lakebase. +5. Is a fork of production rows acceptable for customer data by default, or should init default to an empty schema? + Does Lakebase offer schema-only branches? Reviewer: Lakebase, security. +6. How does an app with a new service principal take ownership back, after a recreate or a resource re-add? Options + are Apps transferring ownership on re-add, or an `appkit db adopt --prod` built on the role API. Reviewer: Lakebase, + Apps. +7. What scale to zero do child branch computes get by default? Production gets 24 hours. Reviewer: Lakebase. +8. How does Genie App Builder drive this flow? Its apps have no per-developer `.env` and no CI, and they start on + every cold start. Reviewer: App Builder. +9. Ship on drizzle-kit 0.31's journal, or wait for drizzle-kit v1's folder format, which removes the journal and its + merge conflicts? Reviewer: AppKit. +10. Which CLI release runs `appkit db init`, with which flags, before or after the dependency install? Reviewer: + Databricks CLI. +11. This design has `appkit db init` install drizzle-kit. Should the CLI support per-plugin dependencies instead, so + apps that select the plugin get it at scaffold time? Reviewer: AppKit, Databricks CLI. From a31b0287087b7b17ca3cbac81b32756604daa6ab Mon Sep 17 00:00:00 2001 From: ditadi Date: Mon, 19 Oct 2026 13:11:49 +0200 Subject: [PATCH 2/3] docs: cover run-local, CI branches and the .env placeholder in design 001 Signed-off-by: ditadi --- designs/001_DatabaseDevelopmentFlow.md | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/designs/001_DatabaseDevelopmentFlow.md b/designs/001_DatabaseDevelopmentFlow.md index 184e23d3d..653102f7f 100644 --- a/designs/001_DatabaseDevelopmentFlow.md +++ b/designs/001_DatabaseDevelopmentFlow.md @@ -5,7 +5,7 @@ | **Status** | Draft | | **Owner** | Victor Ditadi | -The [Google Doc](https://docs.google.com/document/d/14jcyp2b0YTuwMg9SHU9RShIlWyJPaP_H7UUr6ONWLNk/edit) is the copy for comments and edits. Changes there are synced to this file. +The [Google Doc](https://docs.google.com/document/d/1bsMsYsTfoFBSYS3YbXn3FvOOUQl21X829Z4u1I-fBzg/edit) is the copy for comments and edits. Changes there are synced to this file. ## Background and Motivation @@ -100,6 +100,9 @@ Branches count against the project's limits, so init keeps them cheap. It create limit, init stops before creating anything and lists your other branches with the command that deletes them. With the default limit of 10, a project holds the deploy branch and about eight developers. +CI needs no branch, because `appkit db check` runs offline. A CI job that wants a live branch runs +`appkit db init --session --ttl 1h` and deletes it at the end with `appkit db clean --delete-branch`. + ### The deployed app applies its migrations when it starts The app applies the committed migrations its deploy branch has not recorded, as its service principal, before it @@ -125,7 +128,7 @@ it. If the startup check finds every missing table in `public`, its error says t | `appkit db init` | Point `.env` at your dev or session branch, creating it when missing. | Yes | | `appkit db migrate` | Write a migration when `schema.ts` changed, then apply pending migrations to your branch. | Yes | | `appkit db introspect` | Write `schema.ts` and a baseline from tables that already exist on your branch. | Yes | -| `appkit db clean` | Drop the app schema on your branch. | Yes | +| `appkit db clean` | Drop the app schema on your branch, or with `--delete-branch` the branch. | Yes | | `appkit db seed` | Run `config/database/seed.sql` on your branch. | Yes | | `appkit db verify` | Read-only check of your branch, or with `--prod` of the deployed app's binding. | Yes, or API only | | `appkit db check` | Offline check of the committed history, for CI. | No | @@ -141,6 +144,10 @@ It runs only when `APPKIT_DB_MIGRATE_ON_START=true` and the process has service (`DATABRICKS_CLIENT_ID` and `DATABRICKS_CLIENT_SECRET`). `apps init` writes the flag into `app.yaml` for apps that select the plugin. `.env` never carries it, so `npm start` on a laptop never migrates. +`databricks apps run-local` reads `app.yaml`, so it sets the flag, but it sets no service principal credentials. It +migrates only when the shell exports them. As a second guard, migrate on start refuses when `LAKEBASE_ENDPOINT` names +a `dev-` or `session-` branch, because only a developer migrates those. + The stack today infers "deployed" from `NODE_ENV`, `DATABRICKS_APP_NAME` and the credentials. The template's `start` script sets `NODE_ENV=production` and its `.env` sets `DATABRICKS_APP_NAME`, so a local `npm start` with M2M credentials migrates the dev branch as the service principal and breaks the one-owner rule. @@ -337,7 +344,8 @@ and column already exists. A partial match fails start and changes nothing. - `defineSchema()` defaults `schemaName` to `"app"`. - Package export `@databricks/appkit/database-tooling`. It is internal, with no semver guarantee outside the CLI. - App env: `APPKIT_DB_MIGRATE_ON_START`. Local env written by init: `LAKEBASE_PROJECT`, `LAKEBASE_ENDPOINT`, - `PGHOST`, `PGDATABASE`, `APPKIT_DB_SESSION`. + `PGHOST`, `PGDATABASE`, `APPKIT_DB_SESSION`, and `DATABRICKS_CONFIG_PROFILE` or `DATABRICKS_HOST`. +- New flags: `appkit db init --ttl ` for session branches, `appkit db clean --delete-branch`. - Optional peer dependency: `drizzle-kit` `0.31.10`. ## Observability @@ -362,7 +370,9 @@ the schema and the role. vector, including CR and U+2029. - **No driver text leaves the server.** Startup errors carry the SQLSTATE and the migration tag. Clients see `Database setup failed`. -- **`.env`.** Init writes it with mode 0600 through a temporary file and a rename, and refuses a symlink. +- **`.env`.** Init writes it with mode 0600 through a temporary file and a rename, and refuses a symlink. When `.env` + is missing, init writes its own keys and the profile or host it authenticated with. It never copies `.env.example`, + whose `DATABRICKS_HOST` is a placeholder. - **Session branches are bound to a user.** The stack today lets anyone with a copy of `.env` pass the guard for its session branch. With the user hash in the name, a copied `.env` fails the guard for everyone but its creator. @@ -397,8 +407,8 @@ the schema and the role. 2. [#654](https://github.com/databricks/appkit/pull/654) Migrations on deploy start. Changes from this design: the flag, per-migration transactions, the reconcile states with fail-closed `stranded`, the bounded lock wait, `application_name`, `client_connection_check_interval`, the 5s lock timeout, the lock-free read on a current - branch, the `-pooler` host refusal, the schema owner in the startup error, structured migration events, and a - real-Postgres suite. + branch, the `-pooler` host refusal, the dev and session branch refusal, the schema owner in the startup error, + structured migration events, and a real-Postgres suite. 3. [#655](https://github.com/databricks/appkit/pull/655) `app` default schema. Changes: the `public` hint in the startup check and an upgrade note. 4. [#666](https://github.com/databricks/appkit/pull/666) `appkit db migrate`. Changes: discard the run's migration @@ -408,8 +418,8 @@ the schema and the role. refuse on branches you do not own, a drop refusal that matches the actual targets, and catalog tests that run SQL. 6. [#672](https://github.com/databricks/appkit/pull/672) `appkit db init`, `clean`, `seed`. Changes: the hand-off scoped to the app schema's owner, the 2BP01 dependency report, resolve before create, session ids with the user - hash, the 5-minute scale to zero, the branch-limit stop, atomic `.env` writes, extension-aware `public` clean, and - drizzle-kit install. + hash, the 5-minute scale to zero, the branch-limit stop, `--ttl` and `--delete-branch`, atomic `.env` writes + without the `.env.example` placeholder, extension-aware `public` clean, and drizzle-kit install. 7. [#674](https://github.com/databricks/appkit/pull/674) `appkit db verify`. Changes: report `ahead` rows, `--target`, and compare the deployed journal with the local one. 8. New: `appkit db check`. From 34bcd908e06597468c82bf68007c18fa0f4078e1 Mon Sep 17 00:00:00 2001 From: ditadi Date: Tue, 20 Oct 2026 12:02:23 +0200 Subject: [PATCH 3/3] docs: align design 001 with the reviewed stack Signed-off-by: ditadi --- designs/001_DatabaseDevelopmentFlow.md | 154 +++++++++++++------------ 1 file changed, 79 insertions(+), 75 deletions(-) diff --git a/designs/001_DatabaseDevelopmentFlow.md b/designs/001_DatabaseDevelopmentFlow.md index 653102f7f..ef275d562 100644 --- a/designs/001_DatabaseDevelopmentFlow.md +++ b/designs/001_DatabaseDevelopmentFlow.md @@ -32,7 +32,9 @@ Lakebase sets the constraints any design has to live with: database can create and own a schema of its own. - A role is created and deleted through the Lakebase role API. Deleting a role can reassign what it owns (`reassign_owned_to`). -- A project allows 10 unarchived branches by default and 20 active computes outside the default branch. +- A project holds 500 branches, archived ones included, and an internal quota of 25 unarchived branches + ([LKB-15175](https://databricks.atlassian.net/browse/LKB-15175)). 20 computes run at once outside the default + branch. - The pooled endpoint is not available to OAuth roles and runs PgBouncer in transaction mode, which has no advisory locks. Apps connect to the direct endpoint. @@ -45,14 +47,6 @@ Apps sets two more: - Genie App Builder builds on AppKit. Its apps scale to zero after 30 idle minutes, so start runs on every cold start, and they deploy from the UI with no CI step. -### Relation to the July design - -The approved [AppKit Database Plugin design](https://docs.google.com/document/d/12PNcx6e8dvzaSyuBODW9wznlLV3em_3QqJ1-FLdxuyE/edit) -had an admin run `appkit db migrate --prod` and `appkit db grant-access --prod`, with the service principal as a -DML grantee. This design moves deploy-branch migrations into the app. The admin step is a human touchpoint on every -schema change, needs a person with DDL rights on the production branch, and has no place to run for App Builder apps. -The costs of the move are in the architecture decisions below. - The stack below this design was built and verified against a Lakebase project with a throwaway deploy branch and app. Its PR descriptions record the live results. @@ -98,7 +92,8 @@ The developer gets a Postgres role with `DATABRICKS_SUPERUSER` on that branch, a Branches count against the project's limits, so init keeps them cheap. It creates the branch's endpoint with a 5-minute scale to zero, because an idle compute counts against the 20 active ones. When the project is at its branch limit, init stops before creating anything and lists your other branches with the command that deletes them. With the -default limit of 10, a project holds the deploy branch and about eight developers. +quota of 25, a project holds the deploy branch and about 20 developers with one branch each, fewer when they also keep +session branches. The 20 active computes cap how many of them work at the same time. CI needs no branch, because `appkit db check` runs offline. A CI job that wants a live branch runs `appkit db init --session --ttl 1h` and deletes it at the end with `appkit db clean --delete-branch`. @@ -130,10 +125,12 @@ it. If the startup check finds every missing table in `public`, its error says t | `appkit db introspect` | Write `schema.ts` and a baseline from tables that already exist on your branch. | Yes | | `appkit db clean` | Drop the app schema on your branch, or with `--delete-branch` the branch. | Yes | | `appkit db seed` | Run `config/database/seed.sql` on your branch. | Yes | -| `appkit db verify` | Read-only check of your branch, or with `--prod` of the deployed app's binding. | Yes, or API only | +| `appkit db verify` | Read-only check of your branch, or with `--deployed -t ` of a target's deployed app. | Yes, or API only | | `appkit db check` | Offline check of the committed history, for CI. | No | -`appkit db check` is new in this design. It is the only check that runs in CI without Lakebase credentials. +`appkit db init` asks nothing. It keeps this checkout's session branch when `.env` points at one and otherwise uses +your dev branch. `--session` and `--dev` switch between them. `appkit db check` is the only check that runs in CI +without Lakebase credentials. ## Architecture Decisions @@ -147,17 +144,12 @@ select the plugin. `.env` never carries it, so `npm start` on a laptop never mig `databricks apps run-local` reads `app.yaml`, so it sets the flag, but it sets no service principal credentials. It migrates only when the shell exports them. As a second guard, migrate on start refuses when `LAKEBASE_ENDPOINT` names a `dev-` or `session-` branch, because only a developer migrates those. - -The stack today infers "deployed" from `NODE_ENV`, `DATABRICKS_APP_NAME` and the credentials. The template's `start` -script sets `NODE_ENV=production` and its `.env` sets `DATABRICKS_APP_NAME`, so a local `npm start` with M2M -credentials migrates the dev branch as the service principal and breaks the one-owner rule. - Rejected alternatives: | Option | Why not | |---|---| | Infer deployment from the environment | False positive on a local `npm start` with M2M credentials. No way to turn it off. | -| An admin runs `appkit db migrate --prod` (the July design) | A human step on every schema change, by a person with DDL rights on the production branch. App Builder has nobody to run it. | +| An admin migrates the deploy branch from the CLI | A human step on every schema change, by a person with DDL rights on the production branch. App Builder has nobody to run it. | | A shared owner role, with CI migrating on merge ([tmm Lakebase Agentic CI](https://github.com/databricks/tmm/tree/main/Lakebase-Agentic-CI)) | Works. It needs a one-time bootstrap on the deploy branch by someone with `ADMIN OPTION`, database credentials in CI, and a grant to every new app service principal. Developers skip the fork hand-off only if they are members on the deploy branch, which gives them DDL rights there. App Builder apps have no CI. | | A one-shot migration job per deploy | Adds a resource to every app, and still needs a shared owner role and its bootstrap so the app can use what the job created. | | A bundle script hook | Runs as the person or CI identity that deploys, which then owns the objects. | @@ -166,8 +158,8 @@ The decision costs two rules that the docs and `appkit db migrate` carry: - **Each migration works with the previous release.** A horizontally scaled rollout updates one instance first while the old instances keep serving. A dropped, renamed or retyped column breaks them. Drops go in a later deploy than - the code that stops using the column (expand, then contract). `appkit db migrate` warns when it writes `DROP`, - `RENAME` or `ALTER ... TYPE`. + the code that stops using the column (expand, then contract). `appkit db migrate` warns when it writes a drop, a + rename, a type change or `SET NOT NULL`. - **A migration fits in the start window.** A migration that runs longer than the platform's start timeout can never land. Long backfills run as a separate job against the deploy branch, not as a migration. @@ -191,7 +183,7 @@ A session advisory lock keyed on the schema name serializes instances that start `pg_try_advisory_lock` until a 5-minute deadline, and each poll logs the holder from `pg_locks` and `pg_stat_activity`. The migrating session sets `application_name = 'appkit-migrate'` so an operator can find it, and `client_connection_check_interval` so Postgres cancels a statement whose instance was killed. Without it a killed -instance's statement keeps running and holds the lock. `lock_timeout` drops from 30s to 5s, with up to three tries per +instance's statement keeps running and holds the lock. `lock_timeout` is 5s, with up to three tries per migration. A waiting `ACCESS EXCLUSIVE` request queues every later query on its table, so the timeout bounds how long traffic stalls, not only the migration. @@ -200,7 +192,7 @@ rows first without the lock. It takes the lock only when an entry is pending, th branch then costs one connection and one read of the journal table on the start path. A transaction-mode pooler would void a session lock. The pooled Lakebase endpoint is closed to OAuth roles, so the -plugin connects direct today. The connector refuses a `-pooler` host so that stays true. +plugin connects direct. The connector refuses a `-pooler` host so that stays true. ### Fail closed on a history the migrator would skip @@ -221,10 +213,13 @@ The migrator reconciles the committed history with the recorded rows once, under | `ahead` | A recorded row with no committed entry, for example after a code rollback. | Warn. | | `partial-baseline` | `0000_baseline` tables exist only in part. | Fail. Nothing changes. | -`appkit db check` fails in CI when `when` does not increase with `idx`, when the snapshot `prevId` chain forks, when a -listed file is missing, or when `schema.ts` differs from the latest snapshot. A journal on `main` that only grows can -then never strand an entry on the deploy branch, because the deploy branch applies a prefix of it. To fix a failed -check, the developer deletes their own migration, merges `main`, and runs `appkit db migrate` again. +`appkit db check` fails in CI when `when` does not increase with `idx` or an `idx` is not its position, when the +snapshot `prevId` chain forks, when a listed file is missing or a `.sql` is not listed, or when `schema.ts` differs +from the latest snapshot. A journal on `main` that only grows can then never strand an entry on the deploy branch, +because the deploy branch applies a prefix of it. Two pull requests that both add `0001_migration` conflict in git +first, and the check catches a resolution that keeps both. To fix a failed check, the developer merges `main`, resets +`config/database/migrations` to `main`'s, and runs `appkit db clean` and `appkit db migrate`. Migrate writes their +change again after `main`'s migrations. The clean is there because their branch already ran the old migration. Two other fixes lose. Applying by hash instead of by `when` runs DDL in an order nobody reviewed. A custom journal forks drizzle-kit's format, and drizzle-kit v1 already replaces the journal. @@ -237,8 +232,7 @@ because Lakebase rejects `GRANT ... TO databricks_superuser`. A fork copies objects with their owners, so `appkit db init` hands the fork's app objects to the developer. It deletes the fork's copy of the role that owns schema `app` and reassigns its objects (`reassign_owned_to`). It touches only -that role, on the fork, and says so before it does. The stack today deletes every role-API role that owns anything in -any schema of the fork. +that role, on the fork, and says so before it does. The role delete returns HTTP 500 when Postgres refuses the drop with SQLSTATE 2BP01, for example when the role still owns objects in another database of the branch @@ -247,8 +241,10 @@ dependencies from `pg_shdepend` before the delete, and on a 500 prints them with Single ownership by the service principal breaks when that principal changes. A recreated app has a new one, and removing and re-adding the database resource hands the objects to the app manager (XTA-18463). Either way the app -can no longer use or alter its schema, and its next start fails. That recovery is an open question, not part of -this stack. The startup error names the current owner of schema `app` and the recovery doc. +can no longer use or alter its schema, and its next start fails. The startup error names the current owner of schema +`app` and the manual recovery, which the docs repeat. When the owner belonged to a deleted app, someone with CAN MANAGE +on the project runs `databricks postgres delete-role --reassign-owned-to `, which moves the +objects and deletes the old role. Otherwise the owner transfers the schema. A tool for it is an open question. ### Branch policy is one resolver and one table @@ -261,7 +257,9 @@ One function resolves the `.env` branch to a kind, in this order: project defaul | deploy target, project default, protected, other | Refuse | Write only | Refuse | Refuse | Read | `init` resolves the kind of the branch it would create or reuse before creating it or writing `.env`. A deploy target -is any `postgres_branch` value in `databricks.yml`, from every target and from the variable default. +is the branch any `databricks.yml` target binds, on the app's `postgres` resource or in `postgres_branch`. One reader +serves the guard, `init` and `verify`. It follows `include:` and resolves variables per target. `verify` also warns +when `.env` points at a branch that is not yours. The name allowlist is the safety boundary. Only `dev--` and `session--` are writable, and the denylist checks come first. Before connecting, the CLI checks that `PGHOST` is a host of @@ -277,8 +275,8 @@ the installed version and the `drizzle-orm` it resolves, and prints the install template dev dependency would add about 32 MB, 12 downloads and two deprecation warnings to every app, including apps that never use it. The template ships one lockfile, so it cannot be dropped per app. -The shared CLI's view of the tooling API is an interface exported from `shared` that the tooling `satisfies`. Drift -then fails typecheck. The stack today keeps a 131-line hand-written declaration with no check. +The shared CLI's view of the tooling API is an interface exported from `shared`, and an appkit test assigns the +tooling's exports to it. Drift then fails typecheck. ### Introspect renders the AppKit builder, not drizzle's DSL @@ -311,21 +309,23 @@ Not flagged: 1. `databricks apps init` scaffolds the app with the plugin. It asks for the Lakebase project and the deploy branch and database, which go into `databricks.yml`. -2. `apps init` runs `appkit db init --yes`. It forks the project's default branch into `dev--`, adds a - read-write endpoint and the developer's role, hands the fork's app objects to the developer, writes `.env`, and - starts the schema (migrate when `schema.ts` exists, introspect otherwise). +2. `apps init` runs `appkit db init`, which asks nothing. It forks the project's default branch into + `dev--`, adds a read-write endpoint and the developer's role, hands the fork's app objects to the + developer, writes `.env`, and starts the schema (migrate when `schema.ts` exists, introspect otherwise). 3. The developer declares a table in `schema.ts` and runs `appkit db migrate`. It writes `0000_migration.sql`, regenerates `shared/appkit-types/database.d.ts`, and applies the migration to the dev branch. 4. The developer commits `config/database/` and runs `databricks apps deploy`. 5. The app starts with `APPKIT_DB_MIGRATE_ON_START=true`. Under the lock it reconciles, applies `0000`, creates and owns schema `app`, checks the declared tables, and listens. -Until the CLI release that runs `appkit db init` ships, step 2 is a command the README tells the developer to run. +Until the CLI release that runs `appkit db init` ships, the developer runs it once after scaffolding, as the README +says. Until then `.env` points at the branch picked in the prompts, usually the deploy branch, and `appkit db verify` +warns about it. ### A table change Edit `schema.ts`, run `appkit db migrate`, commit, deploy. CI runs `appkit db check`. After the deploy, -`appkit db verify --prod` reports whether the app is running the commit's newest migration. +`appkit db verify --deployed -t ` reports whether the app is running with every committed migration. ### A second developer @@ -345,7 +345,9 @@ and column already exists. A partial match fails start and changes nothing. - Package export `@databricks/appkit/database-tooling`. It is internal, with no semver guarantee outside the CLI. - App env: `APPKIT_DB_MIGRATE_ON_START`. Local env written by init: `LAKEBASE_PROJECT`, `LAKEBASE_ENDPOINT`, `PGHOST`, `PGDATABASE`, `APPKIT_DB_SESSION`, and `DATABRICKS_CONFIG_PROFILE` or `DATABRICKS_HOST`. -- New flags: `appkit db init --ttl ` for session branches, `appkit db clean --delete-branch`. +- Flags: `appkit db init --dev | --session`, `--ttl ` for session branches, + `--schema ` and `-y, --yes`. `appkit db clean --delete-branch`. + `appkit db verify --deployed` and `-t, --target `. - Optional peer dependency: `drizzle-kit` `0.31.10`. ## Observability @@ -373,20 +375,20 @@ the schema and the role. - **`.env`.** Init writes it with mode 0600 through a temporary file and a rename, and refuses a symlink. When `.env` is missing, init writes its own keys and the profile or host it authenticated with. It never copies `.env.example`, whose `DATABRICKS_HOST` is a placeholder. -- **Session branches are bound to a user.** The stack today lets anyone with a copy of `.env` pass the guard for its - session branch. With the user hash in the name, a copied `.env` fails the guard for everyone but its creator. +- **Session branches are bound to a user.** With the user hash in the name, a copied `.env` fails the guard for + everyone but its creator. ## Platform Dependencies | Ask | Why | |---|---| -| Databricks CLI: `apps init` runs `appkit db init --yes` after install. | Removes the manual init step. Without it the README tells the developer to run init. | -| Databricks CLI: a minimum version recorded in the template. | An older CLI scaffolds without init, and `.env` points at the deploy branch. | +| Databricks CLI: `apps init` runs `appkit db init` after install. | Removes the manual init step. Without it the README tells the developer to run init. | +| Databricks CLI: a minimum version recorded in the template. | An older CLI scaffolds without init, and `.env` points at the deploy branch until the developer runs it. `appkit db verify` warns meanwhile. | | Lakebase: `client_connection_check_interval` allowed for app roles. | Cancels the statement of a killed instance that still holds the migration lock. | | Lakebase: role delete returns the 2BP01 dependency error as a 4xx, not a 500 ([ES-2265656](https://databricks.atlassian.net/browse/ES-2265656), open). | Init's hand-off fails on every rerun when it hits this, with no reason given. | -| Lakebase: raise the default of 10 unarchived branches per project ([LKB-12894](https://databricks.atlassian.net/browse/LKB-12894), backlog, proposes 100). | One branch per developer caps a project at about eight developers. | +| Lakebase: raise the quota of 25 unarchived branches per project ([LKB-12894](https://databricks.atlassian.net/browse/LKB-12894), backlog, proposes 100). | One branch per developer caps a project at about 20 developers. | | Apps: keep object ownership with the app across a database resource remove and re-add ([XTA-18463](https://databricks.atlassian.net/browse/XTA-18463), to do). | The app cannot use its own schema after a re-add, and start fails. | -| Apps: the start timeout and whether a non-zero exit restarts the app. | Sets the lock wait and the longest migration. The template exits 0 on a failed start today. | +| Apps: the start timeout and whether a non-zero exit restarts the app. | Sets the lock wait and the longest migration. The template exits 1 on a failed start. | ## Rollout Plan @@ -395,37 +397,38 @@ the schema and the role. 2. **Runtime, dark.** Land #654 and #655 with the flag, the per-migration transactions, the reconcile states and the bounded lock. Done when a real-Postgres test suite passes in CI, and the dev-playground deployed with the flag on applies `0000` and then `0001`. -3. **Commands, hidden.** Land #666 to #674 and `appkit db check`. Done when two people on the dev-playground add - migrations on parallel git branches, `appkit db check` fails the second PR, and the documented fix passes. -4. **Reveal.** Land #675 once the CLI release that runs `appkit db init` is out. Done when `apps init` with the - plugin, one table and a deploy need no step outside the README. +3. **Commands, hidden.** Land #666 to #674. Done when a developer on the dev-playground runs `init`, `migrate` and + `verify` on their own branch, and the deployed app applies the migration. +4. **Reveal.** Land #675, then #678 (`appkit db check`). Until a CLI release runs `appkit db init`, the README says to + run it once. Done when `apps init` with the plugin, one table and a deploy need no step outside the README, and + when two people on the dev-playground add migrations on parallel git branches, `appkit db check` fails a merge + that keeps both, and the documented fix passes. 5. **GA of the plugin** is a separate plan. ## PR Stack 1. `00-design`: this doc. -2. [#654](https://github.com/databricks/appkit/pull/654) Migrations on deploy start. Changes from this design: the - flag, per-migration transactions, the reconcile states with fail-closed `stranded`, the bounded lock wait, - `application_name`, `client_connection_check_interval`, the 5s lock timeout, the lock-free read on a current - branch, the `-pooler` host refusal, the dev and session branch refusal, the schema owner in the startup error, - structured migration events, and a real-Postgres suite. -3. [#655](https://github.com/databricks/appkit/pull/655) `app` default schema. Changes: the `public` hint in the +2. [#654](https://github.com/databricks/appkit/pull/654) Migrations on deploy start: the flag, one transaction per + migration, the reconcile states with fail-closed `stranded`, the bounded lock wait, a 5s lock timeout, the + lock-free read on a current branch, the `-pooler` and developer-branch refusals, the schema owner in the startup + error, structured migration events, and a real-Postgres suite. +3. [#655](https://github.com/databricks/appkit/pull/655) The `app` default schema, with the `public` hint in the startup check and an upgrade note. -4. [#666](https://github.com/databricks/appkit/pull/666) `appkit db migrate`. Changes: discard the run's migration - when applying it fails, the drizzle-kit version check, the destructive-statement warning, the `satisfies`-checked - contract, and the branch resolver. -5. [#670](https://github.com/databricks/appkit/pull/670) `appkit db introspect`. Changes: compare-and-set recording, - refuse on branches you do not own, a drop refusal that matches the actual targets, and catalog tests that run SQL. -6. [#672](https://github.com/databricks/appkit/pull/672) `appkit db init`, `clean`, `seed`. Changes: the hand-off - scoped to the app schema's owner, the 2BP01 dependency report, resolve before create, session ids with the user - hash, the 5-minute scale to zero, the branch-limit stop, `--ttl` and `--delete-branch`, atomic `.env` writes - without the `.env.example` placeholder, extension-aware `public` clean, and drizzle-kit install. -7. [#674](https://github.com/databricks/appkit/pull/674) `appkit db verify`. Changes: report `ahead` rows, - `--target`, and compare the deployed journal with the local one. -8. New: `appkit db check`. -9. [#675](https://github.com/databricks/appkit/pull/675) Offer the plugin in `apps init`. Changes: the flag in - `app.yaml`, no drizzle-kit in the template, a non-zero exit on a failed start, README wording that works before - the CLI release, and docs for existing apps, rollback and CI. +4. [#666](https://github.com/databricks/appkit/pull/666) `appkit db migrate`: the branch resolver, the drizzle-kit + version check, the destructive-statement warning, and a checked tooling contract. A migration the run wrote is + discarded when applying it fails. +5. [#670](https://github.com/databricks/appkit/pull/670) `appkit db introspect`: compare-and-set recording, only on + your own branch, and catalog tests that run SQL. +6. [#672](https://github.com/databricks/appkit/pull/672) `appkit db init`, `clean`, `seed`: dev and session branches + with no prompts, the hand-off of the app schema, the 2BP01 dependency report, the 5-minute scale to zero, the + branch-limit stop, `--ttl` and `--delete-branch`, atomic `.env` writes, and the drizzle-kit install. +7. [#674](https://github.com/databricks/appkit/pull/674) `appkit db verify`: the local check, including `ahead` rows, + and `--deployed` with `-t, --target`, which checks the host and compares the deployed journal with the local one. + One `databricks.yml` reader serves the guard, `init` and `verify`. +8. [#675](https://github.com/databricks/appkit/pull/675) Offer the plugin in `apps init`: the flag in `app.yaml`, no + drizzle-kit in the template, exit 1 on a failed start, a README that works before the CLI release, `verify`'s + warning for a `.env` branch that is not yours, the recreated-app recovery in the owner error, and the docs. +9. [#678](https://github.com/databricks/appkit/pull/678) `appkit db check`. ## Open Questions @@ -439,15 +442,16 @@ the schema and the role. CAN MANAGE also allows deleting the deploy branch. Reviewer: Lakebase. 5. Is a fork of production rows acceptable for customer data by default, or should init default to an empty schema? Does Lakebase offer schema-only branches? Reviewer: Lakebase, security. -6. How does an app with a new service principal take ownership back, after a recreate or a resource re-add? Options - are Apps transferring ownership on re-add, or an `appkit db adopt --prod` built on the role API. Reviewer: Lakebase, - Apps. +6. How does an app with a new service principal take ownership back, after a recreate or a resource re-add? The + stack documents a manual recovery with `databricks postgres delete-role --reassign-owned-to`. Options for a + zero-touch one are Apps transferring ownership on re-add, or an `appkit db adopt --deployed` built on the role API. + Reviewer: Lakebase, Apps. 7. What scale to zero do child branch computes get by default? Production gets 24 hours. Reviewer: Lakebase. 8. How does Genie App Builder drive this flow? Its apps have no per-developer `.env` and no CI, and they start on every cold start. Reviewer: App Builder. 9. Ship on drizzle-kit 0.31's journal, or wait for drizzle-kit v1's folder format, which removes the journal and its merge conflicts? Reviewer: AppKit. -10. Which CLI release runs `appkit db init`, with which flags, before or after the dependency install? Reviewer: - Databricks CLI. +10. Which CLI release runs `appkit db init`, and does it run before or after the dependency install? Init asks nothing, + so it needs no flags. Reviewer: Databricks CLI. 11. This design has `appkit db init` install drizzle-kit. Should the CLI support per-plugin dependencies instead, so apps that select the plugin get it at scaffold time? Reviewer: AppKit, Databricks CLI.