Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 80 additions & 7 deletions docs/docs/plugins/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,9 @@ not support OBO and does not apply per-user or per-row authorization. Every call
who can reach the generated API can perform every enabled operation on every
exposed row, including delete.

Restrict access to the app and grant its service principal only the database
permissions it needs. If users need different permissions or row ownership checks,
disable the relevant generated routes and implement authorization in custom
server routes. App admission alone does not provide row-level isolation.
Restrict access to the app. If users need different permissions or row ownership
checks, disable the relevant generated routes and implement authorization in
custom server routes. App admission alone does not provide row-level isolation.
:::

## Basic usage
Expand All @@ -39,9 +38,10 @@ identifier-only messages for known missing-column or missing-table errors
details, and hints are omitted because they may contain row values.

Apps scaffolded with the Database plugin selected include an empty
`config/database/schema.ts`, so `database()` can start without requiring sample
tables. Replace the empty declaration with your models when their PostgreSQL
tables are ready.
`config/database/schema.ts` in the Postgres schema `app`, and `databricks apps init`
points `.env` at a development branch of your own. Declare your tables, then run
`npx appkit db migrate` to create them there (see
[Development workflow](#development-workflow)).

For local development, the PostgreSQL username is resolved from your Databricks
credentials when `PGUSER` and `DATABRICKS_CLIENT_ID` are absent. An explicitly
Expand Down Expand Up @@ -89,6 +89,79 @@ Tables live in the Postgres schema `app` unless the second argument of
default is not `public` because the deployed app's service principal creates and
owns `app`, but has no `CREATE` privilege on `public`.

## Development workflow

You develop against a Lakebase branch of your own, never the deployed one. Tables
are created and changed through migrations: generated locally from `schema.ts`,
committed with your code, and applied by the deployed app when it starts.

| Command | What it does |
| --- | --- |
| `npx appkit db init` | Points `.env` at your branch: a dev branch of yours (`--new dev`) or a 7-day branch for this checkout (`--new session`), forked from the project's default branch. |
| `npx appkit db migrate` | Writes `config/database/migrations/NNNN_*.sql` when `schema.ts` changed, and applies pending migrations to your branch. |
| `npx appkit db introspect` | Writes `schema.ts` from tables that already exist, and a `0000_baseline` migration (or `NNNN_introspect` when migrations already exist). |
| `npx appkit db clean` | Drops the app schema on your branch; `migrate` then rebuilds it. |
| `npx appkit db seed` | Runs `config/database/seed.sql` on your branch. |
| `npx appkit db verify` | Read-only check of your branch: missing tables or columns, pending migrations, and migrations that will never run there. `--prod` checks that the deployed app is set up to use the deploy branch. |

`databricks apps init` runs `db init` for you. Run it again after cloning, or with
`--new` to switch between your dev and session branches. Without `--new`, it keeps
your dev or session branch when `.env` points at one, and otherwise asks (with
`--yes`, or without a terminal, it uses your dev branch).

`migrate`, `clean` and `seed` change only branches `db init` created for you (your
dev branch, or this checkout's session branch). On any other branch, including the
project's default branch, a protected branch, or the branch `databricks.yml`
deploys to, `clean` and `seed` refuse and `migrate` only writes the migration files.

A typical change:

```bash
# edit config/database/schema.ts
npx appkit db migrate # writes and applies 0001_migration.sql on your branch
git add config/database # commit schema.ts with its migrations
databricks apps deploy # the deployed app applies 0001 on start
```

### Migrations on deploy

When the deployed app starts, `database()` applies the committed migrations the
deploy branch has not recorded yet, before it checks the tables and serves
requests. They run as the app's service principal, one instance at a time, in one
transaction. If one fails, none are applied and the app does not start; the server
log names the migration and the SQLSTATE, or the missing migration file. A
migration that waits more than 30 seconds for a table lock, for example behind a
long transaction of the running app, fails instead of blocking that table. Local
runs (`npm run dev`) never apply migrations: use `npx appkit db migrate`.

A `0000_baseline` from `introspect` is recorded without running when its tables
already exist on the deploy branch and no migration was ever applied there. If
only some of them exist, the app does not start, and nothing is changed.

### One owner per branch

Each branch has one owner of the app schema, and only the owner can change its
tables:

- On the deploy branch, the app's service principal creates schema `app` on its
first start and owns everything in it. The `postgres` resource's
`CAN_CONNECT_AND_CREATE` permission is all it needs; no grants are required.
- On your dev or session branch, you own it. A fork copies its parent's objects
with their owners, so `db init` hands them over to you.

Do not apply migrations to the deploy branch yourself: you would own those objects,
and the app could no longer change them. If schema `app` on the deploy branch
already belongs to someone else, the app cannot migrate it. Use another
`schemaName` in `defineSchema()`, or have the owner transfer the schema.

`npx appkit db verify --prod` checks, through the Databricks API, that the deployed
app is set up to use the deploy branch: the app exists, its `postgres` resource
names the branch and database in `databricks.yml` with `CAN_CONNECT_AND_CREATE`, its
service principal has a role on the branch, and the app is running. A running app
has applied its migrations and checked its schema on start; otherwise the report
names `databricks apps logs`, `start` or `deploy`. It needs no access to the deploy
branch's database and changes nothing.

## Schema discovery and overrides

`database()` and `database({})` use the same defaults. During setup, the plugin
Expand Down
1 change: 0 additions & 1 deletion packages/appkit/src/beta.ts
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,6 @@ export type {
SearchResult,
} from "./plugins/ai-search/types";
export * from "./plugins/beta-exports.generated";
export { database } from "./plugins/database";
export type {
DatabaseApiConfig,
DatabaseApiWriteOperation,
Expand Down
1 change: 1 addition & 0 deletions packages/appkit/src/plugins/beta-exports.generated.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion packages/appkit/src/plugins/database/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
"displayName": "Database (Beta)",
"description": "Schema-driven typed access to Databricks Lakebase PostgreSQL",
"stability": "beta",
"hidden": true,
"resources": {
"required": [
{
Expand All @@ -17,6 +16,7 @@
"project": {
"description": "Lakebase project resource name",
"examples": ["projects/{project-id}"],
"bundleIgnore": true,
"discovery": {
"type": "kind",
"resourceKind": "postgres_project",
Expand Down
3 changes: 1 addition & 2 deletions packages/shared/src/cli/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,7 @@ cmd.addCommand(docsCommand);
cmd.addCommand(pluginCommand);
cmd.addCommand(codemodCommand);
cmd.addCommand(doctorCommand);
// Hidden from --help until the database plugin is offered by `databricks apps init`.
cmd.addCommand(dbCommand, { hidden: true });
cmd.addCommand(dbCommand);
// Registry commands are executable but hidden from --help while the feature
// is still in development (registry + add work end-to-end but aren't announced).
cmd.addCommand(registryCommand, { hidden: true });
Expand Down
39 changes: 32 additions & 7 deletions template/README.md.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,33 @@ The Lakebase plugin requires additional environment variables for PostgreSQL con

{{- if .plugins.database}}

#### Database schema
#### Database

`database()` loads `config/database/schema.ts`. The generated schema starts empty,
so initialization does not require sample tables or expose existing data.
Add your table declarations after creating the corresponding PostgreSQL tables;
AppKit does not create or migrate them. CRUD routes are enabled for declared tables
by default, and can be restricted with `api` in the plugin configuration.
`database()` loads `config/database/schema.ts`, whose tables live in the Postgres
schema `app`. CRUD routes are enabled for declared tables by default, and can be
restricted with `api` in the plugin configuration.

Local development runs against a Lakebase branch of your own, never the deployed one.
`databricks apps init` set it up by running `npx appkit db init`; run it again at any
time (for example after cloning) to point `.env` at your branch:

```bash
npx appkit db init # keep or pick your branch, then set up its schema
npx appkit db init --new dev # your own branch, forked from the default branch
npx appkit db init --new session # a branch for this checkout that expires after 7 days
```

To change the schema, edit `config/database/schema.ts`, then:

```bash
npx appkit db migrate # writes config/database/migrations/NNNN_*.sql and applies it to your branch
```

Commit the migrations. To adopt tables that already exist, run `npx appkit db introspect`:
it writes `schema.ts` from the branch's tables, and a `0000_baseline` migration (or
`NNNN_introspect` when migrations already exist). `npx appkit db clean` drops the app
schema on your branch, and `npx appkit db seed` runs `config/database/seed.sql` on it;
both refuse any branch `db init` did not create for you, such as the deployed one.
{{- end}}

### CLI Authentication
Expand Down Expand Up @@ -196,10 +216,15 @@ databricks apps deploy -t prod
* server.ts # Server entry point
* routes/ # Routes
* shared/ # Shared types
{{- if .plugins.analytics}}
{{- if or .plugins.analytics .plugins.database}}
* config/ # Configuration
{{- end}}
{{- if .plugins.analytics}}
* queries/ # SQL query files
{{- end}}
{{- if .plugins.database}}
* database/ # schema.ts and its migrations
{{- end}}
* databricks.yml # Bundle configuration
* app.yaml # App configuration
* .env.example # Environment variables example
Expand Down
3 changes: 3 additions & 0 deletions template/_gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ build/
test-results/
playwright-report/

# `appkit db introspect` keeps the previous schema.ts here
config/database/*.bak

# Auto-generated types (endpoint-specific, varies per developer)
shared/appkit-types/serving.d.ts

109 changes: 109 additions & 0 deletions template/appkit.plugins.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions template/config/database/schema.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{{if .plugins.database -}}
import { defineSchema } from '@databricks/appkit/beta';

// database() discovers this file automatically. Start with no exposed tables.
// Add declarations here after creating the corresponding PostgreSQL tables.
// database() discovers this file automatically. Edit tables here, then run
// `npx appkit db migrate` on your dev branch. The deployed app's service principal
// owns schema "app" and applies committed migrations on start.
export const schema = defineSchema(() => ({}));
{{- end}}
Loading