diff --git a/CHANGELOG.md b/CHANGELOG.md index 686a1705..b391c2c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,65 @@ - Date format: YYYY-MM-dd +## 2.5.0 / 2026-10-11 + +### All + +* Update `Kotlin`'s version to `2.4.21` +* Rebuild the SQLite static libraries in `libs`, which the native tests on Linux and Windows link, from SQLite 3.51.1 with FTS3, FTS4 (with the enhanced query syntax), FTS5, R*Tree, Geopoly, the math functions, `dbstat` and column metadata enabled, and without debug information, which held paths of the machine that built them + +### sqllin-dsl + +* New API: the opt-in annotation `@PlatformDependentSQLiteAPI`, which marks the APIs that only work with some SQLite versions or compile-time options, which some platforms SQLlin supports don't have, such as Android's system SQLite at lower API levels. Using one is a compile error unless it is accepted with `@OptIn(PlatformDependentSQLiteAPI::class)` +* **Breaking change**: `RENAME_COLUMN`, which needs SQLite 3.25.0 (Android API 30), and `DROP_COLUMN`, which needs SQLite 3.35.0 (Android API 34), are marked with `@PlatformDependentSQLiteAPI`, so code using them has to opt in +* New experimental DSL API: joins of relations. `FROM(relation)` starts a join, which `INNER_JOIN`, `LEFT_OUTER_JOIN`, `RIGHT_OUTER_JOIN`, `FULL_OUTER_JOIN`, `CROSS_JOIN` and the NATURAL joins extend, each with its own `ON` or `USING`, as in `(FROM(PersonTable) LEFT_OUTER_JOIN BookTable ON (PersonTable.id EQ BookTable.authorId)) SELECT listOf(...)`. A join is selected from with a projection, a clause function such as `WHERE(...)`, or result columns, which can now select from a join, and the type its rows are read into is checked against the joined relations when the statement is built: every property has to name a column of one of them, of a single one unless USING or a NATURAL join merges them, and has to be nullable where an outer join can leave its column or expression NULL. Columns are written with their relations' names, and an observed query of a join watches all its relations. `RIGHT_OUTER_JOIN`, `FULL_OUTER_JOIN` and their NATURAL forms need SQLite 3.39.0 (Android API 34), so they are marked with `@PlatformDependentSQLiteAPI` +* The joins written as clauses of SELECT, such as `table SELECT INNER_JOIN(other) ON (...)`, which don't check the type their rows are read into, will be removed in the next version after 2.5.0: use the joins of relations instead +* `CREATE_INDEX` and `CREATE_UNIQUE_INDEX` document that a function of the table's columns makes an expression index, as `lower(email)` for case-insensitive lookups or uniqueness, and reject a column or function of another table +* New experimental DSL API: `IF NOT EXISTS` and `IF EXISTS`, with `CREATE_IF_NOT_EXISTS`, `CREATE_INDEX_IF_NOT_EXISTS`, `CREATE_UNIQUE_INDEX_IF_NOT_EXISTS`, `CREATE_VIEW_IF_NOT_EXISTS`, `DROP_IF_EXISTS` and `DROP_INDEX_IF_EXISTS`, which do nothing when the table, FTS table, index or view exists, or doesn't, so that a migration step can run more than once +* New experimental DSL API: `DROP_INDEX(name)`, for SQL syntax `DROP INDEX`, which drops an index that `CREATE_INDEX` or `CREATE_UNIQUE_INDEX` created. The guide to modifying the database now describes indexes +* New experimental DSL API: partial indexes. `WHERE` after `CREATE_INDEX`, `CREATE_UNIQUE_INDEX` or their `IF_NOT_EXISTS` forms indexes only the rows that satisfy a condition, as in `UserTable.CREATE_UNIQUE_INDEX("idx_user_email", UserTable.email) WHERE (UserTable.isDeleted IS false)`. SQLite allows neither parameters nor subqueries there, so the values of the condition are written into the SQL, and a subquery is rejected with an `IllegalArgumentException`. To support it, these functions now return an `IndexStatement`, where they returned `Unit`; this is source-compatible, but a library compiled against an earlier version has to be recompiled +* New experimental DSL API: `DISTINCT(element)`, for aggregate functions of the distinct values, as in `count(DISTINCT(author))` for `count(DISTINCT book.author)`. `count`, `avg`, `sum` and `group_concat` take it, and `group_concat` separates the distinct values with commas, as SQLite allows no other separator with DISTINCT. What it returns, a `Distinct`, can't be used anywhere else, which is checked at compile time +* New experimental DSL API: `ESCAPE` after `LIKE`, for SQL syntax `LIKE pattern ESCAPE character`, as in `name LIKE "100\\%%" ESCAPE '\\'`, after which `%`, `_` and the escape character match only themselves. To support it, `LIKE` now returns a `LikeCondition`, which is a `SelectCondition`; this is source-compatible, but a library compiled against an earlier version has to be recompiled +* New experimental DSL API: `VACUUM()` and `VACUUM_INTO(file)`, for SQL syntax `VACUUM` and `VACUUM INTO`, which rebuild the database file into the minimum disk space, or write a vacuumed copy of the database to another file, a consistent backup of a database in use. Both throw an `IllegalStateException` inside a transaction, where SQLite can't vacuum. `VACUUM_INTO` needs SQLite 3.27.0 (Android API 30), so it is marked with `@PlatformDependentSQLiteAPI` +* New experimental DSL API: expressions. `+`, `-`, `*`, `/`, the unary `-` and, of integer types, `%` compute with numbers of the same type, and with values of their type, as in `pages + 1`, giving a number of that type, and `+` concatenates strings, SQL's `||`. They work in result columns, conditions and ORDER BY. SQLite gives NULL for a division by zero, so `/` and `%` can be NULL unless the divisor is a value other than zero. `literal(value)` makes a value an expression, and `CAST(x AS type)` converts an expression to another type, named as SQLlin names the columns of that Kotlin type: `TINYINT`, `SMALLINT`, `INT`, `BIGINT`, `FLOAT`, `DOUBLE`, `TEXT` and `BLOB`. The values in an expression are written into its SQL, so that it can be part of a view, a trigger or an index, which SQLite doesn't let take parameters. A literal on its own in ORDER BY or GROUP BY is rejected, as SQLite would read it as the number of a result column +* New experimental DSL API: more functions, whose arguments are expressions: `coalesce`, `ifnull` and `nullif`, the scalar `max` and `min` of several expressions, `hex`, `quote`, `unicode`, `randomblob`, `zeroblob`, `trim`, `ltrim` and `rtrim` with the characters to remove, `substr` from a position, `printf` of several values, `substr`, `replace` and `instr` of expressions, `round` to an integer, the aggregate `total`, and the date and time functions `date`, `time`, `datetime`, `julianday` and `strftime`. A function can only be NULL where its result can: `coalesce` where all its arguments can, and a date and time function always, but `datetime("now")`. `sign` (SQLite 3.35.0, Android API 34), `unixepoch` (SQLite 3.38.0, Android API 34) and the math functions `ceil`, `floor`, `trunc`, `sqrt`, `exp`, `ln`, `log10`, `log2`, `pow`, `mod` and `pi`, which Android's SQLite has at no API level, are marked with `@PlatformDependentSQLiteAPI` +* New experimental DSL API: more expression syntax: `CASE(WHEN(condition) THEN value, ..., ELSE = value)`, SQL's searched CASE, which is NULL where `ELSE` is omitted; scalar subqueries, `select[R::property]`, a SELECT of a single column used as a value, which is checked when the statement is built, and whose tables an observed query watches; `IS` and `ISNOT` of two expressions, the null-safe comparisons, which do what `IS DISTINCT FROM` does on every SQLite version; the COLLATE operator with the collations `BINARY`, `NOCASE` and `RTRIM`; and the bitwise operators `and`, `or`, `inv()`, `shl` and `shr` of `Int` and `Long` +* New experimental DSL API: SET and INSERT of expressions. `SET(listOf(expression AS Row::property))` sets columns to expressions, as in `table UPDATE SET(listOf((visits + 1) AS Person::visits)) WHERE (...)`, and `INSERT`, `INSERT_OR_IGNORE` and `INSERT_OR_REPLACE` take a row of expressions, `table INSERT listOf(literal("purge") AS Audit::action, datetime("now") AS Audit::at)`, for SQL syntax `INSERT INTO ... VALUES`, leaving the other columns to their default values or NULL. `AS` checks the type of each expression at compile time, and the statement checks, when it is built, that an expression that can be NULL is only written to a nullable column, that it isn't an aggregate function, and that it only reads the columns of the updated table, or none for INSERT, where SQLite allows none +* New experimental DSL API: triggers. `CREATE_TRIGGER(name) AFTER DELETE ON PersonTable WHEN (condition) BEGIN { ... }`, for SQL syntax `CREATE TRIGGER`, with `BEFORE` and `AFTER`, the events `INSERT`, `DELETE`, `UPDATE` and `UPDATE_OF(columns)`, and `CREATE_TRIGGER_IF_NOT_EXISTS`. `new(column)` and `old(column)` read the row being written, and a row the trigger doesn't have, which SQLite would only report when the trigger fires, is rejected when the statement is built. The body inserts, updates and deletes rows, with expressions, and `RAISE(ABORT, message)`, `RAISE(FAIL, message)`, `RAISE(ROLLBACK, message)` and `RAISE(IGNORE)` stop the statement that fired the trigger or skip its row. The values of a trigger are written into its SQL, as SQLite doesn't let a trigger take parameters. A trigger can't be on an FTS table or a view; INSTEAD OF triggers on views are left to another item of the roadmap. `DROP_TRIGGER(name)` and `DROP_TRIGGER_IF_EXISTS(name)` drop one. An observed query of an FTS table now counts the writes to any table that has triggers as changes of the FTS table too, as a trigger can write it +* New experimental DSL API: `table INSERT DEFAULT_VALUES`, for SQL syntax `INSERT INTO ... DEFAULT VALUES`, which inserts a row of the default values of all columns, with `INSERT_OR_IGNORE` and `INSERT_OR_REPLACE` too +* New experimental DSL API: `ANALYZE()`, `ANALYZE(table)` and `ANALYZE(name)`, for SQL syntax `ANALYZE`, which gathers the statistics the query planner chooses indexes with; `REINDEX()`, `REINDEX(table)`, `REINDEX(name)` and `REINDEX(collation)`, for `REINDEX`, which rebuilds indexes; and `PRAGMA_OPTIMIZE()`, for `PRAGMA optimize`, which needs SQLite 3.18.0 (Android API 26), so it is marked with `@PlatformDependentSQLiteAPI` +* New experimental DSL API: the orders of the columns of an index, ASC and DESC, as in `PersonTable.CREATE_INDEX("idx_person_age", PersonTable.age to DESC)`, with `CREATE_UNIQUE_INDEX` and the `IF_NOT_EXISTS` forms too +* New experimental DSL API: `NULLS FIRST` and `NULLS LAST` in ORDER BY, as in `ORDER_BY(age to ASC NULLS LAST)`, which need SQLite 3.30.0 (Android API 31), so they are marked with `@PlatformDependentSQLiteAPI`. To support them, ORDER BY takes an `Ordering`, which `OrderByWay` is, rather than an `OrderByWay`; this is source-compatible, but a library compiled against an earlier version has to be recompiled +* New API: `deferred = true` on `@References` and `@ForeignKeyGroup`, which makes a foreign key `DEFERRABLE INITIALLY DEFERRED`: checked when the transaction commits, rather than after each statement +* New experimental API: the `onConflict` of `@PrimaryKey`, `@CompositePrimaryKey`, `@Unique` and `@CompositeUnique`, with the enum `OnConflict`, for the `ON CONFLICT` clause of the constraint, as in `UNIQUE ON CONFLICT REPLACE`. A composite key or group takes the one given on any of its properties, and the processor rejects a key or group whose properties give different ones. ABORT, the default, writes no clause +* `CREATE_INDEX` and `CREATE_UNIQUE_INDEX` reject an expression with a subquery or an aggregate function, or a function that doesn't always give the same result for a row, such as `random()`, `datetime('now')` or a date and time function with the `localtime` modifier, and the WHERE of a partial index rejects such a function, where SQLite would only fail once the table has rows +* New experimental DSL API: subqueries. `select AS SomeView` makes a SELECT a derived table, `(SELECT ...) AS name`, named after the object generated for a `@DBView` class, which gives it its name and column properties, though no such view has to exist in the database. A derived table is read like a table, in the FROM clause and in joins, which can join a table with itself through a derived table of its rows. `IN` takes a SELECT of a single column of values of the same kind, which is checked when the statement is built, and `EXISTS(select)` checks whether a SELECT returns any row; it can refer to the columns of the query it is in. They also work in the WHERE of UPDATE and DELETE. Parameters are bound in the order the SQL has them, and an observed query watches the tables its subqueries read +* New experimental DSL API: `NOT(condition)`, which negates a condition in parentheses, as in `NOT(EXISTS(select))` +* New experimental API: the repeatable annotation `@Check`, which adds a `CHECK` constraint with an SQL expression every row has to satisfy: on a property, to its column, and on a `@DBRow` class, to the table. `constraintName` names it, which SQLite reports when a row fails it; SQLite before 3.34, as on Android below API 34, reports an unnamed one by the table's name. The processor rejects an empty expression, a name that isn't an SQL identifier, and two constraints of a table with the same name, and views and FTS tables reject `@Check` +* New experimental DSL API: FTS4 and FTS3 full-text search tables, which SQLite has on every platform SQLlin supports, Android's included. A `@DBRow` class annotated with `@Fts4` or `@Fts3` gets a generated `FtsTable` object, which `CREATE` creates with `CREATE VIRTUAL TABLE`. Its columns are the class's `String` properties, and a `@PrimaryKey` property of type `Long` or `Long?` named `rowid` or `docid` reads and writes the rowid. `@Fts4` takes the tokenizer and its arguments, the prefix lengths to index and the columns not to index, and `@Fts3` the tokenizer and its arguments. `MATCH` searches the table or one of its columns, and `snippet`, `offsets` and `matchinfo` describe the matches. An observed query of an FTS table counts its changes from SQLlin's INSERT, UPDATE and DELETE on it, as a virtual table can't have triggers. `withName` on an FTS table returns an FTS table, so it is rebuilt as a table is +* New experimental DSL API: views. A class annotated with `@DBView` gets a generated view object, such as `AdultView` for `Adult`, which `CREATE_VIEW(AdultView) AS (select)` creates from a SELECT of its row type, checked at compile time, and `DROP(AdultView)` drops. SQLite doesn't let a view take parameters, so the SELECT's values are written into the view's SQL. A view is queried like a table, in joins and compound SELECTs too, and an observed query of it watches the tables it reads, which SQLite lists through `EXPLAIN`. To keep views from being written to, the new `Relation` is the base of `Table` and `View`: SELECT, joins, the SQL functions and the clause elements take a `Relation`, while INSERT, UPDATE, DELETE, CREATE and ALTER keep taking a `Table`. This is source-compatible, but on Kotlin/Native a library compiled against an earlier version has to be recompiled +* New experimental API: `Database#observe`, which turns a SELECT into a `Flow` of its results, emitted when it is collected and again whenever a statement run through the database changes a table the SELECT reads, if the results differ. A join or a compound SELECT watches all of its tables. The changes are counted by SQLite itself, through TEMP triggers on the observed tables, so rows changed by foreign key actions count, while rolled back transactions and statements that match no rows don't. Changes made through another `Database` instance, connection or process aren't seen. Using it needs `@OptIn(ExperimentalDSLDatabaseAPI::class)` +* `kotlinx.coroutines` is now an `api` dependency of `sqllin-dsl`, as `Database#observe` returns a `Flow` +* A `Database` creates the tracker of observed queries when it observes its first query, so that an app that observes none doesn't reach its code, which code shrinkers then remove: on Kotlin/Native, a release executable of an app that only inserts, queries, updates and deletes is about 418 KB smaller, and with R8, the code of `sqllin-dsl` it keeps is a quarter smaller +* New experimental DSL API: compound SELECTs, with `UNION`, `UNION_ALL`, `INTERSECT` and `EXCEPT` as infix operators between two SELECTs of the same result type, as in `(table SELECT WHERE(age LT 18)) UNION (table SELECT WHERE(age GTE 65))`. The result type is checked at compile time, so the SELECTs can combine the rows of different tables through projections, result columns or joins. Chained operators are evaluated from left to right, as in SQL, and parentheses group them. A SELECT that ends with ORDER BY or LIMIT keeps them to itself, and the compound can be followed by its own ORDER BY and LIMIT. Using it needs `@OptIn(ExperimentalDSLDatabaseAPI::class)` +* `DSLDBConfiguration`, `Database(DSLDBConfiguration)` and the DSL APIs `CREATE`, `CREATE_INDEX`, `CREATE_UNIQUE_INDEX`, `DROP`, `ALTER_ADD_COLUMN`, `ALTER_RENAME_TABLE_TO`, `RENAME_COLUMN`, `DROP_COLUMN` and `PRAGMA_FOREIGN_KEYS` are no longer experimental, so using them no longer needs `@OptIn(ExperimentalDSLDatabaseAPI::class)` +* Deprecated: the `UNION {}` and `UNION_ALL {}` blocks, and the `DatabaseScope#beginUnion`, `createUnionSelectStatement` and `endUnion` functions they use. A block returns the table's row type but reads the rows into its first SELECT's result type, so a union of projections, result columns or joins fails with a `ClassCastException` when its results are used, and a nested block is flattened into the SQL, which changes its meaning unless it is the block's first statement. These bugs won't be fixed: use the infix operators instead. The blocks will be removed in a future version +* Fix documentation: the guide to rebuilding a table didn't drop the views that read it, so on the JVM, Linux and Windows, where SQLite's `legacy_alter_table` setting is off by default, renaming the new table failed while a view read the dropped one, with "error in view ...: no such table". The guide now drops the views before dropping the old table and creates them again after the rename, which works on every platform, and explains that only where the setting is off do the foreign keys of other tables follow a renamed table regardless of whether foreign keys are enforced +* Fix: a column given to a function, to GROUP BY or to ORDER BY, or selected into a property with `AS`, was written without its table's name, so in a join of tables that both have a column of that name, SQLite failed with "ambiguous column name". Such columns are now written with their tables' names, as conditions already were, except in the ORDER BY of a compound SELECT, which can only name the columns of its results +* Fix: AND and OR combined conditions without parentheses. They apply in the order they are written, as Kotlin's infix functions do, so `(age EQ 1) OR (age EQ 2) AND (name EQ "x")` means `(age = 1 OR age = 2) AND name = 'x'`, but SQL gives AND precedence over OR, so SQLite read it as `age = 1 OR (age = 2 AND name = 'x')` and returned other rows. A condition that combines others with the other operator is now put in parentheses +* Fix: combining a condition with AND or OR added the other condition's parameters to its own, so a condition kept in a variable and used again, alone or combined, bound parameters its SQL doesn't have, which failed, as with an `ArrayIndexOutOfBoundsException` on the JVM. Combining conditions now leaves their parameters as they were +* Fix: unsigned values were stored inconsistently. INSERT stored a `UByte`, `UShort` or `UInt` as the bits of a signed number on every platform, UPDATE did so on the JVM, but stored it as its number on Kotlin/Native and Android, so a `UByte` of 200 read back as 200, but compared, ordered and summed as -56 or as 200, depending on how it was written. They are now always stored as their numbers, which SQLite's 64-bit integers hold, so that SQL compares, orders and sums them as Kotlin does, as `sum` and `max` of such a column assumed. A `ULong` keeps the bits of a `Long`, as no 64-bit signed integer holds the values above `Long.MAX_VALUE`. Reading is unchanged and reads the old rows right, but in SQL an old value above the signed range stays negative until it is written again, which one UPDATE per column does: `UPDATE t SET c = c + 256 WHERE c < 0` for a `UByte` column, with `65536` for a `UShort` one and `4294967296` for a `UInt` one + +### sqllin-driver + +* Fix: the JVM driver bound a `UInt`, `UShort` or `UByte` parameter as the bits of a signed number, unlike the Kotlin/Native driver, and the Android driver bound one written through `execSQL` as its string. All of them now bind it as its number, and a `ULong` as the `Long` of the same bits +* Fix: on the JVM, Kotlin/Native and Android, a transaction whose COMMIT failed, as one does while a deferred foreign key isn't satisfied, wasn't rolled back, but left open in SQLite: the statements after it ran in it, and the next transaction failed to begin, with "cannot start a transaction within a transaction". It is now rolled back. Android's framework lets go of such a transaction and doesn't pass a ROLLBACK through, so the driver raises one with a TEMP trigger + +### sqllin-processor + +* Fix: the generated `createSQL` of a table didn't escape `"`, `\` and `$`, which a Kotlin string literal treats specially, so a `@Default` value holding one could make the generated code fail to compile, or hold another value +* **Breaking change**: Fix: reading a column's property in `SET {}`, as in `SET { visits = visits + 1 }`, compiled, but its getter only returns a placeholder, such as `0` or `""`, so the column was set to a value computed from it, here `1`. The getter is now deprecated with the level `ERROR`, so reading the property is a compile error, while assigning it is unchanged + ## 2.4.0 / 2026-10-03 ### All diff --git a/ROADMAP.md b/ROADMAP.md index 2a023b43..30a695ef 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,32 +1,81 @@ # SQLlin Roadmap +Each item has an ID, `Fx`, which stays the same when the item moves to another priority, and under Supported once it is +done. A new item takes the next number. + ## High Priority -* Support observable queries that return a `Flow` re-emitting whenever the tables they read change, e.g. to invalidate a Paging source -* Support UNION of SELECTs whose result type isn't the table's row type, such as projections, result columns and joins. `UNION` is typed by the table's row type but reads the rows into its first SELECT's result type, so using the results of such a union fails with a `ClassCastException` +* F7: Remove the joins written as clauses of SELECT, such as `table SELECT INNER_JOIN(other)`, in the next version after 2.5.0, then let a join start with its first relation, as in `PersonTable INNER_JOIN BookTable`, without `FROM` ## Medium Priority -* Support WASM platform DSL -* Support CREATE VIRTUAL TABLE DSL -* Support CREATE VIEW DSL -* Support CREATE TRIGGER DSL -* Support JOIN sub-query DSL -* Support more functions, such as `coalesce` and `ifnull`, as well as `CAST` and literal values in expressions, e.g. to convert data while copying it to a rebuilt table with `INSERT INTO ... SELECT` -* Support type converters: store a property of any type through a serializer that encodes it to a type SQLite supports, with type-safe WHERE and SET on its column, e.g. to store instances of kotlinx.datetime +* F8: Support WASM platform DSL +* F12: Support type converters: store a property of any type through a serializer that encodes it to a type SQLite supports, with type-safe WHERE and SET on its column, e.g. to store instances of kotlinx.datetime +* F13: Provide a `PagingSource` for androidx.paging built on observable queries, like Room's `LimitOffsetPagingSource`, so that paging a query doesn't need a hand-written bridge +* F14: Support constraints in `ALTER_ADD_COLUMN`, which adds a column with its type only: DEFAULT, NOT NULL with DEFAULT, CHECK, COLLATE NOCASE and REFERENCES +* F15: Support WITHOUT ROWID tables, which save space and a lookup when the primary key isn't an INTEGER +* F16: Support STRICT tables, which reject values of other types than their columns' (SQLite 3.37.0, Android API 34) +* F19: Support the collations BINARY and RTRIM besides NOCASE, and COLLATE on the columns of an index and in ORDER BY +* F21: Support FILTER in aggregate functions (SQLite 3.30.0, Android API 31), and ORDER BY in them, as for `group_concat` (SQLite 3.44.0, Android API 35) +* F23: Support the other conflict resolutions: INSERT OR ABORT, OR FAIL and OR ROLLBACK, and UPDATE OR IGNORE, OR REPLACE and the others +* F25: Support ORDER BY and LIMIT in UPDATE and DELETE, which need SQLite's `SQLITE_ENABLE_UPDATE_DELETE_LIMIT` option, which Android's lacks +* F26: Support INDEXED BY and NOT INDEXED +* F27: Support choosing how a transaction begins, DEFERRED, IMMEDIATE or EXCLUSIVE, which differs between the platforms now +* F29: Support EXPLAIN QUERY PLAN, to see which indexes a query uses +* F30: Support more PRAGMAs, such as integrity_check, quick_check, table_info, index_list, wal_checkpoint, defer_foreign_keys and case_sensitive_like ## Low Priority -* Support CHECK keyword -* Support using a query's results within the same transaction, so that a read-modify-write is one transaction -* Support upsert, `INSERT ... ON CONFLICT (target) DO UPDATE` and `DO NOTHING`, which updates the conflicting row in place where `INSERT OR REPLACE` deletes and re-inserts it +* F31: Support using a query's results within the same transaction, so that a read-modify-write is one transaction +* F32: Support upsert, `INSERT ... ON CONFLICT (target) DO UPDATE` and `DO NOTHING`, which updates the conflicting row in place where `INSERT OR REPLACE` deletes and re-inserts it +* F33: Use a SQLite bundled with SQLlin on Android, such as androidx.sqlite's sqlite-bundled, rather than the system's, which has no FTS5 or R*Tree, and on older Android versions lacks newer SQL such as DROP COLUMN (before API 34) and upsert (before API 30) +* F34: Support FTS5 and R*Tree virtual tables, which Android's system SQLite lacks +* F35: Support common table expressions, WITH and WITH RECURSIVE, which hierarchical data such as trees needs, in SELECT and in INSERT, UPDATE and DELETE +* F36: Support window functions, with OVER, PARTITION BY and WINDOW (SQLite 3.25.0, Android API 30) +* F37: Support RETURNING in INSERT, UPDATE and DELETE, which returns the rows they write, such as the ids SQLite assigns (SQLite 3.35.0, Android API 34) +* F38: Support savepoints, SAVEPOINT, RELEASE and ROLLBACK TO, for nested transactions +* F39: Support ATTACH and DETACH, to query several databases together +* F40: Support generated columns, `GENERATED ALWAYS AS (...)` (SQLite 3.31.0, Android API 31) +* F41: Support CREATE TABLE ... AS SELECT +* F42: Support TEMP tables and views +* F43: Support custom collations, which the driver registers +* F44: Support VALUES as a query, and SELECT without FROM, such as `SELECT datetime('now')` +* F45: Support table-valued functions in FROM, such as json_each, json_tree and pragma_table_info (the JSON ones need SQLite 3.38.0, Android API 34) +* F47: Support UPDATE ... FROM, which updates rows with the values of another table, and needs the expressions of F9 (SQLite 3.33.0, Android API 34) +* F48: Support virtual tables other than FTS and R*Tree, such as dbstat +* F63: Support INSTEAD OF triggers on views, together with INSERT, UPDATE and DELETE on a view, which the DSL doesn't allow, as a view isn't a `Table`, so that such a trigger can be used; left out of the first version of CREATE TRIGGER (F10) +* F64: Support the JSON functions and JSON's `->` and `->>` operators (SQLite 3.38.0, Android API 34), left out of F46 +* F65: Support the conflict clause of NOT NULL, as `NOT NULL ON CONFLICT REPLACE`, which writes the default value instead of NULL, left out of F17: a property's type declares NOT NULL, so no annotation can hold the clause yet; it could be a new annotation, or a parameter of `@Default`. SQLlin's statements rarely write NULL to a non-null column, so it is of little use for now ## Supported -* Support INSERT INTO ... SELECT (2.4.0 ✅) -* Support SQL functions in SELECT results (2.4.0 ✅) -* Support SELECT projection (2.4.0 ✅) -* Support INSERT OR IGNORE (2.4.0 ✅) -* Support INSERT OR REPLACE (2.3.0 ✅) -* Support FOREIGN KEY DSL (2.2.0 ✅) -* Support CREATE INDEX DSL (2.2.0 ✅) \ No newline at end of file +* F17: Support conflict clauses on PRIMARY KEY and UNIQUE constraints (2.5.0 ✅) +* F18: Support DEFERRABLE INITIALLY DEFERRED foreign keys (2.5.0 ✅) +* F22: Support NULLS FIRST and NULLS LAST in ORDER BY (2.5.0 ✅) +* F20: Support ASC and DESC on the columns of an index (2.5.0 ✅) +* F28: Support ANALYZE, REINDEX and PRAGMA optimize (2.5.0 ✅) +* F24: Support INSERT ... DEFAULT VALUES (2.5.0 ✅) +* F10: Support CREATE TRIGGER and DROP TRIGGER, with BEFORE and AFTER triggers on tables (2.5.0 ✅) +* F9: Support expressions in UPDATE's SET and in INSERT (2.5.0 ✅) +* F46: Support more expression syntax: CASE, scalar subqueries, COLLATE, bitwise operators and null-safe IS comparisons; REGEXP is left out, as its function differs between the platforms, and the JVM's SQLite has none (2.5.0 ✅) +* F11: Support more functions, CAST, literal values and arithmetic in expressions (2.5.0 ✅) +* F6: Support VACUUM and VACUUM INTO (2.5.0 ✅) +* F5: Support ESCAPE in LIKE (2.5.0 ✅) +* F3: Support DISTINCT in aggregate functions (2.5.0 ✅) +* F4: Support partial indexes, `CREATE INDEX ... WHERE` (2.5.0 ✅) +* F2: Support IF NOT EXISTS and IF EXISTS on CREATE and DROP of tables, views and indexes (2.5.0 ✅) +* F1: Support DROP INDEX (2.5.0 ✅) +* F49: Support joins of relations: checked result types, result columns, more than two tables, and RIGHT and FULL OUTER JOIN (2.5.0 ✅) +* F50: Support subqueries: derived tables in FROM and JOIN, IN and EXISTS (2.5.0 ✅) +* F51: Support CHECK constraints (2.5.0 ✅) +* F52: Support CREATE VIRTUAL TABLE for FTS4 and FTS3 full-text search (2.5.0 ✅) +* F53: Support CREATE VIEW (2.5.0 ✅) +* F54: Support observable queries that return a `Flow` of their results (2.5.0 ✅) +* F55: Support compound SELECTs: UNION, UNION ALL, INTERSECT and EXCEPT (2.5.0 ✅) +* F56: Support INSERT INTO ... SELECT (2.4.0 ✅) +* F57: Support SQL functions in SELECT results (2.4.0 ✅) +* F58: Support SELECT projection (2.4.0 ✅) +* F59: Support INSERT OR IGNORE (2.4.0 ✅) +* F60: Support INSERT OR REPLACE (2.3.0 ✅) +* F61: Support FOREIGN KEY DSL (2.2.0 ✅) +* F62: Support CREATE INDEX DSL (2.2.0 ✅) diff --git a/gradle.properties b/gradle.properties index 6f07e8f2..1ae382fe 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,4 +1,4 @@ -VERSION=2.4.0 +VERSION=2.5.0 GROUP_ID=com.ctrip.kotlin #Maven Publishing Information diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 787c12ec..e6da5f81 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -1,6 +1,6 @@ [versions] -kotlin = "2.4.20" +kotlin = "2.4.21" agp = "9.4.1" ksp = "2.3.12" serialization = "1.11.0" diff --git a/libs/linux/libsqlite3.a b/libs/linux/libsqlite3.a index d18f4729..ba62a50d 100644 Binary files a/libs/linux/libsqlite3.a and b/libs/linux/libsqlite3.a differ diff --git a/libs/windows/libsqlite3.a b/libs/windows/libsqlite3.a index 46d288a1..d300b091 100644 Binary files a/libs/windows/libsqlite3.a and b/libs/windows/libsqlite3.a differ diff --git a/sample/src/commonMain/kotlin/com/ctrip/sqllin/sample/Sample.kt b/sample/src/commonMain/kotlin/com/ctrip/sqllin/sample/Sample.kt index a823e4c6..cc6bdf74 100644 --- a/sample/src/commonMain/kotlin/com/ctrip/sqllin/sample/Sample.kt +++ b/sample/src/commonMain/kotlin/com/ctrip/sqllin/sample/Sample.kt @@ -19,7 +19,6 @@ package com.ctrip.sqllin.sample import com.ctrip.sqllin.dsl.DSLDBConfiguration import com.ctrip.sqllin.dsl.Database import com.ctrip.sqllin.dsl.annotation.DBRow -import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import com.ctrip.sqllin.dsl.annotation.PrimaryKey import com.ctrip.sqllin.dsl.sql.clause.* import com.ctrip.sqllin.dsl.sql.clause.OrderByWay.DESC @@ -37,7 +36,6 @@ import kotlinx.serialization.Serializable object Sample { - @OptIn(ExperimentalDSLDatabaseAPI::class) private val db by lazy { Database( DSLDBConfiguration( diff --git a/sqllin-driver/build.gradle.kts b/sqllin-driver/build.gradle.kts index d2c19862..49fdfe22 100644 --- a/sqllin-driver/build.gradle.kts +++ b/sqllin-driver/build.gradle.kts @@ -103,8 +103,8 @@ gradle.taskGraph.whenReady { } fun KotlinNativeTarget.setupNativeConfig() { - val main by compilations.getting - val sqlite3 by main.cinterops.creating { + val main = compilations.getByName("main") + main.cinterops.create("sqlite3") { includeDirs("$projectDir/src/include") } binaries.all { diff --git a/sqllin-driver/src/androidHostTest/kotlin/com/ctrip/sqllin/driver/test/AndroidTest.kt b/sqllin-driver/src/androidHostTest/kotlin/com/ctrip/sqllin/driver/test/AndroidTest.kt index 84ca5eab..b716587c 100644 --- a/sqllin-driver/src/androidHostTest/kotlin/com/ctrip/sqllin/driver/test/AndroidTest.kt +++ b/sqllin-driver/src/androidHostTest/kotlin/com/ctrip/sqllin/driver/test/AndroidTest.kt @@ -54,6 +54,9 @@ class AndroidTest { @Test fun testTransaction() = commonTest.testTransaction() + @Test + fun testFailedCommit() = commonTest.testFailedCommit() + @Test fun testConcurrency() = commonTest.testConcurrency() diff --git a/sqllin-driver/src/androidMain/kotlin/com/ctrip/sqllin/driver/AndroidDatabaseConnection.kt b/sqllin-driver/src/androidMain/kotlin/com/ctrip/sqllin/driver/AndroidDatabaseConnection.kt index cf3f3ee4..a2f010df 100644 --- a/sqllin-driver/src/androidMain/kotlin/com/ctrip/sqllin/driver/AndroidDatabaseConnection.kt +++ b/sqllin-driver/src/androidMain/kotlin/com/ctrip/sqllin/driver/AndroidDatabaseConnection.kt @@ -31,7 +31,19 @@ internal class AndroidDatabaseConnection(private val database: SQLiteDatabase) : if (bindParams == null) database.execSQL(sql) else - database.execSQL(sql, bindParams) + database.execSQL(sql, Array(bindParams.size) { unsignedAsLong(bindParams[it]) }) + + /** + * The framework binds a value it doesn't know, such as a `UInt`, as its string, so unsigned values are bound as + * the numbers they are, as [bindTypedParameters] does for queries. A `ULong` keeps the bits of a `Long`. + */ + private fun unsignedAsLong(param: Any?): Any? = when (param) { + is ULong -> param.toLong() + is UInt -> param.toLong() + is UShort -> param.toLong() + is UByte -> param.toLong() + else -> param + } override fun executeInsert(sql: String, bindParams: Array?) = execSQL(sql, bindParams) @@ -85,7 +97,31 @@ internal class AndroidDatabaseConnection(private val database: SQLiteDatabase) : } override fun beginTransaction() = database.beginTransaction() - override fun endTransaction() = database.endTransaction() + override fun endTransaction() { + try { + database.endTransaction() + } catch (e: Exception) { + rollBackOpenTransaction() + throw e + } + } + + /** + * Rolls back the transaction that a failed COMMIT leaves open in SQLite, as one does while a deferred foreign key + * isn't satisfied: the framework has already let go of it, so that the next one would fail to begin. + * + * The framework turns a ROLLBACK given to it into its own end of a transaction, which it has none of, so a TEMP + * trigger raises the ROLLBACK. The table and the trigger, created in the transaction, are rolled back with it; the + * DROP only removes them where SQLite had already rolled back, as it does for some errors. + */ + private fun rollBackOpenTransaction() { + runCatching { + database.execSQL("CREATE TEMP TABLE IF NOT EXISTS sqllin_rollback(x)") + database.execSQL("CREATE TEMP TRIGGER IF NOT EXISTS sqllin_rollback BEFORE INSERT ON sqllin_rollback BEGIN SELECT RAISE(ROLLBACK, 'rollback'); END") + database.execSQL("INSERT INTO sqllin_rollback VALUES (1)") + } + runCatching { database.execSQL("DROP TABLE IF EXISTS temp.sqllin_rollback") } + } override fun setTransactionSuccessful() = database.setTransactionSuccessful() override fun close() = database.close() diff --git a/sqllin-driver/src/commonTest/kotlin/com/ctrip/sqllin/driver/test/CommonBasicTest.kt b/sqllin-driver/src/commonTest/kotlin/com/ctrip/sqllin/driver/test/CommonBasicTest.kt index 74f34d3a..c338b8df 100644 --- a/sqllin-driver/src/commonTest/kotlin/com/ctrip/sqllin/driver/test/CommonBasicTest.kt +++ b/sqllin-driver/src/commonTest/kotlin/com/ctrip/sqllin/driver/test/CommonBasicTest.kt @@ -24,6 +24,7 @@ import com.ctrip.sqllin.driver.withTransaction import kotlinx.coroutines.* import kotlinx.coroutines.test.runTest import kotlin.test.assertEquals +import kotlin.test.assertFails /** * The sqllin-driver common basic test. @@ -199,6 +200,30 @@ class CommonBasicTest(private val path: DatabasePath) { } } + /** + * A COMMIT that fails, as one does while a deferred foreign key isn't satisfied, leaves SQLite's transaction open, so + * it has to be rolled back: its rows are gone, and the next transaction can begin. + */ + fun testFailedCommit() { + openDatabase(getDefaultDBConfig(false)) { + it.execSQL("PRAGMA foreign_keys = ON") + it.execSQL("CREATE TABLE parent(id INTEGER PRIMARY KEY)") + it.execSQL("CREATE TABLE child(parentId INT REFERENCES parent(id) DEFERRABLE INITIALLY DEFERRED)") + assertFails { + it.withTransaction { connection -> + connection.executeInsert("INSERT INTO child VALUES (?)", arrayOf(1L)) + } + } + fun children(): Long = it.withQuery("SELECT count(*) FROM child") { cursor -> cursor.next(); cursor.getLong(0) } + assertEquals(0L, children()) + it.withTransaction { connection -> + connection.executeInsert("INSERT INTO child VALUES (?)", arrayOf(1L)) + connection.executeInsert("INSERT INTO parent VALUES (?)", arrayOf(1L)) + } + assertEquals(1L, children()) + } + } + @OptIn(DelicateCoroutinesApi::class, ExperimentalCoroutinesApi::class) fun testConcurrency() = runTest { val readWriteConfig = getDefaultDBConfig(false) diff --git a/sqllin-driver/src/jvmMain/kotlin/com/ctrip/sqllin/driver/AbstractJdbcDatabaseConnection.kt b/sqllin-driver/src/jvmMain/kotlin/com/ctrip/sqllin/driver/AbstractJdbcDatabaseConnection.kt index 24153536..5e02fb0a 100644 --- a/sqllin-driver/src/jvmMain/kotlin/com/ctrip/sqllin/driver/AbstractJdbcDatabaseConnection.kt +++ b/sqllin-driver/src/jvmMain/kotlin/com/ctrip/sqllin/driver/AbstractJdbcDatabaseConnection.kt @@ -56,10 +56,12 @@ internal abstract class AbstractJdbcDatabaseConnection : DatabaseConnection { is Short -> setShort(realIndex, any) is Byte -> setByte(realIndex, any) + // As the numbers they are, as the native driver does: setInt(any.toInt()) stored a UInt + // above Int.MAX_VALUE as a negative number. A ULong keeps the bits of a Long. is ULong -> setLong(realIndex, any.toLong()) - is UInt -> setInt(realIndex, any.toInt()) - is UShort -> setShort(realIndex, any.toShort()) - is UByte -> setByte(realIndex, any.toByte()) + is UInt -> setLong(realIndex, any.toLong()) + is UShort -> setLong(realIndex, any.toLong()) + is UByte -> setLong(realIndex, any.toLong()) is BigDecimal -> setBigDecimal(realIndex, any) diff --git a/sqllin-driver/src/jvmMain/kotlin/com/ctrip/sqllin/driver/JdbcDatabaseConnection.kt b/sqllin-driver/src/jvmMain/kotlin/com/ctrip/sqllin/driver/JdbcDatabaseConnection.kt index 7eb8028a..b5326d54 100644 --- a/sqllin-driver/src/jvmMain/kotlin/com/ctrip/sqllin/driver/JdbcDatabaseConnection.kt +++ b/sqllin-driver/src/jvmMain/kotlin/com/ctrip/sqllin/driver/JdbcDatabaseConnection.kt @@ -67,9 +67,16 @@ internal class JdbcDatabaseConnection(private val connection: Connection) : Abst } override fun endTransaction() = try { - if (isTransactionSuccess.get()) - connection.commit() - else + if (isTransactionSuccess.get()) { + try { + connection.commit() + } catch (e: Exception) { + // A COMMIT that fails, as one does while a deferred foreign key isn't satisfied, leaves the + // transaction open, so that the statements after it would run in it + runCatching { connection.rollback() } + throw e + } + } else connection.rollback() } finally { connection.autoCommit = true diff --git a/sqllin-driver/src/jvmTest/kotlin/com/ctrip/sqllin/driver/test/JvmTest.kt b/sqllin-driver/src/jvmTest/kotlin/com/ctrip/sqllin/driver/test/JvmTest.kt index 4e8fc8cf..6dac3f4f 100644 --- a/sqllin-driver/src/jvmTest/kotlin/com/ctrip/sqllin/driver/test/JvmTest.kt +++ b/sqllin-driver/src/jvmTest/kotlin/com/ctrip/sqllin/driver/test/JvmTest.kt @@ -46,6 +46,9 @@ class JvmTest { @Test fun testTransaction() = commonTest.testTransaction() + @Test + fun testFailedCommit() = commonTest.testFailedCommit() + @Test fun testConcurrency() = commonTest.testConcurrency() diff --git a/sqllin-driver/src/nativeMain/kotlin/com/ctrip/sqllin/driver/RealDatabaseConnection.kt b/sqllin-driver/src/nativeMain/kotlin/com/ctrip/sqllin/driver/RealDatabaseConnection.kt index 47a164af..4d60f72c 100644 --- a/sqllin-driver/src/nativeMain/kotlin/com/ctrip/sqllin/driver/RealDatabaseConnection.kt +++ b/sqllin-driver/src/nativeMain/kotlin/com/ctrip/sqllin/driver/RealDatabaseConnection.kt @@ -84,8 +84,18 @@ internal class RealDatabaseConnection( override fun endTransaction() = transactionLock.withLock { try { - val sql = if (checkFailTransaction.isSuccessful) "COMMIT;" else "ROLLBACK;" - database.rawExecSql(sql) + if (checkFailTransaction.isSuccessful) { + try { + database.rawExecSql("COMMIT;") + } catch (e: Exception) { + // A COMMIT that fails, as one does while a deferred foreign key isn't satisfied, leaves the + // transaction open, so that the statements after it would run in it + runCatching { database.rawExecSql("ROLLBACK;") } + throw e + } + } else { + database.rawExecSql("ROLLBACK;") + } } finally { transaction.value = null } diff --git a/sqllin-driver/src/nativeTest/kotlin/com/ctrip/sqllin/driver/test/NativeTest.kt b/sqllin-driver/src/nativeTest/kotlin/com/ctrip/sqllin/driver/test/NativeTest.kt index 7c7a50b8..283c768d 100644 --- a/sqllin-driver/src/nativeTest/kotlin/com/ctrip/sqllin/driver/test/NativeTest.kt +++ b/sqllin-driver/src/nativeTest/kotlin/com/ctrip/sqllin/driver/test/NativeTest.kt @@ -46,6 +46,9 @@ class NativeTest { @Test fun testTransaction() = commonTest.testTransaction() + @Test + fun testFailedCommit() = commonTest.testFailedCommit() + @Test fun testConcurrency() = commonTest.testConcurrency() diff --git a/sqllin-dsl-test/src/androidHostTest/kotlin/com/ctrip/sqllin/dsl/test/AndroidTest.kt b/sqllin-dsl-test/src/androidHostTest/kotlin/com/ctrip/sqllin/dsl/test/AndroidTest.kt index dfca3725..d5e14c47 100644 --- a/sqllin-dsl-test/src/androidHostTest/kotlin/com/ctrip/sqllin/dsl/test/AndroidTest.kt +++ b/sqllin-dsl-test/src/androidHostTest/kotlin/com/ctrip/sqllin/dsl/test/AndroidTest.kt @@ -48,6 +48,126 @@ class AndroidTest { @Test fun testUnionSelect() = commonTest.testUnionSelect() + @Test + fun testCompoundSelect() = commonTest.testCompoundSelect() + + @Test + fun testObserve() = commonTest.testObserve() + + @Test + fun testObserveForeignKeyAction() = commonTest.testObserveForeignKeyAction() + + @Test + fun testView() = commonTest.testView() + + @Test + fun testUnsignedValues() = commonTest.testUnsignedValues() + + @Test + fun testFts4() = commonTest.testFts4() + + @Test + fun testFts3() = commonTest.testFts3() + + @Test + fun testCheckConstraint() = commonTest.testCheckConstraint() + + @Test + fun testSubquery() = commonTest.testSubquery() + + @Test + fun testConditionPrecedence() = commonTest.testConditionPrecedence() + + @Test + fun testConditionReuse() = commonTest.testConditionReuse() + + @Test + fun testJoinedRelation() = commonTest.testJoinedRelation() + + @Test + fun testExpressionIndex() = commonTest.testExpressionIndex() + + @Test + fun testDropIndex() = commonTest.testDropIndex() + + @Test + fun testIfNotExists() = commonTest.testIfNotExists() + + @Test + fun testPartialIndex() = commonTest.testPartialIndex() + + @Test + fun testDistinctAggregate() = commonTest.testDistinctAggregate() + + @Test + fun testLikeEscape() = commonTest.testLikeEscape() + + @Test + fun testVacuum() = commonTest.testVacuum() + + @Test + fun testArithmetic() = commonTest.testArithmetic() + + @Test + fun testCast() = commonTest.testCast() + + @Test + fun testNullFunctions() = commonTest.testNullFunctions() + + @Test + fun testMoreFunctions() = commonTest.testMoreFunctions() + + @Test + fun testDateTimeFunctions() = commonTest.testDateTimeFunctions() + + @Test + fun testPlatformDependentFunctions() = commonTest.testPlatformDependentFunctions() + + @Test + fun testCaseExpression() = commonTest.testCaseExpression() + + @Test + fun testScalarSubquery() = commonTest.testScalarSubquery() + + @Test + fun testExpressionOperators() = commonTest.testExpressionOperators() + + @Test + fun testSetExpressions() = commonTest.testSetExpressions() + + @Test + fun testInsertExpressions() = commonTest.testInsertExpressions() + + @Test + fun testTrigger() = commonTest.testTrigger() + + @Test + fun testTriggerSelectingNewRow() = commonTest.testTriggerSelectingNewRow() + + @Test + fun testTriggerChecks() = commonTest.testTriggerChecks() + + @Test + fun testTriggerWritingFts() = commonTest.testTriggerWritingFts() + + @Test + fun testInsertDefaultValues() = commonTest.testInsertDefaultValues() + + @Test + fun testAnalyzeAndReindex() = commonTest.testAnalyzeAndReindex() + + @Test + fun testIndexOrder() = commonTest.testIndexOrder() + + @Test + fun testNullsOrdering() = commonTest.testNullsOrdering() + + @Test + fun testDeferredForeignKey() = commonTest.testDeferredForeignKey() + + @Test + fun testConflictClauses() = commonTest.testConflictClauses() + @Test fun testFunction() = commonTest.testFunction() diff --git a/sqllin-dsl-test/src/commonMain/kotlin/com/ctrip/sqllin/dsl/test/CommonBasicTest.kt b/sqllin-dsl-test/src/commonMain/kotlin/com/ctrip/sqllin/dsl/test/CommonBasicTest.kt index 460f73ce..8483ed38 100644 --- a/sqllin-dsl-test/src/commonMain/kotlin/com/ctrip/sqllin/dsl/test/CommonBasicTest.kt +++ b/sqllin-dsl-test/src/commonMain/kotlin/com/ctrip/sqllin/dsl/test/CommonBasicTest.kt @@ -18,11 +18,16 @@ package com.ctrip.sqllin.dsl.test import com.ctrip.sqllin.driver.DatabaseConfiguration import com.ctrip.sqllin.driver.DatabasePath +import com.ctrip.sqllin.driver.deleteDatabase +import com.ctrip.sqllin.driver.openDatabase +import com.ctrip.sqllin.driver.withQuery import com.ctrip.sqllin.dsl.DSLDBConfiguration import com.ctrip.sqllin.dsl.Database import com.ctrip.sqllin.dsl.DatabaseScope import com.ctrip.sqllin.dsl.annotation.AdvancedInsertAPI import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.annotation.PlatformDependentSQLiteAPI +import com.ctrip.sqllin.dsl.sql.FtsTable import com.ctrip.sqllin.dsl.sql.X import com.ctrip.sqllin.dsl.sql.withName import com.ctrip.sqllin.dsl.sql.clause.* @@ -31,9 +36,12 @@ import com.ctrip.sqllin.dsl.sql.clause.OrderByWay.DESC import com.ctrip.sqllin.dsl.sql.statement.SelectStatement import kotlinx.coroutines.DelicateCoroutinesApi import kotlinx.coroutines.ExperimentalCoroutinesApi +import kotlinx.coroutines.channels.Channel import kotlinx.coroutines.launch import kotlinx.coroutines.newSingleThreadContext +import kotlinx.coroutines.test.runCurrent import kotlinx.coroutines.test.runTest +import kotlin.coroutines.EmptyCoroutineContext import kotlin.test.assertEquals import kotlin.test.assertFails import kotlin.test.assertFailsWith @@ -44,11 +52,11 @@ import kotlin.test.assertNotEquals * @author Yuang Qiao */ -@OptIn(ExperimentalDSLDatabaseAPI::class) class CommonBasicTest(private val path: DatabasePath) { companion object { const val DATABASE_NAME = "BookStore.db" + const val BACKUP_DATABASE_NAME = "BookStoreBackup.db" const val SQL_CREATE_BOOK = "create table book (id integer primary key autoincrement, name text, author text, pages integer, price real)" const val SQL_CREATE_CATEGORY = "create table category (id integer primary key autoincrement, name text, code integer)" } @@ -271,6 +279,109 @@ class CommonBasicTest(private val path: DatabasePath) { assertEquals("Ken Kousen", resultOfGroupByAndHaving.first().author) } + /** + * Covers compound SELECTs: UNION, UNION ALL, INTERSECT and EXCEPT combine two SELECTs of the same result type, as + * in SQL, from any tables. A member that ends with ORDER BY or LIMIT keeps them to itself, a compound on the right + * is combined as a whole, and the compound takes its own ORDER BY and LIMIT. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testCompoundSelect() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> + database { + BookTable INSERT listOf( + Book(name = "The Da Vinci Code", author = "Dan Brown", price = 16.96, pages = 454), + Book(name = "The Lost Symbol", author = "Dan Brown", price = 19.95, pages = 510), + Book(name = "Kotlin Cookbook", author = "Ken Kousen", price = 37.72, pages = 251), + ) + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "Ann", age = 30), + PersonWithId(id = null, name = "Kotlin Cookbook", age = 40), + ) + } + + // The four operators, on projections. The INTERSECT only gives Dan Brown if the WHERE parameters keep their + // order; swapped, both SELECTs would match every book. + lateinit var member: SelectStatement + lateinit var union: SelectStatement + lateinit var unionAll: SelectStatement + lateinit var intersect: SelectStatement + lateinit var except: SelectStatement + database { + BookTable { table -> + member = table SELECT WHERE(price LT 20.0) + union = member UNION (table SELECT WHERE(price GT 10.0)) + unionAll = (table SELECT WHERE(price LT 20.0)) UNION_ALL (table SELECT WHERE(price GT 30.0)) + intersect = (table SELECT WHERE(price LT 20.0)) INTERSECT (table SELECT WHERE(pages GT 500)) + except = (table SELECT WHERE(price GT 0.0)) EXCEPT (table SELECT WHERE(price LT 20.0)) + } + } + assertEquals(listOf("Dan Brown", "Ken Kousen"), union.getResults().map { it.author }.sorted()) + assertEquals(listOf("Dan Brown", "Dan Brown", "Ken Kousen"), unionAll.getResults().map { it.author }.sorted()) + assertEquals(listOf("Dan Brown"), intersect.getResults().map { it.author }) + assertEquals(listOf("Ken Kousen"), except.getResults().map { it.author }) + // A SELECT that became part of a compound no longer runs on its own + assertFailsWith { member.getResults() } + + // Rows of different tables, projected into the same type + lateinit var names: SelectStatement + database { + names = (BookTable SELECT X()) INTERSECT (PersonWithIdTable SELECT X()) + } + assertEquals(listOf(NameOnly("Kotlin Cookbook")), names.getResults()) + + // A member's ORDER BY and LIMIT stay its own, here giving the longest and the shortest book. Written as they + // are, SQLite would reject the first ORDER BY, and apply the last one to the whole compound. + lateinit var extremes: SelectStatement + database { + BookTable { table -> + extremes = (table SELECT ORDER_BY(pages to DESC) LIMIT 1) UNION_ALL (table SELECT ORDER_BY(pages to ASC) LIMIT 1) + } + } + assertEquals(listOf("Kotlin Cookbook", "The Lost Symbol"), extremes.getResults().map { it.name }.sorted()) + + // The compound's own ORDER BY, LIMIT and OFFSET apply to all of it + lateinit var page: SelectStatement + database { + page = (BookTable SELECT X()) UNION (PersonWithIdTable SELECT X()) ORDER_BY (BookTable.name to ASC) LIMIT 2 OFFSET 1 + } + assertEquals(listOf("Kotlin Cookbook", "The Da Vinci Code"), page.getResults().map { it.name }) + + // Chained operators are evaluated from left to right, as in SQL, and parentheses group them + lateinit var leftToRight: SelectStatement + lateinit var grouped: SelectStatement + database { + PersonWithIdTable { table -> + val ann = { table SELECT WHERE(name EQ "Ann") } + leftToRight = ann() UNION ann() UNION_ALL ann() // (Ann UNION Ann) UNION ALL Ann + grouped = ann() UNION (ann() UNION_ALL ann()) // Ann UNION (Ann UNION ALL Ann) + } + } + assertEquals(2, leftToRight.getResults().size) + assertEquals(1, grouped.getResults().size) + + // Result columns combine too, and an incomplete member is rejected when it is combined: without GROUP BY, + // 'author' would be NULL when no rows match + lateinit var counts: SelectStatement + database { + BookTable { table -> + counts = (table SELECT (count(X) AS BookCount::books) WHERE (author EQ "Dan Brown")) UNION_ALL + (table SELECT (count(X) AS BookCount::books) WHERE (author EQ "Ken Kousen")) + } + } + assertEquals(listOf(1L, 2L), counts.getResults().map { it.books }.sorted()) + val ungrouped = assertFailsWith { + database { + BookTable { table -> + (table SELECT (count(X) AS AuthorBookCount::books)) UNION (table SELECT (count(X) AS AuthorBookCount::books) GROUP_BY author) + } + } + } + assertEquals(true, ungrouped.message!!.contains("without GROUP BY")) + } + + /** + * Covers the deprecated `UNION {}` block, which keeps working for the table's own rows until it is removed. + */ + @Suppress("DEPRECATION") fun testUnionSelect() = Database(getDefaultDBConfig(), true).databaseAutoClose { database -> val book0 = Book(name = "The Da Vinci Code", author = "Dan Brown", pages = 454, price = 16.96) val book1 = Book(name = "Kotlin Cookbook", author = "Ken Kousen", pages = 251, price = 37.72) @@ -411,445 +522,2147 @@ class CommonBasicTest(private val path: DatabasePath) { assertEquals(outerJoinStatementWithOn?.getResults()?.size, books.size) } - @OptIn(DelicateCoroutinesApi::class, ExperimentalCoroutinesApi::class) - fun testConcurrency() = Database(getDefaultDBConfig(), true).databaseAutoClose { database -> + /** + * Covers observed queries: the flow emits the SELECT's results when it is collected, and again when a statement run + * through the database changes a table the SELECT reads and the results differ. A join watches all of its tables. + * A change to another table, an UPDATE that matches no rows or leaves the results as they were, and a rolled back + * transaction emit nothing, and an observed table that is dropped and created again is still watched. + */ + @OptIn(ExperimentalCoroutinesApi::class, ExperimentalDSLDatabaseAPI::class) + fun testObserve() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> runTest { - val book1 = Book(name = "The Da Vinci Code", author = "Dan Brown", pages = 454, price = 16.96) - val book2 = Book(name = "The Lost Symbol", author = "Dan Brown", pages = 510, price = 19.95) - launch(newSingleThreadContext("test0")) { - lateinit var statement: SelectStatement - database suspendedScope { - statement = BookTable { table -> - table INSERT listOf(book1, book2) - table SELECT X + val books = Channel>(Channel.UNLIMITED) + val joined = Channel>(Channel.UNLIMITED) + // The queries run in the test's own context, so that runCurrent() lets them catch up + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { BookTable SELECT X }.collect { books.send(it) } + } + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { + BookTable SELECT INNER_JOIN(CategoryTable) USING BookTable.name + }.collect { joined.send(it) } + } + fun assertNoNewBooks() { + runCurrent() + assertEquals(true, books.tryReceive().isFailure) + } + + assertEquals(emptyList(), books.receive()) + assertEquals(emptyList(), joined.receive()) + + val book = Book(name = "Kotlin Cookbook", author = "Ken Kousen", price = 37.72, pages = 251) + database { BookTable INSERT book } + assertEquals(listOf(book), books.receive()) + + // Nothing new: a change to another table, an UPDATE that matches no rows, one that leaves the results as + // they were, and a transaction that is rolled back + database { PersonWithIdTable INSERT PersonWithId(id = null, name = "Ann", age = 30) } + assertNoNewBooks() + database { BookTable { table -> table UPDATE SET { pages = 1 } WHERE (name EQ "No Such Book") } } + assertNoNewBooks() + database { BookTable { table -> table UPDATE SET { pages = 251 } WHERE (name EQ "Kotlin Cookbook") } } + assertNoNewBooks() + assertFails { + database { + transaction { + BookTable INSERT Book(name = "Rolled Back", author = "Nobody", price = 1.0, pages = 1) + UniqueEmailTestTable INSERT listOf( + UniqueEmailTest(id = null, email = "same@example.com", name = "One"), + UniqueEmailTest(id = null, email = "same@example.com", name = "Two"), + ) } } - assertEquals(true, statement.getResults().any { it == book1 }) - assertEquals(true, statement.getResults().any { it == book2 }) } - launch(newSingleThreadContext("test1")) { - val book1NewPrice = 18.96 - val book2NewPrice = 21.95 - val newBook1 = Book(name = "The Da Vinci Code", author = "Dan Brown", pages = 454, price = book1NewPrice) - val newBook2 = Book(name = "The Lost Symbol", author = "Dan Brown", pages = 510, price = book2NewPrice) - lateinit var statement: SelectStatement - database suspendedScope { - statement = transaction { - BookTable { table -> - table INSERT listOf(newBook1, newBook2) - table SELECT X - } - } - } - assertEquals(true, statement.getResults().any { it == newBook1 }) - assertEquals(true, statement.getResults().any { it == newBook2 }) + assertNoNewBooks() + + database { BookTable { table -> table UPDATE SET { pages = 300 } WHERE (name EQ "Kotlin Cookbook") } } + assertEquals(listOf(book.copy(pages = 300)), books.receive()) + + // The join watches the category table too + database { CategoryTable INSERT Category(name = "Kotlin Cookbook", code = 1) } + assertEquals(listOf("Kotlin Cookbook"), joined.receive().map { it.name }) + + // A dropped and created table loses its triggers, which come back + database { + DROP(BookTable) + CREATE(BookTable) } + assertEquals(emptyList(), books.receive()) + database { BookTable INSERT book } + assertEquals(listOf(book), books.receive()) } } - fun testPrimitiveTypeForKSP() { - TestPrimitiveTypeForKSPTable { - SET { - assertEquals(0, testInt) - assertEquals(0L, testLong) - assertEquals(0, testShort) - assertEquals(0, testByte) - assertEquals(0F, testFloat) - assertEquals(0.0, testDouble) - assertEquals(0U, testUInt) - assertEquals(0UL, testULong) - assertEquals(0U, testUShort) - assertEquals(0U, testUByte) - assertEquals(false, testBoolean) - assertEquals('0', testChar) - assertEquals("", testString) + /** + * Covers an observed query of a table that only a foreign key action changes: SQLite reports the rows that + * ON DELETE CASCADE deletes, though no statement names their table. + */ + @OptIn(ExperimentalCoroutinesApi::class, ExperimentalDSLDatabaseAPI::class) + fun testObserveForeignKeyAction() = Database(getForeignKeyDBConfig()).databaseAutoClose { database -> + runTest { + database { + PRAGMA_FOREIGN_KEYS(true) } + database { + FKUserTable INSERT FKUser(id = null, email = "alice@example.com", name = "Alice") + FKOrderTable INSERT FKOrder(id = null, userId = 1L, amount = 99.99, orderDate = "2025-01-15") + } + val orders = Channel>(Channel.UNLIMITED) + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { FKOrderTable SELECT X }.collect { orders.send(it) } + } + assertEquals(1, orders.receive().size) + database { FKUserTable DELETE WHERE(FKUserTable.id EQ 1L) } + assertEquals(emptyList(), orders.receive()) } } - fun testNullValue() { + /** + * Covers views: `CREATE_VIEW` creates one from a SELECT of its row type, with the SELECT's values written into its + * SQL, as a view can't take parameters. A SELECT reads the view like a table, an observed query of it watches the + * tables it reads, and `DROP` drops it. + */ + @OptIn(ExperimentalCoroutinesApi::class, ExperimentalDSLDatabaseAPI::class) + fun testView() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> + runTest { + database { + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "Ann", age = 30), + PersonWithId(id = null, name = "Bob", age = 15), + PersonWithId(id = null, name = "Cat?", age = 40), + PersonWithId(id = null, name = "It's", age = 50), + ) + // The literals of the SELECT go into the view: 18, 'Cat' and 'It''s'. The '?' in replace(name,'?','') + // is a string, not a parameter, and stays as it is. + PersonWithIdTable { table -> + CREATE_VIEW(AdultPersonView) AS ( + table SELECT WHERE((age GTE 18) AND (replace(name, "?", "") NEQ "Cat") AND (name NEQ "It's")) + ) + } + } + + lateinit var adults: SelectStatement + lateinit var olderThan25: SelectStatement + database { + adults = AdultPersonView SELECT X + AdultPersonView { view -> + olderThan25 = view SELECT WHERE(age GT 25) + } + } + assertEquals(listOf(AdultPerson("Ann", 30)), adults.getResults()) + assertEquals(listOf(AdultPerson("Ann", 30)), olderThan25.getResults()) + + // An observed query of the view watches the person table + val observed = Channel>(Channel.UNLIMITED) + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { AdultPersonView SELECT X }.collect { observed.send(it) } + } + assertEquals(listOf(AdultPerson("Ann", 30)), observed.receive()) + database { PersonWithIdTable INSERT PersonWithId(id = null, name = "Dan", age = 20) } + assertEquals(listOf("Ann", "Dan"), observed.receive().map { it.name }.sorted()) + + database { DROP(AdultPersonView) } + assertEquals(true, database.selectFails { AdultPersonView SELECT X }) + } + } + + /** + * Covers how unsigned values are stored: as the numbers they are, which SQLite's 64-bit integers hold, so that SQL + * compares, orders and sums them as Kotlin does, whether an INSERT or an UPDATE wrote them. Stored as the bits of + * signed numbers, a UByte of 200 read back as 200, but compared as -56. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testUnsignedValues() { val config = DSLDBConfiguration( name = DATABASE_NAME, path = path, version = 1, - create = { - CREATE(NullTesterTable) - } + create = { CREATE(TestPrimitiveTypeForKSPTable) }, ) - Database(config, true).databaseAutoClose { database -> - lateinit var selectStatement: SelectStatement - // INSERT & SELECT - database { - selectStatement = NullTesterTable { table -> - table INSERT listOf( - NullTester(null, null, null), - NullTester(8, "888", 8.8), - ) - table SELECT X + fun row(testInt: Int, uByte: UByte, uShort: UShort, uInt: UInt) = TestPrimitiveTypeForKSP( + testInt = testInt, testLong = 1, testShort = 1, testByte = 1, testFloat = 1f, testDouble = 1.0, + testUInt = uInt, testULong = 1u, testUShort = uShort, testUByte = uByte, testBoolean = true, testChar = 'c', + testString = "s", testByteArray = byteArrayOf(1), testEnum = Priority.LOW, testTypeAlias = 1, + ) + Database(config).databaseAutoClose { database -> + database { + TestPrimitiveTypeForKSPTable INSERT listOf(row(1, 200u, 40_000u, 3_000_000_000u), row(2, 100u, 100u, 100u)) + } + lateinit var all: SelectStatement + lateinit var bigBytes: SelectStatement + lateinit var bigShorts: SelectStatement + lateinit var bigInts: SelectStatement + lateinit var byUInt: SelectStatement + lateinit var totals: SelectStatement + database { + TestPrimitiveTypeForKSPTable { table -> + all = table SELECT X + bigBytes = table SELECT WHERE(testUByte GT 150) + bigShorts = table SELECT WHERE(testUShort GT 30_000) + bigInts = table SELECT WHERE(testUInt GT 2_000_000_000L) + byUInt = table SELECT ORDER_BY(testUInt to DESC) + totals = table SELECT listOf(sum(testUInt) AS UIntTotals::total, max(testUInt) AS UIntTotals::highest) } } + val first = all.getResults().single { it.testInt == 1 } + assertEquals(200u.toUByte(), first.testUByte) + assertEquals(40_000u.toUShort(), first.testUShort) + assertEquals(3_000_000_000u, first.testUInt) + assertEquals(listOf(1), bigBytes.getResults().map { it.testInt }) + assertEquals(listOf(1), bigShorts.getResults().map { it.testInt }) + assertEquals(listOf(1), bigInts.getResults().map { it.testInt }) + assertEquals(listOf(1, 2), byUInt.getResults().map { it.testInt }) + assertEquals(UIntTotals(total = 3_000_000_100L, highest = 3_000_000_000u), totals.getResults().single()) - selectStatement.getResults().forEachIndexed { i, tester -> - when (i) { - 0 -> { - assertEquals(null, tester.paramInt) - assertEquals(null, tester.paramString) - assertEquals(null, tester.paramDouble) - } - 1 -> { - assertEquals(8, tester.paramInt) - assertEquals("888", tester.paramString) - assertEquals(8.8, tester.paramDouble) - } + // An UPDATE stores them the same way + database { + TestPrimitiveTypeForKSPTable { table -> + table UPDATE SET { testUByte = 250u; testUInt = 4_000_000_000u } WHERE (testInt EQ 2) + } + } + lateinit var updatedBytes: SelectStatement + lateinit var updatedInts: SelectStatement + database { + TestPrimitiveTypeForKSPTable { table -> + updatedBytes = table SELECT WHERE(testUByte GT 150) + updatedInts = table SELECT WHERE(testUInt GT 3_500_000_000L) } } + assertEquals(listOf(1, 2), updatedBytes.getResults().map { it.testInt }.sorted()) + assertEquals(listOf(2), updatedInts.getResults().map { it.testInt }) + } + } - // UPDATE & SELECT + /** + * Covers FTS4 tables: their CREATE VIRTUAL TABLE statement, rowids SQLite assigns, MATCH over the whole table and + * over one column with terms, stems, prefixes, OR and phrases, a column that isn't indexed, snippet, offsets and + * matchinfo, UPDATE and DELETE, an observed query, and rebuilding the table under another name. + */ + @OptIn(ExperimentalCoroutinesApi::class, ExperimentalDSLDatabaseAPI::class) + fun testFts4() = Database(getFtsDBConfig()).databaseAutoClose { database -> + runTest { + assertEquals( + "CREATE VIRTUAL TABLE articles USING fts4(title,body,note,tokenize=porter,prefix=\"2\",notindexed=note)", + ArticleTable.createSQL, + ) database { - selectStatement = NullTesterTable { table -> - table UPDATE SET { paramString = null } WHERE (paramDouble EQ 8.8) - table SELECT WHERE (paramInt NEQ null) + ArticleTable INSERT listOf( + Article(rowid = null, title = "Kotlin Coroutines", body = "Running suspending functions concurrently", note = "draft"), + Article(rowid = null, title = "SQLite Full-Text Search", body = "Indexes documents for searching", note = "kotlin"), + Article(rowid = null, title = "Kotlin Multiplatform", body = "Sharing code between platforms", note = null), + ) + } + fun matches(block: ArticleTable.() -> SelectCondition): List { + lateinit var statement: SelectStatement
+ database { + statement = ArticleTable SELECT WHERE(ArticleTable.block()) + } + return statement.getResults().map { it.rowid } + } + // "kotlin" is only in a column that isn't indexed of article 2 + assertEquals(listOf(1L, 3L), matches { this MATCH "kotlin" }) + assertEquals(listOf(1L), matches { this MATCH "run" }) + assertEquals(listOf(1L), matches { this MATCH "corout*" }) + assertEquals(listOf(1L, 2L), matches { this MATCH "coroutines OR search" }) + assertEquals(listOf(3L), matches { this MATCH "\"kotlin multiplatform\"" }) + assertEquals(listOf(3L), matches { this MATCH "body:sharing" }) + assertEquals(emptyList(), matches { this MATCH "draft" }) + assertEquals(listOf(3L), matches { body MATCH "platform" }) + assertEquals(emptyList(), matches { title MATCH "code" }) + assertFailsWith { BookTable.name MATCH "kotlin" } + + lateinit var all: SelectStatement
+ lateinit var snippets: SelectStatement + lateinit var bodySnippets: SelectStatement + lateinit var offsets: SelectStatement + lateinit var matchInfo: SelectStatement + database { + ArticleTable { table -> + all = table SELECT X + snippets = table SELECT listOf(table.snippet() AS ArticleMatch::excerpt) WHERE (table MATCH "search") + bodySnippets = table SELECT listOf(table.snippet("[", "]", column = 1) AS ArticleMatch::excerpt) WHERE (table MATCH "run") + offsets = table SELECT listOf(table.offsets() AS ArticleMatch::excerpt) WHERE (table MATCH "coroutines") + matchInfo = table SELECT listOf(table.matchinfo() AS ArticleMatchInfo::info) WHERE (table MATCH "kotlin") } } - val result1 = selectStatement.getResults().first() - assertEquals(1, selectStatement.getResults().size) - assertEquals(8, result1.paramInt) - assertEquals(null, result1.paramString) - assertEquals(8.8, result1.paramDouble) + assertEquals("draft", all.getResults().first().note) + assertEquals(listOf(ArticleMatch("SQLite Full-Text Search", "SQLite Full-Text Search")), snippets.getResults()) + assertEquals(listOf(ArticleMatch("Kotlin Coroutines", "[Running] suspending functions concurrently")), bodySnippets.getResults()) + // Column 0, term 0 of the query, at byte 7, 10 bytes long + assertEquals(listOf(ArticleMatch("Kotlin Coroutines", "0 0 7 10")), offsets.getResults()) + // "pcx": 1 phrase, 3 columns, and 3 numbers for each of them, of 4 bytes each + assertEquals(listOf(44, 44), matchInfo.getResults().map { it.info.size }) - // DELETE & SELECT database { - selectStatement = NullTesterTable { table -> - table DELETE WHERE (paramInt EQ null OR (paramDouble EQ null)) - table SELECT X + ArticleTable { table -> + table UPDATE SET { body = "Structured concurrency" } WHERE (rowid EQ 1L) + table DELETE WHERE (table MATCH "multiplatform") } } - val result2 = selectStatement.getResults().first() - assertEquals(1, selectStatement.getResults().size) - assertEquals(8, result2.paramInt) - assertEquals(null, result2.paramString) - assertEquals(8.8, result2.paramDouble) + assertEquals(listOf(1L), matches { this MATCH "structured" }) + assertEquals(emptyList(), matches { this MATCH "running" }) + assertEquals(listOf(1L), matches { this MATCH "kotlin" }) + + // A virtual table has no triggers, but its observed queries see SQLlin's statements that write to it + val kotlinArticles = Channel>(Channel.UNLIMITED) + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { ArticleTable SELECT WHERE(ArticleTable MATCH "kotlin") } + .collect { kotlinArticles.send(it) } + } + assertEquals(listOf(1L), kotlinArticles.receive().map { it.rowid }) + database { ArticleTable INSERT Article(rowid = null, title = "Kotlin Flows", body = "Cold streams", note = null) } + assertEquals(listOf("Kotlin Coroutines", "Kotlin Flows"), kotlinArticles.receive().map { it.title }) + database { BookTable INSERT Book(name = "Kotlin in Action", author = "Dmitry Jemerov", price = 40.0, pages = 360) } + runCurrent() + assertEquals(true, kotlinArticles.tryReceive().isFailure) + + // Rebuilt under another name, with its rowids, and renamed back, it is still watched. The rebuild itself + // leaves the results as they were, so it emits nothing. + val newArticles = ArticleTable.withName("articles_new") + assertEquals(true, newArticles is FtsTable<*>) + assertEquals(ArticleTable.createSQL.replace(" articles ", " articles_new "), newArticles.createSQL) + database { + CREATE(newArticles) + newArticles INSERT (ArticleTable SELECT X) + DROP(ArticleTable) + "articles_new" ALTER_RENAME_TABLE_TO ArticleTable + } + database { ArticleTable DELETE WHERE (ArticleTable MATCH "flows") } + assertEquals(listOf(1L), kotlinArticles.receive().map { it.rowid }) } } - fun testPrimaryKeyVariations() { - Database(getNewAPIDBConfig()).databaseAutoClose { database -> - // Test 1: Long primary key - val person1 = PersonWithId(id = null, name = "Alice", age = 25) - val person2 = PersonWithId(id = null, name = "Bob", age = 30) - - lateinit var personStatement: SelectStatement + /** + * Covers FTS3 tables: a tokenizer with an argument, which here removes diacritics, and a rowid named docid that the + * caller supplies, through INSERT, INSERT_OR_REPLACE, UPDATE and DELETE. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testFts3() = Database(getFtsDBConfig()).databaseAutoClose { database -> + assertEquals("CREATE VIRTUAL TABLE notes USING fts3(text,tokenize=unicode61 \"remove_diacritics=1\")", NoteTable.createSQL) + fun matches(block: NoteTable.() -> SelectCondition): List { + lateinit var statement: SelectStatement database { - PersonWithIdTable { table -> - table INSERT listOf(person1, person2) - personStatement = table SELECT X - } + statement = NoteTable SELECT WHERE(NoteTable.block()) } + return statement.getResults() + } + database { + NoteTable INSERT listOf(Note(docid = 5, text = "Crème brûlée"), Note(docid = 9, text = "Café au lait")) + } + assertEquals(listOf(Note(docid = 5, text = "Crème brûlée")), matches { this MATCH "creme" }) + assertEquals(listOf(9L), matches { text MATCH "cafe" }.map { it.docid }) - val personResults = personStatement.getResults() - assertEquals(2, personResults.size) - assertEquals("Alice", personResults[0].name) - assertEquals(25, personResults[0].age) - assertEquals("Bob", personResults[1].name) - assertEquals(30, personResults[1].age) + database { + NoteTable { table -> + table INSERT_OR_REPLACE Note(docid = 5, text = "Tarte Tatin") + table UPDATE SET { text = "Crêpe" } WHERE (docid EQ 9L) + } + } + assertEquals(listOf(5L), matches { this MATCH "tatin" }.map { it.docid }) + assertEquals(emptyList(), matches { this MATCH "creme" }) + assertEquals(listOf(9L), matches { this MATCH "crepe" }.map { it.docid }) - // Test 2: String primary key - val product1 = Product(sku = "SKU-WIDGET", name = "Widget", price = 19.99) - val product2 = Product(sku = "SKU-GADGET", name = "Gadget", price = 29.99) + database { NoteTable DELETE WHERE (NoteTable MATCH "tatin") } + lateinit var all: SelectStatement + database { + all = NoteTable SELECT X + } + assertEquals(listOf(Note(docid = 9, text = "Crêpe")), all.getResults()) + } - lateinit var productStatement: SelectStatement - database { - ProductTable { table -> - table INSERT listOf(product1, product2) - productStatement = table SELECT X + /** + * Covers CHECK constraints: their SQL, with a column's unnamed ones before its named ones, rows that pass them, + * NULL included, rows that fail one, which SQLite reports by its name, an UPDATE that fails one and leaves the row + * as it was, and an INSERT_OR_IGNORE that skips the rows failing one. SQLite reports an unnamed one by its + * expression from 3.34 on, and by the table's name before, as on Android below API 34. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testCheckConstraint() { + assertEquals( + "CREATE TABLE CheckedMembership(id INTEGER PRIMARY KEY," + + "name TEXT NOT NULL CHECK (length(trim(name)) > 0) CONSTRAINT lower_case_name CHECK (lower(name) = name)," + + "age INT CHECK (age BETWEEN 0 AND 150),level INT NOT NULL CHECK (level IN (0, 1, 2)),startDate TEXT NOT NULL," + + "endDate TEXT CHECK (endDate IS NULL OR date(endDate) IS endDate)," + + "CONSTRAINT valid_period CHECK (endDate IS NULL OR endDate >= startDate))", + CheckedMembershipTable.createSQL, + ) + val config = DSLDBConfiguration( + name = DATABASE_NAME, + path = path, + version = 1, + create = { CREATE(CheckedMembershipTable) }, + ) + fun member(name: String = "ann", age: Int? = 30, level: Int = 0, endDate: String? = null) = + CheckedMembership(id = null, name = name, age = age, level = level, startDate = "2026-01-01", endDate = endDate) + Database(config).databaseAutoClose { database -> + fun failure(block: DatabaseScope.() -> Unit): String = assertFails { database(block) }.message.orEmpty() + fun assertReports(message: String, name: String?, expression: String?) { + val reported = if (name != null) + name in message + else + expression!! in message || "CHECK constraint failed: CheckedMembership" in message + assertEquals(true, reported, "'$message' doesn't report the constraint '${name ?: expression}'") + } + fun names(): List { + lateinit var statement: SelectStatement + database { + statement = CheckedMembershipTable SELECT X } + return statement.getResults().map { it.name } } - val productResults = productStatement.getResults() - assertEquals(2, productResults.size) - assertEquals("SKU-WIDGET", productResults[0].sku) - assertEquals("Widget", productResults[0].name) - assertEquals(19.99, productResults[0].price) - assertEquals("SKU-GADGET", productResults[1].sku) - assertEquals("Gadget", productResults[1].name) - assertEquals(29.99, productResults[1].price) + // NULL passes a constraint, as it isn't false + database { + CheckedMembershipTable INSERT listOf(member(), member(name = "bob", age = null, level = 2, endDate = "2026-12-31")) + } + assertEquals(listOf("ann", "bob"), names()) - // Test 3: Autoincrement primary key - val student1 = StudentWithAutoincrement(id = null, studentName = "Charlie", grade = 85) - val student2 = StudentWithAutoincrement(id = null, studentName = "Diana", grade = 92) + // The rows failing a named constraint, and those failing an unnamed one, with its expression + val namedFailures = mapOf( + member(name = "Cat") to "lower_case_name", + member(endDate = "2025-12-31") to "valid_period", + ) + val unnamedFailures = mapOf( + member(name = " ") to "length(trim(name)) > 0", + member(age = 200) to "age BETWEEN 0 AND 150", + member(level = 3) to "level IN (0, 1, 2)", + member(endDate = "2026-13-01") to "date(endDate) IS endDate", + ) + for ((row, name) in namedFailures) + assertReports(failure { CheckedMembershipTable INSERT row }, name = name, expression = null) + for ((row, expression) in unnamedFailures) + assertReports(failure { CheckedMembershipTable INSERT row }, name = null, expression = expression) + assertEquals(listOf("ann", "bob"), names()) - lateinit var studentStatement: SelectStatement + val message = failure { + CheckedMembershipTable { table -> table UPDATE SET { level = 5 } WHERE (name EQ "ann") } + } + assertReports(message, name = null, expression = "level IN (0, 1, 2)") + lateinit var levels: SelectStatement database { - StudentWithAutoincrementTable { table -> - table INSERT listOf(student1, student2) - studentStatement = table SELECT X - } + levels = CheckedMembershipTable SELECT ORDER_BY(CheckedMembershipTable.name) } + assertEquals(listOf(0, 2), levels.getResults().map { it.level }) - val studentResults = studentStatement.getResults() - assertEquals(2, studentResults.size) - assertEquals("Charlie", studentResults[0].studentName) - assertEquals(85, studentResults[0].grade) - assertEquals("Diana", studentResults[1].studentName) - assertEquals(92, studentResults[1].grade) + database { + CheckedMembershipTable INSERT_OR_IGNORE listOf(member(name = "dan"), member(name = "Eve"), member(name = "fay", age = -1)) + } + assertEquals(listOf("ann", "bob", "dan"), names()) + } + } - // Test 4: Composite primary key - val enrollment1 = Enrollment(studentId = 1, courseId = 101, semester = "Fall 2025") - val enrollment2 = Enrollment(studentId = 1, courseId = 102, semester = "Fall 2025") - val enrollment3 = Enrollment(studentId = 2, courseId = 101, semester = "Fall 2025") + /** + * Covers subqueries: derived tables in FROM and in a join, one joining a table with itself, IN with a subquery + * combined with other conditions, correlated EXISTS, NOT, subqueries in UPDATE and DELETE, IN subqueries that don't + * fit, and observed queries that watch the tables their subqueries read. Parameters of the subqueries and of the + * queries they are in are bound in the order the SQL has them. + */ + @OptIn(ExperimentalCoroutinesApi::class, ExperimentalDSLDatabaseAPI::class) + fun testSubquery() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> + runTest { + database { + BookTable INSERT listOf( + Book(name = "The Da Vinci Code", author = "Dan Brown", price = 16.96, pages = 454), + Book(name = "The Lost Symbol", author = "Dan Brown", price = 19.95, pages = 510), + Book(name = "Kotlin Cookbook", author = "Ken Kousen", price = 37.72, pages = 251), + Book(name = "Cheap Book", author = "Dan Brown", price = 5.0, pages = 10), + ) + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "Dan Brown", age = 61), + PersonWithId(id = null, name = "Ken Kousen", age = 60), + PersonWithId(id = null, name = "Ann", age = 30), + ) + } - lateinit var enrollmentStatement: SelectStatement + lateinit var prolific: SelectStatement + lateinit var personBooks: SelectStatement + lateinit var pairs: SelectStatement database { - EnrollmentTable { table -> - table INSERT listOf(enrollment1, enrollment2, enrollment3) - enrollmentStatement = table SELECT X - } + // The books over 10 of each author; a derived table can be read by several queries + val authorBooks = BookTable { table -> + table SELECT listOf(count(X) AS AuthorBooks::books) WHERE (price GT 10.0) GROUP_BY author + } AS AuthorBooksView + prolific = authorBooks SELECT WHERE(AuthorBooksView.books GT 1) + personBooks = PersonWithIdTable SELECT INNER_JOIN(authorBooks) ON (PersonWithIdTable.name EQ AuthorBooksView.author) + // The persons under other names join their table with itself + val elders = PersonWithIdTable { table -> + table SELECT listOf(name AS Elder::elderName, age AS Elder::elderAge) + } AS ElderView + pairs = PersonWithIdTable SELECT INNER_JOIN(elders) ON (ElderView.elderAge GT PersonWithIdTable.age) } + assertEquals(listOf(AuthorBooks("Dan Brown", 2)), prolific.getResults()) + assertEquals( + listOf(PersonBooks("Dan Brown", 61, 2), PersonBooks("Ken Kousen", 60, 1)), + personBooks.getResults().sortedBy { it.name }, + ) + assertEquals( + listOf(YoungerAndElder("Ann", "Dan Brown"), YoungerAndElder("Ann", "Ken Kousen"), YoungerAndElder("Ken Kousen", "Dan Brown")), + pairs.getResults().sortedWith(compareBy({ it.name }, { it.elderName })), + ) - val enrollmentResults = enrollmentStatement.getResults() - assertEquals(3, enrollmentResults.size) - assertEquals(true, enrollmentResults.any { it == enrollment1 }) - assertEquals(true, enrollmentResults.any { it == enrollment2 }) - assertEquals(true, enrollmentResults.any { it == enrollment3 }) - } - } + lateinit var expensiveAuthors: SelectStatement + lateinit var shortBookAuthors: SelectStatement + lateinit var withoutBooks: SelectStatement + lateinit var middleAged: SelectStatement + database { + PersonWithIdTable { table -> + expensiveAuthors = table SELECT WHERE((age GT 50) AND (name IN (BookTable SELECT WHERE(BookTable.price GT 30.0)))) + shortBookAuthors = table SELECT WHERE(EXISTS(BookTable SELECT WHERE((BookTable.author EQ name) AND (BookTable.pages LT 300) AND (BookTable.price GT 6.0)))) + withoutBooks = table SELECT WHERE(NOT(EXISTS(BookTable SELECT WHERE(BookTable.author EQ name)))) + middleAged = table SELECT WHERE(NOT((age LT 40) OR (age GT 60))) + } + } + assertEquals(listOf("Ken Kousen"), expensiveAuthors.getResults().map { it.name }) + assertEquals(listOf("Ken Kousen"), shortBookAuthors.getResults().map { it.name }) + assertEquals(listOf("Ann"), withoutBooks.getResults().map { it.name }) + assertEquals(listOf("Ken Kousen"), middleAged.getResults().map { it.name }) - @OptIn(AdvancedInsertAPI::class) - fun testInsertWithId() { - Database(getNewAPIDBConfig()).databaseAutoClose { database -> - val person1 = PersonWithId(id = 100, name = "Eve", age = 28) - val person2 = PersonWithId(id = 200, name = "Frank", age = 35) + // IN compares with a single column of values of the same kind + assertFailsWith { + database { PersonWithIdTable SELECT WHERE(PersonWithIdTable.age IN (BookTable SELECT X())) } + } + assertFailsWith { + database { PersonWithIdTable SELECT WHERE(PersonWithIdTable.name IN (BookTable SELECT X())) } + } - lateinit var selectStatement: SelectStatement + // Subqueries in UPDATE and DELETE, after the parameters of SET database { PersonWithIdTable { table -> - table INSERT_WITH_ID listOf(person1, person2) - selectStatement = table SELECT X + table UPDATE SET { age = 62 } WHERE (name IN (BookTable SELECT WHERE(BookTable.pages GT 500))) } + PersonWithIdTable DELETE WHERE(NOT(PersonWithIdTable.name IN (BookTable SELECT X()))) + } + lateinit var persons: SelectStatement + database { + persons = PersonWithIdTable SELECT X } + assertEquals(listOf("Dan Brown" to 62, "Ken Kousen" to 60), persons.getResults().map { it.name to it.age }.sortedBy { it.first }) - val results = selectStatement.getResults() - assertEquals(2, results.size) - assertEquals(100L, results[0].id) - assertEquals("Eve", results[0].name) - assertEquals(28, results[0].age) - assertEquals(200L, results[1].id) - assertEquals("Frank", results[1].name) - assertEquals(35, results[1].age) + // Observed queries watch the tables their derived tables and IN subqueries read + val books = Channel>(Channel.UNLIMITED) + val oldest = Channel>(Channel.UNLIMITED) + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { + val authorBooks = BookTable { table -> table SELECT listOf(count(X) AS AuthorBooks::books) GROUP_BY author } AS AuthorBooksView + PersonWithIdTable SELECT INNER_JOIN(authorBooks) ON (PersonWithIdTable.name EQ AuthorBooksView.author) + }.collect { books.send(it) } + } + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { + PersonWithIdTable { table -> table SELECT WHERE(age IN (table SELECT listOf(max(age) AS PersonAge::age))) } + }.collect { oldest.send(it) } + } + assertEquals(listOf(3L, 1L), books.receive().sortedBy { it.name }.map { it.books }) + assertEquals(listOf("Dan Brown"), oldest.receive().map { it.name }) + database { BookTable INSERT Book(name = "Inferno", author = "Ken Kousen", price = 18.0, pages = 480) } + assertEquals(listOf(3L, 2L), books.receive().sortedBy { it.name }.map { it.books }) + database { PersonWithIdTable INSERT PersonWithId(id = null, name = "Old Timer", age = 99) } + assertEquals(listOf("Old Timer"), oldest.receive().map { it.name }) } } - @OptIn(AdvancedInsertAPI::class) - fun testInsertOrReplace() { - Database(getNewAPIDBConfig()).databaseAutoClose { database -> - // Insert an initial entity with a known ID - val original = PersonWithId(id = 100L, name = "Eve", age = 28) + /** + * Covers conditions combined with AND and OR, which apply in the order they are written, as Kotlin's infix functions + * do, though SQL gives AND precedence over OR: a condition that combines others with the other operator is put in + * parentheses, while one that combines them with the same operator isn't. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testConditionPrecedence() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> + database { + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "a", age = 1), + PersonWithId(id = null, name = "x", age = 2), + PersonWithId(id = null, name = "y", age = 1), + PersonWithId(id = null, name = "x", age = 3), + ) + } + fun select(condition: PersonWithIdTable.() -> SelectCondition): SelectStatement { + lateinit var statement: SelectStatement database { - PersonWithIdTable { table -> - table INSERT_WITH_ID original - } + statement = PersonWithIdTable SELECT WHERE(PersonWithIdTable.condition()) } + return statement + } + fun names(condition: PersonWithIdTable.() -> SelectCondition): List = + select(condition).getResults().map { it.name }.sorted() - lateinit var selectStatement: SelectStatement - database { - selectStatement = PersonWithIdTable SELECT X + val statement = select { (age EQ 1) OR (age EQ 2) AND (name EQ "x") } + assertEquals( + "SELECT id,name,age FROM person_with_id WHERE (person_with_id.age=? OR person_with_id.age=?) AND person_with_id.name=?", + statement.sqlStr, + ) + assertEquals(listOf("x"), statement.getResults().map { it.name }) + assertEquals(listOf("a", "x", "y"), names { (age EQ 1) OR ((age EQ 2) AND (name EQ "x")) }) + assertEquals(listOf("x"), names { (name EQ "x") AND ((age EQ 2) OR (age EQ 1)) }) + assertEquals(listOf("a", "x", "y"), names { (name EQ "x") AND (age EQ 3) OR (age EQ 1) }) + assertEquals(listOf("a", "x", "x", "y"), names { (age EQ 1) OR (age EQ 2) OR (age EQ 3) }) + assertEquals(listOf("x"), names { NOT((age EQ 1) OR (age EQ 2)) AND (name EQ "x") }) + } + + /** + * Covers a condition kept in a variable and used in several statements: combining it with others leaves its own + * parameters as they were, so that each statement binds the parameters its SQL has. + */ + fun testConditionReuse() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> + database { + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "a", age = 1), + PersonWithId(id = null, name = "x", age = 2), + PersonWithId(id = null, name = "y", age = 3), + ) + } + val older = PersonWithIdTable.age GTE 2 + lateinit var combined: SelectStatement + lateinit var alone: SelectStatement + lateinit var either: SelectStatement + database { + PersonWithIdTable { table -> + combined = table SELECT WHERE(older AND (name EQ "x")) + alone = table SELECT WHERE(older) + either = table SELECT WHERE(older OR (name EQ "a")) } - assertEquals(1, selectStatement.getResults().size) - assertEquals(100L, selectStatement.getResults().first().id) - assertEquals("Eve", selectStatement.getResults().first().name) + } + assertEquals(listOf("x"), combined.getResults().map { it.name }) + assertEquals(listOf("x", "y"), alone.getResults().map { it.name }.sorted()) + assertEquals(listOf("a", "x", "y"), either.getResults().map { it.name }.sorted()) + } - // INSERT_OR_REPLACE with the same PK — should replace the existing row - val replacement = PersonWithId(id = 100L, name = "Eve Updated", age = 29) + /** + * Covers joins of relations: USING and NATURAL joins, which merge the columns of the same name, result columns with + * GROUP BY on an outer join, three joined tables, CROSS JOIN, a derived table joined with parameters in the + * subquery, the ON condition and WHERE, the checks of the type rows are read into, RIGHT and FULL OUTER JOIN where + * SQLite has them, and an observed query of a join. + */ + @OptIn(ExperimentalCoroutinesApi::class, ExperimentalDSLDatabaseAPI::class, PlatformDependentSQLiteAPI::class) + fun testJoinedRelation() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> + runTest { database { - PersonWithIdTable { table -> - table INSERT_OR_REPLACE replacement - } + BookTable INSERT listOf( + Book(name = "The Da Vinci Code", author = "Dan Brown", price = 16.96, pages = 454), + Book(name = "The Lost Symbol", author = "Dan Brown", price = 19.95, pages = 510), + Book(name = "Kotlin Cookbook", author = "Ken Kousen", price = 37.72, pages = 251), + ) + CategoryTable INSERT listOf( + Category(name = "The Da Vinci Code", code = 1), + Category(name = "Kotlin Cookbook", code = 2), + Category(name = "Unwritten", code = 3), + ) + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "Dan Brown", age = 61), + PersonWithId(id = null, name = "Ken Kousen", age = 60), + PersonWithId(id = null, name = "Ann", age = 30), + ) } - database { - selectStatement = PersonWithIdTable SELECT X + lateinit var using: SelectStatement + lateinit var natural: SelectStatement + lateinit var stats: SelectStatement + lateinit var titles: SelectStatement + lateinit var codes: SelectStatement + lateinit var crossCount: SelectStatement + lateinit var prolific: SelectStatement + database { + // USING and NATURAL merge 'name', which book and category both have + using = (FROM(BookTable) INNER_JOIN CategoryTable USING BookTable.name) SELECT WHERE(CategoryTable.code GT 0) + natural = (FROM(BookTable) NATURAL_JOIN CategoryTable) SELECT X() + // Ann has no books: count is 0, while max is NULL + stats = (FROM(PersonWithIdTable) LEFT_OUTER_JOIN BookTable ON (PersonWithIdTable.name EQ BookTable.author)) SELECT listOf( + PersonWithIdTable.name AS PersonBookStats::name, + PersonWithIdTable.count(BookTable.name) AS PersonBookStats::books, + PersonWithIdTable.max(BookTable.price) AS PersonBookStats::highest, + ) GROUP_BY PersonWithIdTable.name + titles = (FROM(PersonWithIdTable) LEFT_OUTER_JOIN BookTable ON (PersonWithIdTable.name EQ BookTable.author)) SELECT listOf( + BookTable.name AS PersonBookTitle::title, + ) WHERE (PersonWithIdTable.age LT 40) + codes = ( + FROM(PersonWithIdTable) + INNER_JOIN BookTable ON (PersonWithIdTable.name EQ BookTable.author) + INNER_JOIN CategoryTable ON (BookTable.name EQ CategoryTable.name) + ) SELECT X() + crossCount = (FROM(BookTable) CROSS_JOIN CategoryTable) SELECT listOf(BookTable.count(X) AS BookCount::books) + // Parameters of the derived table, of ON and of WHERE, in this order + val authorBooks = BookTable { table -> + table SELECT listOf(count(X) AS AuthorBooks::books) WHERE (price GT 10.0) GROUP_BY author + } AS AuthorBooksView + prolific = (FROM(PersonWithIdTable) INNER_JOIN authorBooks ON ((PersonWithIdTable.name EQ AuthorBooksView.author) AND (AuthorBooksView.books GT 1))) SELECT WHERE(PersonWithIdTable.age GT 50) + } + assertEquals(listOf("Kotlin Cookbook" to 2, "The Da Vinci Code" to 1), using.getResults().map { it.name to it.code }.sortedBy { it.first }) + assertEquals(using.getResults().sortedBy { it.name }, natural.getResults().sortedBy { it.name }) + assertEquals( + listOf(PersonBookStats("Ann", 0, null), PersonBookStats("Dan Brown", 2, 19.95), PersonBookStats("Ken Kousen", 1, 37.72)), + stats.getResults().sortedBy { it.name }, + ) + assertEquals(listOf(PersonBookTitle(30, null)), titles.getResults()) + assertEquals(listOf(AuthorCode(60, 2), AuthorCode(61, 1)), codes.getResults().sortedBy { it.age }) + assertEquals(listOf(BookCount(9)), crossCount.getResults()) + assertEquals(listOf(PersonBooks("Dan Brown", 61, 2)), prolific.getResults()) + + // The type rows are read into is checked against the joined relations + val ambiguous = assertFailsWith { + database { (FROM(BookTable) INNER_JOIN CategoryTable ON (BookTable.name EQ CategoryTable.name)) SELECT X() } } - val resultsAfterReplace = selectStatement.getResults() - assertEquals(1, resultsAfterReplace.size) - assertEquals(100L, resultsAfterReplace.first().id) - assertEquals("Eve Updated", resultsAfterReplace.first().name) - assertEquals(29, resultsAfterReplace.first().age) + assertEquals(true, "can't be told apart" in ambiguous.message!!, ambiguous.message) + val outerColumn = assertFailsWith { + database { (FROM(PersonWithIdTable) LEFT_OUTER_JOIN BookTable ON (PersonWithIdTable.name EQ BookTable.author)) SELECT X() } + } + assertEquals(true, "outer join" in outerColumn.message!!, outerColumn.message) + val outerExpression = assertFailsWith { + database { + (FROM(PersonWithIdTable) LEFT_OUTER_JOIN BookTable ON (PersonWithIdTable.name EQ BookTable.author)) SELECT listOf(BookTable.name AS NameOnly::name) + } + } + assertEquals(true, "outer join" in outerExpression.message!!, outerExpression.message) + assertFailsWith { + database { (FROM(BookTable) INNER_JOIN BookTable ON (BookTable.name EQ BookTable.name)) SELECT X() } + } + val otherTable = assertFailsWith { + database { (FROM(BookTable) CROSS_JOIN CategoryTable) SELECT listOf(PersonWithIdTable.name AS NameOnly::name) } + } + assertEquals(true, "belongs to table 'person_with_id'" in otherTable.message!!, otherTable.message) - // INSERT_OR_REPLACE with a new entity (null ID) — should insert without conflict - val newEntity = PersonWithId(id = null, name = "Frank", age = 35) - database { - PersonWithIdTable { table -> - table INSERT_OR_REPLACE newEntity + // RIGHT and FULL OUTER JOIN need SQLite 3.39.0, which Android has from API 34 on + val outerJoins = try { + lateinit var right: SelectStatement + lateinit var full: SelectStatement + database { + right = (FROM(BookTable) RIGHT_OUTER_JOIN CategoryTable USING BookTable.name) SELECT X() + full = (FROM(BookTable) FULL_OUTER_JOIN CategoryTable USING BookTable.name) SELECT X() + } + right.getResults() to full.getResults() + } catch (e: Exception) { + assertEquals(true, "RIGHT and FULL OUTER JOINs are not currently supported" in e.message.orEmpty(), e.message) + null + } + outerJoins?.let { (right, full) -> + assertEquals(listOf(1, 2, 3), right.map { it.code }.sortedBy { it }) + assertEquals(listOf(null), right.filter { it.code == 3 }.map { it.author }) + assertEquals(4, full.size) + } + + // An observed query of a join watches all its tables + val observed = Channel>(Channel.UNLIMITED) + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { + (FROM(PersonWithIdTable) LEFT_OUTER_JOIN BookTable ON (PersonWithIdTable.name EQ BookTable.author)) SELECT listOf( + PersonWithIdTable.name AS PersonBookStats::name, + PersonWithIdTable.count(BookTable.name) AS PersonBookStats::books, + PersonWithIdTable.max(BookTable.price) AS PersonBookStats::highest, + ) GROUP_BY PersonWithIdTable.name + }.collect { observed.send(it) } + } + assertEquals(0L, observed.receive().single { it.name == "Ann" }.books) + database { BookTable INSERT Book(name = "Ann's Book", author = "Ann", price = 9.0, pages = 99) } + assertEquals(1L, observed.receive().single { it.name == "Ann" }.books) + } + } + + @OptIn(DelicateCoroutinesApi::class, ExperimentalCoroutinesApi::class) + fun testConcurrency() = Database(getDefaultDBConfig(), true).databaseAutoClose { database -> + runTest { + val book1 = Book(name = "The Da Vinci Code", author = "Dan Brown", pages = 454, price = 16.96) + val book2 = Book(name = "The Lost Symbol", author = "Dan Brown", pages = 510, price = 19.95) + launch(newSingleThreadContext("test0")) { + lateinit var statement: SelectStatement + database suspendedScope { + statement = BookTable { table -> + table INSERT listOf(book1, book2) + table SELECT X + } + } + assertEquals(true, statement.getResults().any { it == book1 }) + assertEquals(true, statement.getResults().any { it == book2 }) + } + launch(newSingleThreadContext("test1")) { + val book1NewPrice = 18.96 + val book2NewPrice = 21.95 + val newBook1 = Book(name = "The Da Vinci Code", author = "Dan Brown", pages = 454, price = book1NewPrice) + val newBook2 = Book(name = "The Lost Symbol", author = "Dan Brown", pages = 510, price = book2NewPrice) + lateinit var statement: SelectStatement + database suspendedScope { + statement = transaction { + BookTable { table -> + table INSERT listOf(newBook1, newBook2) + table SELECT X + } + } } + assertEquals(true, statement.getResults().any { it == newBook1 }) + assertEquals(true, statement.getResults().any { it == newBook2 }) } + } + } - database { - selectStatement = PersonWithIdTable SELECT X + /** + * Checks that the processor generates a column property in `SET {}` for every supported type, of that type: each + * assignment only compiles while it is. Reading one is a compile error, as its getter only returns a placeholder. + */ + fun testPrimitiveTypeForKSP() { + TestPrimitiveTypeForKSPTable { + SET { + testInt = 1 + testLong = 1L + testShort = 1 + testByte = 1 + testFloat = 1F + testDouble = 1.0 + testUInt = 1U + testULong = 1UL + testUShort = 1U + testUByte = 1U + testBoolean = null + testChar = null + testString = "1" + testByteArray = byteArrayOf(1) + testEnum = Priority.HIGH + testTypeAlias = 1 } - assertEquals(2, selectStatement.getResults().size) - assertEquals(true, selectStatement.getResults().any { it.name == "Frank" }) } } - @OptIn(AdvancedInsertAPI::class) - fun testInsertOrIgnore() { - Database(getNewAPIDBConfig()).databaseAutoClose { database -> - // A conflict on the primary key leaves the existing row exactly as it is, while the entities that - // don't conflict are inserted. Seeing the conflict at all depends on the key being written. - database { - PersonWithIdTable INSERT_WITH_ID PersonWithId(id = 100L, name = "Eve", age = 28) + fun testNullValue() { + val config = DSLDBConfiguration( + name = DATABASE_NAME, + path = path, + version = 1, + create = { + CREATE(NullTesterTable) } + ) + Database(config, true).databaseAutoClose { database -> + lateinit var selectStatement: SelectStatement + // INSERT & SELECT database { - PersonWithIdTable INSERT_OR_IGNORE listOf( - PersonWithId(id = 100L, name = "Eve Updated", age = 29), - PersonWithId(id = 101L, name = "Grace", age = 30), - ) + selectStatement = NullTesterTable { table -> + table INSERT listOf( + NullTester(null, null, null), + NullTester(8, "888", 8.8), + ) + table SELECT X + } } - lateinit var people: SelectStatement - database { - people = PersonWithIdTable SELECT X + + selectStatement.getResults().forEachIndexed { i, tester -> + when (i) { + 0 -> { + assertEquals(null, tester.paramInt) + assertEquals(null, tester.paramString) + assertEquals(null, tester.paramDouble) + } + 1 -> { + assertEquals(8, tester.paramInt) + assertEquals("888", tester.paramString) + assertEquals(8.8, tester.paramDouble) + } + } } - assertEquals(2, people.getResults().size) - val eve = people.getResults().first { it.id == 100L } - assertEquals("Eve", eve.name) - assertEquals(28, eve.age) - assertEquals("Grace", people.getResults().first { it.id == 101L }.name) - // A null ID is still assigned by the database, so it can't conflict on the primary key + // UPDATE & SELECT database { - PersonWithIdTable INSERT_OR_IGNORE PersonWithId(id = null, name = "Frank", age = 35) + selectStatement = NullTesterTable { table -> + table UPDATE SET { paramString = null } WHERE (paramDouble EQ 8.8) + table SELECT WHERE (paramInt NEQ null) + } } + val result1 = selectStatement.getResults().first() + assertEquals(1, selectStatement.getResults().size) + assertEquals(8, result1.paramInt) + assertEquals(null, result1.paramString) + assertEquals(8.8, result1.paramDouble) + + // DELETE & SELECT database { - people = PersonWithIdTable SELECT X + selectStatement = NullTesterTable { table -> + table DELETE WHERE (paramInt EQ null OR (paramDouble EQ null)) + table SELECT X + } } - assertEquals(3, people.getResults().size) - assertNotEquals(null, people.getResults().first { it.name == "Frank" }.id) + val result2 = selectStatement.getResults().first() + assertEquals(1, selectStatement.getResults().size) + assertEquals(8, result2.paramInt) + assertEquals(null, result2.paramString) + assertEquals(8.8, result2.paramDouble) + } + } + + fun testPrimaryKeyVariations() { + Database(getNewAPIDBConfig()).databaseAutoClose { database -> + // Test 1: Long primary key + val person1 = PersonWithId(id = null, name = "Alice", age = 25) + val person2 = PersonWithId(id = null, name = "Bob", age = 30) + + lateinit var personStatement: SelectStatement + database { + PersonWithIdTable { table -> + table INSERT listOf(person1, person2) + personStatement = table SELECT X + } + } + + val personResults = personStatement.getResults() + assertEquals(2, personResults.size) + assertEquals("Alice", personResults[0].name) + assertEquals(25, personResults[0].age) + assertEquals("Bob", personResults[1].name) + assertEquals(30, personResults[1].age) + + // Test 2: String primary key + val product1 = Product(sku = "SKU-WIDGET", name = "Widget", price = 19.99) + val product2 = Product(sku = "SKU-GADGET", name = "Gadget", price = 29.99) + + lateinit var productStatement: SelectStatement + database { + ProductTable { table -> + table INSERT listOf(product1, product2) + productStatement = table SELECT X + } + } + + val productResults = productStatement.getResults() + assertEquals(2, productResults.size) + assertEquals("SKU-WIDGET", productResults[0].sku) + assertEquals("Widget", productResults[0].name) + assertEquals(19.99, productResults[0].price) + assertEquals("SKU-GADGET", productResults[1].sku) + assertEquals("Gadget", productResults[1].name) + assertEquals(29.99, productResults[1].price) + + // Test 3: Autoincrement primary key + val student1 = StudentWithAutoincrement(id = null, studentName = "Charlie", grade = 85) + val student2 = StudentWithAutoincrement(id = null, studentName = "Diana", grade = 92) + + lateinit var studentStatement: SelectStatement + database { + StudentWithAutoincrementTable { table -> + table INSERT listOf(student1, student2) + studentStatement = table SELECT X + } + } + + val studentResults = studentStatement.getResults() + assertEquals(2, studentResults.size) + assertEquals("Charlie", studentResults[0].studentName) + assertEquals(85, studentResults[0].grade) + assertEquals("Diana", studentResults[1].studentName) + assertEquals(92, studentResults[1].grade) + + // Test 4: Composite primary key + val enrollment1 = Enrollment(studentId = 1, courseId = 101, semester = "Fall 2025") + val enrollment2 = Enrollment(studentId = 1, courseId = 102, semester = "Fall 2025") + val enrollment3 = Enrollment(studentId = 2, courseId = 101, semester = "Fall 2025") + + lateinit var enrollmentStatement: SelectStatement + database { + EnrollmentTable { table -> + table INSERT listOf(enrollment1, enrollment2, enrollment3) + enrollmentStatement = table SELECT X + } + } + + val enrollmentResults = enrollmentStatement.getResults() + assertEquals(3, enrollmentResults.size) + assertEquals(true, enrollmentResults.any { it == enrollment1 }) + assertEquals(true, enrollmentResults.any { it == enrollment2 }) + assertEquals(true, enrollmentResults.any { it == enrollment3 }) + } + } + + @OptIn(AdvancedInsertAPI::class) + fun testInsertWithId() { + Database(getNewAPIDBConfig()).databaseAutoClose { database -> + val person1 = PersonWithId(id = 100, name = "Eve", age = 28) + val person2 = PersonWithId(id = 200, name = "Frank", age = 35) + + lateinit var selectStatement: SelectStatement + database { + PersonWithIdTable { table -> + table INSERT_WITH_ID listOf(person1, person2) + selectStatement = table SELECT X + } + } + + val results = selectStatement.getResults() + assertEquals(2, results.size) + assertEquals(100L, results[0].id) + assertEquals("Eve", results[0].name) + assertEquals(28, results[0].age) + assertEquals(200L, results[1].id) + assertEquals("Frank", results[1].name) + assertEquals(35, results[1].age) + } + } + + @OptIn(AdvancedInsertAPI::class) + fun testInsertOrReplace() { + Database(getNewAPIDBConfig()).databaseAutoClose { database -> + // Insert an initial entity with a known ID + val original = PersonWithId(id = 100L, name = "Eve", age = 28) + database { + PersonWithIdTable { table -> + table INSERT_WITH_ID original + } + } + + lateinit var selectStatement: SelectStatement + database { + selectStatement = PersonWithIdTable SELECT X + } + assertEquals(1, selectStatement.getResults().size) + assertEquals(100L, selectStatement.getResults().first().id) + assertEquals("Eve", selectStatement.getResults().first().name) + + // INSERT_OR_REPLACE with the same PK — should replace the existing row + val replacement = PersonWithId(id = 100L, name = "Eve Updated", age = 29) + database { + PersonWithIdTable { table -> + table INSERT_OR_REPLACE replacement + } + } + + database { + selectStatement = PersonWithIdTable SELECT X + } + val resultsAfterReplace = selectStatement.getResults() + assertEquals(1, resultsAfterReplace.size) + assertEquals(100L, resultsAfterReplace.first().id) + assertEquals("Eve Updated", resultsAfterReplace.first().name) + assertEquals(29, resultsAfterReplace.first().age) + + // INSERT_OR_REPLACE with a new entity (null ID) — should insert without conflict + val newEntity = PersonWithId(id = null, name = "Frank", age = 35) + database { + PersonWithIdTable { table -> + table INSERT_OR_REPLACE newEntity + } + } + + database { + selectStatement = PersonWithIdTable SELECT X + } + assertEquals(2, selectStatement.getResults().size) + assertEquals(true, selectStatement.getResults().any { it.name == "Frank" }) + } + } + + @OptIn(AdvancedInsertAPI::class) + fun testInsertOrIgnore() { + Database(getNewAPIDBConfig()).databaseAutoClose { database -> + // A conflict on the primary key leaves the existing row exactly as it is, while the entities that + // don't conflict are inserted. Seeing the conflict at all depends on the key being written. + database { + PersonWithIdTable INSERT_WITH_ID PersonWithId(id = 100L, name = "Eve", age = 28) + } + database { + PersonWithIdTable INSERT_OR_IGNORE listOf( + PersonWithId(id = 100L, name = "Eve Updated", age = 29), + PersonWithId(id = 101L, name = "Grace", age = 30), + ) + } + lateinit var people: SelectStatement + database { + people = PersonWithIdTable SELECT X + } + assertEquals(2, people.getResults().size) + val eve = people.getResults().first { it.id == 100L } + assertEquals("Eve", eve.name) + assertEquals(28, eve.age) + assertEquals("Grace", people.getResults().first { it.id == 101L }.name) + + // A null ID is still assigned by the database, so it can't conflict on the primary key + database { + PersonWithIdTable INSERT_OR_IGNORE PersonWithId(id = null, name = "Frank", age = 35) + } + database { + people = PersonWithIdTable SELECT X + } + assertEquals(3, people.getResults().size) + assertNotEquals(null, people.getResults().first { it.name == "Frank" }.id) + + // A conflict on a UNIQUE column other than the key is ignored as well + database { + UniqueEmailTestTable INSERT UniqueEmailTest(id = null, email = "ivy@example.com", name = "Ivy") + UniqueEmailTestTable INSERT_OR_IGNORE UniqueEmailTest(id = null, email = "ivy@example.com", name = "Ivy Again") + } + lateinit var accounts: SelectStatement + database { + accounts = UniqueEmailTestTable SELECT X + } + assertEquals(1, accounts.getResults().size) + assertEquals("Ivy", accounts.getResults().first().name) + + // With a composite key, inserting a pair again keeps its existing row, other columns included, + // which is what tells INSERT_OR_IGNORE apart from INSERT_OR_REPLACE + database { + EnrollmentTable INSERT Enrollment(studentId = 1, courseId = 101, semester = "Spring") + } + database { + EnrollmentTable INSERT_OR_IGNORE listOf( + Enrollment(studentId = 1, courseId = 101, semester = "Fall"), + Enrollment(studentId = 1, courseId = 102, semester = "Fall"), + ) + } + lateinit var enrollments: SelectStatement + database { + enrollments = EnrollmentTable SELECT X + } + assertEquals(2, enrollments.getResults().size) + assertEquals("Spring", enrollments.getResults().first { it.courseId == 101L }.semester) + assertEquals("Fall", enrollments.getResults().first { it.courseId == 102L }.semester) + } + } + + /** + * Covers projection: a SELECT that reads rows into a narrower @Serializable type than the table's own row type, + * given to the clause function, as in `X()` or `WHERE(...)`. Only the columns that type's + * properties name are selected, and a type that doesn't fit the table is rejected while the statement is built. + */ + fun testProjection() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> + database { + BookTable INSERT listOf( + Book(name = "The Da Vinci Code", author = "Dan Brown", price = 16.96, pages = 454), + Book(name = "The Lost Symbol", author = "Dan Brown", price = 19.95, pages = 510), + Book(name = "Kotlin Cookbook", author = "Ken Kousen", price = 37.72, pages = 251), + ) + } + + // No clause: SELECT name,author FROM book, and SELECT DISTINCT author FROM book. Three books but two authors, + // which only holds if DISTINCT compares the projected column alone, so nothing else is selected. + lateinit var titles: SelectStatement + lateinit var authors: SelectStatement + database { + titles = BookTable SELECT X() + authors = BookTable SELECT_DISTINCT X() + } + assertEquals(3, titles.getResults().size) + assertEquals(true, BookTitle("Kotlin Cookbook", "Ken Kousen") in titles.getResults()) + assertEquals(listOf("Dan Brown", "Ken Kousen"), authors.getResults().map { it.author }.sorted()) + + // Each clause can start a projection, and the projection carries through the rest of the chain + lateinit var longestByBrown: SelectStatement + lateinit var byPages: SelectStatement + lateinit var firstTwo: SelectStatement + lateinit var grouped: SelectStatement + database { + BookTable { table -> + longestByBrown = table SELECT WHERE(author EQ "Dan Brown") ORDER_BY (pages to DESC) LIMIT 1 + byPages = table SELECT ORDER_BY(pages to ASC) + firstTwo = table SELECT LIMIT(2) + grouped = table SELECT GROUP_BY(author) + } + } + assertEquals(listOf(BookTitle("The Lost Symbol", "Dan Brown")), longestByBrown.getResults()) + assertEquals(listOf("Kotlin Cookbook", "The Da Vinci Code", "The Lost Symbol"), byPages.getResults().map { it.name }) + assertEquals(2, firstTwo.getResults().size) + assertEquals(listOf("Dan Brown", "Ken Kousen"), grouped.getResults().map { it.author }.sorted()) + + // The DISTINCT variant of each clause + lateinit var distinctWhere: SelectStatement + lateinit var distinctOrderBy: SelectStatement + lateinit var distinctLimit: SelectStatement + lateinit var distinctGroupBy: SelectStatement + database { + BookTable { table -> + distinctWhere = table SELECT_DISTINCT WHERE(price GT 10.0) + distinctOrderBy = table SELECT_DISTINCT ORDER_BY(author to DESC) + distinctLimit = table SELECT_DISTINCT LIMIT(1) + distinctGroupBy = table SELECT_DISTINCT GROUP_BY(author) + } + } + assertEquals(2, distinctWhere.getResults().size) + assertEquals(listOf("Ken Kousen", "Dan Brown"), distinctOrderBy.getResults().map { it.author }) + assertEquals(1, distinctLimit.getResults().size) + assertEquals(2, distinctGroupBy.getResults().size) + + // A nullable column is read into a nullable property + database { + UserAccountTable INSERT UserAccount( + id = null, + username = "ivy", + email = "ivy@example.com", + status = UserStatus.ACTIVE, + priority = Priority.LOW, + notes = null, + ) + } + lateinit var notes: SelectStatement + database { + notes = UserAccountTable SELECT X() + } + assertEquals(listOf(UserNotes("ivy", null)), notes.getResults()) + + // A type that doesn't fit the table is rejected while the statement is built, before anything runs + val notAColumn = assertFailsWith { + database { BookTable SELECT X() } + } + assertEquals(true, notAColumn.message!!.contains("'isbn' isn't a column")) + val wrongType = assertFailsWith { + database { BookTable SELECT WHERE(BookTable.pages GT 0) } + } + assertEquals(true, wrongType.message!!.contains("'pages' is a kotlin.String, but the column holds a kotlin.Int")) + val notNullable = assertFailsWith { + database { UserAccountTable SELECT X() } + } + assertEquals(true, notNullable.message!!.contains("column 'notes' is nullable")) + } + + /** + * Covers DISTINCT in aggregate functions: `count`, `sum`, `avg` and `group_concat` of the distinct values, next to + * the aggregates of all values. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testDistinctAggregate() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + BookTable INSERT listOf( + Book(name = "The Da Vinci Code", author = "Dan Brown", price = 10.0, pages = 100), + Book(name = "The Lost Symbol", author = "Dan Brown", price = 10.0, pages = 200), + Book(name = "Kotlin Cookbook", author = "Ken Kousen", price = 20.0, pages = 100), + ) + } + lateinit var totals: SelectStatement + database { + BookTable { table -> + totals = table SELECT listOf( + count(DISTINCT(author)) AS DistinctBookTotals::authors, + count(author) AS DistinctBookTotals::books, + sum(DISTINCT(price)) AS DistinctBookTotals::distinctPrice, + sum(price) AS DistinctBookTotals::totalPrice, + sum(DISTINCT(pages)) AS DistinctBookTotals::distinctPages, + avg(DISTINCT(pages)) AS DistinctBookTotals::averageDistinctPages, + group_concat(DISTINCT(author)) AS DistinctBookTotals::authorNames, + ) + } + } + val total = totals.getResults().single() + assertEquals(2L, total.authors) + assertEquals(3L, total.books) + assertEquals(30.0, total.distinctPrice!!, 1e-9) + assertEquals(40.0, total.totalPrice!!, 1e-9) + assertEquals(300L, total.distinctPages) + assertEquals(150.0, total.averageDistinctPages!!, 1e-9) + val authorNames = total.authorNames!!.split(',') + assertEquals(setOf("Dan Brown", "Ken Kousen"), authorNames.toSet()) + assertEquals(2, authorNames.size) + + // In HAVING, as any aggregate function + lateinit var prolificAuthors: SelectStatement + database { + prolificAuthors = BookTable SELECT GROUP_BY(BookTable.author) HAVING (BookTable.count(DISTINCT(BookTable.pages)) GT 1) + } + assertEquals(listOf("Dan Brown"), prolificAuthors.getResults().map { it.author }) + } + + /** + * Covers LIKE with ESCAPE: `%` and `_` after the escape character match only themselves, while without it they are + * wildcards; the escaped LIKE combines with other conditions as any condition does. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testLikeEscape() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + ProductTable INSERT listOf( + Product(sku = "SKU-1", name = "100% cotton", price = 1.0), + Product(sku = "SKU-2", name = "1000 threads", price = 2.0), + Product(sku = "SKU-3", name = "a_b", price = 3.0), + Product(sku = "SKU-4", name = "axb", price = 4.0), + ) + } + lateinit var percent: SelectStatement + lateinit var wildcard: SelectStatement + lateinit var underscore: SelectStatement + lateinit var combined: SelectStatement + database { + percent = ProductTable SELECT WHERE (ProductTable.name LIKE "100\\%%" ESCAPE '\\') + wildcard = ProductTable SELECT WHERE (ProductTable.name LIKE "100%") ORDER_BY (ProductTable.sku to ASC) + underscore = ProductTable SELECT WHERE (ProductTable.name LIKE "a!_b" ESCAPE '!') + combined = ProductTable SELECT WHERE ((ProductTable.price GT 3.5) OR (ProductTable.name LIKE "a!_%" ESCAPE '!')) ORDER_BY (ProductTable.sku to ASC) + } + assertEquals(listOf("SKU-1"), percent.getResults().map { it.sku }) + assertEquals(listOf("SKU-1", "SKU-2"), wildcard.getResults().map { it.sku }) + assertEquals(listOf("SKU-3"), underscore.getResults().map { it.sku }) + assertEquals(listOf("SKU-3", "SKU-4"), combined.getResults().map { it.sku }) + } + + /** + * Covers arithmetic and concatenation: the operators of numbers of the same type, with values too, a division by + * zero being NULL, a negative value written so as not to start a comment, and `+` of strings. Expressions work in + * WHERE and ORDER BY too, but a literal on its own in ORDER BY or GROUP BY is rejected, as SQLite would read it as + * the number of a result column, and an expression that can be NULL in a non-null property is rejected. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testArithmetic() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + BookTable INSERT listOf( + Book(name = "Kotlin", author = "Ken", price = 10.0, pages = 100), + Book(name = "Swift", author = "Sam", price = 20.0, pages = 30), + ) + } + lateinit var math: SelectStatement + lateinit var expensive: SelectStatement + lateinit var byNegatedPrice: SelectStatement + database { + BookTable { table -> + math = table SELECT listOf( + (price * 2.0) AS BookMath::doubledPrice, + (CAST(pages AS DOUBLE) / price) AS BookMath::pagesPerPrice, + (-pages) AS BookMath::negatedPages, + (pages - (-1)) AS BookMath::nextPage, + (name + " by " + author) AS BookMath::label, + (pages % 7) AS BookMath::remainder, + (pages / 0) AS BookMath::perZero, + ) ORDER_BY (name to ASC) + expensive = table SELECT WHERE((price * 2.0) GT 30.0) + byNegatedPrice = table SELECT ORDER_BY((price * -1.0) to ASC) + } + } + assertEquals( + listOf( + BookMath("Kotlin", doubledPrice = 20.0, pagesPerPrice = 10.0, negatedPages = -100, nextPage = 101, label = "Kotlin by Ken", remainder = 2, perZero = null), + BookMath("Swift", doubledPrice = 40.0, pagesPerPrice = 1.5, negatedPages = -30, nextPage = 31, label = "Swift by Sam", remainder = 2, perZero = null), + ), + math.getResults(), + ) + assertEquals(listOf("Swift"), expensive.getResults().map { it.name }) + assertEquals(listOf("Swift", "Kotlin"), byNegatedPrice.getResults().map { it.name }) + + assertFailsWith { + database { BookTable SELECT ORDER_BY(literal(1) to ASC) } + } + assertFailsWith { + database { BookTable SELECT GROUP_BY(literal(1)) } + } + // A division by zero is NULL, so it can't be read into a non-null property + assertFailsWith { + database { BookTable { table -> table SELECT ((pages / 0) AS BookPages::pages) } } + } + } + + /** + * Covers CAST to the types SQLlin names the columns of: a real number is truncated, a string read as far as it is a + * number, and numbers of different types computed with after a CAST. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testCast() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + BookTable INSERT Book(name = "Kotlin", author = "Ken", price = 10.5, pages = 100) + } + lateinit var casts: SelectStatement + database { + BookTable { table -> + casts = table SELECT listOf( + CAST(price AS INT) AS BookCasts::wholePrice, + CAST(pages AS TEXT) AS BookCasts::pagesText, + CAST(name AS BLOB) AS BookCasts::nameBytes, + CAST(literal("12abc") AS INT) AS BookCasts::leadingDigits, + (CAST(pages AS DOUBLE) * price) AS BookCasts::weighted, + ) + } + } + val cast = casts.getResults().single() + assertEquals(10, cast.wholePrice) + assertEquals("100", cast.pagesText) + assertEquals("Kotlin", cast.nameBytes.decodeToString()) + assertEquals(12, cast.leadingDigits) + assertEquals(1050.0, cast.weighted, 1e-9) + } + + /** + * Covers the functions that replace NULL, with a value or another expression, and are NULL only where all their + * arguments can be, NULLIF, which can always be NULL, and the scalar max and min of several expressions. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testNullFunctions() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + UserAccountTable INSERT listOf( + UserAccount(id = null, username = "ann", email = "ann@mail", status = UserStatus.ACTIVE, priority = Priority.LOW, notes = null), + UserAccount(id = null, username = "bob", email = "bob@mail", status = UserStatus.ACTIVE, priority = Priority.HIGH, notes = "vip"), + ) + } + lateinit var results: SelectStatement + database { + UserAccountTable { table -> + results = table SELECT listOf( + coalesce(notes, "none") AS NullFunctionResults::notesOrNone, + coalesce(notes, email) AS NullFunctionResults::notesOrEmail, + ifnull(notes, "none") AS NullFunctionResults::ifNullNotes, + nullif(username, "bob") AS NullFunctionResults::notBob, + max(username, email) AS NullFunctionResults::greater, + min(username, email) AS NullFunctionResults::lesser, + ) ORDER_BY (username to ASC) + } + } + assertEquals( + listOf( + NullFunctionResults("ann", notesOrNone = "none", notesOrEmail = "ann@mail", ifNullNotes = "none", notBob = "ann", greater = "ann@mail", lesser = "ann"), + NullFunctionResults("bob", notesOrNone = "vip", notesOrEmail = "vip", ifNullNotes = "vip", notBob = null, greater = "bob@mail", lesser = "bob"), + ), + results.getResults(), + ) + // NULLIF can be NULL, and so can COALESCE of expressions that all can be + assertFailsWith { + database { UserAccountTable { table -> table SELECT (nullif(username, "bob") AS NullFunctionResults::greater) } } + } + assertFailsWith { + database { UserAccountTable { table -> table SELECT (coalesce(notes, notes) AS NullFunctionResults::greater) } } + } + } + + /** + * Covers more string, BLOB and number functions, whose arguments are expressions as well as values, and `total`, + * which is 0.0 rather than NULL when no rows match. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testMoreFunctions() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + lateinit var noTotals: SelectStatement + database { + BookTable { table -> + noTotals = table SELECT listOf(total(price) AS PriceTotals::total, total(DISTINCT(price)) AS PriceTotals::distinctTotal) + } + } + assertEquals(listOf(PriceTotals(0.0, 0.0)), noTotals.getResults()) + + database { + BookTable INSERT listOf( + Book(name = "Kotlin", author = "0Ken0", price = 10.5, pages = 100), + Book(name = "Swift", author = "Sam", price = 10.5, pages = 30), + Book(name = "Rust", author = "Rob", price = 20.0, pages = 50), + ) + } + lateinit var texts: SelectStatement + lateinit var totals: SelectStatement + database { + BookTable { table -> + texts = table SELECT listOf( + hex(name) AS TextFunctions::hexName, + quote(name) AS TextFunctions::quoted, + unicode(name) AS TextFunctions::firstCode, + trim(author, "0") AS TextFunctions::trimmed, + substr(name, 3) AS TextFunctions::tail, + substr(name, literal(2), literal(3)) AS TextFunctions::middle, + replace(name, literal("Kot"), author) AS TextFunctions::replaced, + instr(name, literal("lin")) AS TextFunctions::position, + printf("%s has %d pages", name, pages) AS TextFunctions::formatted, + round(price) AS TextFunctions::rounded, + length(zeroblob(3)) AS TextFunctions::zeros, + length(randomblob(4)) AS TextFunctions::randomBytes, + ) WHERE (name EQ "Kotlin") + totals = table SELECT listOf(total(price) AS PriceTotals::total, total(DISTINCT(price)) AS PriceTotals::distinctTotal) + } + } + assertEquals( + TextFunctions( + hexName = "4B6F746C696E", + quoted = "'Kotlin'", + firstCode = 75, + trimmed = "Ken", + tail = "tlin", + middle = "otl", + replaced = "0Ken0lin", + position = 4, + formatted = "Kotlin has 100 pages", + rounded = 11.0, + zeros = 3, + randomBytes = 4, + ), + texts.getResults().single(), + ) + assertEquals(listOf(PriceTotals(total = 41.0, distinctTotal = 30.5)), totals.getResults()) + } + + /** + * Covers the date and time functions, of strings, string and number expressions, with modifiers, and that a function + * of the current time, or of the local time, which isn't deterministic, can't be in an index or a partial index's + * WHERE, while one of a column can. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testDateTimeFunctions() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + BookTable INSERT Book(name = "2026-01-31", author = "Ken", price = 10.0, pages = 100) + } + lateinit var times: SelectStatement + database { + BookTable { table -> + times = table SELECT listOf( + date("2026-10-07 12:30:00") AS DateTimes::day, + date(name, "+1 day") AS DateTimes::nextDay, + time("2026-10-07 12:30:00", "+90 minutes") AS DateTimes::later, + datetime(literal(0), "unixepoch") AS DateTimes::epochStart, + strftime("%Y/%m", "2026-10-07") AS DateTimes::yearMonth, + julianday("2000-01-01 12:00:00") AS DateTimes::julian, + datetime("now") AS DateTimes::now, + ) + } + } + val result = times.getResults().single() + assertEquals( + DateTimes(day = "2026-10-07", nextDay = "2026-02-01", later = "14:00:00", epochStart = "1970-01-01 00:00:00", yearMonth = "2026/10", julian = 2451545.0, now = result.now), + result, + ) + assertEquals(19, result.now.length) + // With a modifier, which may not be valid, it can be NULL + assertFailsWith { + database { BookTable { table -> table SELECT (datetime("now", "localtime") AS DateTimes::now) } } + } + + database { BookTable.CREATE_INDEX("idx_book_day", BookTable.date(BookTable.name)) } + assertFailsWith { + database { BookTable.CREATE_INDEX("idx_book_random", BookTable.random()) } + } + assertFailsWith { + database { BookTable.CREATE_INDEX("idx_book_local_day", BookTable.date(BookTable.name, "localtime")) } + } + assertFailsWith { + database { BookTable.CREATE_INDEX("idx_book_recent", BookTable.name) WHERE (BookTable.name GT BookTable.datetime("now")) } + } + } + + /** + * Covers the functions that need a newer SQLite, or its math functions, each checked only where the SQLite has it: + * Android has no math functions at any API level. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class, PlatformDependentSQLiteAPI::class) + fun testPlatformDependentFunctions() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + BookTable INSERT Book(name = "Kotlin", author = "Ken", price = 16.0, pages = 10) + } + if (sqliteHas("SELECT sqrt(4)")) { + lateinit var math: SelectStatement + database { + BookTable { table -> + math = table SELECT listOf( + sqrt(price) AS MathResults::root, + ceil(literal(1.2)) AS MathResults::ceiling, + floor(pages) AS MathResults::flooredPages, + trunc(literal(-1.5)) AS MathResults::truncated, + pow(pages, literal(2)) AS MathResults::power, + pow(pages, 3.0) AS MathResults::powerOf, + mod(pages, literal(3)) AS MathResults::remainder, + mod(price, 5.0) AS MathResults::remainderOf, + pi() AS MathResults::piValue, + exp(literal(0)) AS MathResults::exponential, + ln(literal(1)) AS MathResults::natural, + log10(literal(100)) AS MathResults::decimal, + log2(literal(8)) AS MathResults::binary, + ) + } + } + val result = math.getResults().single() + assertEquals( + MathResults(root = 4.0, ceiling = 2.0, flooredPages = 10, truncated = -1.0, power = 100.0, powerOf = 1000.0, remainder = 1.0, remainderOf = 1.0, piValue = result.piValue, exponential = 1.0, natural = 0.0, decimal = 2.0, binary = 3.0), + result, + ) + assertEquals(kotlin.math.PI, result.piValue, 1e-12) + } + if (sqliteHas("SELECT sign(1)")) { + lateinit var signs: SelectStatement + database { BookTable { table -> signs = table SELECT (sign(literal(-3)) AS SignResult::signOf) } } + assertEquals(listOf(SignResult(-1)), signs.getResults()) + } + if (sqliteHas("SELECT unixepoch('2020-01-01')")) { + lateinit var epochs: SelectStatement + database { BookTable { table -> epochs = table SELECT (unixepoch("2020-01-01") AS EpochResult::epoch) } } + assertEquals(listOf(EpochResult(1577836800)), epochs.getResults()) + } + } + + /** + * The string in the first column of the first row that [sql] returns from the database, which has to exist, or null. + */ + private fun sqliteString(sql: String): String? { + val connection = openDatabase(DatabaseConfiguration(name = DATABASE_NAME, path = path, version = 1, create = {})) + return try { + connection.withQuery(sql) { cursor -> if (cursor.next()) cursor.getString(0) else null } + } finally { + connection.close() + } + } + + /** + * Whether the SQLite of the database, which has to exist, can run [sql], as it can't use a function it doesn't have. + */ + private fun sqliteHas(sql: String): Boolean { + val connection = openDatabase(DatabaseConfiguration(name = DATABASE_NAME, path = path, version = 1, create = {})) + return try { + connection.withQuery(sql) { cursor -> cursor.forEachRow { } } + true + } catch (e: Exception) { + false + } finally { + connection.close() + } + } + + /** + * Covers CASE: the value of the first branch whose condition holds, ELSE where none does, and NULL without ELSE, so + * that it can only be read into a nullable property. CASE works in conditions and ORDER BY too, and the values of + * its conditions, a quote among them, are written into the SQL. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testCaseExpression() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + BookTable INSERT listOf( + Book(name = "Kotlin", author = "Ken", price = 20.0, pages = 300), + Book(name = "Swift", author = "Sam", price = 10.0, pages = 30), + Book(name = "It's Go", author = "Gus", price = 12.0, pages = 120), + ) + } + lateinit var categories: SelectStatement + lateinit var shortBooks: SelectStatement + lateinit var quotedFirst: SelectStatement + database { + BookTable { table -> + categories = table SELECT listOf( + CASE( + WHEN(pages LT 50) THEN literal("short"), + WHEN(pages LT 200) THEN literal("medium"), + ELSE = literal("long"), + ) AS BookCategory::size, + CASE(WHEN(price GT 15.0) THEN (price * 0.1)) AS BookCategory::discount, + ) ORDER_BY (name to ASC) + shortBooks = table SELECT WHERE(CASE(WHEN(pages LT 50) THEN literal("short"), ELSE = literal("long")) EQ "short") + quotedFirst = table SELECT ORDER_BY(CASE(WHEN(name EQ "It's Go") THEN literal(0), ELSE = literal(1)) to ASC) + } + } + assertEquals( + listOf( + BookCategory("It's Go", size = "medium", discount = null), + BookCategory("Kotlin", size = "long", discount = 2.0), + BookCategory("Swift", size = "short", discount = null), + ), + categories.getResults(), + ) + assertEquals(listOf("Swift"), shortBooks.getResults().map { it.name }) + assertEquals("It's Go", quotedFirst.getResults().first().name) + + // Without ELSE, CASE can be NULL + assertFailsWith { + database { BookTable { table -> table SELECT (CASE(WHEN(pages LT 50) THEN pages) AS BookPages::pages) } } + } + } + + /** + * Covers scalar subqueries: a SELECT of a single column used as a value, which is NULL when the SELECT returns no + * rows, in conditions and result columns. An observed query watches the tables of its scalar subqueries. A SELECT of + * more than one column is rejected. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testScalarSubquery() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + runTest { + val olderThanPages = Channel>(Channel.UNLIMITED) + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { + PersonWithIdTable SELECT WHERE(PersonWithIdTable.age GT (BookTable SELECT listOf(BookTable.max(BookTable.pages) AS MaxPages::value))[MaxPages::value]) + }.collect { olderThanPages.send(it) } + } + database { + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "Ann", age = 30), + PersonWithId(id = null, name = "Bob", age = 10), + ) + } + // No books: the subquery is NULL, so no one is older + assertEquals(emptyList(), olderThanPages.receive()) + runCurrent() + while (olderThanPages.tryReceive().isSuccess) Unit + + database { + BookTable INSERT listOf( + Book(name = "Kotlin", author = "Ken", price = 20.0, pages = 20), + Book(name = "Swift", author = "Sam", price = 10.0, pages = 5), + ) + } + // A change to the subquery's table is seen + assertEquals(listOf("Ann"), olderThanPages.receive().map { it.name }) + + lateinit var expensive: SelectStatement + lateinit var withMax: SelectStatement + database { + expensive = BookTable SELECT WHERE(BookTable.price GT (BookTable SELECT listOf(BookTable.avg(BookTable.price) AS AveragePrice::value))[AveragePrice::value]) + BookTable { table -> + withMax = table SELECT ((BookTable SELECT listOf(max(pages) AS MaxPages::value))[MaxPages::value] AS BookWithMaxPages::maxPages) ORDER_BY (name to ASC) + } + } + assertEquals(listOf("Kotlin"), expensive.getResults().map { it.name }) + assertEquals(listOf(BookWithMaxPages("Kotlin", 20), BookWithMaxPages("Swift", 20)), withMax.getResults()) + + assertFailsWith { + database { BookTable SELECT WHERE(BookTable.name EQ (BookTable SELECT X)[Book::name]) } + } + } + } + + /** + * Covers the null-safe comparisons of expressions, IS and IS NOT, which are true or false where EQ and NEQ would be + * NULL; the COLLATE operator, in a condition and in ORDER BY; and the bitwise operators. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testExpressionOperators() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + UserAccountTable INSERT listOf( + UserAccount(id = null, username = "ann", email = "ann@mail", status = UserStatus.ACTIVE, priority = Priority.LOW, notes = null), + UserAccount(id = null, username = "Bob", email = "bob@mail", status = UserStatus.ACTIVE, priority = Priority.HIGH, notes = "vip"), + UserAccount(id = null, username = "cat", email = "cat@mail", status = UserStatus.ACTIVE, priority = Priority.HIGH, notes = "new"), + ) + PersonWithIdTable INSERT PersonWithId(id = null, name = "Ann", age = 13) + } + lateinit var vip: SelectStatement + lateinit var notVip: SelectStatement + lateinit var notVipByNeq: SelectStatement + lateinit var bob: SelectStatement + lateinit var byName: SelectStatement + lateinit var bits: SelectStatement + database { + UserAccountTable { table -> + vip = table SELECT WHERE(notes IS literal("vip")) + notVip = table SELECT WHERE(notes ISNOT literal("vip")) ORDER_BY (username to ASC) + notVipByNeq = table SELECT WHERE(notes NEQ literal("vip")) + bob = table SELECT WHERE((username COLLATE NOCASE) EQ "BOB") + byName = table SELECT ORDER_BY((username COLLATE NOCASE) to ASC) + } + PersonWithIdTable { table -> + bits = table SELECT listOf( + (age and 4) AS AgeBits::anded, + (age or 2) AS AgeBits::ored, + (age shl 2) AS AgeBits::shifted, + (age shr 1) AS AgeBits::halved, + age.inv() AS AgeBits::inverted, + ) + } + } + assertEquals(listOf("Bob"), vip.getResults().map { it.username }) + // IS NOT is true for NULL, while != is NULL, so it leaves 'ann' out + assertEquals(listOf("ann", "cat"), notVip.getResults().map { it.username }) + assertEquals(listOf("cat"), notVipByNeq.getResults().map { it.username }) + assertEquals(listOf("Bob"), bob.getResults().map { it.username }) + assertEquals(listOf("ann", "Bob", "cat"), byName.getResults().map { it.username }) + assertEquals(listOf(AgeBits(anded = 4, ored = 15, shifted = 52, halved = 6, inverted = -14)), bits.getResults()) + } + + /** + * Covers SET with expressions: columns set from the row's own columns, functions, literals and a scalar subquery + * made non-null with coalesce. An expression that can be NULL can't set a non-null column, nor can an aggregate + * function or a column of another table. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testSetExpressions() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "Ann", age = 30), + PersonWithId(id = null, name = "Bob", age = 10), + ) + BookTable INSERT Book(name = "Kotlin", author = "Ken", price = 20.0, pages = 99) + } + database { + PersonWithIdTable { table -> + table UPDATE SET(listOf((age + 1) AS PersonWithId::age, (name + "!") AS PersonWithId::name)) WHERE (name EQ "Ann") + table UPDATE SET(coalesce((BookTable SELECT listOf(BookTable.max(BookTable.pages) AS MaxPages::value))[MaxPages::value], 0) AS PersonWithId::age) WHERE (name EQ "Bob") + } + } + lateinit var people: SelectStatement + database { people = PersonWithIdTable SELECT ORDER_BY(PersonWithIdTable.id to ASC) } + assertEquals(listOf("Ann!" to 31, "Bob" to 99), people.getResults().map { it.name to it.age }) + + database { PersonWithIdTable { table -> table UPDATE SET(literal(5) AS PersonWithId::age) } } + database { people = PersonWithIdTable SELECT X } + assertEquals(listOf(5, 5), people.getResults().map { it.age }) + + // A scalar subquery can be NULL, and 'age' isn't nullable + assertFailsWith { + database { + PersonWithIdTable UPDATE SET((BookTable SELECT listOf(BookTable.max(BookTable.pages) AS MaxPages::value))[MaxPages::value] AS PersonWithId::age) + } + } + assertFailsWith { + database { PersonWithIdTable UPDATE SET(PersonWithIdTable.max(PersonWithIdTable.age) AS PersonWithId::age) } + } + assertFailsWith { + database { PersonWithIdTable UPDATE SET(BookTable.pages AS PersonWithId::age) } + } + } + + /** + * Covers INSERT of expressions: a row of literals and expressions of them, its other columns taking their default + * values, and INSERT OR IGNORE, which skips a row that conflicts. The values can't read a table, and an expression + * that can be NULL can't be written to a non-null column. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testInsertExpressions() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + PersonWithIdTable INSERT listOf(literal("Zed") AS PersonWithId::name, (literal(20) * 2) AS PersonWithId::age) + UserAccountTable INSERT listOf( + literal("ann") AS UserAccount::username, + (literal("ann") + "@mail") AS UserAccount::email, + literal(UserStatus.SUSPENDED) AS UserAccount::status, + literal(Priority.HIGH) AS UserAccount::priority, + ) + } + lateinit var people: SelectStatement + lateinit var accounts: SelectStatement + database { + people = PersonWithIdTable SELECT X + accounts = UserAccountTable SELECT X + } + assertEquals(listOf(PersonWithId(id = 1, name = "Zed", age = 40)), people.getResults()) + assertEquals( + listOf(UserAccount(id = 1, username = "ann", email = "ann@mail", status = UserStatus.SUSPENDED, priority = Priority.HIGH, notes = null)), + accounts.getResults(), + ) + + // The key exists, so the row is skipped + database { + PersonWithIdTable INSERT_OR_IGNORE listOf(literal(1L) AS PersonWithId::id, literal("Dup") AS PersonWithId::name, literal(1) AS PersonWithId::age) + } + database { people = PersonWithIdTable SELECT X } + assertEquals(listOf("Zed"), people.getResults().map { it.name }) + + // 'age' is NOT NULL and has no default, which SQLite reports when the statement runs + assertFails { + database { PersonWithIdTable INSERT (literal("Nobody") AS PersonWithId::name) } + } + assertFailsWith { + database { PersonWithIdTable INSERT listOf(PersonWithIdTable.name AS PersonWithId::name, literal(1) AS PersonWithId::age) } + } + assertFailsWith { + database { PersonWithIdTable INSERT listOf(PersonWithIdTable.nullif(literal("a"), "a") AS PersonWithId::name, literal(1) AS PersonWithId::age) } + } + } + + /** + * Covers triggers: on INSERT, keeping a count with an UPDATE of an expression and logging the new row with an + * INSERT of expressions; on DELETE, logging the old row; BEFORE INSERT with WHEN and RAISE(ABORT), which stops the + * statement with its message, and RAISE(IGNORE), which skips the row; and on UPDATE OF a column, with WHEN comparing + * the new and old rows. IF NOT EXISTS skips a trigger that exists, and DROP_TRIGGER drops one. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testTrigger() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + CREATE(TriggerAuditTable) + CREATE(TriggerCounterTable) + TriggerCounterTable INSERT TriggerCounter(id = 1, count = 0) + CREATE_TRIGGER("person_inserted") AFTER INSERT ON PersonWithIdTable BEGIN { + TriggerCounterTable { table -> table UPDATE SET((count + 1) AS TriggerCounter::count) WHERE (id EQ 1) } + TriggerAuditTable { table -> + table INSERT listOf( + new(PersonWithIdTable.id) AS TriggerAudit::personId, + literal("insert") AS TriggerAudit::action, + datetime("now") AS TriggerAudit::at, + ) + } + } + CREATE_TRIGGER("person_deleted") AFTER DELETE ON PersonWithIdTable BEGIN { + TriggerAuditTable INSERT listOf(old(PersonWithIdTable.id) AS TriggerAudit::personId, literal("delete") AS TriggerAudit::action) + } + CREATE_TRIGGER("no_negative_age") BEFORE INSERT ON PersonWithIdTable WHEN (new(PersonWithIdTable.age) LT 0) BEGIN { + RAISE(ABORT, "age can't be negative") + } + CREATE_TRIGGER("no_babies") BEFORE INSERT ON PersonWithIdTable WHEN ((new(PersonWithIdTable.age) GTE 0) AND (new(PersonWithIdTable.age) LT 1)) BEGIN { + RAISE(IGNORE) + } + CREATE_TRIGGER("age_changed") AFTER UPDATE_OF(PersonWithIdTable.age) ON PersonWithIdTable WHEN (new(PersonWithIdTable.age) ISNOT old(PersonWithIdTable.age)) BEGIN { + TriggerAuditTable INSERT listOf(new(PersonWithIdTable.id) AS TriggerAudit::personId, literal("age") AS TriggerAudit::action) + } + } + database { + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "Ann", age = 30), + PersonWithId(id = null, name = "Bob", age = 10), + ) + } + val error = assertFails { database { PersonWithIdTable INSERT PersonWithId(id = null, name = "Neg", age = -1) } } + assertEquals(true, error.message?.contains("age can't be negative"), error.message) + // Skipped, without an error + database { PersonWithIdTable INSERT PersonWithId(id = null, name = "Baby", age = 0) } + database { PersonWithIdTable { table -> table UPDATE SET { name = "Bobby" } WHERE (name EQ "Bob") } } + database { PersonWithIdTable { table -> table UPDATE SET { age = 11 } WHERE (name EQ "Bobby") } } + database { PersonWithIdTable { table -> table UPDATE SET { age = 11 } WHERE (name EQ "Bobby") } } + database { PersonWithIdTable DELETE WHERE (PersonWithIdTable.name EQ "Ann") } + + lateinit var audits: SelectStatement + lateinit var counters: SelectStatement + lateinit var people: SelectStatement + database { + audits = TriggerAuditTable SELECT ORDER_BY(TriggerAuditTable.id to ASC) + counters = TriggerCounterTable SELECT X + people = PersonWithIdTable SELECT X + } + assertEquals(listOf(1L to "insert", 2L to "insert", 2L to "age", 1L to "delete"), audits.getResults().map { it.personId to it.action }) + assertEquals(true, audits.getResults().first().at != null) + assertEquals(listOf(TriggerCounter(id = 1, count = 2)), counters.getResults()) + assertEquals(listOf("Bobby"), people.getResults().map { it.name }) + + // It exists, so creating it again fails, unless IF NOT EXISTS + assertFails { + database { CREATE_TRIGGER("person_deleted") AFTER DELETE ON PersonWithIdTable BEGIN { RAISE(IGNORE) } } + } + database { + CREATE_TRIGGER_IF_NOT_EXISTS("person_deleted") AFTER DELETE ON PersonWithIdTable BEGIN { RAISE(IGNORE) } + DROP_TRIGGER("person_inserted") + DROP_TRIGGER_IF_EXISTS("person_inserted") + } + database { PersonWithIdTable INSERT PersonWithId(id = null, name = "Cat", age = 5) } + database { counters = TriggerCounterTable SELECT X } + assertEquals(2, counters.getResults().single().count) + assertFails { database { DROP_TRIGGER("person_inserted") } } + } - // A conflict on a UNIQUE column other than the key is ignored as well - database { - UniqueEmailTestTable INSERT UniqueEmailTest(id = null, email = "ivy@example.com", name = "Ivy") - UniqueEmailTestTable INSERT_OR_IGNORE UniqueEmailTest(id = null, email = "ivy@example.com", name = "Ivy Again") - } - lateinit var accounts: SelectStatement - database { - accounts = UniqueEmailTestTable SELECT X + /** + * Covers a trigger whose body inserts the result of a SELECT that reads the trigger's new row in a result column. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testTriggerSelectingNewRow() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + CREATE(TriggerCounterTable) + TriggerCounterTable INSERT TriggerCounter(id = 1, count = 0) + CREATE_TRIGGER("copy_age") AFTER INSERT ON PersonWithIdTable BEGIN { + TriggerCounterTable INSERT_OR_REPLACE (TriggerCounterTable SELECT (new(PersonWithIdTable.age) AS TriggerCounter::count) WHERE (TriggerCounterTable.id EQ 1)) } - assertEquals(1, accounts.getResults().size) - assertEquals("Ivy", accounts.getResults().first().name) + } + database { PersonWithIdTable INSERT PersonWithId(id = null, name = "Ann", age = 42) } + lateinit var counters: SelectStatement + database { counters = TriggerCounterTable SELECT X } + assertEquals(listOf(TriggerCounter(id = 1, count = 42)), counters.getResults()) + } - // With a composite key, inserting a pair again keeps its existing row, other columns included, - // which is what tells INSERT_OR_IGNORE apart from INSERT_OR_REPLACE + /** + * Covers what a trigger rejects when it is built: `new()` in a trigger on DELETE and `old()` in one on INSERT, which + * SQLite would only report when the trigger fires; `new()` outside a trigger, or of a column of another table; a + * trigger on an FTS table; a body without statements, or with a SELECT or a transaction; `RAISE` outside a trigger; + * UPDATE OF a column of another table; and a trigger without BEGIN, which would never be created. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testTriggerChecks() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + assertFailsWith { + database { CREATE_TRIGGER("t") AFTER DELETE ON PersonWithIdTable WHEN (new(PersonWithIdTable.age) GT 1) BEGIN { RAISE(IGNORE) } } + } + assertFailsWith { database { - EnrollmentTable INSERT Enrollment(studentId = 1, courseId = 101, semester = "Spring") + CREATE_TRIGGER("t") AFTER INSERT ON PersonWithIdTable BEGIN { + PersonWithIdTable UPDATE SET(old(PersonWithIdTable.age) AS PersonWithId::age) + } } + } + assertFailsWith { + database { PersonWithIdTable SELECT WHERE(new(PersonWithIdTable.age) GT 1) } + } + assertFailsWith { + database { CREATE_TRIGGER("t") AFTER INSERT ON PersonWithIdTable WHEN (new(BookTable.pages) GT 1) BEGIN { RAISE(IGNORE) } } + } + assertFailsWith { + database { CREATE_TRIGGER("t") AFTER INSERT ON ArticleTable BEGIN { RAISE(IGNORE) } } + } + assertFailsWith { + database { CREATE_TRIGGER("t") AFTER INSERT ON PersonWithIdTable BEGIN { } } + } + assertFailsWith { + database { CREATE_TRIGGER("t") AFTER INSERT ON PersonWithIdTable BEGIN { BookTable SELECT X } } + } + assertFailsWith { + database { CREATE_TRIGGER("t") AFTER INSERT ON PersonWithIdTable BEGIN { transaction { RAISE(IGNORE) } } } + } + assertFailsWith { + database { RAISE(ABORT, "outside") } + } + assertFailsWith { + database { CREATE_TRIGGER("t") AFTER UPDATE_OF(BookTable.pages) ON PersonWithIdTable BEGIN { RAISE(IGNORE) } } + } + assertFailsWith { + database { CREATE_TRIGGER("t") AFTER INSERT ON PersonWithIdTable } + } + } + + /** + * Covers a trigger that keeps an FTS table in step with a table, as an observed query of the FTS table sees: it is a + * virtual table, which can't have triggers, so the tracker counts its changes through the triggers of the table. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testTriggerWritingFts() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + runTest { database { - EnrollmentTable INSERT_OR_IGNORE listOf( - Enrollment(studentId = 1, courseId = 101, semester = "Fall"), - Enrollment(studentId = 1, courseId = 102, semester = "Fall"), - ) + CREATE(ArticleTable) + CREATE_TRIGGER("person_article") AFTER INSERT ON PersonWithIdTable BEGIN { + ArticleTable INSERT listOf(literal("person") AS Article::title, new(PersonWithIdTable.name) AS Article::body) + } } - lateinit var enrollments: SelectStatement - database { - enrollments = EnrollmentTable SELECT X + val articles = Channel>(Channel.UNLIMITED) + backgroundScope.launch { + database.observe(EmptyCoroutineContext) { ArticleTable SELECT X }.collect { articles.send(it) } } - assertEquals(2, enrollments.getResults().size) - assertEquals("Spring", enrollments.getResults().first { it.courseId == 101L }.semester) - assertEquals("Fall", enrollments.getResults().first { it.courseId == 102L }.semester) + assertEquals(emptyList(), articles.receive()) + database { PersonWithIdTable INSERT PersonWithId(id = null, name = "Ann", age = 30) } + assertEquals(listOf("Ann"), articles.receive().map { it.body }) } } /** - * Covers projection: a SELECT that reads rows into a narrower @Serializable type than the table's own row type, - * given to the clause function, as in `X()` or `WHERE(...)`. Only the columns that type's - * properties name are selected, and a type that doesn't fit the table is rejected while the statement is built. + * Covers INSERT DEFAULT VALUES: a row of the default values of all columns, NULL where a column has none, and the + * next key where the database assigns it. A NOT NULL column without a default rejects it. */ - fun testProjection() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testInsertDefaultValues() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + CREATE(DefaultRowTable) + DefaultRowTable INSERT DEFAULT_VALUES + DefaultRowTable INSERT_OR_IGNORE DEFAULT_VALUES + } + lateinit var rows: SelectStatement + database { rows = DefaultRowTable SELECT X } + assertEquals(listOf(DefaultRow(1, "new", 0, null), DefaultRow(2, "new", 0, null)), rows.getResults()) + assertFails { database { PersonWithIdTable INSERT DEFAULT_VALUES } } + } + + /** + * Covers ANALYZE, which creates the statistics table `sqlite_stat1`, REINDEX of all indexes, of a table, of an + * index and of a collation, and PRAGMA optimize. A name of no table or index fails. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class, PlatformDependentSQLiteAPI::class) + fun testAnalyzeAndReindex() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> database { + BookTable.CREATE_INDEX("idx_book_author", BookTable.author) BookTable INSERT listOf( - Book(name = "The Da Vinci Code", author = "Dan Brown", price = 16.96, pages = 454), - Book(name = "The Lost Symbol", author = "Dan Brown", price = 19.95, pages = 510), - Book(name = "Kotlin Cookbook", author = "Ken Kousen", price = 37.72, pages = 251), + Book(name = "Kotlin", author = "Ken", price = 10.0, pages = 100), + Book(name = "Swift", author = "Sam", price = 20.0, pages = 30), ) } + assertEquals(false, sqliteHas("SELECT * FROM sqlite_stat1")) + database { ANALYZE(BookTable) } + assertEquals(true, sqliteHas("SELECT * FROM sqlite_stat1")) + database { + ANALYZE() + ANALYZE("idx_book_author") + REINDEX() + REINDEX(BookTable) + REINDEX("idx_book_author") + REINDEX(NOCASE) + PRAGMA_OPTIMIZE() + } + lateinit var books: SelectStatement + database { books = BookTable SELECT WHERE(BookTable.author EQ "Sam") } + assertEquals(listOf("Swift"), books.getResults().map { it.name }) + assertFails { database { REINDEX("no_such_index") } } + } - // No clause: SELECT name,author FROM book, and SELECT DISTINCT author FROM book. Three books but two authors, - // which only holds if DISTINCT compares the projected column alone, so nothing else is selected. - lateinit var titles: SelectStatement - lateinit var authors: SelectStatement + /** + * Covers the orders of the columns of an index, ASC and DESC, in a unique index too, with WHERE and IF NOT EXISTS. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testIndexOrder() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> database { - titles = BookTable SELECT X() - authors = BookTable SELECT_DISTINCT X() + PersonWithIdTable.CREATE_INDEX("idx_person_age_name", PersonWithIdTable.age to DESC, PersonWithIdTable.name to ASC) + PersonWithIdTable.CREATE_UNIQUE_INDEX_IF_NOT_EXISTS("idx_person_name", PersonWithIdTable.name to DESC) WHERE (PersonWithIdTable.age GT 0) + PersonWithIdTable.CREATE_UNIQUE_INDEX_IF_NOT_EXISTS("idx_person_name", PersonWithIdTable.name to DESC) + PersonWithIdTable INSERT listOf( + PersonWithId(id = null, name = "Ann", age = 30), + PersonWithId(id = null, name = "Bob", age = 40), + ) } - assertEquals(3, titles.getResults().size) - assertEquals(true, BookTitle("Kotlin Cookbook", "Ken Kousen") in titles.getResults()) - assertEquals(listOf("Dan Brown", "Ken Kousen"), authors.getResults().map { it.author }.sorted()) + assertEquals(true, sqliteString("SELECT sql FROM sqlite_master WHERE name = 'idx_person_age_name'")?.endsWith("person_with_id(age DESC,name ASC)")) + assertEquals(true, sqliteString("SELECT sql FROM sqlite_master WHERE name = 'idx_person_name'")?.endsWith("person_with_id(name DESC) WHERE age>0")) + lateinit var people: SelectStatement + database { people = PersonWithIdTable SELECT ORDER_BY(PersonWithIdTable.age to DESC) } + assertEquals(listOf("Bob", "Ann"), people.getResults().map { it.name }) + assertFails { database { PersonWithIdTable INSERT PersonWithId(id = null, name = "Ann", age = 1) } } + } - // Each clause can start a projection, and the projection carries through the rest of the chain - lateinit var longestByBrown: SelectStatement - lateinit var byPages: SelectStatement - lateinit var firstTwo: SelectStatement - lateinit var grouped: SelectStatement + /** + * Covers NULLS FIRST and NULLS LAST in ORDER BY, after a Pair, after ASC or DESC in parentheses, and in a map of an + * ORDER_BY after WHERE with the plain ASC and DESC, where the SQLite has them, from 3.30.0. Without them, NULL values sort first in ascending + * order. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class, PlatformDependentSQLiteAPI::class) + fun testNullsOrdering() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> database { - BookTable { table -> - longestByBrown = table SELECT WHERE(author EQ "Dan Brown") ORDER_BY (pages to DESC) LIMIT 1 - byPages = table SELECT ORDER_BY(pages to ASC) - firstTwo = table SELECT LIMIT(2) - grouped = table SELECT GROUP_BY(author) - } + UserAccountTable INSERT listOf( + UserAccount(id = null, username = "ann", email = "ann@mail", status = UserStatus.ACTIVE, priority = Priority.LOW, notes = null), + UserAccount(id = null, username = "bob", email = "bob@mail", status = UserStatus.ACTIVE, priority = Priority.HIGH, notes = "b"), + UserAccount(id = null, username = "cat", email = "cat@mail", status = UserStatus.INACTIVE, priority = Priority.HIGH, notes = "a"), + ) } - assertEquals(listOf(BookTitle("The Lost Symbol", "Dan Brown")), longestByBrown.getResults()) - assertEquals(listOf("Kotlin Cookbook", "The Da Vinci Code", "The Lost Symbol"), byPages.getResults().map { it.name }) - assertEquals(2, firstTwo.getResults().size) - assertEquals(listOf("Dan Brown", "Ken Kousen"), grouped.getResults().map { it.author }.sorted()) - - // The DISTINCT variant of each clause - lateinit var distinctWhere: SelectStatement - lateinit var distinctOrderBy: SelectStatement - lateinit var distinctLimit: SelectStatement - lateinit var distinctGroupBy: SelectStatement + lateinit var ascending: SelectStatement + database { ascending = UserAccountTable SELECT ORDER_BY(UserAccountTable.notes to ASC) } + assertEquals(listOf("ann", "cat", "bob"), ascending.getResults().map { it.username }) + if (!sqliteHas("SELECT 1 ORDER BY 1 NULLS LAST")) + return@databaseAutoClose + lateinit var nullsLast: SelectStatement + lateinit var nullsFirst: SelectStatement + lateinit var mixed: SelectStatement database { - BookTable { table -> - distinctWhere = table SELECT_DISTINCT WHERE(price GT 10.0) - distinctOrderBy = table SELECT_DISTINCT ORDER_BY(author to DESC) - distinctLimit = table SELECT_DISTINCT LIMIT(1) - distinctGroupBy = table SELECT_DISTINCT GROUP_BY(author) + UserAccountTable { table -> + nullsLast = table SELECT ORDER_BY(notes to ASC NULLS LAST) + nullsFirst = table SELECT ORDER_BY(notes to (DESC NULLS FIRST)) + mixed = table SELECT WHERE(username NEQ "") ORDER_BY mapOf(status to DESC, notes to (ASC NULLS LAST)) } } - assertEquals(2, distinctWhere.getResults().size) - assertEquals(listOf("Ken Kousen", "Dan Brown"), distinctOrderBy.getResults().map { it.author }) - assertEquals(1, distinctLimit.getResults().size) - assertEquals(2, distinctGroupBy.getResults().size) + assertEquals(listOf("cat", "bob", "ann"), nullsLast.getResults().map { it.username }) + assertEquals(listOf("ann", "bob", "cat"), nullsFirst.getResults().map { it.username }) + assertEquals(listOf("cat", "bob", "ann"), mixed.getResults().map { it.username }) + } - // A nullable column is read into a nullable property + /** + * Covers deferred foreign keys, of a column and of a group: children inserted before their parent in a transaction, + * which is checked when it commits, and a transaction that commits without the parent, which fails and is rolled + * back, so that the next transaction can begin. + */ + fun testDeferredForeignKey() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + assertEquals(true, DeferredChildTable.createSQL.contains("REFERENCES deferred_parent(id) ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED"), DeferredChildTable.createSQL) + assertEquals(true, DeferredGroupChildTable.createSQL.contains("FOREIGN KEY (parentId) REFERENCES deferred_parent(id) DEFERRABLE INITIALLY DEFERRED"), DeferredGroupChildTable.createSQL) database { - UserAccountTable INSERT UserAccount( - id = null, - username = "ivy", - email = "ivy@example.com", - status = UserStatus.ACTIVE, - priority = Priority.LOW, - notes = null, - ) + CREATE(DeferredParentTable) + CREATE(DeferredChildTable) + CREATE(DeferredGroupChildTable) } - lateinit var notes: SelectStatement + database { PRAGMA_FOREIGN_KEYS(true) } database { - notes = UserAccountTable SELECT X() + transaction { + DeferredChildTable INSERT DeferredChild(id = 1, parentId = 10) + DeferredGroupChildTable INSERT DeferredGroupChild(id = 1, parentId = 10) + DeferredParentTable INSERT DeferredParent(id = 10) + } } - assertEquals(listOf(UserNotes("ivy", null)), notes.getResults()) - - // A type that doesn't fit the table is rejected while the statement is built, before anything runs - val notAColumn = assertFailsWith { - database { BookTable SELECT X() } + assertFails { + database { transaction { DeferredChildTable INSERT DeferredChild(id = 2, parentId = 99) } } } - assertEquals(true, notAColumn.message!!.contains("'isbn' isn't a column")) - val wrongType = assertFailsWith { - database { BookTable SELECT WHERE(BookTable.pages GT 0) } + // The failed transaction is rolled back, so the next one can begin + database { transaction { DeferredParentTable INSERT DeferredParent(id = 20) } } + lateinit var children: SelectStatement + lateinit var groupChildren: SelectStatement + database { + children = DeferredChildTable SELECT X + groupChildren = DeferredGroupChildTable SELECT X } - assertEquals(true, wrongType.message!!.contains("'pages' is a kotlin.String, but the column holds a kotlin.Int")) - val notNullable = assertFailsWith { - database { UserAccountTable SELECT X() } + assertEquals(listOf(DeferredChild(1, 10)), children.getResults()) + assertEquals(listOf(DeferredGroupChild(1, 10)), groupChildren.getResults()) + } + + /** + * Covers the conflict clauses of constraints: REPLACE on a key and on a composite unique group, which delete the + * rows a row conflicts with, and IGNORE on a unique column and on a composite key, which skip the row. ABORT, the + * default, writes no clause. + */ + fun testConflictClauses() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + assertEquals( + "CREATE TABLE conflict_test(id INTEGER PRIMARY KEY ON CONFLICT REPLACE,email TEXT NOT NULL UNIQUE ON CONFLICT IGNORE,x INT NOT NULL,y INT NOT NULL,UNIQUE(x,y) ON CONFLICT REPLACE)", + ConflictTestTable.createSQL, + ) + assertEquals("CREATE TABLE conflict_pair(a TEXT NOT NULL,b TEXT NOT NULL,note TEXT,PRIMARY KEY(a,b) ON CONFLICT IGNORE)", ConflictPairTable.createSQL) + database { + CREATE(ConflictTestTable) + CREATE(ConflictPairTable) + ConflictTestTable INSERT ConflictTest(id = 1, email = "a@mail", x = 1, y = 1) + // The same email: skipped + ConflictTestTable INSERT ConflictTest(id = 2, email = "a@mail", x = 2, y = 2) + // The same key: the row 1 is replaced + ConflictTestTable INSERT ConflictTest(id = 1, email = "b@mail", x = 3, y = 3) + // The same (x, y): the row 1 is replaced again + ConflictTestTable INSERT ConflictTest(id = 4, email = "c@mail", x = 3, y = 3) + ConflictPairTable INSERT ConflictPair(a = "k", b = "v", note = "first") + // The same key: skipped + ConflictPairTable INSERT ConflictPair(a = "k", b = "v", note = "second") + } + lateinit var rows: SelectStatement + lateinit var pairs: SelectStatement + database { + rows = ConflictTestTable SELECT X + pairs = ConflictPairTable SELECT X } - assertEquals(true, notNullable.message!!.contains("column 'notes' is nullable")) + assertEquals(listOf(ConflictTest(id = 4, email = "c@mail", x = 3, y = 3)), rows.getResults()) + assertEquals(listOf(ConflictPair(a = "k", b = "v", note = "first")), pairs.getResults()) } /** @@ -1022,7 +2835,7 @@ class CommonBasicTest(private val path: DatabasePath) { } } } - assertEquals(true, nullInGroup.message!!.contains("'group_concat(notes,',')' can be NULL, so property 'notes' has to be nullable")) + assertEquals(true, nullInGroup.message!!.contains("'group_concat(user_account.notes,',')' can be NULL, so property 'notes' has to be nullable")) val twice = assertFailsWith { database { BookTable { table -> @@ -1136,7 +2949,6 @@ class CommonBasicTest(private val path: DatabasePath) { * table's row type insert the rows it returns, and the SELECT no longer runs on its own. The target is a copy of a * table made with `withName`. */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testInsertSelect() = Database(getNewAPIDBConfig()).databaseAutoClose { database -> val bookCopy = BookTable.withName("book_copy") assertEquals(true, bookCopy.createSQL.startsWith("CREATE TABLE book_copy(")) @@ -1226,7 +3038,8 @@ class CommonBasicTest(private val path: DatabasePath) { * Covers rebuilding a table in a migration, for a change `ALTER TABLE` can't make: the new structure is created * under a temporary name with `withName`, filled with `INSERT INTO ... SELECT` from the old one, which is dropped, * and renamed to the table's name. Renaming the new table rather than the old one leaves the foreign keys of other - * tables pointing at the rebuilt table. + * tables pointing at the rebuilt table. A view that reads the table is dropped before and created again after: had + * it been kept, the rename would fail where SQLite's `legacy_alter_table` is off, as on the JVM. */ @OptIn(ExperimentalDSLDatabaseAPI::class) fun testTableRebuild() { @@ -1237,6 +3050,7 @@ class CommonBasicTest(private val path: DatabasePath) { create = { CREATE(RebuildPersonV1Table) CREATE(RebuildPetTable) + CREATE_VIEW(RebuildPersonIdView) AS (RebuildPersonV1Table SELECT X()) }, ) Database(version1).databaseAutoClose { database -> @@ -1256,9 +3070,11 @@ class CommonBasicTest(private val path: DatabasePath) { create = { CREATE(RebuildPersonTable) CREATE(RebuildPetTable) + CREATE_VIEW(RebuildPersonIdView) AS (RebuildPersonTable SELECT X()) }, upgrade = { oldVersion, _ -> if (oldVersion < 2) { + DROP(RebuildPersonIdView) val newPerson = RebuildPersonTable.withName("rebuild_person_new") CREATE(newPerson) RebuildPersonV1Table { table -> @@ -1266,6 +3082,7 @@ class CommonBasicTest(private val path: DatabasePath) { } DROP(RebuildPersonV1Table) "rebuild_person_new" ALTER_RENAME_TABLE_TO RebuildPersonTable + CREATE_VIEW(RebuildPersonIdView) AS (RebuildPersonTable SELECT X()) } }, ) @@ -1278,6 +3095,13 @@ class CommonBasicTest(private val path: DatabasePath) { assertEquals(listOf(RebuildPerson(1, "Ann"), RebuildPerson(2, "Bob")), people.getResults().sortedBy { it.id }) assertEquals(true, database.selectFails { RebuildPersonV1Table SELECT X }) + // The view, created again, reads the rebuilt table + lateinit var ids: SelectStatement + database { + ids = RebuildPersonIdView SELECT X + } + assertEquals(listOf(1L, 2L), ids.getResults().map { it.id }.sortedBy { it }) + // The new constraint holds assertFails { database { RebuildPersonTable INSERT RebuildPerson(id = null, fullName = "Ann") } @@ -1614,7 +3438,6 @@ class CommonBasicTest(private val path: DatabasePath) { } } - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testDropAndCreateTable() { Database(getNewAPIDBConfig()).databaseAutoClose { database -> // Test 1: DROP using global function @@ -1719,7 +3542,7 @@ class CommonBasicTest(private val path: DatabasePath) { * table through the entity matching the shape it should have at that point, so a step that does * not actually run makes the test fail instead of passing quietly. */ - @OptIn(ExperimentalDSLDatabaseAPI::class) + @OptIn(PlatformDependentSQLiteAPI::class) fun testSchemaModification() { Database(getNewAPIDBConfig()).databaseAutoClose { database -> database { @@ -1839,7 +3662,8 @@ class CommonBasicTest(private val path: DatabasePath) { * non-null columns, which is the order that used to leak the key's nullability into every later * column. These assignments only compile while `name` and `age` are generated as non-null. */ - @Suppress("unused", "UNUSED_VARIABLE") + // Reading the properties is an error, as their getters only return placeholders, but their types are what is checked + @Suppress("unused", "UNUSED_VARIABLE", "DEPRECATION_ERROR") private fun checkSetClauseNullability(clause: SetClause): Unit = with(PersonWithIdTable) { val id: Long? = clause.id val name: String = clause.name @@ -1852,7 +3676,6 @@ class CommonBasicTest(private val path: DatabasePath) { * and a key of any other type is supplied by the caller and declared `NOT NULL`, which SQLite would * otherwise not imply for it. */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testPrimaryKeyNullability() { // Both Long keys are rowid aliases; only the non-Long key needs NOT NULL spelled out. assertEquals(true, PersonWithIdTable.createSQL.contains("id INTEGER PRIMARY KEY,")) @@ -2329,11 +4152,211 @@ class CommonBasicTest(private val path: DatabasePath) { assertEquals(listOf(UserStatus.ACTIVE), enums.getResults().map { it.status }) } + /** + * Covers expression indexes: a function of the table's columns, written without the table's name, which SQLite + * doesn't allow in an index, while a string literal holding that name keeps it. Unique indexes of `lower(name)` + * and of `replace(name, 'product.', '')` show that the expressions are indexed, and an expression of another table + * is rejected. + */ + fun testExpressionIndex() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + ProductTable.CREATE_UNIQUE_INDEX("idx_product_lower_name", ProductTable.lower(ProductTable.name)) + ProductTable.CREATE_UNIQUE_INDEX("idx_product_bare_name", ProductTable.replace(ProductTable.name, "product.", "")) + ProductTable INSERT Product(sku = "SKU-1", name = "Widget", price = 1.0) + } + fun insertFails(product: Product): Boolean = try { + database { ProductTable INSERT product } + false + } catch (e: Exception) { + true + } + assertEquals(true, insertFails(Product(sku = "SKU-2", name = "WIDGET", price = 2.0))) + assertEquals(true, insertFails(Product(sku = "SKU-3", name = "product.Widget", price = 3.0))) + assertEquals(false, insertFails(Product(sku = "SKU-4", name = "Gadget", price = 4.0))) + assertFailsWith { + database { ProductTable.CREATE_INDEX("idx_other_table", BookTable.lower(BookTable.name)) } + } + } + + /** + * Covers DROP_INDEX: a unique index keeps out a duplicate until it is dropped, and dropping an index that doesn't + * exist fails. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testDropIndex() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + fun insertFails(product: Product): Boolean = try { + database { ProductTable INSERT product } + false + } catch (e: Exception) { + true + } + database { + ProductTable.CREATE_UNIQUE_INDEX("idx_product_name", ProductTable.name) + ProductTable INSERT Product(sku = "SKU-1", name = "Widget", price = 1.0) + } + assertEquals(true, insertFails(Product(sku = "SKU-2", name = "Widget", price = 2.0))) + database { DROP_INDEX("idx_product_name") } + assertEquals(false, insertFails(Product(sku = "SKU-3", name = "Widget", price = 3.0))) + assertFails { database { DROP_INDEX("idx_product_name") } } + } + + /** + * Covers IF NOT EXISTS and IF EXISTS: creating a table, an FTS table, indexes and a view that exist does nothing, as + * does dropping a table, a view and an index that don't, while the statements without them fail. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testIfNotExists() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + // BookTable exists already + assertFails { database { CREATE(BookTable) } } + database { + CREATE_IF_NOT_EXISTS(BookTable) + ArticleTable.CREATE_IF_NOT_EXISTS() + ArticleTable.CREATE_IF_NOT_EXISTS() + BookTable.CREATE_INDEX_IF_NOT_EXISTS("idx_book_author", BookTable.author) + BookTable.CREATE_INDEX_IF_NOT_EXISTS("idx_book_author", BookTable.author) + ProductTable.CREATE_UNIQUE_INDEX_IF_NOT_EXISTS("idx_product_name", ProductTable.name) + ProductTable.CREATE_UNIQUE_INDEX_IF_NOT_EXISTS("idx_product_name", ProductTable.name) + CREATE_VIEW_IF_NOT_EXISTS(AdultPersonView) AS (PersonWithIdTable SELECT WHERE(PersonWithIdTable.age GTE 18)) + CREATE_VIEW_IF_NOT_EXISTS(AdultPersonView) AS (PersonWithIdTable SELECT WHERE(PersonWithIdTable.age GTE 18)) + ArticleTable INSERT Article(rowid = null, title = "Kotlin", body = "Coroutines", note = null) + PersonWithIdTable INSERT PersonWithId(id = null, name = "Ann", age = 30) + } + lateinit var articles: SelectStatement
+ lateinit var adults: SelectStatement + database { + articles = ArticleTable SELECT X + adults = AdultPersonView SELECT X + } + assertEquals(listOf("Kotlin"), articles.getResults().map { it.title }) + assertEquals(listOf("Ann"), adults.getResults().map { it.name }) + assertFails { database { BookTable.CREATE_INDEX("idx_book_author", BookTable.author) } } + + database { + DROP_IF_EXISTS(AdultPersonView) + DROP_IF_EXISTS(AdultPersonView) + DROP_INDEX_IF_EXISTS("idx_book_author") + DROP_INDEX_IF_EXISTS("idx_book_author") + ArticleTable.DROP_IF_EXISTS() + DROP_IF_EXISTS(ArticleTable) + } + assertFails { database { DROP(ArticleTable) } } + assertEquals(true, database.selectFails { ArticleTable SELECT X }) + assertEquals(true, database.selectFails { AdultPersonView SELECT X }) + } + + /** + * Covers partial indexes: a unique index with WHERE keeps out the duplicates of the rows that satisfy its condition + * only, with the values of the condition, a quote among them, written into the SQL; a subquery there, and a second + * WHERE, are rejected. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun testPartialIndex() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + fun insertFails(product: Product): Boolean = try { + database { ProductTable INSERT product } + false + } catch (e: Exception) { + true + } + database { + ProductTable.CREATE_UNIQUE_INDEX("idx_product_name_on_sale", ProductTable.name) WHERE + ((ProductTable.price GT 0.0) AND (ProductTable.name NEQ "It's free")) + ProductTable INSERT Product(sku = "SKU-1", name = "Widget", price = 1.0) + // Not on sale, so not in the index + ProductTable INSERT Product(sku = "SKU-2", name = "Widget", price = 0.0) + ProductTable INSERT Product(sku = "SKU-3", name = "Widget", price = 0.0) + ProductTable INSERT Product(sku = "SKU-4", name = "It's free", price = 1.0) + } + assertEquals(true, insertFails(Product(sku = "SKU-5", name = "Widget", price = 2.0))) + assertEquals(false, insertFails(Product(sku = "SKU-6", name = "It's free", price = 2.0))) + lateinit var widgets: SelectStatement + database { + widgets = ProductTable SELECT WHERE ((ProductTable.name EQ "Widget") AND (ProductTable.price GT 0.0)) + } + assertEquals(listOf("SKU-1"), widgets.getResults().map { it.sku }) + + assertFailsWith { + database { + ProductTable.CREATE_INDEX("idx_product_book_name", ProductTable.name) WHERE + (ProductTable.name IN (BookTable SELECT WHERE(BookTable.price GT 30.0))) + } + } + assertFailsWith { + database { + val index = ProductTable.CREATE_INDEX("idx_product_price", ProductTable.price) + index WHERE (ProductTable.price GT 0.0) + index WHERE (ProductTable.price GT 1.0) + } + } + } + + /** + * Covers VACUUM and VACUUM INTO: VACUUM keeps the rows, VACUUM INTO writes a copy that a database can be opened + * from, and neither can run inside a transaction. VACUUM INTO needs SQLite 3.27.0, so it is only checked where the + * SQLite is as new. + */ + @OptIn(ExperimentalDSLDatabaseAPI::class, PlatformDependentSQLiteAPI::class) + fun testVacuum() { + Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { + BookTable INSERT List(100) { Book(name = "Book $it", author = "Author", price = 1.0, pages = it) } + } + database { BookTable DELETE WHERE (BookTable.pages GTE 10) } + database { VACUUM() } + lateinit var books: SelectStatement + database { books = BookTable SELECT X } + assertEquals((0 until 10).toList(), books.getResults().map { it.pages }) + assertFailsWith { + database { transaction { VACUUM() } } + } + assertFailsWith { + database { transaction { VACUUM_INTO("never.db") } } + } + } + + // The file of the database, and the SQLite version, from a connection of the driver + val connection = openDatabase(DatabaseConfiguration(name = DATABASE_NAME, path = path, version = 1, create = {})) + val (file, version) = try { + val file = connection.withQuery("PRAGMA database_list") { cursor -> + var main: String? = null + cursor.forEachRow { if (cursor.getString(1) == "main") main = cursor.getString(2) } + main!! + } + val version = connection.withQuery("SELECT sqlite_version()") { cursor -> + var version = "" + cursor.forEachRow { version = cursor.getString(0)!! } + version + } + file to version + } finally { + connection.close() + } + val (major, minor) = version.split('.').map { it.toInt() } + if (major == 3 && minor < 27) + return + val separator = maxOf(file.lastIndexOf('/'), file.lastIndexOf('\\')) + val backup = file.substring(0, separator + 1) + BACKUP_DATABASE_NAME + deleteDatabase(path, BACKUP_DATABASE_NAME) + try { + Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> + database { VACUUM_INTO(backup) } + // The file exists now + assertFails { database { VACUUM_INTO(backup) } } + } + val backupConfig = DSLDBConfiguration(name = BACKUP_DATABASE_NAME, path = path, version = 1, create = {}) + Database(backupConfig, true).databaseAutoClose { database -> + lateinit var books: SelectStatement + database { books = BookTable SELECT X } + assertEquals((0 until 10).toList(), books.getResults().map { it.pages }) + } + } finally { + deleteDatabase(path, BACKUP_DATABASE_NAME) + } + } + /** * Test for CREATE_INDEX and CREATE_UNIQUE_INDEX operations * Verifies index creation functionality */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testIndexOperations() = Database(getNewAPIDBConfig(), true).databaseAutoClose { database -> // Test 1: CREATE_INDEX on single column database { @@ -2909,7 +4932,6 @@ class CommonBasicTest(private val path: DatabasePath) { * Test PRAGMA_FOREIGN_KEYS function * Verifies foreign key enforcement can be enabled/disabled */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testPragmaForeignKeys() { Database(getForeignKeyDBConfig(), true).databaseAutoClose { database -> // Test 1: Enable foreign keys @@ -2959,7 +4981,6 @@ class CommonBasicTest(private val path: DatabasePath) { * Test CASCADE delete behavior with @References * Verifies that child rows are automatically deleted when parent is deleted */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testForeignKeyCascadeDelete() { Database(getForeignKeyDBConfig(), true).databaseAutoClose { database -> // Enable foreign keys @@ -3022,7 +5043,6 @@ class CommonBasicTest(private val path: DatabasePath) { * Test SET_NULL delete behavior with @References * Verifies that child foreign keys are set to NULL when parent is deleted */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testForeignKeySetNullDelete() { Database(getForeignKeyDBConfig(), true).databaseAutoClose { database -> // Enable foreign keys @@ -3081,7 +5101,6 @@ class CommonBasicTest(private val path: DatabasePath) { * Test RESTRICT delete behavior with @References * Verifies that parent deletion is prevented when child rows exist */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testForeignKeyRestrictDelete() { Database(getForeignKeyDBConfig(), true).databaseAutoClose { database -> // Enable foreign keys @@ -3151,7 +5170,6 @@ class CommonBasicTest(private val path: DatabasePath) { * Test composite foreign keys with @ForeignKey annotation * Verifies multi-column foreign key constraints work correctly */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testCompositeForeignKey() { Database(getForeignKeyDBConfig(), true).databaseAutoClose { database -> // Enable foreign keys @@ -3219,7 +5237,6 @@ class CommonBasicTest(private val path: DatabasePath) { * Test multiple foreign keys to different tables * Verifies a table can have foreign keys to multiple parent tables */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testMultipleForeignKeys() { Database(getForeignKeyDBConfig(), true).databaseAutoClose { database -> // Enable foreign keys @@ -3335,7 +5352,6 @@ class CommonBasicTest(private val path: DatabasePath) { * Test foreign key constraint without PRAGMA_FOREIGN_KEYS enabled * Verifies that constraints are not enforced when PRAGMA is not enabled */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testForeignKeyWithoutPragma() { Database(getForeignKeyDBConfig(), true).databaseAutoClose { database -> // Note: NOT enabling PRAGMA_FOREIGN_KEYS @@ -3471,7 +5487,6 @@ class CommonBasicTest(private val path: DatabasePath) { * Test for @Default annotation with foreign key ON_DELETE_SET_DEFAULT trigger * Verifies that default values are correctly included in CREATE TABLE statements with foreign keys */ - @OptIn(ExperimentalDSLDatabaseAPI::class) fun testDefaultValuesWithForeignKey() { val config = DSLDBConfiguration( name = DATABASE_NAME, @@ -3561,7 +5576,6 @@ class CommonBasicTest(private val path: DatabasePath) { } ) - @OptIn(ExperimentalDSLDatabaseAPI::class) private fun getResultColumnDBConfig(): DSLDBConfiguration = DSLDBConfiguration( name = DATABASE_NAME, @@ -3575,6 +5589,18 @@ class CommonBasicTest(private val path: DatabasePath) { ) @OptIn(ExperimentalDSLDatabaseAPI::class) + private fun getFtsDBConfig(): DSLDBConfiguration = + DSLDBConfiguration( + name = DATABASE_NAME, + path = path, + version = 1, + create = { + CREATE(ArticleTable) + CREATE(NoteTable) + CREATE(BookTable) + } + ) + private fun getForeignKeyDBConfig(): DSLDBConfiguration = DSLDBConfiguration( name = DATABASE_NAME, diff --git a/sqllin-dsl-test/src/commonMain/kotlin/com/ctrip/sqllin/dsl/test/Entities.kt b/sqllin-dsl-test/src/commonMain/kotlin/com/ctrip/sqllin/dsl/test/Entities.kt index 7964b03e..2996f930 100644 --- a/sqllin-dsl-test/src/commonMain/kotlin/com/ctrip/sqllin/dsl/test/Entities.kt +++ b/sqllin-dsl-test/src/commonMain/kotlin/com/ctrip/sqllin/dsl/test/Entities.kt @@ -16,10 +16,17 @@ package com.ctrip.sqllin.dsl.test +import com.ctrip.sqllin.dsl.annotation.Check import com.ctrip.sqllin.dsl.annotation.CollateNoCase import com.ctrip.sqllin.dsl.annotation.CompositePrimaryKey import com.ctrip.sqllin.dsl.annotation.CompositeUnique import com.ctrip.sqllin.dsl.annotation.DBRow +import com.ctrip.sqllin.dsl.annotation.DBView +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.annotation.Fts3 +import com.ctrip.sqllin.dsl.annotation.Fts4 +import com.ctrip.sqllin.dsl.annotation.FtsTokenizer +import com.ctrip.sqllin.dsl.annotation.OnConflict import com.ctrip.sqllin.dsl.annotation.PrimaryKey import com.ctrip.sqllin.dsl.annotation.Unique import kotlinx.serialization.SerialName @@ -534,6 +541,12 @@ data class BookTitle(val name: String, val author: String) @Serializable data class BookAuthor(val author: String) +/** + * The `name` of a [Book] or a [PersonWithId]: a projection both tables fit, so that a compound SELECT can combine them. + */ +@Serializable +data class NameOnly(val name: String) + /** * A projection of [UserAccount] that reads its nullable `notes` column into a nullable property. */ @@ -574,6 +587,159 @@ data class BookTotals(val books: Long, val maxPages: PageCount?, val averagePric @Serializable data class BookCount(val books: Long) +/** + * Arithmetic and concatenation of the columns of a [Book], next to its `name`, read from its column. + */ +@Serializable +data class BookMath( + val name: String, + val doubledPrice: Double, + val pagesPerPrice: Double?, + val negatedPages: PageCount, + val nextPage: PageCount, + val label: String, + val remainder: PageCount, + val perZero: PageCount?, +) + +/** + * The pages of a [Book], to select an expression into. + */ +@Serializable +data class BookPages(val pages: PageCount) + +/** + * The columns of a [Book] converted to other types with CAST. + */ +@Serializable +data class BookCasts( + val wholePrice: Int, + val pagesText: String, + val nameBytes: ByteArray, + val leadingDigits: Int, + val weighted: Double, +) + +/** + * The functions that replace NULL, and the scalar max and min, of the columns of a [UserAccount]. + */ +@Serializable +data class NullFunctionResults( + val username: String, + val notesOrNone: String, + val notesOrEmail: String, + val ifNullNotes: String, + val notBob: String?, + val greater: String, + val lesser: String, +) + +/** + * String, BLOB and number functions of the columns of a [Book]. + */ +@Serializable +data class TextFunctions( + val hexName: String, + val quoted: String, + val firstCode: Long?, + val trimmed: String, + val tail: String, + val middle: String, + val replaced: String, + val position: Long, + val formatted: String, + val rounded: Double, + val zeros: Long, + val randomBytes: Long, +) + +/** + * The `total` of the prices of the books, which is 0.0 rather than NULL when no rows match. + */ +@Serializable +data class PriceTotals(val total: Double, val distinctTotal: Double) + +/** + * Date and time functions: all can be NULL, but `datetime('now')`. + */ +@Serializable +data class DateTimes( + val day: String?, + val nextDay: String?, + val later: String?, + val epochStart: String?, + val yearMonth: String?, + val julian: Double?, + val now: String, +) + +/** + * The math functions, which only some platforms' SQLite has. + */ +@Serializable +data class MathResults( + val root: Double?, + val ceiling: Double, + val flooredPages: PageCount, + val truncated: Double, + val power: Double?, + val powerOf: Double?, + val remainder: Double?, + val remainderOf: Double?, + val piValue: Double, + val exponential: Double?, + val natural: Double?, + val decimal: Double?, + val binary: Double?, +) + +@Serializable +data class SignResult(val signOf: Long) + +@Serializable +data class EpochResult(val epoch: Long?) + +/** + * A [Book] sorted into categories with CASE. + */ +@Serializable +data class BookCategory(val name: String, val size: String, val discount: Double?) + +/** + * The single column of a SELECT used as a value, a scalar subquery. + */ +@Serializable +data class AveragePrice(val value: Double?) + +@Serializable +data class MaxPages(val value: PageCount?) + +/** + * A [Book] next to the most pages of any book, from a scalar subquery. + */ +@Serializable +data class BookWithMaxPages(val name: String, val maxPages: PageCount?) + +/** + * Bitwise operators of the age of a [PersonWithId]. + */ +@Serializable +data class AgeBits(val anded: Age, val ored: Age, val shifted: Age, val halved: Age, val inverted: Age) + +/** + * Aggregates of the distinct values of the columns of the books, next to those of all values. + */ +@Serializable +data class DistinctBookTotals( + val authors: Long, + val books: Long, + val distinctPrice: Double?, + val totalPrice: Double?, + val distinctPages: Long?, + val averageDistinctPages: Double?, + val authorNames: String?, +) + /** * Scalar functions of the columns of a book, next to its `name`, which is read from its column. */ @@ -633,6 +799,101 @@ data class RebuildPerson( @Unique val fullName: String, ) +/** + * A table whose constraints have conflict clauses: the key replaces the row it conflicts with, a duplicate email is + * skipped, and the group (x, y) replaces. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow("conflict_test") +@Serializable +data class ConflictTest( + @PrimaryKey(onConflict = OnConflict.REPLACE) val id: Long, + @Unique(onConflict = OnConflict.IGNORE) val email: String, + @CompositeUnique(0, onConflict = OnConflict.REPLACE) val x: Int, + @CompositeUnique(0) val y: Int, +) + +/** + * A table whose composite key skips the rows that conflict with it. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow("conflict_pair") +@Serializable +data class ConflictPair( + @CompositePrimaryKey(onConflict = OnConflict.IGNORE) val a: String, + @CompositePrimaryKey val b: String, + val note: String?, +) + +/** + * A parent of [DeferredChild] and [DeferredGroupChild], whose foreign keys are deferred: checked when the transaction + * commits. + */ +@DBRow("deferred_parent") +@Serializable +data class DeferredParent(@PrimaryKey val id: Long) + +@DBRow("deferred_child") +@Serializable +data class DeferredChild( + @PrimaryKey val id: Long, + @com.ctrip.sqllin.dsl.annotation.References( + tableName = "deferred_parent", + trigger = com.ctrip.sqllin.dsl.annotation.Trigger.ON_DELETE_CASCADE, + foreignKeys = ["id"], + deferred = true, + ) + val parentId: Long, +) + +@DBRow("deferred_group_child") +@Serializable +@com.ctrip.sqllin.dsl.annotation.ForeignKeyGroup(group = 0, tableName = "deferred_parent", deferred = true) +data class DeferredGroupChild( + @PrimaryKey val id: Long, + @com.ctrip.sqllin.dsl.annotation.ForeignKey(group = 0, reference = "id") val parentId: Long, +) + +/** + * A row whose columns all have default values, or are nullable, so that `INSERT DEFAULT VALUES` can insert it. + */ +@DBRow("default_row") +@Serializable +data class DefaultRow( + @PrimaryKey(autoIncrement = true) val id: Long?, + @com.ctrip.sqllin.dsl.annotation.Default("'new'") val status: String, + @com.ctrip.sqllin.dsl.annotation.Default("0") val count: Int, + val note: String?, +) + +/** + * A log that triggers write the changes of [PersonWithId]s to. + */ +@DBRow("trigger_audit") +@Serializable +data class TriggerAudit( + @PrimaryKey(autoIncrement = true) val id: Long?, + val personId: Long?, + val action: String, + val at: String?, +) + +/** + * A count that a trigger keeps. + */ +@DBRow("trigger_counter") +@Serializable +data class TriggerCounter(@PrimaryKey val id: Long, val count: Int) + +/** + * A view of the ids in `rebuild_person`, which a rebuild of the table has to drop before dropping the old table, and + * create again after the rename. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBView("rebuild_person_ids") +@Serializable +data class RebuildPersonId(val id: Long?) + /** * References `rebuild_person`, so its foreign key shows whether the rebuild left the reference in place. */ @@ -643,3 +904,118 @@ data class RebuildPet( @com.ctrip.sqllin.dsl.annotation.References(tableName = "rebuild_person", foreignKeys = ["id"]) val ownerId: Long, ) + +/** + * A view of the [PersonWithId]s who are adults, which `CREATE_VIEW` creates from a SELECT of this type. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBView("adult_person") +@Serializable +data class AdultPerson(val name: String, val age: Age) + +/** + * Aggregates of the unsigned `testUInt` column of [TestPrimitiveTypeForKSP], which are only right if it holds the + * numbers, not their bits as signed numbers. + */ +@Serializable +data class UIntTotals(val total: Long?, val highest: UInt?) + +/** + * An FTS4 table: the porter tokenizer, so that "running" matches "run", prefixes of two letters indexed, and a column + * it stores but doesn't index. Its rowid is assigned by SQLite. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow("articles") +@Fts4(tokenizer = FtsTokenizer.PORTER, prefix = [2], notIndexed = ["note"]) +@Serializable +data class Article(@PrimaryKey val rowid: Long?, val title: String, val body: String, val note: String?) + +/** + * A match of [Article] with a text that describes it: a snippet, or offsets. + */ +@Serializable +data class ArticleMatch(val title: String, val excerpt: String) + +/** + * A match of [Article] with its matchinfo. + */ +@Serializable +class ArticleMatchInfo(val title: String, val info: ByteArray) + +/** + * An FTS3 table whose tokenizer takes an argument, with its rowid named docid and supplied by the caller. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow("notes") +@Fts3(tokenizer = FtsTokenizer.UNICODE61, tokenizerArgs = ["remove_diacritics=1"]) +@Serializable +data class Note(@PrimaryKey val docid: Long, val text: String) + +/** + * A table with CHECK constraints: of columns, two of them on one column, written named first to check that the unnamed + * one comes first, and a named one of the table that compares two columns. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow +@Check("endDate IS NULL OR endDate >= startDate", constraintName = "valid_period") +@Serializable +data class CheckedMembership( + @PrimaryKey val id: Long?, + @Check("lower(name) = name", constraintName = "lower_case_name") @Check("length(trim(name)) > 0") val name: String, + @Check("age BETWEEN 0 AND 150") val age: Int?, + @Check("level IN (0, 1, 2)") val level: Int, + val startDate: String, + @Check("endDate IS NULL OR date(endDate) IS endDate") val endDate: String?, +) + +/** + * The number of books of each author, declared as a view to name a derived table of book: no such view is created. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBView("author_books") +@Serializable +data class AuthorBooks(val author: String, val books: Long) + +/** + * A person with the number of their books, from a join with the derived table [AuthorBooks]. + */ +@Serializable +data class PersonBooks(val name: String, val age: Age, val books: Long) + +/** + * The persons under other names, as a derived table that joins person_with_id with itself. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBView("elder") +@Serializable +data class Elder(val elderName: String, val elderAge: Age) + +/** + * A person with someone older, from person_with_id joined with itself. + */ +@Serializable +data class YoungerAndElder(val name: String, val elderName: String) + +/** + * The age of the oldest person: a SELECT of a single column, which an IN subquery compares with. + */ +@Serializable +data class PersonAge(val age: Age?) + +/** + * A person with the number of their books and the highest price of one, from person_with_id outer joined with book. + */ +@Serializable +data class PersonBookStats(val name: String, val books: Long, val highest: Double?) + +/** + * The age of a person with the name of one of their books, which an outer join leaves NULL for a person without one. + */ +@Serializable +data class PersonBookTitle(val age: Age, val title: String?) + +/** + * The age of an author with the code of the category of one of their books, from three joined tables. + */ +@Serializable +data class AuthorCode(val age: Age, val code: Code) diff --git a/sqllin-dsl-test/src/jvmTest/kotlin/com/ctrip/sqllin/dsl/test/JvmTest.kt b/sqllin-dsl-test/src/jvmTest/kotlin/com/ctrip/sqllin/dsl/test/JvmTest.kt index 28b93304..3d277003 100644 --- a/sqllin-dsl-test/src/jvmTest/kotlin/com/ctrip/sqllin/dsl/test/JvmTest.kt +++ b/sqllin-dsl-test/src/jvmTest/kotlin/com/ctrip/sqllin/dsl/test/JvmTest.kt @@ -40,6 +40,126 @@ class JvmTest { @Test fun testUnionSelect() = commonTest.testUnionSelect() + @Test + fun testCompoundSelect() = commonTest.testCompoundSelect() + + @Test + fun testObserve() = commonTest.testObserve() + + @Test + fun testObserveForeignKeyAction() = commonTest.testObserveForeignKeyAction() + + @Test + fun testView() = commonTest.testView() + + @Test + fun testUnsignedValues() = commonTest.testUnsignedValues() + + @Test + fun testFts4() = commonTest.testFts4() + + @Test + fun testFts3() = commonTest.testFts3() + + @Test + fun testCheckConstraint() = commonTest.testCheckConstraint() + + @Test + fun testSubquery() = commonTest.testSubquery() + + @Test + fun testConditionPrecedence() = commonTest.testConditionPrecedence() + + @Test + fun testConditionReuse() = commonTest.testConditionReuse() + + @Test + fun testJoinedRelation() = commonTest.testJoinedRelation() + + @Test + fun testExpressionIndex() = commonTest.testExpressionIndex() + + @Test + fun testDropIndex() = commonTest.testDropIndex() + + @Test + fun testIfNotExists() = commonTest.testIfNotExists() + + @Test + fun testPartialIndex() = commonTest.testPartialIndex() + + @Test + fun testDistinctAggregate() = commonTest.testDistinctAggregate() + + @Test + fun testLikeEscape() = commonTest.testLikeEscape() + + @Test + fun testVacuum() = commonTest.testVacuum() + + @Test + fun testArithmetic() = commonTest.testArithmetic() + + @Test + fun testCast() = commonTest.testCast() + + @Test + fun testNullFunctions() = commonTest.testNullFunctions() + + @Test + fun testMoreFunctions() = commonTest.testMoreFunctions() + + @Test + fun testDateTimeFunctions() = commonTest.testDateTimeFunctions() + + @Test + fun testPlatformDependentFunctions() = commonTest.testPlatformDependentFunctions() + + @Test + fun testCaseExpression() = commonTest.testCaseExpression() + + @Test + fun testScalarSubquery() = commonTest.testScalarSubquery() + + @Test + fun testExpressionOperators() = commonTest.testExpressionOperators() + + @Test + fun testSetExpressions() = commonTest.testSetExpressions() + + @Test + fun testInsertExpressions() = commonTest.testInsertExpressions() + + @Test + fun testTrigger() = commonTest.testTrigger() + + @Test + fun testTriggerSelectingNewRow() = commonTest.testTriggerSelectingNewRow() + + @Test + fun testTriggerChecks() = commonTest.testTriggerChecks() + + @Test + fun testTriggerWritingFts() = commonTest.testTriggerWritingFts() + + @Test + fun testInsertDefaultValues() = commonTest.testInsertDefaultValues() + + @Test + fun testAnalyzeAndReindex() = commonTest.testAnalyzeAndReindex() + + @Test + fun testIndexOrder() = commonTest.testIndexOrder() + + @Test + fun testNullsOrdering() = commonTest.testNullsOrdering() + + @Test + fun testDeferredForeignKey() = commonTest.testDeferredForeignKey() + + @Test + fun testConflictClauses() = commonTest.testConflictClauses() + @Test fun testFunction() = commonTest.testFunction() diff --git a/sqllin-dsl-test/src/nativeTest/kotlin/com/ctrip/sqllin/dsl/test/NativeTest.kt b/sqllin-dsl-test/src/nativeTest/kotlin/com/ctrip/sqllin/dsl/test/NativeTest.kt index 5dd36280..ef0ee1d6 100644 --- a/sqllin-dsl-test/src/nativeTest/kotlin/com/ctrip/sqllin/dsl/test/NativeTest.kt +++ b/sqllin-dsl-test/src/nativeTest/kotlin/com/ctrip/sqllin/dsl/test/NativeTest.kt @@ -56,6 +56,126 @@ class NativeTest { @Test fun testUnionSelect() = commonTest.testUnionSelect() + @Test + fun testCompoundSelect() = commonTest.testCompoundSelect() + + @Test + fun testObserve() = commonTest.testObserve() + + @Test + fun testObserveForeignKeyAction() = commonTest.testObserveForeignKeyAction() + + @Test + fun testView() = commonTest.testView() + + @Test + fun testUnsignedValues() = commonTest.testUnsignedValues() + + @Test + fun testFts4() = commonTest.testFts4() + + @Test + fun testFts3() = commonTest.testFts3() + + @Test + fun testCheckConstraint() = commonTest.testCheckConstraint() + + @Test + fun testSubquery() = commonTest.testSubquery() + + @Test + fun testConditionPrecedence() = commonTest.testConditionPrecedence() + + @Test + fun testConditionReuse() = commonTest.testConditionReuse() + + @Test + fun testJoinedRelation() = commonTest.testJoinedRelation() + + @Test + fun testExpressionIndex() = commonTest.testExpressionIndex() + + @Test + fun testDropIndex() = commonTest.testDropIndex() + + @Test + fun testIfNotExists() = commonTest.testIfNotExists() + + @Test + fun testPartialIndex() = commonTest.testPartialIndex() + + @Test + fun testDistinctAggregate() = commonTest.testDistinctAggregate() + + @Test + fun testLikeEscape() = commonTest.testLikeEscape() + + @Test + fun testVacuum() = commonTest.testVacuum() + + @Test + fun testArithmetic() = commonTest.testArithmetic() + + @Test + fun testCast() = commonTest.testCast() + + @Test + fun testNullFunctions() = commonTest.testNullFunctions() + + @Test + fun testMoreFunctions() = commonTest.testMoreFunctions() + + @Test + fun testDateTimeFunctions() = commonTest.testDateTimeFunctions() + + @Test + fun testPlatformDependentFunctions() = commonTest.testPlatformDependentFunctions() + + @Test + fun testCaseExpression() = commonTest.testCaseExpression() + + @Test + fun testScalarSubquery() = commonTest.testScalarSubquery() + + @Test + fun testExpressionOperators() = commonTest.testExpressionOperators() + + @Test + fun testSetExpressions() = commonTest.testSetExpressions() + + @Test + fun testInsertExpressions() = commonTest.testInsertExpressions() + + @Test + fun testTrigger() = commonTest.testTrigger() + + @Test + fun testTriggerSelectingNewRow() = commonTest.testTriggerSelectingNewRow() + + @Test + fun testTriggerChecks() = commonTest.testTriggerChecks() + + @Test + fun testTriggerWritingFts() = commonTest.testTriggerWritingFts() + + @Test + fun testInsertDefaultValues() = commonTest.testInsertDefaultValues() + + @Test + fun testAnalyzeAndReindex() = commonTest.testAnalyzeAndReindex() + + @Test + fun testIndexOrder() = commonTest.testIndexOrder() + + @Test + fun testNullsOrdering() = commonTest.testNullsOrdering() + + @Test + fun testDeferredForeignKey() = commonTest.testDeferredForeignKey() + + @Test + fun testConflictClauses() = commonTest.testConflictClauses() + @Test fun testFunction() = commonTest.testFunction() diff --git a/sqllin-dsl/build.gradle.kts b/sqllin-dsl/build.gradle.kts index cc05bb08..2c8f3e71 100644 --- a/sqllin-dsl/build.gradle.kts +++ b/sqllin-dsl/build.gradle.kts @@ -59,7 +59,8 @@ kotlin { commonMain.dependencies { api(project(":sqllin-driver")) implementation(libs.kotlinx.serialization) - implementation(libs.kotlinx.coroutines.core) + // Database.observe returns a Flow + api(libs.kotlinx.coroutines.core) } } } diff --git a/sqllin-dsl/doc/advanced-query-cn.md b/sqllin-dsl/doc/advanced-query-cn.md index b23eefd4..d55cd7f3 100644 --- a/sqllin-dsl/doc/advanced-query-cn.md +++ b/sqllin-dsl/doc/advanced-query-cn.md @@ -2,155 +2,76 @@ 我们已经学习了基础查询和在条件查询中使用 SQL 函数。现在我们来学习一些更高级的查询技巧。 -## Union +## 组合查询 -_UNION_ 操作符用于合并两个查询结果相同的 _SELECT_ 语句。 - -在 SQL 中,*UNION* 操作符位于两个 _SELECT_ 语句中间,但是在 SQLlin 中,我们使用一个高阶函数来实现 *UNION*: +组合查询把两条 _SELECT_ 语句的结果行合并起来,对应 SQL 的 `UNION`、`UNION ALL`、`INTERSECT` 和 `EXCEPT`。在 SQLlin 中, +它们是写在两条语句之间的中缀函数,写法和 SQL 一样。它们是实验性 API,使用时需要 `@OptIn(ExperimentalDSLDatabaseAPI::class)`: ```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) fun sample() { lateinit var selectStatement: SelectStatement database { PersonTable { table -> - selectStatement = UNION { - table SELECT WHERE (age GTE 5) - table SELECT WHERE (length(name) LTE 8) - } + // SELECT name,age FROM person WHERE age < ? UNION SELECT name,age FROM person WHERE age >= ? + selectStatement = (table SELECT WHERE(age LT 18)) UNION (table SELECT WHERE(age GTE 65)) } } } ``` -你只需要将你的 _SELECT_ 语句写在 `UNION {...}` 块内部。 `UNION {...}` 块内部至少要有两个 _SELECT_ -语句,否则你将会在运行时得到一个 `IllegalStateException` 异常。 +`UNION` 合并结果并去重,`UNION_ALL` 保留重复行,`INTERSECT` 只保留两条语句都返回的行,`EXCEPT` 保留第一条语句返回、而第二条 +语句没有返回的行。每条 _SELECT_ 都要加括号,因为 _SELECT_ 和这些运算符都是同优先级的中缀函数;不加括号的话,代码无法编译。 -如果你想要交替使用 _UNION_ 和 _UNION ALL_ ,请使用 `UNION {...}` 或 `UNION_ALL {...}` 块嵌套: +两条语句读出的行必须是同一种类型,这一点在编译期检查。它们可以来自不同的表,借助投影、结果列或 Join,详见下文: ```kotlin -fun sample() { - lateinit var selectStatement: SelectStatement - database { - PersonTable { table -> - selectStatement = UNION { - table SELECT WHERE (age GTE 5) - UNION_ALL { - table SELECT WHERE (length(name) LTE 8) - table SELECT WHERE (name EQ "Tom") - } - } - } - } -} -``` - -前面的代码等价于: - -```roomsql -SELECT * FROM person WHERE age >= 5 -UNION -SELECT * FROM person WHERE length(name) <= 8 -UNION ALL -SELECT * FROM person WHERE name = "Tom" -``` - -## 子查询 - -SQLlin 还不支持子查询,我们将会尽快开发该功能。 - -## Join - -SQLlin 目前支持 join 表。 - -我们需要另外两个数据库实体: - -```kotlin -@DBRow("transcript") -@Serializable -data class Transcript( - val name: String?, - val math: Int, - val english: Int, -) - -@Serializable -data class Student( - val name: String?, - val age: Int?, - val math: Int, - val english: Int, -) - -@Serializable -data class CrossJoinStudent( - val age: Int?, - val math: Int, - val english: Int, -) +// 既是学生又是老师的人的名字 +(StudentTable SELECT X()) INTERSECT (TeacherTable SELECT X()) ``` -`Transcript` 代表另一张表,`Student` 表示 join 的查询结果的类型(所以 `Student` 不需要被添加 `@DBRow` 注解),它拥有所有 `Person` 和 `Transcript` -所拥有的列名。 - -### Cross Join +连续的运算符按从左到右的顺序计算,和 SQLite 一致,括号可以改变分组。SQLite 的组合查询里不能写括号,所以 SQLlin 会把分组后的 +组合写成子查询: ```kotlin -fun joinSample() { - db { - PersonTable { table -> - table SELECT CROSS_JOIN(TranscriptTable) - } - } -} +a UNION b UNION_ALL c // (a UNION b) UNION ALL c +a UNION (b UNION_ALL c) // a UNION SELECT * FROM (b UNION ALL c) ``` -`CROSS_JOIN` 函数接收一个或多个 `Table` 作为参数。在普通的 _SELECT_ 语句中,该语句的查询结果的类型由 _sqllin-processor_ 生成的 -`Table` 决定,但是 _JOIN_ 操作符将会将其改变为指定的类型。在前面的示例中, `CROSS_JOIN` 将该类型改变为了 `CrossJoinStudent`。 - -注意,由于 SQL 中 _CROSS JOIN_ 自身的特性,如果附带 _CROSS JOIN_ 子句的 _SELECT_ 语句查询的列包含两个表中的同名列,这会导致查询失败。因为 -class 中不能包含两个同名的属性。因此请确保 `CROSS_JOIN` 函数转换后的结果类型不包含两个表中的同名列。 - -### Inner Join +写在组合之后的 _ORDER BY_ 和 _LIMIT_ 作用于整个组合。某条语句自己末尾的 _ORDER BY_ 或 _LIMIT_ 只作用于它自己,因为 SQLlin 同样会 +把它写成子查询: ```kotlin -fun joinSample() { - db { - PersonTable { table -> - table SELECT INNER_JOIN(TranscriptTable) USING name - table SELECT NATURAL_INNER_JOIN(TranscriptTable) - table SELECT INNER_JOIN(TranscriptTable) ON (name EQ TranscriptTable.name) - } - } -} +// 年龄最大的 3 个人和年龄最小的 3 个人 +(table SELECT ORDER_BY(age to DESC) LIMIT 3) UNION_ALL (table SELECT ORDER_BY(age to ASC) LIMIT 3) +// 儿童和老人合在一起,按名字排序,每页 10 个 +(table SELECT WHERE(age LT 18)) UNION (table SELECT WHERE(age GTE 65)) ORDER_BY (name to ASC) LIMIT 10 OFFSET 20 ``` -`INNER_JOIN` 与 `CROSS_JOIN` 非常相似,不同之处在于 `INNER_JOIN` 需要连接一个 `USING` 或 `ON` 子句。如果一个 _INNER JOIN_ 语句没有 -`USING` 或 `ON` 子句,那么它是不完整的,但是你的代码仍然可以编译,但它在运行时不会做任何事情。 +参与组合的语句会成为组合的一部分,因此不再单独执行,也无法读取它们各自的结果。 -`NATURAL_INNER_JOIN` 将会产生一个完整的 _SELECT_ 语句(与 `CROSS_JOIN` 相似)。所以,你不能再它末尾连接 `USING` 或 `ON` 子句,这将由 -Kotlin 编译器来保证。 +### 已废弃的 `UNION {...}` 块 -注意,带有 `ON` 子句的 `INNER_JOIN` 子句的行为与 `CROSS_JOIN` 相同,你不能 select 在两个表中拥有相同名字的列。 +以前的版本用 `UNION {...}` 和 `UNION_ALL {...}` 块来合并 _SELECT_ 语句。**它们已经废弃,将在未来的版本中移除**,因为它们存在 +不会再修复的 bug: -`INNER_JOIN` 拥有一个别名——`JOIN`, `NATURAL_INNER_JOIN` 也拥有一个别名——`NATURAL_JOIN` 。这就像你在 SQL 的 inner join -查询中可以省略 `INNER` 关键字一样。 +- 块的结果类型取自表的行类型,结果行却按第一条 _SELECT_ 的结果类型读取。所以对投影、结果列或 Join 做 UNION 时,代码能编译通过, + 但使用结果时会抛出 `ClassCastException`。 +- 嵌套的块在生成 SQL 时会被展平,而 SQLite 按从左到右的顺序计算,所以 `UNION { a; UNION_ALL { b; c } }` 实际执行的是 + `(a UNION b) UNION ALL c`,而不是嵌套所表达的含义。 -### Left Outer Join +请改用中缀运算符: ```kotlin -fun joinSample() { - db { - PersonTable { table -> - table SELECT LEFT_OUTER_JOIN(TranscriptTable) USING name - table SELECT NATURAL_LEFT_OUTER_JOIN(TranscriptTable) - table SELECT LEFT_OUTER_JOIN(TranscriptTable) ON (name EQ TranscriptTable.name) - } - } +// 已废弃 +UNION { + table SELECT WHERE(age LT 18) + table SELECT WHERE(age GTE 65) } +// 改为 +(table SELECT WHERE(age LT 18)) UNION (table SELECT WHERE(age GTE 65)) ``` -`LEFT_OUTER_JOIN` 的用法与 `INNER_JOIN` 非常相似,不同之处仅仅是它们的 API 名字。 - ## 投影 `SELECT` 默认把每一行读成表自己的行类型。如果只想读取其中一部分列,可以声明一个更窄的 `@Serializable` 类型,用它的属性名 @@ -169,7 +90,7 @@ fun sample() { PersonTable { table -> // SELECT name FROM person names = table SELECT X() - // SELECT name FROM person WHERE age >= ? ORDER BY name LIMIT 10 + // SELECT name FROM person WHERE age >= ? ORDER BY person.name LIMIT 10 adultNames = table SELECT WHERE(age GTE 18) ORDER_BY name LIMIT 10 } } @@ -209,7 +130,7 @@ fun sample() { lateinit var adults: SelectStatement database { PersonTable { table -> - // SELECT name,count(*) AS people,max(age) AS maxAge FROM person GROUP BY name + // SELECT name,count(*) AS people,max(person.age) AS maxAge FROM person GROUP BY person.name stats = table SELECT listOf(count(X) AS NameStats::people, max(age) AS NameStats::maxAge) GROUP_BY name // SELECT count(*) AS people FROM person WHERE age >= ? adults = table SELECT (count(X) AS PersonCount::people) WHERE (age GTE 18) @@ -245,8 +166,361 @@ fun sample() { `IllegalArgumentException`。唯独后面是否还会接 `GROUP_BY`,在构建时还无法得知,所以需要 `GROUP_BY` 的属性会在数据库作用域 结束时报错,此时作用域中的任何语句都还没有执行。 -交给表达式的属性按属性名查找,所以不能用 `@SerialName` 重命名。结果列只能查询单张表:目前还不能和 Join 一起使用, -也不支持算术运算、`CASE` 和子查询。 +交给表达式的属性按属性名查找,所以不能用 `@SerialName` 重命名。结果列也可以从 [Join](#join) 中查询。算术运算、`CASE` 和只返回 +单个值的子查询暂时还不能作为表达式使用。 + +## Join + +Join 把多个关系(表、视图或派生表)的行合在一起读取。用 `FROM(relation)` 开始一个 Join,用 Join 运算符扩展它,然后像查询表一样 +从中查询。Join 是实验性 API,使用时需要 `@OptIn(ExperimentalDSLDatabaseAPI::class)`。 + +示例用到另一张表,以及几个表示 Join 结果的类型,它们不需要 `@DBRow`: + +```kotlin +@DBRow("transcript") +@Serializable +data class Transcript( + val name: String?, + val math: Int, + val english: Int, +) + +@Serializable +data class Student( + val name: String?, + val age: Int?, + val math: Int, + val english: Int, +) + +@Serializable +data class StudentScore( + val name: String?, + val tests: Long, + val bestMath: Int?, +) +``` + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun joinSample() { + lateinit var students: SelectStatement + lateinit var scores: SelectStatement + database { + // SELECT name,person.age,transcript.math,transcript.english FROM person INNER JOIN transcript USING (name) + students = (FROM(PersonTable) INNER_JOIN TranscriptTable USING PersonTable.name) SELECT X() + // SELECT person.name AS name,count(transcript.math) AS tests,max(transcript.math) AS bestMath + // FROM person LEFT OUTER JOIN transcript ON person.name = transcript.name GROUP BY person.name + scores = (FROM(PersonTable) LEFT_OUTER_JOIN TranscriptTable ON (PersonTable.name EQ TranscriptTable.name)) SELECT listOf( + PersonTable.name AS StudentScore::name, + PersonTable.count(TranscriptTable.math) AS StudentScore::tests, + PersonTable.max(TranscriptTable.math) AS StudentScore::bestMath, + ) GROUP_BY PersonTable.name + } +} +``` + +Join 运算符有: + +| 运算符 | 后接 | 返回 | +| --- | --- | --- | +| `INNER_JOIN` | `ON` 或 `USING` | 两边匹配的行 | +| `LEFT_OUTER_JOIN` | `ON` 或 `USING` | 上述行,加上它之前的关系中没有匹配的行 | +| `RIGHT_OUTER_JOIN` | `ON` 或 `USING` | 上述行,加上被连接的关系中没有匹配的行 | +| `FULL_OUTER_JOIN` | `ON` 或 `USING` | 上述行,加上两边各自没有匹配的行 | +| `CROSS_JOIN` | 无 | 它之前的关系的每一行与被连接关系的每一行的组合 | +| `NATURAL_JOIN`、`NATURAL_LEFT_OUTER_JOIN`、`NATURAL_RIGHT_OUTER_JOIN`、`NATURAL_FULL_OUTER_JOIN` | 无 | 同上,按同名的列匹配 | + +`RIGHT_OUTER_JOIN`、`FULL_OUTER_JOIN` 和它们的 NATURAL 形式需要 SQLite 3.39.0,Android 从 API 34 起才具备,所以它们标有 +`@PlatformDependentSQLiteAPI`,需要按[依赖 SQLite 版本的 API](modify-database-and-transaction-cn.md#依赖-sqlite-版本的-api) 所说的 +方式声明同意。 + +Join 按书写的顺序执行,每一个都可以有自己的条件:`FROM(a) INNER_JOIN b ON (...) LEFT_OUTER_JOIN c ON (...)`。一个关系只能被连接 +一次:要让表和自身 Join,请连接由它的行构成的[派生表](#派生表),这相当于给它起了另一个名字。 + +Join 的行没有自己的类型,所以从 Join 查询时要指定读取成的类型:用 `X()`、`WHERE(...)` 这类子句函数,或者结果列。构建语句时, +会对照参与 Join 的关系检查这个类型: + +* 每个属性都必须是某个关系的列。两个关系中同名的列无法区分,除非 `USING` 或 NATURAL Join 合并了它们,所以请用 `AS` 选取其中一个, + 就像上面的 `PersonTable.name AS StudentScore::name`。 +* 列或表达式可能为 NULL 时,属性必须可空。外连接在找不到匹配行的地方,会让一个关系的列为 NULL,因此除 `count` 以外,关于这些列的 + 表达式也是如此。 + +SQLlin 写列时都带上关系的名字,所以条件、函数、_GROUP BY_ 和 _ORDER BY_ 都能使用其他关系中也有的列名。可观察查询会观察 Join 中的 +所有关系。 + +### 旧的 Join API + +在 2.5.0 之前,Join 写作 _SELECT_ 的子句:`PersonTable SELECT INNER_JOIN(TranscriptTable) USING name`,以及 `CROSS_JOIN`、 +`NATURAL_JOIN`、`LEFT_OUTER_JOIN` 等。它们仍然可用,但不会检查行被读取成的类型,无法连接两个以上各带条件的关系,也不能选取结果列。 +**它们将在 2.5.0 之后的下一个版本中删除**,请按上面的方式书写 Join。 + +## 子查询 + +一条 _SELECT_ 可以成为另一条查询的一部分:作为派生表,在 FROM 或 Join 中像表一样被读取;或者作为 `IN`、`EXISTS` 条件里的子查询。 +它会成为那条查询的一部分,所以不会再单独执行。子查询是实验性 API,使用时需要 `@OptIn(ExperimentalDSLDatabaseAPI::class)`。 + +示例使用这两张表: + +```kotlin +@DBRow("person") +@Serializable +data class Person(@PrimaryKey val id: Long?, val name: String, val age: Int) + +@DBRow("book") +@Serializable +data class Book(val title: String, val authorId: Long, val price: Double) +``` + +### 派生表 + +派生表以一个视图命名:像[视图](modify-database-and-transaction-cn.md#create-view---创建视图)一样用 `@DBView` 声明它的行,但数据库里 +不需要真的有这个视图,视图对象只是为派生表提供名字和列。`AS` 把一条返回该视图行类型的 _SELECT_ 变成派生表: + +```kotlin +@DBView("author_books") +@Serializable +data class AuthorBooks(val authorId: Long, val books: Long) + +@Serializable +data class PersonBooks(val name: String, val books: Long) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + lateinit var prolific: SelectStatement + lateinit var personBooks: SelectStatement + database { + val authorBooks = BookTable { table -> + table SELECT listOf(count(X) AS AuthorBooks::books) GROUP_BY authorId + } AS AuthorBooksView + // SELECT authorId,books FROM (SELECT authorId,count(*) AS books FROM book GROUP BY book.authorId) AS author_books + // WHERE author_books.books>? + prolific = authorBooks SELECT WHERE(AuthorBooksView.books GT 1) + // SELECT person.name,author_books.books FROM person INNER JOIN (SELECT ...) AS author_books ON person.id=author_books.authorId + personBooks = (FROM(PersonTable) INNER_JOIN authorBooks ON (PersonTable.id EQ AuthorBooksView.authorId)) SELECT X() + } +} +``` + +_SELECT_ 必须返回视图的行类型,这在编译期检查。派生表可以被多条查询读取:可以用任意子句、投影或结果列从中查询,也可以参与 Join, +但不能写入。一条查询不能读取同一个视图的两个派生表,因为它们的名字相同。 + +由一张表的行构成的派生表相当于给这张表起了另一个名字,表和自身 Join 时就需要这样做。同时用结果列给它的列也起别的名字,让查询能 +把它们和表自己的列区分开: + +```kotlin +@DBView("elder") +@Serializable +data class Elder(val elderName: String, val elderAge: Int) + +@Serializable +data class YoungerAndElder(val name: String, val elderName: String) + +database { + val elders = PersonTable { table -> + table SELECT listOf(name AS Elder::elderName, age AS Elder::elderAge) + } AS ElderView + // SELECT person.name,elder.elderName FROM person + // INNER JOIN (SELECT person.name AS elderName,person.age AS elderAge FROM person) AS elder ON elder.elderAge>person.age + pairs = (FROM(PersonTable) INNER_JOIN elders ON (ElderView.elderAge GT PersonTable.age)) SELECT X() +} +``` + +### IN 和 EXISTS + +带 _SELECT_ 的 `IN` 检查一个值是否在 _SELECT_ 返回的值之中。_SELECT_ 必须只返回一列、并且是同一类的值(数字、文本或 BLOB), +这在构建语句时检查,所以请用投影或结果列选取一个只有一个属性的类型: + +```kotlin +@Serializable +data class AuthorId(val authorId: Long) + +// SELECT id,name,age FROM person WHERE person.id IN (SELECT authorId FROM book WHERE book.price>?) +PersonTable SELECT WHERE(PersonTable.id IN (BookTable SELECT WHERE(BookTable.price GT 30.0))) +``` + +`EXISTS` 检查一条 _SELECT_ 是否返回了任意一行。由于 SQLlin 写列时都带上表名,_SELECT_ 可以引用外层查询的列,这会让它对外层查询的 +每一行执行一次: + +```kotlin +// SELECT id,name,age FROM person WHERE EXISTS (SELECT title,authorId,price FROM book WHERE book.authorId=person.id) +PersonTable SELECT WHERE(EXISTS(BookTable SELECT WHERE(BookTable.authorId EQ PersonTable.id))) +``` + +在 _SELECT_ 内部,两条查询都读取的表的列指的是 _SELECT_ 自己的,因为两者名字相同,所以它不能引用外层查询中这张表的行。 + +`NOT` 对条件取反,比如用 `NOT(EXISTS(select))` 找出没有书的人,或者 `NOT(PersonTable.id IN (select))`。它会给条件加上括号: +`NOT((age LT 18) OR (age GT 65))` 生成的是 `NOT (person.age?)`。 + +子查询同样可以用在 _UPDATE_ 和 _DELETE_ 的 _WHERE_ 中,可观察查询也会观察子查询读取的表。公用表表达式(`WITH`)暂不支持。 + +### 标量子查询 + +只返回一列的 _SELECT_ 可以作为一个值,也就是这一列在第一行的值;没有返回任何行时为 NULL。用 `select[R::property]` 指明这一列, +这个值就有了属性的类型,可以像任何[表达式](sql-functions-cn.md#表达式)一样参与比较、运算或被选取: + +```kotlin +@Serializable +data class AveragePrice(val value: Double?) + +// SELECT title,authorId,price FROM book WHERE book.price>(SELECT avg(book.price) AS value FROM book) +BookTable SELECT WHERE(BookTable.price GT (BookTable SELECT listOf(BookTable.avg(BookTable.price) AS AveragePrice::value))[AveragePrice::value]) +``` + +它的结果类型只能有这一个属性,这在构建语句时检查。它可能为 NULL,所以 `AS` 只能把它放进可空的属性。和其他表达式一样,它里面的 +值会写进 SQL。 + +## 可观察查询 + +`Database#observe` 把一条 _SELECT_ 变成它的结果的 `Flow`:开始收集时发出当前结果;之后每当经由这个数据库执行的语句修改了 +这条 _SELECT_ 读取的表,并且结果确实有变化时,再次发出。它是实验性 API,使用时需要 `@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample(scope: CoroutineScope) { + val adults: Flow> = database.observe { + PersonTable SELECT WHERE(PersonTable.age GTE 18) + } + scope.launch { + adults.collect { people -> + // 当前的成年人,以及之后 person 表每次改变了这个结果之后的成年人 + } + } +} +``` + +代码块用来构建要观察的 _SELECT_,并且只应该构建它,因为每次查询结果时它都会重新执行。任何 _SELECT_ 都可以:Join 和组合查询会 +观察它们读取的所有表。查询默认在 `Dispatchers.IO` 上执行,也可以传入别的上下文:`database.observe(context) { ... }`。 +连续的多次修改可能只引起一次查询。 + +修改由 SQLite 自己计数:某张表第一次被观察时,SQLlin 会在它上面创建 TEMP 触发器。所以外键动作(比如 `ON DELETE CASCADE`) +修改了被观察的表中的行,即使没有任何语句提到这张表,也会被计入;而回滚的事务不会被计入,没有匹配到任何行的 _UPDATE_ 或 +_DELETE_ 也不会。 + +只有经由同一个 `Database` 实例的修改才能被感知:同一个文件上的其他实例或连接(比如用 _sqllin-driver_ 打开的连接),以及其他 +进程所做的修改,都感知不到。 + +要组合几条 _SELECT_ 的结果,可以用 kotlinx.coroutines 的 `combine` 组合观察它们的几个 Flow。由于每个 Flow 各自执行查询,一次 +同时影响两边的表的修改,可能会短暂地呈现一边的新结果和另一边的旧结果。如果结果来自几张表,用一条带 Join 的 _SELECT_ 可以一次 +得到。 + +## 全文搜索 + +全文搜索(FTS)表会为文本建立索引,让查询像搜索引擎一样找出包含某些词的行。SQLlin 支持 SQLite 的 FTS4 和 FTS3 表,SQLlin +支持的所有平台上的 SQLite 都有它们,Android 也不例外。它们是实验性 API,使用时需要 `@OptIn(ExperimentalDSLDatabaseAPI::class)`。 + +### 声明 FTS 表 + +给 `@DBRow` 类加上 `@Fts4`,_sqllin-processor_ 就会像为普通表一样为它生成表对象: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow("articles") +@Fts4(tokenizer = FtsTokenizer.PORTER, prefix = [2], notIndexed = ["note"]) +@Serializable +data class Article( + @PrimaryKey val rowid: Long?, + val title: String, + val body: String, + val note: String?, +) +``` + +`CREATE(ArticleTable)` 用下面的语句创建它: + +```sql +CREATE VIRTUAL TABLE articles USING fts4(title,body,note,tokenize=porter,prefix="2",notindexed=note) +``` + +FTS 为文本建立索引,所以列就是类中 `String` 和 `String?` 类型的属性;FTS 表的列没有类型和约束,所以它们也不接受约束注解。 +行的 id,也就是 rowid,通过一个名为 `rowid` 或 `docid`、类型为 `Long` 或 `Long?` 的 `@PrimaryKey` 属性读写,它不是一列: +`Long?` 表示由 SQLite 分配,和普通表的 `INTEGER PRIMARY KEY` 一样;`Long` 表示由你提供。没有这个属性时,行仍然有 rowid, +只是你读不到。 + +`@Fts4` 的选项有: + +* `tokenizer`,把文本切分成词的分词器:`FtsTokenizer.SIMPLE` 是默认值,把 ASCII 转为小写;`PORTER` 还会把英文单词还原为 + 词干,于是 "running" 能匹配 "run";`UNICODE61` 按 Unicode 切分并转为小写,还会去掉变音符号 +* `tokenizerArgs`,分词器的参数,比如给 `UNICODE61` 的 `["remove_diacritics=2"]` +* `prefix`,要建立索引的前缀长度,可以加快 `kot*` 这样的前缀查询 +* `notIndexed`,只存储值、不参与搜索的列 + +`@Fts3` 声明 FTS3 表,它是 FTS4 的旧版本,只有 `tokenizer` 和 `tokenizerArgs` 两个选项。除非表必须是 FTS3(比如你的应用的 +早期版本创建的表),否则请使用 FTS4。 + +FTS 表和普通表一样,用 _INSERT_、_UPDATE_、_DELETE_ 写入,用 _SELECT_ 读取。SQLite 不允许修改虚拟表的结构或给它建索引, +所以 `ALTER_ADD_COLUMN` 和 `CREATE_INDEX` 用在它上面会失败。要修改它的列或选项,请按照[重建表](modify-database-and-transaction-cn.md#重建表) +的方法重建:对于 FTS 表,`withName` 返回的是选项相同的 FTS 表。 + +### MATCH + +`MATCH` 找出在任意一列或某一列中匹配查询的行: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + lateinit var anyColumn: SelectStatement
+ lateinit var titles: SelectStatement
+ database { + ArticleTable { table -> + // SELECT rowid,title,body,note FROM articles WHERE articles MATCH ? + anyColumn = table SELECT WHERE(table MATCH "kotlin coroutines") + // SELECT rowid,title,body,note FROM articles WHERE articles.title MATCH ? + titles = table SELECT WHERE(title MATCH "kotlin") + } + } +} +``` + +查询用的是 FTS 自己的语言,其中的词会和文本一样经过表的分词器切分和转换: + +* `kotlin coroutines`:同时包含这两个词的行 +* `kotlin OR java`:包含其中任意一个词的行;`OR` 必须大写 +* `"kotlin multiplatform"`:包含这个短语的行 +* `corout*`:包含以 `corout` 开头的词的行 +* `title:kotlin`:`title` 列中包含这个词的行 +* `kotlin NEAR java`:这两个词彼此靠近的行 + +只用这些写法的查询在所有平台上的效果都一样。除此之外,FTS 有两种语法。Android 的 SQLite 使用标准语法,其中 `-java` 表示 +排除包含 `java` 的行;JVM 驱动和 Apple 平台的 SQLite 使用增强语法,它增加了 `AND`、`NOT` 和括号,而 `-java` 只被当作 `java` +这个词本身。在 Linux 和 Windows 上,取决于你的应用链接的 SQLite 是如何编译的。 + +### 描述匹配 + +在带 `MATCH` 的 _SELECT_ 中,FTS 表对象的函数 `snippet`、`offsets` 和 `matchinfo` 描述每一行是如何匹配的。它们通过 +[结果列](#结果列)选取: + +```kotlin +@Serializable +data class SearchResult(val title: String, val excerpt: String) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + lateinit var results: SelectStatement + database { + ArticleTable { table -> + // SELECT title,snippet(articles,'','','...',-1,-15) AS excerpt FROM articles WHERE articles MATCH ? + results = table SELECT listOf(table.snippet() AS SearchResult::excerpt) WHERE (table MATCH "coroutines") + } + } + // 例如:SearchResult(title = "Kotlin Coroutines", excerpt = "Kotlin Coroutines") +} +``` + +* `snippet(start, end, ellipsis, column, tokens)`:匹配到的词周围的文本,词的前后分别加上 `start` 和 `end`;取自 `column` + 列,默认取自匹配得最好的列 +* `offsets()`:匹配到的词所在的位置,是一段文本,每处匹配四个数字:列、查询中的第几个词、匹配的字节偏移量和字节数 +* `matchinfo(format)`:匹配的统计数据,是由本机字节序的 32 位无符号整数组成的 `ByteArray`,其含义见 SQLite 的 `matchinfo` + 文档 + +FTS4 和 FTS3 不会给匹配结果排序:_SELECT_ 按 rowid 的顺序返回它们。要排序的话,请在 Kotlin 中根据 `matchinfo` 计算分数。 +能给匹配结果排序的 FTS5 和 R*Tree 表暂不支持,因为 Android 的 SQLite 没有它们。 + +### 观察 FTS 表 + +FTS 表是虚拟表,无法拥有[可观察查询](#可观察查询)用来计数修改的触发器。所以 `Database#observe` 改为根据 SQLlin 对它执行的 +_INSERT_、_UPDATE_ 和 _DELETE_ 来计数;由于触发器可能写入它,对任何带触发器的表执行的这些语句也会计入:用其他 SQL 所做的修改 +感知不到,视图所读取的 FTS 表的修改也感知不到。回滚的事务仍会计入其中的语句,这只会让查询多执行一次。 ## 最后 diff --git a/sqllin-dsl/doc/advanced-query.md b/sqllin-dsl/doc/advanced-query.md index 00e426e6..e346551a 100644 --- a/sqllin-dsl/doc/advanced-query.md +++ b/sqllin-dsl/doc/advanced-query.md @@ -4,159 +4,80 @@ We have learned basic querying and using SQL functions in querying conditions. Let's learn some advanced skills of querying. -## Unions +## Compound Queries -The _UNION_ operator is used for merge two _SELECT_ statements' results and these results must be of the same type. - -In SQL, the _UNION_ operator between with the two _SELECT_ statements, but in SQLlin, we use a higher-order function to -implement _UNION_: +A compound query combines the rows of two _SELECT_ statements, as SQL's `UNION`, `UNION ALL`, `INTERSECT` and +`EXCEPT` do. In SQLlin they are infix functions written between the two statements, as in SQL. They are experimental, +so opt in with `@OptIn(ExperimentalDSLDatabaseAPI::class)`: ```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) fun sample() { lateinit var selectStatement: SelectStatement database { PersonTable { table -> - selectStatement = UNION { - table SELECT WHERE (age GTE 5) - table SELECT WHERE (length(name) LTE 8) - } + // SELECT name,age FROM person WHERE age < ? UNION SELECT name,age FROM person WHERE age >= ? + selectStatement = (table SELECT WHERE(age LT 18)) UNION (table SELECT WHERE(age GTE 65)) } } } ``` -You just need to write your _SELECT_ statements in the `UNION {...}` block. There must be at least two _SELECT_ statements -inside the `UNION {...}` block, if not, you will get a `IllegalStateException` at runtime. +`UNION` combines the rows without duplicates and `UNION_ALL` keeps them, `INTERSECT` keeps the rows both statements +return, and `EXCEPT` the rows of the first that the second doesn't return. Put each _SELECT_ in parentheses, as +_SELECT_ and these operators are infix functions of the same precedence; without them, the code doesn't compile. -If you want to use _UNION_ and _UNION ALL_ interchangeably, just use `UNION {...}` or `UNION_ALL {...}` block nesting: +Both statements have to read rows of the same type, which is checked at compile time. They can come from different +tables, through a projection, result columns or a join, as described below: ```kotlin -fun sample() { - lateinit var selectStatement: SelectStatement - database { - PersonTable { table -> - selectStatement = UNION { - table SELECT WHERE (age GTE 5) - UNION_ALL { - table SELECT WHERE (length(name) LTE 8) - table SELECT WHERE (name EQ "Tom") - } - } - } - } -} -``` - -Above code equals the SQL: - -```roomsql -SELECT * FROM person WHERE age >= 5 -UNION -SELECT * FROM person WHERE length(name) <= 8 -UNION ALL -SELECT * FROM person WHERE name = "Tom" -``` - -## Subqueries - -SQLlin doesn't support subqueries yet, we will develop as soon as possible. - -## Join - -SQLlin supports joining tables now. - -We need other two database entities: - -```kotlin -@DBRow("transcript") -@Serializable -data class Transcript( - val name: String?, - val math: Int, - val english: Int, -) - -@Serializable -data class Student( - val name: String?, - val age: Int?, - val math: Int, - val english: Int, -) - -@Serializable -data class CrossJoinStudent( - val age: Int?, - val math: Int, - val english: Int, -) +// The names of the people who are both students and teachers +(StudentTable SELECT X()) INTERSECT (TeacherTable SELECT X()) ``` -The `Transcript` represents an other table. And the `Student` represents the join querying results' type (so `Student` -doesn't need to be annotated `@DBRow`), it owns all column names that belong to `Person` and `Transcript`. - -### Cross Join +Chained operators are evaluated from left to right, as SQLite does, and parentheses group them. SQLite has no +parentheses in compound queries, so SQLlin writes a grouped compound as a subquery: ```kotlin -fun joinSample() { - db { - PersonTable { table -> - table SELECT CROSS_JOIN(TranscriptTable) - } - } -} +a UNION b UNION_ALL c // (a UNION b) UNION ALL c +a UNION (b UNION_ALL c) // a UNION SELECT * FROM (b UNION ALL c) ``` -The `CROSS_JOIN` function receives one or multiple `Table`s as parameters. In normal _SELECT_ statements, the statements' querying results' type is -depended on the `Table` that be generated by _sqllin-processor_, but _JOIN_ operator will change it to specific type. In above sample, `CROSS_JOIN` changes -the type to `CrossJoinStudent`. - -Note, because _CROSS JOIN_ owns feature in SQL. If the columns that be queried by a _SELECT_ statement that with _CROSS JOIN_ clause include the same -name columns in the two tables, this will cause the querying to fail. Because a class isn't allowed to have multiple properties those have same names, _sqllin-dsl_ -doesn't support the _CROSS JOIN_ with columns of the same name. - -### Inner Join +_ORDER BY_ and _LIMIT_ written after the compound apply to all of it. A statement that ends with its own _ORDER BY_ or +_LIMIT_ keeps them to itself, as SQLlin writes it as a subquery too: ```kotlin -fun joinSample() { - db { - PersonTable { table -> - table SELECT INNER_JOIN(TranscriptTable) USING name - table SELECT NATURAL_INNER_JOIN(TranscriptTable) - table SELECT INNER_JOIN(TranscriptTable) ON (name EQ TranscriptTable.name) - } - } -} +// The 3 oldest and the 3 youngest people +(table SELECT ORDER_BY(age to DESC) LIMIT 3) UNION_ALL (table SELECT ORDER_BY(age to ASC) LIMIT 3) +// The children and the seniors together, by name, 10 at a time +(table SELECT WHERE(age LT 18)) UNION (table SELECT WHERE(age GTE 65)) ORDER_BY (name to ASC) LIMIT 10 OFFSET 20 ``` -The `INNER_JOIN` is similar to `CROSS_JOIN`, the deference is `INNER_JOIN` need to connect a `USING` or `ON` clause. If a _INNER JOIN_ statement -without the `USING` or `ON` clause, it is incomplete, but your code still be compiled and will do nothing at runtime. - -The `NATURAL_INNER_JOIN` will produce a complete _SELECT_ statement (same with `CROSS_JOIN`). So, you can't add a `USING` or `ON` clause behind it, this is -guaranteed by Kotlin compiler. +The statements become part of the compound, so they no longer run on their own, and their results can't be read. -Note, the behavior of a `INNER_JOIN` clause with a `ON` clause is same with `CROSS_JOIN`, you can't select a column that has same name in two tables. +### The deprecated `UNION {...}` block -The `INNER_JOIN` have an alias that named `JOIN`, and `NATURAL_INNER_JOIN` also have an alias that named `NATURAL_JOIN`. This is just like you can -bypass a `INNER` keyword in SQL's inner join querying. +Earlier versions combined _SELECT_ statements with `UNION {...}` and `UNION_ALL {...}` blocks. **They are deprecated +and will be removed in a future version**, because they have bugs that won't be fixed: +- A block's result is typed by the table's row type, but its rows are read into the first _SELECT_'s result type. So + a union of projections, result columns or joins compiles, and then fails with a `ClassCastException` when its + results are used. +- A nested block is flattened into the SQL, which SQLite evaluates from left to right, so + `UNION { a; UNION_ALL { b; c } }` runs as `(a UNION b) UNION ALL c`, not as its nesting suggests. -### Left Outer Join +Use the infix operators instead: ```kotlin -fun joinSample() { - db { - PersonTable { table -> - table SELECT LEFT_OUTER_JOIN(TranscriptTable) USING name - table SELECT NATURAL_LEFT_OUTER_JOIN(TranscriptTable) - table SELECT LEFT_OUTER_JOIN(TranscriptTable) ON (name EQ TranscriptTable.name) - } - } +// Deprecated +UNION { + table SELECT WHERE(age LT 18) + table SELECT WHERE(age GTE 65) } +// Instead +(table SELECT WHERE(age LT 18)) UNION (table SELECT WHERE(age GTE 65)) ``` -The `LEFT_OUTER_JOIN`'s usage is very similar with `INNER_JOIN`, the difference just is their API names. - ## Projection A `SELECT` reads each row into the table's own row type. To read only some of the columns, declare a narrower @@ -175,7 +96,7 @@ fun sample() { PersonTable { table -> // SELECT name FROM person names = table SELECT X() - // SELECT name FROM person WHERE age >= ? ORDER BY name LIMIT 10 + // SELECT name FROM person WHERE age >= ? ORDER BY person.name LIMIT 10 adultNames = table SELECT WHERE(age GTE 18) ORDER_BY name LIMIT 10 } } @@ -219,7 +140,7 @@ fun sample() { lateinit var adults: SelectStatement database { PersonTable { table -> - // SELECT name,count(*) AS people,max(age) AS maxAge FROM person GROUP BY name + // SELECT name,count(*) AS people,max(person.age) AS maxAge FROM person GROUP BY person.name stats = table SELECT listOf(count(X) AS NameStats::people, max(age) AS NameStats::maxAge) GROUP_BY name // SELECT count(*) AS people FROM person WHERE age >= ? adults = table SELECT (count(X) AS PersonCount::people) WHERE (age GTE 18) @@ -257,8 +178,387 @@ A result type that breaks one of these rules makes the `SELECT` throw an `Illega built, before it runs, as does a property given two expressions. Only whether `GROUP_BY` follows can't be known yet then, so a property that needs it is reported when the database scope ends, before any statement of the scope runs. -A property given an expression is found by its name, so it can't be renamed with `@SerialName`. Result columns select from -a single table: they can't be used with a join yet, nor can arithmetic, `CASE` or subqueries. +A property given an expression is found by its name, so it can't be renamed with `@SerialName`. Result columns can select +from a [join](#join) too. Arithmetic, `CASE` and subqueries that give a single value can't be used as expressions yet. + +## Join + +A join reads the rows of several relations, tables, views or derived tables, together. Start one with +`FROM(relation)`, extend it with join operators, and select from it as from a table. Joins are experimental, so opt in +with `@OptIn(ExperimentalDSLDatabaseAPI::class)`. + +The examples use another table, and types for the rows of joins, which don't need `@DBRow`: + +```kotlin +@DBRow("transcript") +@Serializable +data class Transcript( + val name: String?, + val math: Int, + val english: Int, +) + +@Serializable +data class Student( + val name: String?, + val age: Int?, + val math: Int, + val english: Int, +) + +@Serializable +data class StudentScore( + val name: String?, + val tests: Long, + val bestMath: Int?, +) +``` + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun joinSample() { + lateinit var students: SelectStatement + lateinit var scores: SelectStatement + database { + // SELECT name,person.age,transcript.math,transcript.english FROM person INNER JOIN transcript USING (name) + students = (FROM(PersonTable) INNER_JOIN TranscriptTable USING PersonTable.name) SELECT X() + // SELECT person.name AS name,count(transcript.math) AS tests,max(transcript.math) AS bestMath + // FROM person LEFT OUTER JOIN transcript ON person.name = transcript.name GROUP BY person.name + scores = (FROM(PersonTable) LEFT_OUTER_JOIN TranscriptTable ON (PersonTable.name EQ TranscriptTable.name)) SELECT listOf( + PersonTable.name AS StudentScore::name, + PersonTable.count(TranscriptTable.math) AS StudentScore::tests, + PersonTable.max(TranscriptTable.math) AS StudentScore::bestMath, + ) GROUP_BY PersonTable.name + } +} +``` + +The join operators are: + +| Operator | Followed by | Returns | +| --- | --- | --- | +| `INNER_JOIN` | `ON` or `USING` | the rows of both that match | +| `LEFT_OUTER_JOIN` | `ON` or `USING` | those, and the rows of the relations before it that match none | +| `RIGHT_OUTER_JOIN` | `ON` or `USING` | those, and the rows of the joined relation that match none | +| `FULL_OUTER_JOIN` | `ON` or `USING` | those, and the rows of each that match none | +| `CROSS_JOIN` | nothing | every row of the relations before it with every row of the joined one | +| `NATURAL_JOIN`, `NATURAL_LEFT_OUTER_JOIN`, `NATURAL_RIGHT_OUTER_JOIN`, `NATURAL_FULL_OUTER_JOIN` | nothing | as above, matching the columns of the same names | + +`RIGHT_OUTER_JOIN`, `FULL_OUTER_JOIN` and their NATURAL forms need SQLite 3.39.0, which Android has from API 34 on, so +they are marked with `@PlatformDependentSQLiteAPI`, which you have to opt in to, as described in +[APIs That Depend on the SQLite Version](modify-database-and-transaction.md#apis-that-depend-on-the-sqlite-version). + +Joins are applied in the order they are written, and each can have its own condition: +`FROM(a) INNER_JOIN b ON (...) LEFT_OUTER_JOIN c ON (...)`. A relation can only be joined once: to join a table with +itself, join a [derived table](#derived-tables) of its rows, which gives it another name. + +A join's rows have no type of their own, so a _SELECT_ of a join names the type it reads them into: with +`X()`, with a clause function such as `WHERE(...)`, or with result columns. That type is checked against the +joined relations when the statement is built: + +* Every property has to name a column of one of them. Columns of the same name in two relations can't be told apart, + unless `USING` or a NATURAL join merges them, so select one of them with `AS`, as `PersonTable.name AS StudentScore::name` + above. +* A property has to be nullable when its column or expression can be NULL. An outer join leaves the columns of a + relation NULL in the rows where it finds no matching row, and so every expression of them but `count`. + +SQLlin writes the columns with their relations' names, so that a condition, a function, _GROUP BY_ or _ORDER BY_ can +name a column that another relation has too. An observed query of a join watches all its relations. + +### The Old Join API + +Before 2.5.0, joins were written as clauses of _SELECT_: `PersonTable SELECT INNER_JOIN(TranscriptTable) USING name`, +with `CROSS_JOIN`, `NATURAL_JOIN`, `LEFT_OUTER_JOIN` and the others. They still work, but they don't check the type +their rows are read into, can't join more than two relations each with its own condition, and can't select result +columns. **They will be removed in the next version after 2.5.0**, so write joins as above. + +## Subqueries + +A _SELECT_ can be part of another query: as a derived table, which the FROM clause or a join reads like a table, or as +the subquery of an `IN` or `EXISTS` condition. It becomes part of that query, so it no longer runs on its own. +Subqueries are experimental, so opt in with `@OptIn(ExperimentalDSLDatabaseAPI::class)`. + +The examples use these tables: + +```kotlin +@DBRow("person") +@Serializable +data class Person(@PrimaryKey val id: Long?, val name: String, val age: Int) + +@DBRow("book") +@Serializable +data class Book(val title: String, val authorId: Long, val price: Double) +``` + +### Derived Tables + +A derived table is named after a view: declare its rows with `@DBView`, as for a [view](modify-database-and-transaction.md#create-view---creating-views), +but no such view has to exist in the database, as the view object only gives the derived table its name and its +columns. `AS` makes a _SELECT_ of the view's rows a derived table: + +```kotlin +@DBView("author_books") +@Serializable +data class AuthorBooks(val authorId: Long, val books: Long) + +@Serializable +data class PersonBooks(val name: String, val books: Long) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + lateinit var prolific: SelectStatement + lateinit var personBooks: SelectStatement + database { + val authorBooks = BookTable { table -> + table SELECT listOf(count(X) AS AuthorBooks::books) GROUP_BY authorId + } AS AuthorBooksView + // SELECT authorId,books FROM (SELECT authorId,count(*) AS books FROM book GROUP BY book.authorId) AS author_books + // WHERE author_books.books>? + prolific = authorBooks SELECT WHERE(AuthorBooksView.books GT 1) + // SELECT person.name,author_books.books FROM person INNER JOIN (SELECT ...) AS author_books ON person.id=author_books.authorId + personBooks = (FROM(PersonTable) INNER_JOIN authorBooks ON (PersonTable.id EQ AuthorBooksView.authorId)) SELECT X() + } +} +``` + +The _SELECT_ has to return the view's rows, which is checked at compile time. A derived table can be selected from, +with any clause, a projection or result columns, and joined, by several queries, but not written to. A query can't read +two derived tables of the same view, as they would have the same name. + +A derived table of a table's rows gives the table another name, which a query that joins the table with itself needs. +Give its columns other names too, with result columns, so that the query can tell them from the table's: + +```kotlin +@DBView("elder") +@Serializable +data class Elder(val elderName: String, val elderAge: Int) + +@Serializable +data class YoungerAndElder(val name: String, val elderName: String) + +database { + val elders = PersonTable { table -> + table SELECT listOf(name AS Elder::elderName, age AS Elder::elderAge) + } AS ElderView + // SELECT person.name,elder.elderName FROM person + // INNER JOIN (SELECT person.name AS elderName,person.age AS elderAge FROM person) AS elder ON elder.elderAge>person.age + pairs = (FROM(PersonTable) INNER_JOIN elders ON (ElderView.elderAge GT PersonTable.age)) SELECT X() +} +``` + +### IN and EXISTS + +`IN` with a _SELECT_ checks whether a value is among those the _SELECT_ returns. The _SELECT_ has to return a single +column of values of the same kind, numbers, text or BLOBs, which is checked when the statement is built, so select a +type with a single property, with a projection or with result columns: + +```kotlin +@Serializable +data class AuthorId(val authorId: Long) + +// SELECT id,name,age FROM person WHERE person.id IN (SELECT authorId FROM book WHERE book.price>?) +PersonTable SELECT WHERE(PersonTable.id IN (BookTable SELECT WHERE(BookTable.price GT 30.0))) +``` + +`EXISTS` checks whether a _SELECT_ returns any row. The _SELECT_ can refer to the columns of the query it is in, which +makes it run for each row of that query, as SQLlin writes columns with their tables' names: + +```kotlin +// SELECT id,name,age FROM person WHERE EXISTS (SELECT title,authorId,price FROM book WHERE book.authorId=person.id) +PersonTable SELECT WHERE(EXISTS(BookTable SELECT WHERE(BookTable.authorId EQ PersonTable.id))) +``` + +Inside the _SELECT_, a column of a table that both queries read is the _SELECT_'s own, as both have the same name, so it +can't refer to the other query's rows of that table. + +`NOT` negates a condition, such as `NOT(EXISTS(select))` for the persons without books, or +`NOT(PersonTable.id IN (select))`. It puts the condition in parentheses: `NOT((age LT 18) OR (age GT 65))` is +`NOT (person.age?)`. + +Subqueries work in the _WHERE_ of _UPDATE_ and _DELETE_ too, and an observed query watches the tables its subqueries +read. Common table expressions (`WITH`) aren't supported yet. + +### Scalar Subqueries + +A _SELECT_ of a single column can be a value, which is the column's value in its first row, or NULL if it returns no +rows. `select[R::property]` names that column, so that the value has the property's type, and can be compared, +computed with or selected like any [expression](sql-functions.md#expressions): + +```kotlin +@Serializable +data class AveragePrice(val value: Double?) + +// SELECT title,authorId,price FROM book WHERE book.price>(SELECT avg(book.price) AS value FROM book) +BookTable SELECT WHERE(BookTable.price GT (BookTable SELECT listOf(BookTable.avg(BookTable.price) AS AveragePrice::value))[AveragePrice::value]) +``` + +Its result type has to have that property only, which is checked when the statement is built. As it can be NULL, `AS` +only selects it into a nullable property. Its values are written into the SQL, as the values of any expression are. + +## Observed Queries + +`Database#observe` turns a _SELECT_ into a `Flow` of its results: it emits them when it is collected, and again whenever +a statement run through the database changes a table the _SELECT_ reads, if the results differ. It is experimental, so +opt in with `@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample(scope: CoroutineScope) { + val adults: Flow> = database.observe { + PersonTable SELECT WHERE(PersonTable.age GTE 18) + } + scope.launch { + adults.collect { people -> + // The adults now, and after every change to the person table that changes them + } + } +} +``` + +The block builds the _SELECT_ to observe, and should only build it, as it runs again each time the results are queried. +Any _SELECT_ works: a join or a compound query watches all of the tables it reads. The queries run on `Dispatchers.IO`, +unless you pass another context: `database.observe(context) { ... }`. Several changes in a row may lead to a single +query. + +The changes are counted by SQLite itself, through TEMP triggers SQLlin creates on a table the first time a query that +reads it is observed. So the rows that a foreign key action, such as `ON DELETE CASCADE`, changes in an observed table +count, though no statement names it, while a rolled back transaction doesn't, and neither does an _UPDATE_ or _DELETE_ +that matches no rows. + +Only the changes made through the same `Database` instance are seen: not those of another instance or connection on +the same file, such as one opened with _sqllin-driver_, nor those of another process. + +To combine the results of several _SELECT_ statements, combine the flows that observe them, with `combine` from +kotlinx.coroutines. As each runs its query on its own, a change to the tables of both may briefly show the new results of +one with the old results of the other. When the results come from several tables, a single _SELECT_ with a join gives +them at once. + +## Full-Text Search + +A full-text search (FTS) table indexes text, so that a query finds the rows containing some words, as a search engine +does. SQLlin supports SQLite's FTS4 and FTS3 tables, which SQLite has on every platform SQLlin supports, Android's +included. They are experimental, so opt in with `@OptIn(ExperimentalDSLDatabaseAPI::class)`. + +### Declaring an FTS Table + +Annotate a `@DBRow` class with `@Fts4`, and _sqllin-processor_ generates a table object for it, as for any table: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow("articles") +@Fts4(tokenizer = FtsTokenizer.PORTER, prefix = [2], notIndexed = ["note"]) +@Serializable +data class Article( + @PrimaryKey val rowid: Long?, + val title: String, + val body: String, + val note: String?, +) +``` + +`CREATE(ArticleTable)` creates it with: + +```sql +CREATE VIRTUAL TABLE articles USING fts4(title,body,note,tokenize=porter,prefix="2",notindexed=note) +``` + +FTS indexes text, so the columns are the class's `String` and `String?` properties, and as an FTS table's columns have +no types or constraints, they take no constraint annotations. A row's id, its rowid, is read and written through a +`@PrimaryKey` property of type `Long` or `Long?` named `rowid` or `docid`, which isn't a column: `Long?` leaves it for +SQLite to assign, as for a table's `INTEGER PRIMARY KEY`, while `Long` has you supply it. Without one, the rows still +have rowids, which you can't read. + +The options of `@Fts4` are: + +* `tokenizer`, which splits the text into terms: `FtsTokenizer.SIMPLE`, the default, which folds ASCII to lower case; + `PORTER`, which also reduces English words to their stems, so that "running" matches "run"; and `UNICODE61`, which + splits and folds all of Unicode, and removes diacritics +* `tokenizerArgs`, the tokenizer's arguments, such as `["remove_diacritics=2"]` for `UNICODE61` +* `prefix`, the lengths of the prefixes to index, which make prefix queries such as `kot*` faster +* `notIndexed`, the columns whose values are stored, but not searched + +`@Fts3` declares an FTS3 table, the older version of FTS4, with only `tokenizer` and `tokenizerArgs`. Prefer FTS4, +unless the table has to be FTS3, such as one an earlier version of your app created. + +An FTS table is written to with _INSERT_, _UPDATE_ and _DELETE_, and read with _SELECT_, like any table. SQLite doesn't +let a virtual table be altered or indexed, so `ALTER_ADD_COLUMN` and `CREATE_INDEX` fail on it. To change its columns or +options, rebuild it as described in [Rebuilding a Table](modify-database-and-transaction.md#rebuilding-a-table): for an +FTS table, `withName` returns an FTS table with the same options. + +### MATCH + +`MATCH` finds the rows that match a query, in any column, or in one: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + lateinit var anyColumn: SelectStatement
+ lateinit var titles: SelectStatement
+ database { + ArticleTable { table -> + // SELECT rowid,title,body,note FROM articles WHERE articles MATCH ? + anyColumn = table SELECT WHERE(table MATCH "kotlin coroutines") + // SELECT rowid,title,body,note FROM articles WHERE articles.title MATCH ? + titles = table SELECT WHERE(title MATCH "kotlin") + } + } +} +``` + +The query is written in FTS's own language. Its terms are split and folded by the table's tokenizer, as the text is: + +* `kotlin coroutines`: rows with both terms +* `kotlin OR java`: rows with either term; `OR` has to be in upper case +* `"kotlin multiplatform"`: rows with the phrase +* `corout*`: rows with a term starting with `corout` +* `title:kotlin`: rows with the term in the column `title` +* `kotlin NEAR java`: rows with the terms close to each other + +Queries made of these work the same on every platform. Beyond them, FTS has two syntaxes. Android's SQLite has the +standard one, where `-java` leaves out the rows with the term `java`. The SQLite of the JVM driver and of Apple's +platforms has the enhanced one, which adds `AND`, `NOT` and parentheses, and reads `-java` as the term `java` itself. On +Linux and Windows, it depends on how the SQLite your app links was compiled. + +### Describing the Matches + +In a _SELECT_ with `MATCH`, the functions `snippet`, `offsets` and `matchinfo` of an FTS table object describe how each +row matched. They are selected with [result columns](#result-columns): + +```kotlin +@Serializable +data class SearchResult(val title: String, val excerpt: String) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + lateinit var results: SelectStatement + database { + ArticleTable { table -> + // SELECT title,snippet(articles,'','','...',-1,-15) AS excerpt FROM articles WHERE articles MATCH ? + results = table SELECT listOf(table.snippet() AS SearchResult::excerpt) WHERE (table MATCH "coroutines") + } + } + // For example: SearchResult(title = "Kotlin Coroutines", excerpt = "Kotlin Coroutines") +} +``` + +* `snippet(start, end, ellipsis, column, tokens)`: the text around the matched terms, with `start` and `end` around + them, from `column`, or by default from the column that matches best +* `offsets()`: where the matched terms are, as a text of four numbers per match: the column, the term of the query, and + the byte offset and size of the match +* `matchinfo(format)`: statistics of the match, as a `ByteArray` of 32-bit unsigned integers in the machine's byte + order, described by SQLite's documentation of `matchinfo` + +FTS4 and FTS3 don't rank the matches: a _SELECT_ returns them in the order of their rowids. To rank them, compute a +score from `matchinfo` in Kotlin. FTS5, which ranks matches, and R*Tree tables aren't supported yet, as Android's +SQLite doesn't have them. + +### Observing an FTS Table + +An FTS table is a virtual table, which can't have the triggers [observed queries](#observed-queries) count changes with. +So `Database#observe` counts the changes to it from SQLlin's _INSERT_, _UPDATE_ and _DELETE_ on it instead, and, as a +trigger can write it, from those on any table that has triggers: a change made with other SQL isn't seen, and neither +is a change to an FTS table that a view reads. A rolled back transaction still counts its statements, which only runs +the query once more. ## Finally diff --git a/sqllin-dsl/doc/getting-start-cn.md b/sqllin-dsl/doc/getting-start-cn.md index f08a727e..22c3b643 100644 --- a/sqllin-dsl/doc/getting-start-cn.md +++ b/sqllin-dsl/doc/getting-start-cn.md @@ -138,7 +138,7 @@ val database = Database( ### 使用 DSLDBConfiguration 进行类型安全的模式管理 -除此之外,你还可以使用新的试验性 API `DSLDBConfiguration`,它允许你在 `create` 和 `upgrade` 回调中使用类型安全的 SQL DSL,而不是原始的 SQL 字符串: +除此之外,你还可以使用 `DSLDBConfiguration`,它允许你在 `create` 和 `upgrade` 回调中使用类型安全的 SQL DSL,而不是原始的 SQL 字符串: ```kotlin import com.ctrip.sqllin.driver.DSLDBConfiguration @@ -176,7 +176,7 @@ val database = Database( 通常你只需要在你的组件的生命周期内创建一个 `Database` 对象,所以你需要在组件的生命周期结束时手动关闭数据库: -> 注意: `DSLDBConfiguration` 处于实验性阶段,但当其稳定后会彻底取代 `DatabaseConfiguration`, 也就是说在未来版本中 _sqllin-dsl_ 将不再支持使用 `DatabaseConfiguration` 创建 `Database` 实例。 +> 注意: `DSLDBConfiguration` 将会彻底取代 `DatabaseConfiguration`,也就是说在未来版本中 _sqllin-dsl_ 将不再支持使用 `DatabaseConfiguration` 创建 `Database` 实例。 ```kotlin override fun onDestroy() { @@ -429,6 +429,26 @@ data class Product( ) ``` +#### 冲突子句 + +`@PrimaryKey`、`@CompositePrimaryKey`、`@Unique` 和 `@CompositeUnique` 的 `onConflict` 参数,像约束的 `ON CONFLICT` 子句一样, +指定 SQLite 怎样处理会违反该约束的行。它是实验性 API: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow +@Serializable +data class Account( + @PrimaryKey(onConflict = OnConflict.REPLACE) val id: Long, + @Unique(onConflict = OnConflict.IGNORE) val email: String, +) +// Generated SQL: CREATE TABLE Account(id INTEGER PRIMARY KEY ON CONFLICT REPLACE,email TEXT NOT NULL UNIQUE ON CONFLICT IGNORE) +``` + +`REPLACE` 删除与新行冲突的行,`IGNORE` 跳过新行,`FAIL` 终止语句、但保留它之前做的修改,`ROLLBACK` 回滚整个事务。默认的 +`ABORT` 终止语句并撤销它的修改。复合主键或复合唯一组取任一属性上给出的值,组内属性不能给出不同的值。语句自己的 OR 子句(比如 +`INSERT_OR_IGNORE`)优先于它。 + #### @Default - 列默认值 使用 `@Default` 为 CREATE TABLE 语句中的列指定默认值。当插入行时未显式提供这些列的值时,SQLite 会自动使用这些默认值: @@ -505,6 +525,77 @@ val status: String val status: String ``` +#### @Check - 校验列值 + +使用 `@Check` 添加 `CHECK` 约束:一个每一行都必须满足的 SQL 表达式。每当 _INSERT_ 或 _UPDATE_ 写入一行时,SQLite 都会对它 +求值,结果为假就拒绝这一行。这样无论是哪里的代码写入,不合法的数据都进不了这张表。`@Check` 是实验性 API,使用时需要 +`@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow +@Check("endDate IS NULL OR endDate >= startDate", constraintName = "valid_period") +@Serializable +data class Membership( + @PrimaryKey(autoIncrement = true) val id: Long?, + @Check("length(trim(name)) > 0") val name: String, + @Check("level IN (0, 1, 2)") val level: Int, + @Check("age BETWEEN 0 AND 150") val age: Int?, + val startDate: String, + val endDate: String?, +) +// 生成的 SQL:CREATE TABLE Membership( +// id INTEGER PRIMARY KEY AUTOINCREMENT, +// name TEXT NOT NULL CHECK (length(trim(name)) > 0), +// level INT NOT NULL CHECK (level IN (0, 1, 2)), +// age INT CHECK (age BETWEEN 0 AND 150), +// startDate TEXT NOT NULL, +// endDate TEXT, +// CONSTRAINT valid_period CHECK (endDate IS NULL OR endDate >= startDate) +// ) +``` + +`@Check` 写在属性上,就是给这一列加约束;写在类上,就是给整张表加约束。SQLite 对两者的检查方式相同,两者也都可以引用任意列, +所以比较多列的约束写在类上更清楚。两者都可以重复使用,每个约束都会被检查。 + +**给约束命名:** + +`constraintName` 给约束命名,之后某一行违反它时,SQLite 会报出这个名字:`CHECK constraint failed: valid_period`。没有名字的 +约束,SQLite 3.34 起报的是它的表达式,更早的版本报的则是表名,Android 在 API 34 之前就是这样,这时只有名字能区分一张表的各个 +约束。名字必须是 SQL 标识符,同一张表的 CHECK 约束名不能重复。 + +**NULL 会通过:** + +只有表达式的结果为假,这一行才算违反约束;结果为 NULL 时,这一行会通过。所以 `@Check("age BETWEEN 0 AND 150")` 会放行为 +NULL 的 age;要拦住 NULL,请把属性声明为非空类型。看起来正确的表达式也可能因此失效: + +```kotlin +// ❌ 不是日期的文本都能通过:对它们 date(day) 是 NULL,比较的结果也是 NULL +@Check("date(day) = day") +val day: String? + +// ✅ 能拦住它们:IS 把 NULL 当作一个值来比较 +@Check("day IS NULL OR date(day) IS day") +val day: String? +``` + +**表达式:** +- 它是 SQL,原样写进 CREATE TABLE 语句,所以其中的错误要到建表时才会暴露 +- 用列在表中的名字引用列 +- 不能包含子查询,所以不能引用其他表,这种需求请用[外键](#外键约束) +- 只能调用确定性函数:和当前时间比较的表达式,比如 `expires > date('now')`,会在写入行时失败 + +**语句的行为:** +- 写入违反约束的行的 _INSERT_ 或 _UPDATE_ 会失败,抛出驱动的异常,_UPDATE_ 不会修改这一行 +- `INSERT_OR_IGNORE` 会跳过违反约束的行,插入其余的行 +- `INSERT_OR_REPLACE` 和 _INSERT_ 一样会失败:替换已有的行并不能让新行满足约束 + +**已有的表:** + +SQLite 的 `ALTER TABLE` 不能给表添加约束,`ALTER_ADD_COLUMN` 添加的列也不带约束。要给已有的表添加约束,请 +[重建表](modify-database-and-transaction-cn.md#重建表)。删除某一列时,它自己的约束会随之删除,但如果其他列的约束或表的约束 +引用了这一列,SQLite 会拒绝删除。 + ### 支持的类型 SQLlin 支持以下 Kotlin 类型用于 `@DBRow` 数据类的属性。其他任何类型的属性都会导致编译错误;如果想让这样的属性不进入表中,请为它加上 `kotlinx.serialization.Transient` 注解: @@ -842,6 +933,24 @@ data class Order( // Generated SQL: userId BIGINT CONSTRAINT fk_order_user REFERENCES User(id) ON DELETE CASCADE ``` +#### 延迟检查的外键 + +SQLite 在每条语句之后检查外键。在 `@References` 或 `@ForeignKeyGroup` 上加 `deferred = true`,就会像 +`DEFERRABLE INITIALLY DEFERRED` 那样改为在事务提交时检查,这样一个事务里可以先插入子行再插入父行,或者插入互相引用的行: + +```kotlin +@DBRow +@Serializable +data class Order( + @PrimaryKey(autoIncrement = true) val id: Long?, + @References(tableName = "User", foreignKeys = ["id"], deferred = true) + val userId: Long, +) +// Generated SQL: userId BIGINT REFERENCES User(id) DEFERRABLE INITIALLY DEFERRED +``` + +提交时如果还有这样的外键不满足,事务会失败并被回滚。在事务之外,每条语句本身就是一个事务,所以会立即检查。 + #### 最佳实践 1. **始终启用外键**:在每个数据库会话开始时调用 `PRAGMA_FOREIGN_KEYS(true)` diff --git a/sqllin-dsl/doc/getting-start.md b/sqllin-dsl/doc/getting-start.md index 5687b3fa..a81105bf 100644 --- a/sqllin-dsl/doc/getting-start.md +++ b/sqllin-dsl/doc/getting-start.md @@ -185,7 +185,7 @@ With `DSLDBConfiguration`, you can use CREATE, DROP, and ALTER operations direct Usually, you just need to create one `Database` instance in your component lifecycle. So, you need to close database manually when the lifecycle ended: -> Notice: `DSLDBConfiguration` is experimental, but it will completely replace `DatabaseConfiguration` when it is stable. That means _sqllin-dsl_ will not support to use `DatabaseConfiguration` to create `Database` instances in the future versions. +> Notice: `DSLDBConfiguration` will completely replace `DatabaseConfiguration`. That means _sqllin-dsl_ will not support to use `DatabaseConfiguration` to create `Database` instances in the future versions. ```kotlin override fun onDestroy() { @@ -440,6 +440,27 @@ data class Product( ) ``` +#### Conflict Clauses + +The `onConflict` of `@PrimaryKey`, `@CompositePrimaryKey`, `@Unique` and `@CompositeUnique` says what SQLite does with +a row that would break the constraint, as its `ON CONFLICT` clause does. It is experimental: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow +@Serializable +data class Account( + @PrimaryKey(onConflict = OnConflict.REPLACE) val id: Long, + @Unique(onConflict = OnConflict.IGNORE) val email: String, +) +// Generated SQL: CREATE TABLE Account(id INTEGER PRIMARY KEY ON CONFLICT REPLACE,email TEXT NOT NULL UNIQUE ON CONFLICT IGNORE) +``` + +`REPLACE` deletes the rows the new row conflicts with, `IGNORE` skips the new row, `FAIL` stops the statement keeping the +changes it made before, and `ROLLBACK` rolls back the transaction. `ABORT`, the default, stops the statement and undoes +its changes. A composite key or group takes the one given on any of its properties, and they can't give different +ones. The OR of a statement, as `INSERT_OR_IGNORE`, overrides it. + #### @Default - Column Default Values Use `@Default` to specify default values for columns in your CREATE TABLE statements. SQLite will automatically use these values when inserting rows without explicitly providing values for these columns: @@ -516,6 +537,83 @@ val status: String val status: String ``` +#### @Check - Validating Values + +Use `@Check` to add a `CHECK` constraint: an SQL expression that every row has to satisfy. SQLite evaluates it whenever +an _INSERT_ or _UPDATE_ writes a row, and rejects the row if it is false, so invalid data can't get into the table, +whichever code writes it. `@Check` is experimental, so opt in with `@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBRow +@Check("endDate IS NULL OR endDate >= startDate", constraintName = "valid_period") +@Serializable +data class Membership( + @PrimaryKey(autoIncrement = true) val id: Long?, + @Check("length(trim(name)) > 0") val name: String, + @Check("level IN (0, 1, 2)") val level: Int, + @Check("age BETWEEN 0 AND 150") val age: Int?, + val startDate: String, + val endDate: String?, +) +// Generated SQL: CREATE TABLE Membership( +// id INTEGER PRIMARY KEY AUTOINCREMENT, +// name TEXT NOT NULL CHECK (length(trim(name)) > 0), +// level INT NOT NULL CHECK (level IN (0, 1, 2)), +// age INT CHECK (age BETWEEN 0 AND 150), +// startDate TEXT NOT NULL, +// endDate TEXT, +// CONSTRAINT valid_period CHECK (endDate IS NULL OR endDate >= startDate) +// ) +``` + +On a property, `@Check` adds a constraint to its column; on the class, a constraint to the table. SQLite checks them +alike, and both can refer to any column, so a constraint that compares columns reads best on the class. Both can be +repeated, and every constraint is checked. + +**Naming constraints:** + +`constraintName` names a constraint, which SQLite then reports when a row fails it: +`CHECK constraint failed: valid_period`. An unnamed constraint is reported by its expression from SQLite 3.34 on, but by +the table's name before, as on Android below API 34, where only names tell a table's constraints apart. A name has to +be an SQL identifier, and the names of a table's CHECK constraints have to differ. + +**NULL passes:** + +A row fails a constraint only when its expression is false, and when it is NULL, the row passes. So +`@Check("age BETWEEN 0 AND 150")` lets a NULL age through; to keep NULL out, make the property non-null. This also +catches expressions that look right: + +```kotlin +// ❌ Lets every text that isn't a date through: date(day) is NULL for it, and so is the comparison +@Check("date(day) = day") +val day: String? + +// ✅ Keeps them out: IS compares NULL as a value +@Check("day IS NULL OR date(day) IS day") +val day: String? +``` + +**The expression:** +- It is SQL, written into the CREATE TABLE statement as it is, so a mistake in it only shows when the table is created +- It names columns by their names in the table +- It can't contain a subquery, so it can't refer to other tables: use [foreign keys](#foreign-key-constraints) for that +- It can only call deterministic functions: a comparison with the current time, such as `expires > date('now')`, fails + when a row is written + +**How statements behave:** +- An _INSERT_ or _UPDATE_ that writes a row failing a constraint fails with the driver's exception, and the _UPDATE_ + leaves the row as it was +- `INSERT_OR_IGNORE` skips the rows failing a constraint, and inserts the others +- `INSERT_OR_REPLACE` fails as an _INSERT_ does: replacing a row doesn't help a row satisfy a constraint + +**Existing tables:** + +SQLite's `ALTER TABLE` can't add a constraint to a table, and `ALTER_ADD_COLUMN` adds a column without its constraints. +To add a constraint to an existing table, [rebuild the table](modify-database-and-transaction.md#rebuilding-a-table). +When a column is dropped, its own constraints are dropped with it, but SQLite refuses to drop a column that another +column's constraint or the table's refers to. + ### Supported Types SQLlin supports the following Kotlin types for properties in `@DBRow` data classes. A property of any other type is a compile-time error; to keep such a property out of the table, annotate it with `kotlinx.serialization.Transient`: @@ -853,6 +951,26 @@ data class Order( // Generated SQL: userId BIGINT CONSTRAINT fk_order_user REFERENCES User(id) ON DELETE CASCADE ``` +#### Deferred Foreign Keys + +SQLite checks a foreign key after each statement. With `deferred = true`, on `@References` or `@ForeignKeyGroup`, it +checks it when the transaction commits instead, as `DEFERRABLE INITIALLY DEFERRED` does, so that a transaction can +insert a child before its parent, or rows that refer to each other: + +```kotlin +@DBRow +@Serializable +data class Order( + @PrimaryKey(autoIncrement = true) val id: Long?, + @References(tableName = "User", foreignKeys = ["id"], deferred = true) + val userId: Long, +) +// Generated SQL: userId BIGINT REFERENCES User(id) DEFERRABLE INITIALLY DEFERRED +``` + +A transaction that commits while such a key isn't satisfied fails, and is rolled back. Outside a transaction, each +statement is its own transaction, so it is checked right away. + #### Best Practices 1. **Always enable foreign keys**: Call `PRAGMA_FOREIGN_KEYS(true)` at the start of each database session diff --git a/sqllin-dsl/doc/modify-database-and-transaction-cn.md b/sqllin-dsl/doc/modify-database-and-transaction-cn.md index 95be08f2..c94807cd 100644 --- a/sqllin-dsl/doc/modify-database-and-transaction-cn.md +++ b/sqllin-dsl/doc/modify-database-and-transaction-cn.md @@ -4,7 +4,7 @@ ## 表结构操作 -SQLlin 提供了用于管理表结构的类型安全 DSL 操作:CREATE、DROP 和 ALTER。 +SQLlin 提供了用于管理表结构的类型安全 DSL 操作:CREATE、DROP 和 ALTER,以及视图、索引和触发器的操作。 ### CREATE - 创建表 @@ -59,6 +59,157 @@ fun sample() { **⚠️ 警告**:DROP 是一个破坏性操作。执行后,表及其所有数据将被永久删除。请谨慎使用。 +### CREATE VIEW - 创建视图 + +视图是保存在数据库里的一条 _SELECT_,查询时可以像表一样读取它。用 `@DBView` 声明视图的行,就像用 `@DBRow` 声明表的行一样, +_sqllin-processor_ 会生成一个以类名加 `View` 后缀命名的视图对象。视图是实验性 API,使用时需要 `@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBView("adults") +@Serializable +data class Adult( + val name: String, + val age: Int, +) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + // CREATE VIEW adults(name,age) AS SELECT name,age FROM person WHERE person.age>=18 + CREATE_VIEW(AdultView) AS (PersonTable SELECT WHERE(PersonTable.age GTE 18)) + } + lateinit var olderAdults: SelectStatement + database { + AdultView { view -> + olderAdults = view SELECT WHERE(age GT 60) + } + } +} +``` + +`AS` 后面的 _SELECT_ 读出的必须是视图的行类型,这一点在编译期检查。它可以是任何 _SELECT_:投影、结果列、Join 或组合查询都可以。 +SQLite 不允许视图带参数,所以 SQLlin 会把 _SELECT_ 中的值直接写进视图的 SQL。 + +查询视图的方式和查询表一样,视图对象的属性就是它的列,也可以用在 Join 和组合查询中,`Database#observe` 会观察它读取的表。 +视图不能写入:它的对象不是 `Table`,所以对它使用 `INSERT`、`UPDATE`、`DELETE`、`CREATE` 和 `ALTER` 都无法编译,它的属性也不能 +加约束注解。`DROP(AdultView)` 会删除视图。视图的定义属于数据库结构的一部分:要修改它,就在 `upgrade` 中先删除、再重新创建。 + +### CREATE INDEX - 创建索引 + +索引能让按其列查找行的查询变快。`CREATE_INDEX` 在表的列上创建索引,`CREATE_UNIQUE_INDEX` 创建的索引还会保证任意两行在这些列上的值 +不相同: + +```kotlin +fun sample() { + database { + // CREATE INDEX idx_person_name ON person(name) + PersonTable.CREATE_INDEX("idx_person_name", PersonTable.name) + // CREATE UNIQUE INDEX idx_person_name_age ON person(name,age) + PersonTable.CREATE_UNIQUE_INDEX("idx_person_name_age", PersonTable.name, PersonTable.age) + // CREATE INDEX idx_person_lower_name ON person(lower(name)) + PersonTable.CREATE_INDEX("idx_person_lower_name", PersonTable.lower(PersonTable.name)) + } +} +``` + +用表的列的函数可以创建表达式索引,比较同一个表达式的查询会用到它,例如用 `lower(name)` 做不区分大小写的查找。索引只能包含它自己 +所在表的列。 + +索引的每一列都可以像 `ORDER_BY` 那样指定排序方向:`PersonTable.CREATE_INDEX("idx_person_age", PersonTable.age to DESC)` 就是 +`CREATE INDEX idx_person_age ON person(age DESC)`。它是实验性 API。按这个顺序或相反顺序排序的查询,可以按顺序读取索引。 + +在 `CREATE_INDEX` 或 `CREATE_UNIQUE_INDEX` 后面加上 `WHERE` 就是部分索引:只索引满足条件的行。它比索引所有行的索引更小,唯一的部分 +索引只保证这些行之间的值不相同,例如未删除用户的邮箱。部分索引是实验性 API: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + // CREATE UNIQUE INDEX idx_user_email ON user(email) WHERE isDeleted<=0 + UserTable.CREATE_UNIQUE_INDEX("idx_user_email", UserTable.email) WHERE (UserTable.isDeleted IS false) + } +} +``` + +SQLite 不允许索引的条件中带参数,也不允许子查询,所以 SQLlin 会把条件中的值直接写进 SQL,带子查询的条件会抛出 +`IllegalArgumentException`。只有当查询的 _WHERE_ 能推出索引的条件时,查询才会用到部分索引。 + +`DROP_INDEX` 按名字删除索引,它是实验性 API: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + DROP_INDEX("idx_person_name") + } +} +``` + +### IF NOT EXISTS 与 IF EXISTS + +表已存在时 `CREATE` 会失败,表不存在时 `DROP` 会失败。它们的 `IF NOT EXISTS` 和 `IF EXISTS` 版本则什么也不做,适用于表、视图和索引。 +它们是实验性 API: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + // CREATE TABLE IF NOT EXISTS person(...) + CREATE_IF_NOT_EXISTS(PersonTable) // 或 PersonTable.CREATE_IF_NOT_EXISTS() + PersonTable.CREATE_INDEX_IF_NOT_EXISTS("idx_person_name", PersonTable.name) + PersonTable.CREATE_UNIQUE_INDEX_IF_NOT_EXISTS("idx_person_name_age", PersonTable.name, PersonTable.age) + CREATE_VIEW_IF_NOT_EXISTS(AdultView) AS (PersonTable SELECT WHERE(PersonTable.age GTE 18)) + + DROP_IF_EXISTS(AdultView) // 或 AdultView.DROP_IF_EXISTS() + DROP_INDEX_IF_EXISTS("idx_person_name") + DROP_IF_EXISTS(PersonTable) // 或 PersonTable.DROP_IF_EXISTS() + } +} +``` + +`IF NOT EXISTS` 只检查名字:如果同名的表已经存在但列不同,它会保持原样。 + +### CREATE TRIGGER - 创建触发器 + +触发器会在表的某一行被插入、删除或更新之前或之后执行一组语句,可以带一个条件。触发器是实验性 API: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + // CREATE TRIGGER person_deleted AFTER DELETE ON person + // BEGIN INSERT INTO audit(personId,action,at) VALUES (OLD.id,'delete',datetime('now')); END + CREATE_TRIGGER("person_deleted") AFTER DELETE ON PersonTable BEGIN { + AuditTable { table -> + table INSERT listOf( + old(PersonTable.id) AS Audit::personId, + literal("delete") AS Audit::action, + datetime("now") AS Audit::at, + ) + } + } + // CREATE TRIGGER no_negative_age BEFORE INSERT ON person WHEN NEW.age<0 + // BEGIN SELECT RAISE(ABORT,'age can''t be negative'); END + CREATE_TRIGGER("no_negative_age") BEFORE INSERT ON PersonTable WHEN (new(PersonTable.age) LT 0) BEGIN { + RAISE(ABORT, "age can't be negative") + } + } +} +``` + +- `BEFORE` 或 `AFTER` 指定时机,`INSERT`、`DELETE`、`UPDATE` 或 `UPDATE_OF(列)` 指定事件。`WHEN` 可以指定条件。 +- `new(列)` 和 `old(列)` 读取正在写入的行:INSERT 触发器有新行,DELETE 触发器有旧行,UPDATE 触发器两者都有。读取触发器没有的 + 行会在构建语句时被拒绝,而 SQLite 要等触发器触发时才会报错。 +- 触发器体中可以插入、更新和删除行,并使用 `SET` 和 `INSERT` 的[表达式](sql-functions-cn.md#表达式)。`RAISE(ABORT, 消息)` + 以这条消息终止触发它的语句,`FAIL` 和 `ROLLBACK` 也会终止语句,只是处理方式不同;`RAISE(IGNORE)` 跳过这一行。 +- 这些语句在触发器触发时执行,而不是在作用域结束时。SQLite 不允许触发器带参数,所以触发器 SQL 中的值都是字面量。 + +`CREATE_TRIGGER_IF_NOT_EXISTS` 在触发器已存在时什么也不做,`DROP_TRIGGER(name)` 和 `DROP_TRIGGER_IF_EXISTS(name)` 删除触发器。 +FTS 表是虚拟表,不能有触发器。删除一张表会同时删除它的触发器。 + +触发器写入的行会被[可观察查询](advanced-query-cn.md#可观察查询)计为修改,FTS 表也一样。 + ### ALTER - 修改表结构 SQLlin 提供了多种 ALTER 操作来修改现有的表结构: @@ -99,11 +250,20 @@ fun sample() { } ``` +#### 依赖 SQLite 版本的 API + +有些 API 需要某个版本的 SQLite,或者 SQLite 的某个编译选项,而有的平台没有。SQLlin 运行在哪个 SQLite 上取决于平台:Android 系统的 +SQLite 在 API 24(SQLlin 的最低版本)上是 3.9,API 34 以下是 3.32,API 34 起是 3.39,并且没有数学函数等编译选项;Apple 系统的 SQLite +取决于系统版本;在 Linux 和 Windows 上,则是你的应用链接的 SQLite。这样的 API 在缺少所需条件的 SQLite 上会在运行时失败,所以它们都 +标有 `@PlatformDependentSQLiteAPI`:不用 `@OptIn(PlatformDependentSQLiteAPI::class)` 声明同意就会编译报错。请只在你的应用的所有平台 +都满足 API 文档中写明的条件时才这样做。 + #### 重命名列 -重命名表中的列: +重命名表中的列。`ALTER TABLE ... RENAME COLUMN` 需要 SQLite 3.25.0,Android 从 API 30 起才具备: ```kotlin +@OptIn(PlatformDependentSQLiteAPI::class) fun sample() { database { // Using ClauseElement references (type-safe) @@ -117,9 +277,10 @@ fun sample() { #### 删除列 -从现有表中删除列: +从现有表中删除列。`ALTER TABLE ... DROP COLUMN` 需要 SQLite 3.35.0: ```kotlin +@OptIn(PlatformDependentSQLiteAPI::class) fun sample() { database { PersonTable DROP_COLUMN PersonTable.email @@ -193,7 +354,7 @@ val database = Database( val newPerson = PersonTable.withName("person_new") CREATE(newPerson) PersonV1Table { table -> - // INSERT INTO person_new(id,fullName) SELECT id,name AS fullName FROM person + // INSERT INTO person_new(id,fullName) SELECT id,person.name AS fullName FROM person newPerson INSERT (table SELECT (name AS Person::fullName)) } DROP(PersonV1Table) @@ -204,13 +365,28 @@ val database = Database( ) ``` -要重命名的是新表,而不是旧表。在 SQLite 的默认设置下,重命名一张表时,其他表外键中对它的引用也会跟着改名,所以如果先重命名 -旧表,这些引用就会跟着旧表走,等旧表被删除后,它们就指向了一张不存在的表。索引会随旧表一起删除,所以要在重建后的表上重新创建。 +要重命名的是新表,而不是旧表。SQLite 的 `legacy_alter_table` 设置关闭时(JVM、Linux 和 Windows 上默认如此),重命名一张表, +其他表外键中对它的引用也会跟着改名,所以如果先重命名旧表,这些引用就会跟着旧表走,等旧表被删除后,它们就指向了一张不存在的表。 +Android 和 Apple 平台上这个设置是开启的,只有在启用了外键约束时引用才会跟着改名。索引会随旧表一起删除,所以要在重建后的表上重新 +创建。 + +读取这张表的视图、读取这些视图的视图,以及其他表上写入或读取这张表的触发器,要在删除旧表之前删掉,改名之后再重新创建。 +`legacy_alter_table` 关闭时,重命名表会检查每个视图和触发器读取的表是否都存在,只要还有一个读取已删除的旧表,改名就会失败。 +先删掉它们、之后再重建,在所有平台上都可行。这张表自己的触发器会随旧表一起删除,所以也要重新创建: + +```kotlin +DROP(PersonNameView) // 一个读取 'person' 的视图 +// 按上面的步骤创建新表、拷贝数据、删除旧表、给新表改名 +CREATE_VIEW(PersonNameView) AS (PersonTable SELECT X()) +``` `withName` 返回的表与原表有相同的列、约束和行类型,但没有列属性,因为那些属性指向的是原表。它用于针对整张表的语句: `CREATE`、`INSERT`、`DROP` 和 `ALTER_RENAME_TABLE_TO`。 +对于 [FTS 表](advanced-query-cn.md#全文搜索),它返回的是选项相同的 FTS 表,所以要修改 FTS 表的列或选项,也是用同样的方法 +重建。 -数据的转换由 `SELECT` 完成,可以使用投影或结果列,详见[《高级查询》](advanced-query-cn.md)。转换值类型、替换 `NULL` 的函数目前还不支持。 +数据的转换由 `SELECT` 完成,可以使用投影或结果列,详见[《高级查询》](advanced-query-cn.md),还可以使用[表达式](sql-functions-cn.md#表达式): +`CAST` 转换值的类型,`ifnull` 替换 `NULL`,比如 `ifnull(nickname, "")`。 ## 插入 @@ -269,6 +445,28 @@ fun sample() { 这条 _SELECT_ 会成为 _INSERT_ 的一部分,所以它不再单独执行,也无法读取它的结果。主键按查询出的值原样拷贝。 `INSERT_OR_IGNORE` 和 `INSERT_OR_REPLACE` 同样可以接受 _SELECT_。 +_INSERT_ 还可以像 `INSERT INTO ... VALUES` 那样插入一行[表达式](sql-functions-cn.md#表达式),每个表达式用 `AS` 交给行类型的 +一个属性。它是实验性 API: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + AuditTable { table -> + // INSERT INTO audit(action,at) VALUES ('purge',datetime('now')) + table INSERT listOf(literal("purge") AS Audit::action, datetime("now") AS Audit::at) + } + } +} +``` + +没写到的列取默认值或 NULL;如果是没有默认值的 `NOT NULL` 列,语句执行时会报错。可能为 NULL 的表达式只能写进可空的列;这些值 +不能读取任何表,因为 SQLite 不允许,要插入其他行的值请用 _SELECT_。`INSERT_OR_IGNORE` 和 `INSERT_OR_REPLACE` 同样可以接受表达式。 + +`table INSERT DEFAULT_VALUES` 像 `INSERT INTO ... DEFAULT VALUES` 一样插入一行,所有列都取默认值:没有默认值的列取 NULL, +`NOT NULL` 列会因此报错;由数据库分配的主键取下一个值。它是实验性 API,`INSERT_OR_IGNORE DEFAULT_VALUES` 和 +`INSERT_OR_REPLACE DEFAULT_VALUES` 也一样。 + ## 删除 _DELETE_ 语句将会比 _INSERT_ 语句稍微复杂。SQLlin 不像 [Jetpack Room](https://developer.android.com/training/data-storage/room) @@ -306,6 +504,23 @@ SQL 操作符与 SQLlin 操作符的对应关系如下表: |GLOB|GLOB| |OR|OR| |AND|AND| +|LIKE ... ESCAPE|LIKE ... ESCAPE| + +在 _LIKE_ 的模式中,`%` 匹配任意多个字符,`_` 匹配任意一个字符。要匹配它们本身,可以用 `ESCAPE` 给模式指定一个转义字符(实验性 +API):转义字符之后的 `%`、`_` 和转义字符本身都只匹配它们自己。 + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + PersonTable { table -> + // 删除名字中带下划线的人: + // DELETE FROM person WHERE person.name LIKE ? ESCAPE ?,参数为 "%\_%" 和 "\" + table DELETE WHERE (name LIKE "%\\_%" ESCAPE '\\') + } + } +} +``` 有时候,我们想要删除表中的所有数据,这时 _DELETE_ 语句可以省略 _WHERE_ 子句: @@ -343,6 +558,27 @@ fun sample() { _SET_ 子句与其他子句不同,它接收一个 lambda 表达式作为参数,你可以在 lambda 中给列设置一个新值。lambda 表达式中的 `age` 是一个由 KSP 生成的可写属性,并且它仅在 _SET_ 子句中可用,它与 _WHERE_ 子句中的只读属性 `age` 不同。 +_SET_ lambda 中的属性只能赋值。读取它(比如 `age = age + 1`)会编译报错:它保存的不是列的值,只是一个占位值,读取的话列会被设成由 +占位值算出来的结果。 + +要把列设成一个[表达式](sql-functions-cn.md#表达式),比如由这一行的列算出来的值,可以给 `SET` 传一组表达式,每个表达式用 `AS` +交给行类型的一个属性,`AS` 会在编译期检查类型。它是实验性 API,使用时需要 `@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + PersonTable { table -> + // UPDATE person SET age=(person.age + 1),name=(person.name || '!') WHERE person.name!=? + table UPDATE SET(listOf((age + 1) AS Person::age, (name + "!") AS Person::name)) WHERE (name NEQ "Tom") + } + } +} +``` + +`SET(expression AS Person::age)` 只设置一列。可能为 NULL 的表达式只能设置可空的列;表达式只能读取被更新的表的列,也不能是聚合 +函数,这些在构建语句时检查。表达式中需要单独的值时,用 `literal(5)`。 + 你也可以编写没有 _WHERE_ 子句的 _UPDATE_ 语句用于更新所有的行,但使用它的时候你应该谨慎。 ## 事务 @@ -369,6 +605,47 @@ fun sample() { `transaction {...}` 是 `Database` 的成员函数,将它写在 `TABLE(databaseName) {...}` 函数的内部或外部没有特别限制。 +## VACUUM + +删除行并不会让数据库文件变小,因为 SQLite 会留着空闲页以便复用。`VACUUM()` 会重建数据库文件,把它压缩到最小的磁盘空间; +`VACUUM_INTO(file)` 则把压缩后的数据库副本写到另一个文件:这是正在使用的数据库在某一时刻的一致备份。两者都是实验性 API: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class, PlatformDependentSQLiteAPI::class) +fun sample() { + database { + VACUUM() + } + database { + VACUUM_INTO("/absolute/path/to/backup.db") + } +} +``` + +SQLite 不能在事务中执行 VACUUM,所以在 `transaction {...}` 中调用两者都会抛出 `IllegalStateException`。VACUUM 会重写整个数据库, +所以数据库大时耗时较长,并且需要最多两倍于数据库大小的空闲磁盘空间;它还可能改变没有 INTEGER PRIMARY KEY 的表的 rowid。 +`VACUUM_INTO` 需要 SQLite 3.27.0,Android 从 API 30 起才有,所以它标有 [`@PlatformDependentSQLiteAPI`](#依赖-sqlite-版本的-api)。 +目标文件必须不存在,在 Android 上路径必须是绝对路径。 + +## ANALYZE 与 REINDEX + +`ANALYZE()` 收集所有表和索引的统计信息,查询规划器会据此选择索引;`ANALYZE(table)` 或 `ANALYZE(name)` 只收集一张表或一个索引 +的。`PRAGMA_OPTIMIZE()` 只收集有帮助的统计信息,速度很快,SQLite 建议在关闭数据库前执行,长期打开的数据库每隔几小时执行一次; +它需要 SQLite 3.18.0,Android 从 API 26 起才有,所以标有 [`@PlatformDependentSQLiteAPI`](#依赖-sqlite-版本的-api)。 +`REINDEX()` 重建所有索引,`REINDEX(table)`、`REINDEX(name)`、`REINDEX(collation)` 分别重建一张表、一个索引,或者使用某个排序 +规则(比如 `NOCASE`)的索引。它们都是实验性 API: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class, PlatformDependentSQLiteAPI::class) +fun sample() { + database { + ANALYZE(PersonTable) + REINDEX("idx_person_name") + PRAGMA_OPTIMIZE() + } +} +``` + ## 接下来 你已经学习了如何使用 _INSERT_、_DELETE_ 以及 _UPDATE_ 语句,接下来你将学习 _SELECT_ 语句。 _SELECT_ 语句相比其他语句更复杂,做好准备哦 :)。 diff --git a/sqllin-dsl/doc/modify-database-and-transaction.md b/sqllin-dsl/doc/modify-database-and-transaction.md index bdf15121..318ec34b 100644 --- a/sqllin-dsl/doc/modify-database-and-transaction.md +++ b/sqllin-dsl/doc/modify-database-and-transaction.md @@ -7,7 +7,7 @@ we start to learn how to write SQL statements with SQLlin. ## Table Structure Operations -SQLlin provides type-safe DSL operations for managing table structures: CREATE, DROP, and ALTER. +SQLlin provides type-safe DSL operations for managing table structures: CREATE, DROP, and ALTER, and for views, indexes and triggers. ### CREATE - Creating Tables @@ -62,6 +62,170 @@ fun sample() { **⚠️ WARNING**: DROP is a destructive operation. Once executed, the table and all its data are permanently deleted. Use with caution. +### CREATE VIEW - Creating Views + +A view is a stored _SELECT_ that queries read like a table. Declare its rows with `@DBView`, as you declare a table's +with `@DBRow`, and _sqllin-processor_ generates a view object named after the class with a `View` suffix. Views are +experimental, so opt in with `@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +@DBView("adults") +@Serializable +data class Adult( + val name: String, + val age: Int, +) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + // CREATE VIEW adults(name,age) AS SELECT name,age FROM person WHERE person.age>=18 + CREATE_VIEW(AdultView) AS (PersonTable SELECT WHERE(PersonTable.age GTE 18)) + } + lateinit var olderAdults: SelectStatement + database { + AdultView { view -> + olderAdults = view SELECT WHERE(age GT 60) + } + } +} +``` + +The _SELECT_ after `AS` has to read rows of the view's type, which is checked at compile time. It can be any _SELECT_: +with a projection, result columns, a join or a compound query. SQLite doesn't let a view take parameters, so SQLlin +writes the values of the _SELECT_ into the view's SQL. + +A view is queried like a table, with the properties of its view object as columns, in joins and compound queries too, +and `Database#observe` watches the tables it reads. It can't be written to: its object isn't a `Table`, so `INSERT`, +`UPDATE`, `DELETE`, `CREATE` and `ALTER` don't compile on it, and its properties take no constraint annotations. +`DROP(AdultView)` drops it. A view's definition is part of the schema: to change it, drop it and create it again in an +`upgrade`. + +### CREATE INDEX - Creating Indexes + +An index makes the queries that look rows up by its columns fast. `CREATE_INDEX` creates one on columns of a table, and +`CREATE_UNIQUE_INDEX` one that also keeps any two rows from having the same values in them: + +```kotlin +fun sample() { + database { + // CREATE INDEX idx_person_name ON person(name) + PersonTable.CREATE_INDEX("idx_person_name", PersonTable.name) + // CREATE UNIQUE INDEX idx_person_name_age ON person(name,age) + PersonTable.CREATE_UNIQUE_INDEX("idx_person_name_age", PersonTable.name, PersonTable.age) + // CREATE INDEX idx_person_lower_name ON person(lower(name)) + PersonTable.CREATE_INDEX("idx_person_lower_name", PersonTable.lower(PersonTable.name)) + } +} +``` + +A function of the table's columns makes an expression index, which the queries that compare the same expression use, as +`lower(name)` does for case-insensitive lookups. An index can only hold the columns of its own table. + +Each column of an index can be given the order it is sorted in, as `ORDER_BY` gives it: +`PersonTable.CREATE_INDEX("idx_person_age", PersonTable.age to DESC)` is `CREATE INDEX idx_person_age ON person(age DESC)`. +This is experimental. A query that sorts the columns in that order, or the reverse, reads the index in order. + +`WHERE` after `CREATE_INDEX` or `CREATE_UNIQUE_INDEX` makes a partial index: an index of the rows that satisfy a condition +only. It is smaller than an index of all rows, and a unique one keeps the values unique among those rows only, such as +the emails of the users that aren't deleted. Partial indexes are experimental: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + // CREATE UNIQUE INDEX idx_user_email ON user(email) WHERE isDeleted<=0 + UserTable.CREATE_UNIQUE_INDEX("idx_user_email", UserTable.email) WHERE (UserTable.isDeleted IS false) + } +} +``` + +SQLite allows neither parameters nor subqueries in the condition of an index, so SQLlin writes the values of the +condition into the SQL, and a condition with a subquery throws an `IllegalArgumentException`. A query uses a partial +index only when its _WHERE_ implies the condition of the index. + +`DROP_INDEX`, which is experimental, drops an index by its name: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + DROP_INDEX("idx_person_name") + } +} +``` + +### IF NOT EXISTS and IF EXISTS + +`CREATE` fails if the table already exists, and `DROP` if it doesn't. Their `IF NOT EXISTS` and `IF EXISTS` versions do +nothing instead, for tables, views and indexes. They are experimental: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + // CREATE TABLE IF NOT EXISTS person(...) + CREATE_IF_NOT_EXISTS(PersonTable) // Or PersonTable.CREATE_IF_NOT_EXISTS() + PersonTable.CREATE_INDEX_IF_NOT_EXISTS("idx_person_name", PersonTable.name) + PersonTable.CREATE_UNIQUE_INDEX_IF_NOT_EXISTS("idx_person_name_age", PersonTable.name, PersonTable.age) + CREATE_VIEW_IF_NOT_EXISTS(AdultView) AS (PersonTable SELECT WHERE(PersonTable.age GTE 18)) + + DROP_IF_EXISTS(AdultView) // Or AdultView.DROP_IF_EXISTS() + DROP_INDEX_IF_EXISTS("idx_person_name") + DROP_IF_EXISTS(PersonTable) // Or PersonTable.DROP_IF_EXISTS() + } +} +``` + +`IF NOT EXISTS` only checks the name: if a table of that name exists with other columns, it stays as it is. + +### CREATE TRIGGER - Creating Triggers + +A trigger runs statements whenever a row of a table is inserted, deleted or updated, before or after, where a condition +holds. Triggers are experimental: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + // CREATE TRIGGER person_deleted AFTER DELETE ON person + // BEGIN INSERT INTO audit(personId,action,at) VALUES (OLD.id,'delete',datetime('now')); END + CREATE_TRIGGER("person_deleted") AFTER DELETE ON PersonTable BEGIN { + AuditTable { table -> + table INSERT listOf( + old(PersonTable.id) AS Audit::personId, + literal("delete") AS Audit::action, + datetime("now") AS Audit::at, + ) + } + } + // CREATE TRIGGER no_negative_age BEFORE INSERT ON person WHEN NEW.age<0 + // BEGIN SELECT RAISE(ABORT,'age can''t be negative'); END + CREATE_TRIGGER("no_negative_age") BEFORE INSERT ON PersonTable WHEN (new(PersonTable.age) LT 0) BEGIN { + RAISE(ABORT, "age can't be negative") + } + } +} +``` + +- `BEFORE` or `AFTER` gives the time, and `INSERT`, `DELETE`, `UPDATE` or `UPDATE_OF(columns)` the event. `WHEN` can give + a condition. +- `new(column)` and `old(column)` read the row being written: a trigger on INSERT has a new row, one on DELETE an old + row, and one on UPDATE both. Reading a row the trigger doesn't have is rejected when the statement is built, as + SQLite would only fail when the trigger fires. +- The body inserts, updates and deletes rows, with the [expressions](sql-functions.md#expressions) of `SET` and + `INSERT`. `RAISE(ABORT, message)` stops the statement that fired the trigger with that message, as `FAIL` and + `ROLLBACK` do in their ways, and `RAISE(IGNORE)` skips its row. +- The statements run when the trigger fires, not when the scope ends. SQLite doesn't let a trigger take parameters, so + the values in its SQL are literals. + +`CREATE_TRIGGER_IF_NOT_EXISTS` skips a trigger that exists, and `DROP_TRIGGER(name)` and `DROP_TRIGGER_IF_EXISTS(name)` +drop one. A trigger can't be on an FTS table, which is a virtual table. Dropping a table drops its triggers. + +The rows a trigger writes count as changes for [observed queries](advanced-query.md#observed-queries), FTS tables +included. + ### ALTER - Modifying Table Structure SQLlin provides several ALTER operations for modifying existing table structures: @@ -102,11 +266,22 @@ fun sample() { } ``` +#### APIs That Depend on the SQLite Version + +Some APIs only work with a SQLite version, or a compile-time option of SQLite, that some platforms don't have. The +SQLite SQLlin runs on depends on the platform: Android's system SQLite is 3.9 at API 24, SQLlin's minimum, 3.32 below +API 34 and 3.39 from API 34 on, and lacks options such as the math functions; Apple's system SQLite depends on the OS +version; and on Linux and Windows, it is the SQLite your app links. Such an API fails at runtime where the SQLite lacks +what it needs, so it is marked with `@PlatformDependentSQLiteAPI`, which is a compile error unless you opt in with +`@OptIn(PlatformDependentSQLiteAPI::class)`. Opt in only when all your app's platforms have what the API needs, which +its documentation describes. + #### Rename Column -Rename a column within a table: +Rename a column within a table. `ALTER TABLE ... RENAME COLUMN` needs SQLite 3.25.0, which Android has from API 30 on: ```kotlin +@OptIn(PlatformDependentSQLiteAPI::class) fun sample() { database { // Using ClauseElement references (type-safe) @@ -120,9 +295,10 @@ fun sample() { #### Drop Column -Remove a column from an existing table: +Remove a column from an existing table. `ALTER TABLE ... DROP COLUMN` needs SQLite 3.35.0: ```kotlin +@OptIn(PlatformDependentSQLiteAPI::class) fun sample() { database { PersonTable DROP_COLUMN PersonTable.email @@ -198,7 +374,7 @@ val database = Database( val newPerson = PersonTable.withName("person_new") CREATE(newPerson) PersonV1Table { table -> - // INSERT INTO person_new(id,fullName) SELECT id,name AS fullName FROM person + // INSERT INTO person_new(id,fullName) SELECT id,person.name AS fullName FROM person newPerson INSERT (table SELECT (name AS Person::fullName)) } DROP(PersonV1Table) @@ -209,17 +385,33 @@ val database = Database( ) ``` -Rename the new table, not the old one. Renaming a table also renames the references to it in the foreign keys of -other tables, with SQLite's default settings, so the references would follow the old table, and be left pointing at a -table that no longer exists once it is dropped. Indexes are dropped with the old table, so create them again on the -rebuilt one. +Rename the new table, not the old one. Where SQLite's `legacy_alter_table` setting is off, the default on the JVM, Linux +and Windows, renaming a table also renames the references to it in the foreign keys of other tables, so the references +would follow the old table, and be left pointing at a table that no longer exists once it is dropped. On Android and +Apple's platforms, where it is on, they only follow it while foreign keys are enforced. Indexes are dropped with the +old table, so create them again on the rebuilt one. + +Drop the views that read the table before dropping it, and the views that read those, and the triggers of other tables +that write or read it, and create them again after the rename. Where `legacy_alter_table` is off, renaming a table +checks that every view and trigger still reads tables that exist, so it fails while one reads the dropped table. +Dropping and creating them again works on every platform. The triggers of the table itself are dropped with it, so +create them again too: + +```kotlin +DROP(PersonNameView) // A view that reads 'person' +// Create the new table, copy the rows, drop the old table and rename the new one, as above +CREATE_VIEW(PersonNameView) AS (PersonTable SELECT X()) +``` The table `withName` returns has the columns, constraints and row type of the original, but no column properties, as those name the original table. It is meant for statements on the table as a whole: `CREATE`, `INSERT`, `DROP` and `ALTER_RENAME_TABLE_TO`. +For an [FTS table](advanced-query.md#full-text-search), it is an FTS table with the same +options, so an FTS table is rebuilt the same way, to change its columns or options. The rows are converted by the `SELECT`, with a projection or with result columns, as described in -[Advanced Query](advanced-query.md). Functions to convert a value's type, or to replace a `NULL`, aren't available yet. +[Advanced Query](advanced-query.md), and [expressions](sql-functions.md#expressions): `CAST` converts a value's type, and +`ifnull` replaces a `NULL`, as in `ifnull(nickname, "")`. ## Insert @@ -283,6 +475,30 @@ fun sample() { The _SELECT_ becomes part of the _INSERT_, so it no longer runs on its own, and its results can't be read. The primary key is copied as it is selected. `INSERT_OR_IGNORE` and `INSERT_OR_REPLACE` take a _SELECT_ as well. +_INSERT_ can also insert a row of [expressions](sql-functions.md#expressions), as `INSERT INTO ... VALUES` does, each +given to a property of the row type with `AS`. It is experimental: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + AuditTable { table -> + // INSERT INTO audit(action,at) VALUES ('purge',datetime('now')) + table INSERT listOf(literal("purge") AS Audit::action, datetime("now") AS Audit::at) + } + } +} +``` + +The columns left out take their default values, or NULL, which a `NOT NULL` column without a default rejects when the +statement runs. An expression that can be NULL can only be written to a nullable column, and the values can't read a +table, as SQLite doesn't allow it there: insert the values of other rows with a _SELECT_. `INSERT_OR_IGNORE` and +`INSERT_OR_REPLACE` take expressions as well. + +`table INSERT DEFAULT_VALUES` inserts a row of the default values of all columns, as `INSERT INTO ... DEFAULT VALUES` +does: a column without a default value takes NULL, which a `NOT NULL` column rejects, and a key the database assigns +its next value. It is experimental, as are `INSERT_OR_IGNORE DEFAULT_VALUES` and `INSERT_OR_REPLACE DEFAULT_VALUES`. + ## Delete The _DELETE_ statements will be slightly more complex than _INSERT_. SQLlin doesn't delete objects like @@ -320,6 +536,24 @@ The chart of between SQL operators and SQLlin operators like this: |GLOB|GLOB| |OR|OR| |AND|AND| +|LIKE ... ESCAPE|LIKE ... ESCAPE| + +In a _LIKE_ pattern, `%` matches any characters and `_` any one character. To match them as themselves, give the pattern +an escape character with `ESCAPE`, which is experimental: after it, `%`, `_` and the escape character itself match only +themselves. + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + PersonTable { table -> + // Deletes the people whose names have an underscore: + // DELETE FROM person WHERE person.name LIKE ? ESCAPE ?, with "%\_%" and "\" + table DELETE WHERE (name LIKE "%\\_%" ESCAPE '\\') + } + } +} +``` Sometimes, we want to delete all data in the table. At this time, the _DELETE_ statement doesn't have _WHERE_ clause: @@ -359,6 +593,29 @@ The _SET_ clause is different from other clauses, it receives a lambda as parame lambda. The `age` in the set lambda is a writable property that also be generated by KSP, and it is only available in _SET_ clauses, it is different with readonly property `age` in _WHERE_ clauses. +The properties in the _SET_ lambda can only be assigned. Reading one, as in `age = age + 1`, is a compile error: it +doesn't hold the column's value, only a placeholder, so the column would be set to a value computed from that. + +To set a column to an [expression](sql-functions.md#expressions), such as one computed from the row's columns, give +`SET` a list of expressions, each given to a property of the row type with `AS`, which checks its type at compile time. +It is experimental, so opt in with `@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + database { + PersonTable { table -> + // UPDATE person SET age=(person.age + 1),name=(person.name || '!') WHERE person.name!=? + table UPDATE SET(listOf((age + 1) AS Person::age, (name + "!") AS Person::name)) WHERE (name NEQ "Tom") + } + } +} +``` + +`SET(expression AS Person::age)` sets a single column. An expression that can be NULL can only set a nullable column, +and it can only read the columns of the table it updates, and not be an aggregate function, which is checked when the +statement is built. Use `literal(5)` for a value among expressions. + You also could write the _UPDATE_ statements without the _WHERE_ clause that used for update all rows, but you should use it with caution. ## Transaction @@ -385,6 +642,51 @@ fun sample() { The `transaction {...}` is a member function in `Database`, it is inside or outside of `TABLE(databaseName) {...}` doesn't matter. +## VACUUM + +Deleting rows doesn't shrink the database file, as SQLite keeps the free pages to reuse them. `VACUUM()` rebuilds the +file, repacking it into the minimum disk space, and `VACUUM_INTO(file)` writes a vacuumed copy of the database to another +file: a backup, consistent as of one moment, of a database in use. Both are experimental: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class, PlatformDependentSQLiteAPI::class) +fun sample() { + database { + VACUUM() + } + database { + VACUUM_INTO("/absolute/path/to/backup.db") + } +} +``` + +SQLite can't vacuum inside a transaction, so both throw an `IllegalStateException` inside `transaction {...}`. VACUUM +rewrites the whole database, so it takes long for a large one and needs free disk space of up to twice its size, and it +may change the rowids of the tables without an INTEGER PRIMARY KEY. `VACUUM_INTO` needs SQLite 3.27.0, which Android has +from API 30 on, so it is marked with [`@PlatformDependentSQLiteAPI`](#apis-that-depend-on-the-sqlite-version). Its file +must not exist, and on Android its path has to be absolute. + +## ANALYZE and REINDEX + +`ANALYZE()` gathers the statistics of the tables and indexes, which the query planner chooses indexes with, and +`ANALYZE(table)` or `ANALYZE(name)` those of a table or an index. `PRAGMA_OPTIMIZE()` only gathers those that would help, +quickly, as SQLite advises to run before closing a database, or every few hours in a long-lived one; it needs SQLite +3.18.0, which Android has from API 26 on, so it is marked with +[`@PlatformDependentSQLiteAPI`](#apis-that-depend-on-the-sqlite-version). `REINDEX()` rebuilds all indexes, and +`REINDEX(table)`, `REINDEX(name)` or `REINDEX(collation)` those of a table, an index or a collation, as `NOCASE`. They +are experimental: + +```kotlin +@OptIn(ExperimentalDSLDatabaseAPI::class, PlatformDependentSQLiteAPI::class) +fun sample() { + database { + ANALYZE(PersonTable) + REINDEX("idx_person_name") + PRAGMA_OPTIMIZE() + } +} +``` + ## Next Step You have learned how to use _INSERT_, _DELETE_ and _UPDATE_ statements. Next step you will learn _SELECT_ statements. The diff --git a/sqllin-dsl/doc/query-cn.md b/sqllin-dsl/doc/query-cn.md index 88252d09..2eec4a11 100644 --- a/sqllin-dsl/doc/query-cn.md +++ b/sqllin-dsl/doc/query-cn.md @@ -47,6 +47,10 @@ fun sample() { } ``` +SQLite 在升序时把 NULL 排在最前面,在降序时排在最后面。`NULLS FIRST` 和 `NULLS LAST` 可以指定 NULL 的位置,比如 +`ORDER_BY(age to ASC NULLS LAST)`,或者和其他排序项一起写成 `ORDER_BY(age to (DESC NULLS FIRST))`。它们是实验性 API,并且需要 +SQLite 3.30.0,Android 从 API 31 起才有,所以标有 [`@PlatformDependentSQLiteAPI`](modify-database-and-transaction-cn.md#依赖-sqlite-版本的-api)。 + ## 子句连接 有时我们会一次使用多个子句。在 SQL 中,有一些子句必须跟在另一些子句之后,比如 _HAVING_ 跟在 _GROUP BY_ 后面。SQLlin diff --git a/sqllin-dsl/doc/query.md b/sqllin-dsl/doc/query.md index a207b26f..1c71bb44 100644 --- a/sqllin-dsl/doc/query.md +++ b/sqllin-dsl/doc/query.md @@ -48,6 +48,11 @@ fun sample() { } ``` +SQLite sorts NULL values first in ascending order, and last in descending order. `NULLS FIRST` and `NULLS LAST` put them +where you want, as in `ORDER_BY(age to ASC NULLS LAST)`, or `ORDER_BY(age to (DESC NULLS FIRST))` among other pairs. +They are experimental, and need SQLite 3.30.0, which Android has from API 31 on, so they are marked with +[`@PlatformDependentSQLiteAPI`](modify-database-and-transaction.md#apis-that-depend-on-the-sqlite-version). + ## Clause Connection Sometimes we need to use multiple clauses once. In SQL, some clauses must be added after other clauses. For example, the _HAVING_ behind with diff --git a/sqllin-dsl/doc/sql-functions-cn.md b/sqllin-dsl/doc/sql-functions-cn.md index 95eed4b0..01c0fffe 100644 --- a/sqllin-dsl/doc/sql-functions-cn.md +++ b/sqllin-dsl/doc/sql-functions-cn.md @@ -26,6 +26,10 @@ fun sample() { `count` 函数有一个不同点,它可以接收一个 `X` 作为参数用于表示 SQL 中的 `count(*)`, 如前面的示例所示。 +`DISTINCT` 让聚合函数只计算参数中不重复的值,与 SQL 相同:`count(DISTINCT(name))` 就是 `count(DISTINCT person.name)`,即不同名字的 +个数。`count`、`sum`、`avg` 和 `group_concat` 都可以接收它,`group_concat` 会用逗号分隔这些不重复的值,因为 SQLite 在这里不允许 +其他分隔符。`DISTINCT` 是实验性 API,使用时需要 `@OptIn(ExperimentalDSLDatabaseAPI::class)`。 + 要在 _SELECT_ 关键字之后使用函数,可以用 `AS` 把它们交给结果类型的属性: ```kotlin @@ -40,7 +44,7 @@ fun sample() { lateinit var stats: SelectStatement database { PersonTable { table -> - // SELECT name,count(*) AS people,max(age) AS maxAge FROM person GROUP BY name + // SELECT name,count(*) AS people,max(person.age) AS maxAge FROM person GROUP BY person.name stats = table SELECT listOf(count(X) AS NameStats::people, max(age) AS NameStats::maxAge) GROUP_BY name } } @@ -50,4 +54,99 @@ fun sample() { 每个函数的结果都具有 SQLite 为它返回的值的类型,比如 `count` 为 `Long`,`avg` 为 `Double`,`AS` 只能把它交给这个类型的属性。 结果列的详细用法请见[《高级查询》](advanced-query-cn.md#结果列)。 +## 表达式 + +列和函数可以参与运算,在结果列、条件和 `ORDER BY` 中都能用。运算符是实验性 API,使用时需要 +`@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@Serializable +data class BookLine( + val name: String, + val total: Double, + val label: String, +) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + lateinit var lines: SelectStatement + database { + BookTable { table -> + // SELECT name,(book.price * 1.1) AS total,((book.name || ' by ') || book.author) AS label FROM book + // WHERE (book.price * 1.1)>? ORDER BY (book.pages % 10) ASC + lines = table SELECT listOf( + (price * 1.1) AS BookLine::total, + (name + " by " + author) AS BookLine::label, + ) WHERE ((price * 1.1) GT 20.0) ORDER_BY ((pages % 10) to ASC) + } + } +} +``` + +- `+`、`-`、`*`、`/` 和一元 `-` 用于两个同类型的数,或者一个数和一个同类型的值,结果也是这个类型:`Int` 列的 `pages + 1` + 是 `Int`。`%` 只提供给整数类型,因为 SQLite 会先把小数转成整数再求余。字符串的 `+` 就是 SQL 的 `||` 拼接。 +- SQLite 的计算结果和 Kotlin 一致,只有两处例外:整数溢出时会变成浮点数,而不是回绕;除以零得 NULL,而不是抛异常。所以 + `/` 和 `%` 的结果可能是 NULL(除数是非零常量时除外),`AS` 只能把它们放进可空的属性。 +- 传给运算符或函数的 Kotlin 值会变成字面量。值单独出现时,用 `literal(5)` 把它变成表达式。表达式里的值以字面量的形式写进 + SQL,所以表达式可以用在视图、触发器和索引里,SQLite 不允许这些地方带参数。`ORDER BY` 和 `GROUP BY` 里不能只有一个字面量, + 因为 SQLite 会把 `ORDER BY 2` 理解成第 2 个结果列。 +- `CAST(x AS 类型)` 把表达式转换成另一种类型,用于不同类型的数之间的运算,比如 `CAST(pages AS DOUBLE) * price`。类型名和 + SQLlin 建表时给对应 Kotlin 类型的列用的名字一致:`TINYINT`、`SMALLINT`、`INT`、`BIGINT`、`FLOAT`、`DOUBLE`、`TEXT` 和 + `BLOB`。转成整数类型时,小数会截断,字符串会读取开头的数字部分,比如 `CAST('12abc' AS INT)` 是 `12`。 + +### 更多表达式语法 + +- `CASE` 返回第一个条件成立的分支的值,没有分支成立时返回 `ELSE` 的值,省略 `ELSE` 时为 NULL: + + ```kotlin + // CASE WHEN book.pages<50 THEN 'short' WHEN book.pages<200 THEN 'medium' ELSE 'long' END + CASE( + WHEN(pages LT 50) THEN literal("short"), + WHEN(pages LT 200) THEN literal("medium"), + ELSE = literal("long"), + ) + ``` + + 各分支的值类型相同,`CASE` 的类型也就是它,所以 `AS` 会像检查其他表达式一样检查它。 +- `IS` 和 `ISNOT` 像 `EQ`、`NEQ` 一样比较两个表达式,区别是 NULL 等于 NULL,所以比较结果是真或假,而不会是 NULL: + `notes ISNOT literal("vip")` 也会匹配 `notes` 为 NULL 的行,而 `notes NEQ literal("vip")` 会把它们排除掉。它们的作用和 + `IS DISTINCT FROM` 相同,并且所有版本的 SQLite 都支持。 +- `COLLATE` 按指定的排序规则(`BINARY`、`NOCASE` 或 `RTRIM`)比较和排序字符串:`(name COLLATE NOCASE) EQ "ann"` 也会匹配 + `'Ann'`,`ORDER_BY((name COLLATE NOCASE) to ASC)` 会忽略大小写排序。`NOCASE` 只忽略 ASCII 字母的大小写。 +- `and`、`or`、`inv()`、`shl` 和 `shr` 是 `Int` 和 `Long` 的位运算,名字和 Kotlin 自己的一致。 +- 只返回一列的 _SELECT_ 可以作为一个值,详见[标量子查询](advanced-query-cn.md#标量子查询)。 + +## 更多函数 + +这些函数同样是实验性 API。它们的参数都可以是表达式,需要值的地方也可以传 Kotlin 值: + +> **NULL 处理**:`coalesce`、`ifnull`、`nullif`,比如 `coalesce(nickname, name)`、`ifnull(nickname, "?")` +> +> **多个表达式的标量 max 和 min**:`max(a, b, ...)`、`min(a, b, ...)` +> +> **字符串和 BLOB**:`hex`、`quote`、`unicode`、`randomblob`、`zeroblob`,可以指定要去掉的字符的 `trim`、`ltrim`、`rtrim`, +> 从某个位置开始的 `substr`,多个值的 `printf`,以及参数都可以是表达式的 `substr`、`replace`、`instr` +> +> **数值**:取整的 `round`,以及聚合函数 `total`:结果是 `Double` 的求和,没有匹配的行时是 `0.0` 而不是 NULL +> +> **日期和时间**:`date`、`time`、`datetime`、`julianday`、`strftime`,参数是 `"now"` 这样的时间字符串或表达式,还可以加 +> `"+1 day"` 这样的修饰符 + +只有结果可能为 NULL 时,函数才是可空的:`coalesce` 在所有参数都可能为 NULL 时可空,`nullif` 总是可空;日期时间函数遇到读不懂 +的值或修饰符时返回 NULL,所以总是可空,只有不带修饰符的 `datetime("now")` 除外。`typeof` 是 Kotlin 的保留字,所以没有提供。 + +当前时间、`"localtime"` 和 `"utc"` 修饰符会让日期时间函数的结果随时间变化,`random()` 也是这样,所以索引不能包含这样的函数: +`CREATE_INDEX` 会拒绝它,而 SQLite 要等表里有数据时才会报错。 + +有些函数需要较新的 SQLite,或者 SQLite 的某个编译选项,而有的平台没有,所以它们都标有 +[`@PlatformDependentSQLiteAPI`](modify-database-and-transaction-cn.md#依赖-sqlite-版本的-api): + +| 函数 | 需要 | Android | +|---|---|---| +| `sign` | SQLite 3.35.0 | API 34 | +| `unixepoch` | SQLite 3.38.0 | API 34 | +| `ceil`、`floor`、`trunc`、`sqrt`、`exp`、`ln`、`log10`、`log2`、`pow`、`mod`、`pi` | SQLite 3.35.0,并开启 `SQLITE_ENABLE_MATH_FUNCTIONS` | 所有版本都没有 | + +JVM 和 Apple 平台有数学函数;在 Linux 和 Windows 上,取决于应用链接的 SQLite。 + 最后,让我们来学习[《高级查询》](advanced-query-cn.md)吧。 \ No newline at end of file diff --git a/sqllin-dsl/doc/sql-functions.md b/sqllin-dsl/doc/sql-functions.md index 17203d9a..405e8a93 100644 --- a/sqllin-dsl/doc/sql-functions.md +++ b/sqllin-dsl/doc/sql-functions.md @@ -31,6 +31,11 @@ a `ClauseElement` as the result. The functions supported by SQLlin are as follow The `count` function has a different point, it could receive `X` as parameter be used for representing `count(*)` in SQL, as shown in the example above. +`DISTINCT` makes an aggregate function take only the distinct values of its argument, as in SQL: `count(DISTINCT(name))` +is `count(DISTINCT person.name)`, the number of different names. `count`, `sum`, `avg` and `group_concat` take it, and +`group_concat` separates the distinct values with commas, as SQLite allows no other separator there. `DISTINCT` is +experimental, so opt in with `@OptIn(ExperimentalDSLDatabaseAPI::class)`. + To use functions after the _SELECT_ keyword, select them into properties of a result type with `AS`: ```kotlin @@ -45,7 +50,7 @@ fun sample() { lateinit var stats: SelectStatement database { PersonTable { table -> - // SELECT name,count(*) AS people,max(age) AS maxAge FROM person GROUP BY name + // SELECT name,count(*) AS people,max(person.age) AS maxAge FROM person GROUP BY person.name stats = table SELECT listOf(count(X) AS NameStats::people, max(age) AS NameStats::maxAge) GROUP_BY name } } @@ -56,4 +61,110 @@ Each function's result has the type of the values SQLite returns for it, such as `avg`, and `AS` only selects it into a property of that type. [Advanced Query](advanced-query.md#result-columns) describes result columns in detail. +## Expressions + +Columns and functions can be computed with, in result columns, conditions and `ORDER BY` alike. The operators are +experimental, so opt in with `@OptIn(ExperimentalDSLDatabaseAPI::class)`: + +```kotlin +@Serializable +data class BookLine( + val name: String, + val total: Double, + val label: String, +) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +fun sample() { + lateinit var lines: SelectStatement + database { + BookTable { table -> + // SELECT name,(book.price * 1.1) AS total,((book.name || ' by ') || book.author) AS label FROM book + // WHERE (book.price * 1.1)>? ORDER BY (book.pages % 10) ASC + lines = table SELECT listOf( + (price * 1.1) AS BookLine::total, + (name + " by " + author) AS BookLine::label, + ) WHERE ((price * 1.1) GT 20.0) ORDER_BY ((pages % 10) to ASC) + } + } +} +``` + +- `+`, `-`, `*`, `/` and the unary `-` compute with two numbers of the same type, or a number and a value of its type, + and give a number of that type: `pages + 1` of an `Int` column is an `Int`. `%` is only given to integer types, as + SQLite converts real numbers to integers for it. `+` concatenates strings, SQL's `||`. +- SQLite computes as Kotlin does, except in two cases: an integer that overflows becomes a real number rather than + wrapping around, and a division by zero is NULL rather than an exception. So `/` and `%` can be NULL, unless the + divisor is a value other than zero, and `AS` only selects them into a nullable property. +- A Kotlin value given to an operator or a function becomes a literal. Where a value stands alone, `literal(5)` makes it + an expression. Values in expressions are written into the SQL, as literals, so an expression can be part of a view, a + trigger or an index, which SQLite doesn't let take parameters. A literal on its own can't be in `ORDER BY` or + `GROUP BY`, as SQLite reads `ORDER BY 2` as the second result column. +- `CAST(x AS type)` converts an expression to another type, to compute with numbers of different types, as in + `CAST(pages AS DOUBLE) * price`. The types are named as SQLlin names the columns of that Kotlin type: `TINYINT`, + `SMALLINT`, `INT`, `BIGINT`, `FLOAT`, `DOUBLE`, `TEXT` and `BLOB`. An integer type truncates a real number, and reads a + string as far as it is a number, as `CAST('12abc' AS INT)` is `12`. + +### More Expression Syntax + +- `CASE` gives the value of the first branch whose condition holds, or `ELSE` where none does, and NULL where `ELSE` is + omitted: + + ```kotlin + // CASE WHEN book.pages<50 THEN 'short' WHEN book.pages<200 THEN 'medium' ELSE 'long' END + CASE( + WHEN(pages LT 50) THEN literal("short"), + WHEN(pages LT 200) THEN literal("medium"), + ELSE = literal("long"), + ) + ``` + + The branches have values of the same type, which `CASE` has, so `AS` checks it as it checks any expression. +- `IS` and `ISNOT` compare two expressions as `EQ` and `NEQ` do, except that NULL equals NULL, so the comparison is true + or false rather than NULL: `notes ISNOT literal("vip")` matches the rows whose `notes` are NULL too, which + `notes NEQ literal("vip")` leaves out. They do what `IS DISTINCT FROM` does, on every SQLite version. +- `COLLATE` compares and sorts a string by a collation, `BINARY`, `NOCASE` or `RTRIM`: + `(name COLLATE NOCASE) EQ "ann"` matches `'Ann'` too, and `ORDER_BY((name COLLATE NOCASE) to ASC)` sorts ignoring + case. `NOCASE` only ignores the case of the ASCII letters. +- `and`, `or`, `inv()`, `shl` and `shr` are the bitwise operators of `Int` and `Long`, named as Kotlin names them. +- A _SELECT_ of a single column can be a value, as described in [Scalar Subqueries](advanced-query.md#scalar-subqueries). + +## More Functions + +These functions are experimental too. Their arguments are expressions, and a Kotlin value where one is given: + +> **NULL**: `coalesce`, `ifnull`, `nullif`, as in `coalesce(nickname, name)` or `ifnull(nickname, "?")` +> +> **Scalar max and min** of several expressions: `max(a, b, ...)`, `min(a, b, ...)` +> +> **Strings and BLOBs**: `hex`, `quote`, `unicode`, `randomblob`, `zeroblob`, `trim`, `ltrim` and `rtrim` with the +> characters to remove, `substr` from a position, `printf` of several values, and `substr`, `replace` and `instr` with +> expressions as all their arguments +> +> **Numbers**: `round` to an integer, and the aggregate `total`, a `Double` sum that is `0.0` rather than NULL when no +> rows match +> +> **Date and time**: `date`, `time`, `datetime`, `julianday` and `strftime`, of a time string such as `"now"` or of an +> expression, with modifiers such as `"+1 day"` + +A function is NULL only where its result can be: `coalesce` where all its arguments can be, `nullif` always, and a date +and time function for a value or modifier it can't read, so always, except for `datetime("now")` without modifiers. +`typeof` isn't available, as it is a reserved word in Kotlin. + +The current time, and the `"localtime"` and `"utc"` modifiers, make a date and time function give different results over +time, as `random()` does, so an index can't hold such a function: `CREATE_INDEX` rejects it, where SQLite would only +fail once the table has rows. + +Some functions need a newer SQLite, or a compile-time option of SQLite, than some platforms have, so they are marked +with [`@PlatformDependentSQLiteAPI`](modify-database-and-transaction.md#apis-that-depend-on-the-sqlite-version): + +| Function | Needs | Android | +|---|---|---| +| `sign` | SQLite 3.35.0 | API 34 | +| `unixepoch` | SQLite 3.38.0 | API 34 | +| `ceil`, `floor`, `trunc`, `sqrt`, `exp`, `ln`, `log10`, `log2`, `pow`, `mod`, `pi` | SQLite 3.35.0 with `SQLITE_ENABLE_MATH_FUNCTIONS` | None | + +The math functions are on the JVM and Apple's platforms, while on Linux and Windows they depend on the SQLite the app +links. + Finally, let's learn [Advanced Query](advanced-query.md). \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DSLDBConfiguration.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DSLDBConfiguration.kt index d30fc92d..d7cbb207 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DSLDBConfiguration.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DSLDBConfiguration.kt @@ -20,7 +20,6 @@ import com.ctrip.sqllin.driver.DatabaseConfiguration import com.ctrip.sqllin.driver.DatabasePath import com.ctrip.sqllin.driver.JournalMode import com.ctrip.sqllin.driver.SynchronousMode -import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI /** * DSL-level database configuration with [DatabaseScope] callbacks. @@ -43,7 +42,6 @@ import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI * * @author Yuang Qiao */ -@ExperimentalDSLDatabaseAPI public data class DSLDBConfiguration( val name: String, val path: DatabasePath, diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/Database.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/Database.kt index fb3736db..8ddbfccb 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/Database.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/Database.kt @@ -17,8 +17,18 @@ package com.ctrip.sqllin.dsl import com.ctrip.sqllin.driver.DatabaseConnection +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.sql.statement.SelectStatement +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.IO +import kotlinx.coroutines.flow.Flow +import kotlinx.coroutines.flow.distinctUntilChanged +import kotlinx.coroutines.flow.flow +import kotlinx.coroutines.flow.flowOn import kotlinx.coroutines.sync.Mutex import kotlinx.coroutines.sync.withLock +import kotlin.concurrent.Volatile +import kotlin.coroutines.CoroutineContext /** * High-level database interface for executing SQL operations using type-safe DSL. @@ -63,10 +73,36 @@ public class Database internal constructor( public operator fun invoke(block: DatabaseScope.() -> T): T { val databaseScope = DatabaseScope(databaseConnection, enableSimpleSQLLog) val result = databaseScope.block() - databaseScope.executeAllStatements() + execute(databaseScope) return result } + /** + * The tracker of the tables that observed queries read, which the first observed query creates, so that the code of + * a database that observes no query doesn't reach the tracker, which code shrinkers then remove. + */ + @Volatile + private var invalidationTracker: InvalidationTracker? = null + + private suspend fun invalidationTracker(): InvalidationTracker = + invalidationTracker ?: executiveMutex.withLock { + invalidationTracker ?: InvalidationTracker(databaseConnection).also { invalidationTracker = it } + } + + /** + * Runs the statements of [databaseScope], then tells the observed queries what they changed. They are told even + * when a statement fails, as the ones before it may have changed something. + */ + private fun execute(databaseScope: DatabaseScope) { + val changes = databaseScope.changes + val writtenTables = databaseScope.writtenTables + try { + databaseScope.executeAllStatements() + } finally { + invalidationTracker?.refresh(changes, writtenTables) + } + } + private val executiveMutex by lazy { Mutex() } /** @@ -82,8 +118,55 @@ public class Database internal constructor( val databaseScope = DatabaseScope(databaseConnection, enableSimpleSQLLog) val result = databaseScope.block() executiveMutex.withLock { - databaseScope.executeAllStatements() + execute(databaseScope) } return result } + + /** + * Observes the results of the SELECT that [query] builds: the flow emits them when it is collected, and again + * whenever a statement run through this database changes one of the tables the SELECT reads, if the results differ. + * + * Example: + * ```kotlin + * val adults: Flow> = database.observe { + * PersonTable SELECT WHERE(PersonTable.age GTE 18) + * } + * ``` + * + * [query] runs again to build the SELECT each time, and should only build it. The tables it reads are watched + * whatever the SELECT is: a join or a compound SELECT watches all of its tables, and a view the tables it reads. Changes are counted by SQLite + * itself, so rows that a foreign key action changes count, a rolled back transaction doesn't, and neither does an + * UPDATE or DELETE that matches no rows. Several changes in a row may lead to a single query. + * + * Only changes made through this database are seen, not those of another database instance or connection on the + * same file, or of another process. To combine the results of several SELECTs, combine the flows that observe them; + * as each runs its query on its own, a change to the tables of both may briefly show the new results of one with the + * old results of the other. + * + * @param context The context the queries run in + * @param query Builds the SELECT to observe + * @return A flow of the SELECT's results + */ + @ExperimentalDSLDatabaseAPI + public fun observe(context: CoroutineContext = Dispatchers.IO, query: DatabaseScope.() -> SelectStatement): Flow> = + flow { + val tracker = invalidationTracker() + // The versions of the tables read when the results were last queried + var queriedVersions: Map? = null + tracker.versions.collect { versions -> + val queried = queriedVersions + if (queried != null && queried.all { (table, version) -> (versions[table] ?: 0L) == version }) + return@collect + val databaseScope = DatabaseScope(databaseConnection, enableSimpleSQLLog) + val statement = databaseScope.query() + executiveMutex.withLock { + // Tracked before the SELECT runs, so that no change after it is missed + val tables = tracker.track(statement.tables) + queriedVersions = tables.associateWith { tracker.versions.value[it] ?: 0L } + execute(databaseScope) + } + emit(statement.getResults()) + } + }.distinctUntilChanged().flowOn(context) } \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DatabaseCreators.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DatabaseCreators.kt index 61a015ba..cbaed42a 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DatabaseCreators.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DatabaseCreators.kt @@ -19,7 +19,6 @@ package com.ctrip.sqllin.dsl import com.ctrip.sqllin.driver.DatabaseConfiguration import com.ctrip.sqllin.driver.DatabasePath import com.ctrip.sqllin.driver.openDatabase -import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI /** * Factory functions for creating [Database] instances. @@ -74,7 +73,6 @@ public fun Database( * @param enableSimpleSQLLog Whether to enable simple SQL logging for debugging * @return A new database instance */ -@ExperimentalDSLDatabaseAPI public fun Database( dsldbConfiguration: DSLDBConfiguration, enableSimpleSQLLog: Boolean = false, diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DatabaseScope.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DatabaseScope.kt index c379d9e3..4984d626 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DatabaseScope.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/DatabaseScope.kt @@ -19,25 +19,48 @@ package com.ctrip.sqllin.dsl import com.ctrip.sqllin.driver.DatabaseConnection import com.ctrip.sqllin.dsl.annotation.AdvancedInsertAPI import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.annotation.KeyWordDslMaker +import com.ctrip.sqllin.dsl.annotation.PlatformDependentSQLiteAPI import com.ctrip.sqllin.dsl.annotation.StatementDslMaker -import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.DefaultValues +import com.ctrip.sqllin.dsl.sql.DerivedTable +import com.ctrip.sqllin.dsl.sql.From +import com.ctrip.sqllin.dsl.sql.FtsTable import com.ctrip.sqllin.dsl.sql.ProjectedX +import com.ctrip.sqllin.dsl.sql.RaiseIgnore +import com.ctrip.sqllin.dsl.sql.RaiseResolution +import com.ctrip.sqllin.dsl.sql.Relation +import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.TriggerDefinition +import com.ctrip.sqllin.dsl.sql.TriggerEvent +import com.ctrip.sqllin.dsl.sql.TriggerName +import com.ctrip.sqllin.dsl.sql.TriggerTiming +import com.ctrip.sqllin.dsl.sql.View +import com.ctrip.sqllin.dsl.sql.ViewDefinition import com.ctrip.sqllin.dsl.sql.X import com.ctrip.sqllin.dsl.sql.clause.* +import com.ctrip.sqllin.dsl.sql.compiler.inlineParameters +import com.ctrip.sqllin.dsl.sql.compiler.sqlLiteral import com.ctrip.sqllin.dsl.sql.operation.Alter import com.ctrip.sqllin.dsl.sql.operation.Create import com.ctrip.sqllin.dsl.sql.operation.Delete import com.ctrip.sqllin.dsl.sql.operation.Drop import com.ctrip.sqllin.dsl.sql.operation.Insert +import com.ctrip.sqllin.dsl.sql.operation.Maintenance import com.ctrip.sqllin.dsl.sql.operation.PRAGMA import com.ctrip.sqllin.dsl.sql.operation.Select +import com.ctrip.sqllin.dsl.sql.operation.Trigger import com.ctrip.sqllin.dsl.sql.operation.Update +import com.ctrip.sqllin.dsl.sql.operation.Vacuum import com.ctrip.sqllin.dsl.sql.statement.* +import kotlinx.serialization.ExperimentalSerializationApi import kotlinx.serialization.KSerializer +import kotlinx.serialization.descriptors.SerialDescriptor import kotlinx.serialization.modules.EmptySerializersModule import kotlinx.serialization.serializer import kotlin.concurrent.Volatile import kotlin.jvm.JvmName +import kotlin.reflect.KProperty1 /** * Scope for executing type-safe SQL DSL statements. @@ -119,6 +142,7 @@ public class DatabaseScope internal constructor( * @return `true` if transaction started successfully, `false` if already in a transaction */ public fun beginTransaction(): Boolean { + check(triggerBody == null) { "A transaction can't begin in the body of a trigger, which runs in the statement that fires it." } if (isInTransaction) return false transactionStatementsGroup = TransactionStatementsGroup(databaseConnection, enableSimpleSQLLog) @@ -159,12 +183,20 @@ public class DatabaseScope internal constructor( private val executiveEngine = DatabaseExecuteEngine(enableSimpleSQLLog) private fun addStatement(statement: SingleStatement) { - if (isInTransaction) - transactionStatementsGroup!!.addStatement(statement) - else - executiveEngine.addStatement(statement) + val body = triggerBody + when { + body != null -> body.addStatement(statement) + isInTransaction -> transactionStatementsGroup!!.addStatement(statement) + else -> executiveEngine.addStatement(statement) + } } + /** + * The container the statements are added to: the body of the trigger being defined, the transaction, or the scope. + */ + private val currentContainer: StatementContainer + get() = triggerBody ?: transactionStatementsGroup ?: executiveEngine + private fun addSelectStatement(statement: SelectStatement) { if (unionSelectStatementGroupStack.isNotEmpty()) (unionSelectStatementGroupStack.last() as UnionSelectStatementGroup).addSelectStatement(statement) @@ -172,7 +204,24 @@ public class DatabaseScope internal constructor( addStatement(statement) } - internal fun executeAllStatements() = executiveEngine.executeAllStatement() + internal fun executeAllStatements() { + check(definingTrigger == null) { + "The trigger '${definingTrigger?.timing?.trigger?.name}' has no BEGIN with its statements, so it would never be created." + } + executiveEngine.executeAllStatement() + } + + /** + * What the statements of this scope may change when they run. + */ + internal val changes: Changes + get() = executiveEngine.changes + + /** + * The tables the statements of this scope write rows to. + */ + internal val writtenTables: Set + get() = executiveEngine.writtenTables // ========== INSERT Operations ========== @@ -367,6 +416,106 @@ public class DatabaseScope internal constructor( public infix fun Table.INSERT_OR_IGNORE(entity: T): Unit = INSERT_OR_IGNORE(listOf(entity)) + // ========== INSERT of expressions ========== + + /** + * Inserts a row of expressions, as SQL's `INSERT INTO table(columns) VALUES (expressions)` does, each given to a + * property of the row type with `AS`, which checks its type at compile time: + * ```kotlin + * // INSERT INTO audit(action,at) VALUES ('purge',datetime('now')) + * AuditTable INSERT listOf(literal("purge") AS Audit::action, AuditTable.datetime("now") AS Audit::at) + * ``` + * The columns left out take their default values, or NULL, which a NOT NULL column without a default rejects when + * the statement runs. An expression that can be NULL can only be written to a nullable column, and the values can't + * read any table, but the rows of a trigger: use `INSERT` with a SELECT to insert the values of other rows. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + @JvmName("insertExpressions") + public infix fun Table.INSERT(values: Iterable>) { + addStatement(Insert.insert("INSERT INTO ", this, databaseConnection, values)) + } + + /** + * Inserts a row of a single expression, as `INSERT(listOf(value))` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun Table.INSERT(value: ResultColumn): Unit = INSERT(listOf(value)) + + /** + * Inserts a row of expressions, or nothing if it conflicts with a row, as SQL's `INSERT OR IGNORE` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + @JvmName("insertOrIgnoreExpressions") + public infix fun Table.INSERT_OR_IGNORE(values: Iterable>) { + addStatement(Insert.insert("INSERT OR IGNORE INTO ", this, databaseConnection, values)) + } + + /** + * Inserts a row of a single expression, or nothing if it conflicts with a row. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun Table.INSERT_OR_IGNORE(value: ResultColumn): Unit = INSERT_OR_IGNORE(listOf(value)) + + /** + * Inserts a row of expressions, replacing the rows it conflicts with, as SQL's `INSERT OR REPLACE` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + @JvmName("insertOrReplaceExpressions") + public infix fun Table.INSERT_OR_REPLACE(values: Iterable>) { + addStatement(Insert.insert("INSERT OR REPLACE INTO ", this, databaseConnection, values)) + } + + /** + * Inserts a row of a single expression, replacing the rows it conflicts with. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun Table.INSERT_OR_REPLACE(value: ResultColumn): Unit = INSERT_OR_REPLACE(listOf(value)) + + /** + * `DEFAULT VALUES`, which `table INSERT DEFAULT_VALUES` inserts a row of. + */ + @ExperimentalDSLDatabaseAPI + @KeyWordDslMaker + public val DEFAULT_VALUES: DefaultValues + get() = DefaultValues() + + /** + * Inserts a row of the default values of all columns, as SQL's `INSERT INTO table DEFAULT VALUES` does: + * `table INSERT DEFAULT_VALUES`. A column without a default value takes NULL, which a NOT NULL one rejects when the + * statement runs, and a database-assigned primary key its next value. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun Table.INSERT(defaultValues: DefaultValues) { + addStatement(Insert.insertDefaultValues("INSERT INTO ", this, databaseConnection)) + } + + /** + * Inserts a row of the default values of all columns, or nothing if it conflicts with a row, as SQL's + * `INSERT OR IGNORE INTO table DEFAULT VALUES` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun Table.INSERT_OR_IGNORE(defaultValues: DefaultValues) { + addStatement(Insert.insertDefaultValues("INSERT OR IGNORE INTO ", this, databaseConnection)) + } + + /** + * Inserts a row of the default values of all columns, replacing the rows it conflicts with, as SQL's + * `INSERT OR REPLACE INTO table DEFAULT VALUES` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun Table.INSERT_OR_REPLACE(defaultValues: DefaultValues) { + addStatement(Insert.insertDefaultValues("INSERT OR REPLACE INTO ", this, databaseConnection)) + } + // ========== INSERT INTO ... SELECT ========== // // These insert the rows a SELECT returns, as in `PersonTable INSERT (PersonV1Table SELECT X())`. The @@ -438,13 +587,7 @@ public class DatabaseScope internal constructor( */ @StatementDslMaker public infix fun Table.UPDATE(clause: SetClause): UpdateStatementWithoutWhereClause = - transactionStatementsGroup?.let { - val statement = Update.update(this, databaseConnection, it, clause) - it addStatement statement - statement - } ?: Update.update(this, databaseConnection, executiveEngine, clause).also { - executiveEngine addStatement it - } + Update.update(this, databaseConnection, currentContainer, clause).also { addStatement(it) } // ========== DELETE Operations ========== @@ -487,19 +630,19 @@ public class DatabaseScope internal constructor( * ``` */ @StatementDslMaker - public inline infix fun Table.SELECT(x: X): FinalSelectStatement = + public inline infix fun Relation.SELECT(x: X): FinalSelectStatement = select(kSerializer(), false) /** * Selects distinct records from the table. */ @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(x: X): FinalSelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(x: X): FinalSelectStatement = select(kSerializer(), true) - public fun Table.select(serializer: KSerializer, isDistinct: Boolean): FinalSelectStatement { + public fun Relation.select(serializer: KSerializer, isDistinct: Boolean): FinalSelectStatement { val container = getSelectStatementGroup() - val statement = Select.select(this, isDistinct, serializer, databaseConnection, container) + val statement = Select.select(From.of(this), isDistinct, serializer, databaseConnection, container) addSelectStatement(statement) return statement } @@ -515,16 +658,16 @@ public class DatabaseScope internal constructor( * ``` */ @StatementDslMaker - public inline infix fun Table.SELECT(clause: WhereClause): WhereSelectStatement = + public inline infix fun Relation.SELECT(clause: WhereClause): WhereSelectStatement = select(kSerializer(), clause, false) @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: WhereClause): WhereSelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(clause: WhereClause): WhereSelectStatement = select(kSerializer(), clause, true) - public fun Table.select(serializer: KSerializer, clause: WhereClause, isDistinct: Boolean): WhereSelectStatement { + public fun Relation.select(serializer: KSerializer, clause: WhereClause, isDistinct: Boolean): WhereSelectStatement { val container = getSelectStatementGroup() - val statement = Select.select(this, clause, isDistinct, serializer, databaseConnection, container) + val statement = Select.select(From.of(this), clause, isDistinct, serializer, databaseConnection, container) addSelectStatement(statement) return statement } @@ -538,16 +681,16 @@ public class DatabaseScope internal constructor( * ``` */ @StatementDslMaker - public inline infix fun Table.SELECT(clause: OrderByClause): OrderBySelectStatement = + public inline infix fun Relation.SELECT(clause: OrderByClause): OrderBySelectStatement = select(kSerializer(), clause, false) @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: OrderByClause): OrderBySelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(clause: OrderByClause): OrderBySelectStatement = select(kSerializer(), clause, true) - public fun Table.select(serializer: KSerializer, clause: OrderByClause, isDistinct: Boolean): OrderBySelectStatement { + public fun Relation.select(serializer: KSerializer, clause: OrderByClause, isDistinct: Boolean): OrderBySelectStatement { val container = getSelectStatementGroup() - val statement = Select.select(this, clause, isDistinct, serializer, databaseConnection, container) + val statement = Select.select(From.of(this), clause, isDistinct, serializer, databaseConnection, container) addSelectStatement(statement) return statement } @@ -561,16 +704,16 @@ public class DatabaseScope internal constructor( * ``` */ @StatementDslMaker - public inline infix fun Table.SELECT(clause: LimitClause): LimitSelectStatement = + public inline infix fun Relation.SELECT(clause: LimitClause): LimitSelectStatement = select(kSerializer(), clause, false) @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: LimitClause): LimitSelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(clause: LimitClause): LimitSelectStatement = select(kSerializer(), clause, true) - public fun Table.select(serializer: KSerializer, clause: LimitClause, isDistinct: Boolean): LimitSelectStatement { + public fun Relation.select(serializer: KSerializer, clause: LimitClause, isDistinct: Boolean): LimitSelectStatement { val container = getSelectStatementGroup() - val statement = Select.select(this, clause, isDistinct, serializer, databaseConnection, container) + val statement = Select.select(From.of(this), clause, isDistinct, serializer, databaseConnection, container) addSelectStatement(statement) return statement } @@ -584,16 +727,16 @@ public class DatabaseScope internal constructor( * ``` */ @StatementDslMaker - public inline infix fun Table.SELECT(clause: GroupByClause): GroupBySelectStatement = + public inline infix fun Relation.SELECT(clause: GroupByClause): GroupBySelectStatement = select(kSerializer(), clause, false) @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: GroupByClause): GroupBySelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(clause: GroupByClause): GroupBySelectStatement = select(kSerializer(), clause, true) - public fun Table.select(serializer: KSerializer, clause: GroupByClause, isDistinct: Boolean): GroupBySelectStatement { + public fun Relation.select(serializer: KSerializer, clause: GroupByClause, isDistinct: Boolean): GroupBySelectStatement { val container = getSelectStatementGroup() - val statement = Select.select(this, clause, isDistinct, serializer, databaseConnection, container) + val statement = Select.select(From.of(this), clause, isDistinct, serializer, databaseConnection, container) addSelectStatement(statement) return statement } @@ -624,7 +767,7 @@ public class DatabaseScope internal constructor( * is of a different type from its column, or isn't nullable while its column is */ @StatementDslMaker - public inline infix fun Table.SELECT(x: ProjectedX): FinalSelectStatement = + public inline infix fun Relation.SELECT(x: ProjectedX): FinalSelectStatement = select(getKSerializer(), false) /** @@ -639,7 +782,7 @@ public class DatabaseScope internal constructor( * is of a different type from its column, or isn't nullable while its column is */ @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(x: ProjectedX): FinalSelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(x: ProjectedX): FinalSelectStatement = select(getKSerializer(), true) /** @@ -655,7 +798,7 @@ public class DatabaseScope internal constructor( */ @JvmName("selectWhereProjection") @StatementDslMaker - public inline infix fun Table.SELECT(clause: WhereClause): WhereSelectStatement = + public inline infix fun Relation.SELECT(clause: WhereClause): WhereSelectStatement = select(getKSerializer(), clause, false) /** @@ -666,7 +809,7 @@ public class DatabaseScope internal constructor( */ @JvmName("selectDistinctWhereProjection") @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: WhereClause): WhereSelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(clause: WhereClause): WhereSelectStatement = select(getKSerializer(), clause, true) /** @@ -682,7 +825,7 @@ public class DatabaseScope internal constructor( */ @JvmName("selectOrderByProjection") @StatementDslMaker - public inline infix fun Table.SELECT(clause: OrderByClause): OrderBySelectStatement = + public inline infix fun Relation.SELECT(clause: OrderByClause): OrderBySelectStatement = select(getKSerializer(), clause, false) /** @@ -693,7 +836,7 @@ public class DatabaseScope internal constructor( */ @JvmName("selectDistinctOrderByProjection") @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: OrderByClause): OrderBySelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(clause: OrderByClause): OrderBySelectStatement = select(getKSerializer(), clause, true) /** @@ -709,7 +852,7 @@ public class DatabaseScope internal constructor( */ @JvmName("selectLimitProjection") @StatementDslMaker - public inline infix fun Table.SELECT(clause: LimitClause): LimitSelectStatement = + public inline infix fun Relation.SELECT(clause: LimitClause): LimitSelectStatement = select(getKSerializer(), clause, false) /** @@ -720,7 +863,7 @@ public class DatabaseScope internal constructor( */ @JvmName("selectDistinctLimitProjection") @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: LimitClause): LimitSelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(clause: LimitClause): LimitSelectStatement = select(getKSerializer(), clause, true) /** @@ -736,7 +879,7 @@ public class DatabaseScope internal constructor( */ @JvmName("selectGroupByProjection") @StatementDslMaker - public inline infix fun Table.SELECT(clause: GroupByClause): GroupBySelectStatement = + public inline infix fun Relation.SELECT(clause: GroupByClause): GroupBySelectStatement = select(getKSerializer(), clause, false) /** @@ -747,7 +890,7 @@ public class DatabaseScope internal constructor( */ @JvmName("selectDistinctGroupByProjection") @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: GroupByClause): GroupBySelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(clause: GroupByClause): GroupBySelectStatement = select(getKSerializer(), clause, true) // ========== SELECT with Result Columns ========== @@ -774,7 +917,7 @@ public class DatabaseScope internal constructor( * non-null in a group of GROUP BY is reported when the scope ends, before anything runs, if no GROUP BY follows. */ @StatementDslMaker - public inline infix fun Table.SELECT(column: ResultColumn): ResultColumnSelectStatement = + public inline infix fun Relation.SELECT(column: ResultColumn): ResultColumnSelectStatement = select(getKSerializer(), listOf(column), false) /** @@ -784,7 +927,7 @@ public class DatabaseScope internal constructor( * @throws IllegalArgumentException if [R] doesn't fit the query, as for [SELECT] */ @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(column: ResultColumn): ResultColumnSelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(column: ResultColumn): ResultColumnSelectStatement = select(getKSerializer(), listOf(column), true) /** @@ -798,7 +941,7 @@ public class DatabaseScope internal constructor( * val stats = BookTable { table -> * table SELECT listOf(count(X) AS AuthorStats::books, sum(pages) AS AuthorStats::totalPages) GROUP_BY author * } - * // SELECT author,count(*) AS books,sum(pages) AS totalPages FROM book GROUP BY author + * // SELECT author,count(*) AS books,sum(book.pages) AS totalPages FROM book GROUP BY book.author * ``` * * Can be followed by WHERE, GROUP BY, ORDER BY, or LIMIT. @@ -807,7 +950,7 @@ public class DatabaseScope internal constructor( * expressions, or none at all */ @StatementDslMaker - public inline infix fun Table.SELECT(columns: Iterable>): ResultColumnSelectStatement = + public inline infix fun Relation.SELECT(columns: Iterable>): ResultColumnSelectStatement = select(getKSerializer(), columns, false) /** @@ -817,21 +960,303 @@ public class DatabaseScope internal constructor( * @throws IllegalArgumentException if [R] doesn't fit the query, as for [SELECT] */ @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(columns: Iterable>): ResultColumnSelectStatement = + public inline infix fun Relation.SELECT_DISTINCT(columns: Iterable>): ResultColumnSelectStatement = select(getKSerializer(), columns, true) - public fun Table.select(serializer: KSerializer, columns: Iterable>, isDistinct: Boolean): ResultColumnSelectStatement { + public fun Relation.select(serializer: KSerializer, columns: Iterable>, isDistinct: Boolean): ResultColumnSelectStatement { val container = getSelectStatementGroup() - val statement = Select.select(this, columns, isDistinct, serializer, databaseConnection, container) + val statement = Select.select(From.of(this), columns, isDistinct, serializer, databaseConnection, container) + addSelectStatement(statement) + return statement + } + + // ========== Compound SELECT ========== + // + // These combine two SELECTs of the same result type, as in SQL: `(select1) UNION (select2)`. Each SELECT is put in + // parentheses, as SELECT and these operators are infix functions of the same precedence. Chained, they are + // evaluated from left to right, as SQLite does, and parentheses group them: `a UNION (b UNION_ALL c)`. + + /** + * Combines the rows of this SELECT and [select], without duplicates, as the SQL `select1 UNION select2` does. + * + * Example: + * ```kotlin + * PersonTable { table -> + * (table SELECT WHERE(age LT 18)) UNION (table SELECT WHERE(age GTE 65)) + * } + * ``` + * + * Both SELECTs read rows of the same type, which is checked at compile time, from any tables: with a projection, + * result columns or a join, they can combine rows of different tables. They become part of the compound, so they + * no longer run on their own. A SELECT that ends with ORDER BY or LIMIT keeps them to itself, and the compound can + * be followed by its own ORDER BY and LIMIT. + * + * @return The compound, which can be followed by another compound operator, ORDER BY or LIMIT + * @throws IllegalArgumentException if either SELECT is incomplete, as an aggregate query that needs GROUP BY + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun SelectStatement.UNION(select: SelectStatement): CompoundSelectStatement = + compound(" UNION ", select) + + /** + * Combines the rows of this SELECT and [select], keeping duplicates, as the SQL `select1 UNION ALL select2` does. + * + * @see UNION + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun SelectStatement.UNION_ALL(select: SelectStatement): CompoundSelectStatement = + compound(" UNION ALL ", select) + + /** + * Keeps the rows of this SELECT that [select] also returns, as the SQL `select1 INTERSECT select2` does. + * + * @see UNION + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun SelectStatement.INTERSECT(select: SelectStatement): CompoundSelectStatement = + compound(" INTERSECT ", select) + + /** + * Keeps the rows of this SELECT that [select] doesn't return, as the SQL `select1 EXCEPT select2` does. + * + * @see UNION + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun SelectStatement.EXCEPT(select: SelectStatement): CompoundSelectStatement = + compound(" EXCEPT ", select) + + @OptIn(ExperimentalDSLDatabaseAPI::class) + private fun SelectStatement.compound(operator: String, select: SelectStatement): CompoundSelectStatement { + checkComplete() + select.checkComplete() + container removeStatement this + select.container removeStatement select + val statement = Select.compound(this, operator, select, getSelectStatementGroup()) addSelectStatement(statement) return statement } - // ========== UNION Operations ========== + // ========== Subqueries ========== + // + // A SELECT can be a derived table, in the FROM clause or a JOIN of another query, or the subquery of an IN or + // EXISTS condition. It becomes part of that query, so it no longer runs on its own. + + /** + * Makes this SELECT a derived table named after [view], which a query reads like a table: `(SELECT ...) AS name`. + * + * The derived table has [view]'s rows, which the SELECT has to return, as is checked at compile time, and [view]'s + * column properties name its columns. [view] is the object generated for a class annotated with `@DBView`, which + * only declares the rows and their name here: no such view has to exist in the database. + * + * Example: + * ```kotlin + * @DBView("author_books") + * @Serializable + * data class AuthorBooks(val authorId: Long, val books: Long) + * + * val authorBooks = BookTable { table -> + * table SELECT listOf(count(X) AS AuthorBooks::books) GROUP_BY authorId + * } AS AuthorBooksView + * (FROM(PersonTable) INNER_JOIN authorBooks ON (PersonTable.id EQ AuthorBooksView.authorId)) SELECT X() + * // SELECT ... FROM person INNER JOIN (SELECT authorId,count(*) AS books FROM book GROUP BY book.authorId) AS author_books + * // ON person.id=author_books.authorId + * ``` + * + * As each derived table is named after its view, a query can't read two derived tables of the same view. A + * derived table of a table's rows gives that table another name, which a query joining the table with itself + * needs: `(PersonTable SELECT X) AS ManagerView`. + * + * @return The derived table, which can be selected from and joined like a table or a view, but not written to + * @throws IllegalArgumentException if this SELECT is incomplete, as an aggregate query that needs GROUP BY + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun SelectStatement.AS(view: View): Relation { + takeAsSubquery() + return DerivedTable(view, this) + } + + /** + * Whether this number is among the values the single column of [select] returns: `column IN (SELECT ...)`. + * + * Example: + * ```kotlin + * @Serializable + * data class AuthorId(val authorId: Long) + * + * PersonTable SELECT WHERE(PersonTable.id IN (BookTable SELECT X())) + * // SELECT ... FROM person WHERE person.id IN (SELECT authorId FROM book) + * ``` + * + * @throws IllegalArgumentException if [select] doesn't return one column of numbers, or is incomplete + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun ClauseNumber<*>.IN(select: SelectStatement<*>): SelectCondition = + subqueryIn(this, select, "numbers") { it.serialName.removeSuffix("?") in NUMBER_TYPES } + + /** + * Whether this text is among the values the single column of [select] returns: `column IN (SELECT ...)`. + * + * @throws IllegalArgumentException if [select] doesn't return one column of text, or is incomplete + * @see IN + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun ClauseString<*>.IN(select: SelectStatement<*>): SelectCondition = + subqueryIn(this, select, "text") { it.serialName.removeSuffix("?") in TEXT_TYPES } + + /** + * Whether this BLOB is among the values the single column of [select] returns: `column IN (SELECT ...)`. + * + * @throws IllegalArgumentException if [select] doesn't return one column of BLOBs, or is incomplete + * @see IN + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun ClauseBlob.IN(select: SelectStatement<*>): SelectCondition = + subqueryIn(this, select, "BLOBs") { it.serialName.removeSuffix("?") == BYTE_ARRAY_TYPE } + + /** + * Whether [select] returns any row: `EXISTS (SELECT ...)`. + * + * The SELECT can refer to the columns of the query it is in, as columns are written with their tables' names, + * which makes it run for each of that query's rows: + * ```kotlin + * PersonTable SELECT WHERE(EXISTS(BookTable SELECT WHERE(BookTable.authorId EQ PersonTable.id))) + * // SELECT ... FROM person WHERE EXISTS (SELECT ... FROM book WHERE book.authorId=person.id) + * ``` + * Inside the SELECT, a column of a table that both queries read is the SELECT's own, as both have the same name, + * so it can't refer to the other query's rows of that table. `NOT(EXISTS(select))` checks that it returns no rows. + * + * @throws IllegalArgumentException if [select] is incomplete, as an aggregate query that needs GROUP BY + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun EXISTS(select: SelectStatement<*>): SelectCondition { + select.takeAsSubquery() + return SelectCondition("EXISTS (${select.sqlStr})", select.parameters?.toMutableList(), select.tables) + } + + /** + * Takes this SELECT out of the statements the scope runs, as it becomes a subquery of another one. + */ + // A SELECT of a single column can also be a value, a scalar subquery: `select[R::property]` names the column, so + // that the value has the property's type. + + /** + * This SELECT as a number, a scalar subquery: the value of its single column, [property], in its first row, or NULL + * if it returns no rows, as in `price GT (BookTable SELECT listOf(avg(price) AS AveragePrice::value))[AveragePrice::value]`. + * It can be NULL, and the values of the SELECT are written into the SQL, as the values of any expression are. + * + * @throws IllegalArgumentException if the SELECT returns more than one column + */ + @ExperimentalDSLDatabaseAPI + public operator fun SelectStatement.get(property: KProperty1): ClauseNumber = + scalar(this, property) { sql, tables -> ClauseNumber(sql, ExpressionRelation, isFunction = true, isNullable = true, isAggregate = false, isNullOnNoRows = true, columnTables = emptySet(), tables = tables) } + + /** + * This SELECT as a string, a scalar subquery: the value of its single column, [property], in its first row, or NULL + * if it returns no rows. + * + * @throws IllegalArgumentException if the SELECT returns more than one column + */ + @ExperimentalDSLDatabaseAPI + @JvmName("getString") + public operator fun SelectStatement.get(property: KProperty1): ClauseString = + scalar(this, property) { sql, tables -> ClauseString(sql, ExpressionRelation, isFunction = true, isNullable = true, isAggregate = false, isNullOnNoRows = true, columnTables = emptySet(), tables = tables) } + + /** + * This SELECT as a Boolean, a scalar subquery: the value of its single column, [property], in its first row, or NULL + * if it returns no rows. + * + * @throws IllegalArgumentException if the SELECT returns more than one column + */ + @ExperimentalDSLDatabaseAPI + @JvmName("getBoolean") + public operator fun SelectStatement.get(property: KProperty1): ClauseBoolean = + scalar(this, property) { sql, tables -> ClauseBoolean(sql, ExpressionRelation, isFunction = true, isNullable = true, isAggregate = false, isNullOnNoRows = true, columnTables = emptySet(), tables = tables) } + + /** + * This SELECT as a BLOB, a scalar subquery: the value of its single column, [property], in its first row, or NULL if + * it returns no rows. + * + * @throws IllegalArgumentException if the SELECT returns more than one column + */ + @ExperimentalDSLDatabaseAPI + @JvmName("getBlob") + public operator fun SelectStatement.get(property: KProperty1): ClauseBlob = + scalar(this, property) { sql, tables -> ClauseBlob(sql, ExpressionRelation, isFunction = true, isNullable = true, isAggregate = false, isNullOnNoRows = true, columnTables = emptySet(), tables = tables) } + + /** + * This SELECT as an enum entry, a scalar subquery: the value of its single column, [property], in its first row, or + * NULL if it returns no rows. + * + * @throws IllegalArgumentException if the SELECT returns more than one column + */ + @ExperimentalDSLDatabaseAPI + @JvmName("getEnum") + public operator fun > SelectStatement.get(property: KProperty1): ClauseEnum = + scalar(this, property) { sql, tables -> ClauseEnum(sql, ExpressionRelation, isFunction = true, isNullable = true, isAggregate = false, isNullOnNoRows = true, columnTables = emptySet(), tables = tables) } + + @OptIn(ExperimentalSerializationApi::class) + private fun > scalar(select: SelectStatement<*>, property: KProperty1<*, *>, element: (String, Set) -> E): E { + val descriptor = select.deserializer.descriptor + require(descriptor.elementsCount == 1 && descriptor.getElementName(0) == property.name) { + "A SELECT of '${descriptor.serialName}' can't be the value '${property.name}': a scalar subquery returns a single column, so its result type has to have that property only." + } + select.takeAsSubquery() + return element("(${inlineParameters(select.sqlStr, select.parameters)})", select.tables) + } + + private fun SelectStatement<*>.takeAsSubquery() { + checkComplete() + container removeStatement this + } + + /** + * Returns the condition that [element] is among the values of the single column of [select], checking that the + * column [holds] values [element] can be compared with, which are [values]. + */ + @OptIn(ExperimentalSerializationApi::class) + private fun subqueryIn(element: ClauseElement<*>, select: SelectStatement<*>, values: String, holds: (SerialDescriptor) -> Boolean): SelectCondition { + val descriptor = select.deserializer.descriptor + val prefix = "Can't find '${element.valueName}' IN the results of a SELECT of '${descriptor.serialName}'" + require(descriptor.elementsCount == 1) { + "$prefix: it returns ${descriptor.elementsCount} columns, while IN compares with one. Select a type with a single property." + } + val column = descriptor.getElementDescriptor(0) + require(holds(column)) { + "$prefix: its column '${descriptor.getElementName(0)}' holds ${column.serialName.removeSuffix("?")}, while '${element.valueName}' holds $values." + } + select.takeAsSubquery() + val sql = buildString { + element.appendSQL(this) + append(" IN (") + append(select.sqlStr) + append(')') + } + return SelectCondition(sql, select.parameters?.toMutableList(), select.tables) + } + + private companion object { + val NUMBER_TYPES = setOf( + "kotlin.Byte", "kotlin.Short", "kotlin.Int", "kotlin.Long", "kotlin.Float", "kotlin.Double", + "kotlin.UByte", "kotlin.UShort", "kotlin.UInt", "kotlin.ULong", + ) + val TEXT_TYPES = setOf("kotlin.String", "kotlin.Char") + const val BYTE_ARRAY_TYPE = "kotlin.ByteArray" + } + + // ========== UNION Operations (deprecated) ========== private val unionSelectStatementGroupStack by lazy { ArrayDeque>() } - private fun getSelectStatementGroup(): StatementContainer = unionSelectStatementGroupStack.lastOrNull() ?: transactionStatementsGroup ?: executiveEngine + private fun getSelectStatementGroup(): StatementContainer = unionSelectStatementGroupStack.lastOrNull() ?: currentContainer /** * Combines multiple SELECT statements with UNION (removes duplicates). @@ -843,7 +1268,15 @@ public class DatabaseScope internal constructor( * it SELECT WHERE(age GTE 65) * } * ``` + * + * **Deprecated**: the result is typed by the table's row type, but read into the first SELECT's result type, so + * a union of projections, result columns or joins fails with a `ClassCastException` when its results are used. + * A nested block is flattened into the SQL, which SQLite evaluates from left to right, so it changes its meaning + * when it isn't the first statement. Use the infix [UNION] instead: `(select1) UNION (select2)`. */ + @Deprecated("The UNION {} block reads the rows into its first SELECT's result type while returning the table's row type, so a union of projections, result columns or joins fails with a ClassCastException when its results are used, and a nested block is flattened into the SQL, which changes its meaning. " + + "Use the infix operators instead, as in (select1) UNION (select2), which also offer INTERSECT and EXCEPT. This block will be removed in a future version.") + @Suppress("DEPRECATION") // Its body calls the plumbing deprecated with it public inline fun Table.UNION(block: Table.(Table) -> Unit): FinalSelectStatement { beginUnion() var selectStatement: SelectStatement? = null @@ -858,7 +1291,13 @@ public class DatabaseScope internal constructor( /** * Combines multiple SELECT statements with UNION ALL (keeps duplicates). + * + * **Deprecated** for the reasons given for the `UNION {}` block. Use the infix [UNION_ALL] instead: + * `(select1) UNION_ALL (select2)`. */ + @Deprecated("The UNION {} block reads the rows into its first SELECT's result type while returning the table's row type, so a union of projections, result columns or joins fails with a ClassCastException when its results are used, and a nested block is flattened into the SQL, which changes its meaning. " + + "Use the infix operators instead, as in (select1) UNION (select2), which also offer INTERSECT and EXCEPT. This block will be removed in a future version.") + @Suppress("DEPRECATION") // Its body calls the plumbing deprecated with it @StatementDslMaker public inline fun Table.UNION_ALL(block: Table.(Table) -> Unit): FinalSelectStatement { beginUnion() @@ -875,6 +1314,7 @@ public class DatabaseScope internal constructor( /** * Begins a UNION statement group (for advanced usage). */ + @Deprecated("Part of the deprecated UNION {} block, which will be removed in a future version. Use the infix UNION instead.") public fun beginUnion() { unionSelectStatementGroupStack.add(UnionSelectStatementGroup()) } @@ -882,6 +1322,7 @@ public class DatabaseScope internal constructor( /** * Creates the final UNION select statement from accumulated SELECT statements. */ + @Deprecated("Part of the deprecated UNION {} block, which will be removed in a future version. Use the infix UNION instead.") public fun createUnionSelectStatement(isUnionAll: Boolean): FinalSelectStatement { check(unionSelectStatementGroupStack.isNotEmpty()) { "Please invoke the 'beginUnion' before you invoke this function!!!" } return (unionSelectStatementGroupStack.last() as UnionSelectStatementGroup).unionStatements(isUnionAll) @@ -890,71 +1331,311 @@ public class DatabaseScope internal constructor( /** * Ends the UNION statement group and adds the final statement. */ + @Deprecated("Part of the deprecated UNION {} block, which will be removed in a future version. Use the infix UNION instead.") public fun endUnion(selectStatement: SelectStatement?) { unionSelectStatementGroupStack.removeLast() selectStatement?.let { addSelectStatement(it) } } - // ========== JOIN Operations ========== + // ========== SELECT from Joins ========== + // + // A join of relations, such as `PersonTable INNER_JOIN BookTable ON (...)`, is selected from like a table, but has + // to be given the type its rows are read into, as rows of a join have no type of their own. /** - * Selects with JOIN clause (requires ON condition). + * Selects rows of this join into [R], with X(), checking [R] against the joined relations. * - * Example: - * ```kotlin - * val joined = PersonTable SELECT INNER_JOIN(AddressTable) ON ... - * ``` + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation */ + @ExperimentalDSLDatabaseAPI @StatementDslMaker - public inline infix fun Table.SELECT(clause: JoinClause): JoinStatementWithoutCondition = - select(getKSerializer(), clause, false) + public inline infix fun JoinedRelation.SELECT(x: ProjectedX): FinalSelectStatement = + select(getKSerializer(), isDistinct = false) + /** + * Selects rows of this join into [R], with X(), returning distinct rows, checking [R] against the joined + * relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: JoinClause): JoinStatementWithoutCondition = - select(getKSerializer(), clause, true) - - public fun Table.select(serializer: KSerializer, clause: JoinClause, isDistinct: Boolean): JoinStatementWithoutCondition { - val container = getSelectStatementGroup() - return Select.select(this, clause, isDistinct, serializer, databaseConnection, container, ::addSelectStatement) - } + public inline infix fun JoinedRelation.SELECT_DISTINCT(x: ProjectedX): FinalSelectStatement = + select(getKSerializer(), isDistinct = true) /** - * Selects with NATURAL JOIN (joins on columns with same names). + * Selects rows of this join into [R], with WHERE(...), checking [R] against the joined relations. * - * Example: - * ```kotlin - * val joined = PersonTable SELECT NATURAL_INNER_JOIN(AddressTable) - * ``` + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation */ + @ExperimentalDSLDatabaseAPI @StatementDslMaker - public inline infix fun Table.SELECT(clause: NaturalJoinClause): JoinSelectStatement = - select(getKSerializer(), clause, false) + public inline infix fun JoinedRelation.SELECT(clause: WhereClause): WhereSelectStatement = + select(getKSerializer(), clause, isDistinct = false) + /** + * Selects rows of this join into [R], with WHERE(...), returning distinct rows, checking [R] against the joined + * relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI @StatementDslMaker - public inline infix fun Table.SELECT_DISTINCT(clause: NaturalJoinClause): JoinSelectStatement = - select(getKSerializer(), clause, true) - - public fun Table.select(serializer: KSerializer, clause: NaturalJoinClause, isDistinct: Boolean): JoinSelectStatement { - val container = getSelectStatementGroup() - val statement = Select.select(this, clause, isDistinct, serializer, databaseConnection, container) - addSelectStatement(statement) - return statement - } - - // ========== CREATE Operations ========== + public inline infix fun JoinedRelation.SELECT_DISTINCT(clause: WhereClause): WhereSelectStatement = + select(getKSerializer(), clause, isDistinct = true) /** - * Creates a table from its Table definition. + * Selects rows of this join into [R], with ORDER_BY(...), checking [R] against the joined relations. * - * Example: - * ```kotlin - * CREATE(PersonTable) - * // or - * PersonTable.CREATE() - * ``` + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation */ @ExperimentalDSLDatabaseAPI @StatementDslMaker + public inline infix fun JoinedRelation.SELECT(clause: OrderByClause): OrderBySelectStatement = + select(getKSerializer(), clause, isDistinct = false) + + /** + * Selects rows of this join into [R], with ORDER_BY(...), returning distinct rows, checking [R] against the + * joined relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public inline infix fun JoinedRelation.SELECT_DISTINCT(clause: OrderByClause): OrderBySelectStatement = + select(getKSerializer(), clause, isDistinct = true) + + /** + * Selects rows of this join into [R], with LIMIT(...), checking [R] against the joined relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public inline infix fun JoinedRelation.SELECT(clause: LimitClause): LimitSelectStatement = + select(getKSerializer(), clause, isDistinct = false) + + /** + * Selects rows of this join into [R], with LIMIT(...), returning distinct rows, checking [R] against the joined + * relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public inline infix fun JoinedRelation.SELECT_DISTINCT(clause: LimitClause): LimitSelectStatement = + select(getKSerializer(), clause, isDistinct = true) + + /** + * Selects rows of this join into [R], with GROUP_BY(...), checking [R] against the joined relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public inline infix fun JoinedRelation.SELECT(clause: GroupByClause): GroupBySelectStatement = + select(getKSerializer(), clause, isDistinct = false) + + /** + * Selects rows of this join into [R], with GROUP_BY(...), returning distinct rows, checking [R] against the + * joined relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public inline infix fun JoinedRelation.SELECT_DISTINCT(clause: GroupByClause): GroupBySelectStatement = + select(getKSerializer(), clause, isDistinct = true) + + /** + * Selects rows of this join into [R], with a result column, checking [R] against the joined relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public inline infix fun JoinedRelation.SELECT(column: ResultColumn): ResultColumnSelectStatement = + select(getKSerializer(), listOf(column), isDistinct = false) + + /** + * Selects rows of this join into [R], with a result column, returning distinct rows, checking [R] against the + * joined relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public inline infix fun JoinedRelation.SELECT_DISTINCT(column: ResultColumn): ResultColumnSelectStatement = + select(getKSerializer(), listOf(column), isDistinct = true) + + /** + * Selects rows of this join into [R], with result columns, checking [R] against the joined relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public inline infix fun JoinedRelation.SELECT(columns: Iterable>): ResultColumnSelectStatement = + select(getKSerializer(), columns, isDistinct = false) + + /** + * Selects rows of this join into [R], with result columns, returning distinct rows, checking [R] against the joined + * relations. + * + * @throws IllegalArgumentException if [R] doesn't fit the join + * @see JoinedRelation + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public inline infix fun JoinedRelation.SELECT_DISTINCT(columns: Iterable>): ResultColumnSelectStatement = + select(getKSerializer(), columns, isDistinct = true) + + @ExperimentalDSLDatabaseAPI + public fun JoinedRelation.select(serializer: KSerializer, isDistinct: Boolean): FinalSelectStatement { + val container = getSelectStatementGroup() + val statement = Select.select(from, isDistinct, serializer, databaseConnection, container) + addSelectStatement(statement) + return statement + } + + @ExperimentalDSLDatabaseAPI + public fun JoinedRelation.select(serializer: KSerializer, clause: WhereClause, isDistinct: Boolean): WhereSelectStatement { + val container = getSelectStatementGroup() + val statement = Select.select(from, clause, isDistinct, serializer, databaseConnection, container) + addSelectStatement(statement) + return statement + } + + @ExperimentalDSLDatabaseAPI + public fun JoinedRelation.select(serializer: KSerializer, clause: OrderByClause, isDistinct: Boolean): OrderBySelectStatement { + val container = getSelectStatementGroup() + val statement = Select.select(from, clause, isDistinct, serializer, databaseConnection, container) + addSelectStatement(statement) + return statement + } + + @ExperimentalDSLDatabaseAPI + public fun JoinedRelation.select(serializer: KSerializer, clause: LimitClause, isDistinct: Boolean): LimitSelectStatement { + val container = getSelectStatementGroup() + val statement = Select.select(from, clause, isDistinct, serializer, databaseConnection, container) + addSelectStatement(statement) + return statement + } + + @ExperimentalDSLDatabaseAPI + public fun JoinedRelation.select(serializer: KSerializer, clause: GroupByClause, isDistinct: Boolean): GroupBySelectStatement { + val container = getSelectStatementGroup() + val statement = Select.select(from, clause, isDistinct, serializer, databaseConnection, container) + addSelectStatement(statement) + return statement + } + + @ExperimentalDSLDatabaseAPI + public fun JoinedRelation.select(serializer: KSerializer, columns: Iterable>, isDistinct: Boolean): ResultColumnSelectStatement { + val container = getSelectStatementGroup() + val statement = Select.select(from, columns, isDistinct, serializer, databaseConnection, container) + addSelectStatement(statement) + return statement + } + + // ========== JOIN Operations ========== + + /** + * Selects with JOIN clause (requires ON condition). + * + * Example: + * ```kotlin + * val joined = PersonTable SELECT INNER_JOIN(AddressTable) ON ... + * ``` + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. + */ + @StatementDslMaker + public inline infix fun Relation.SELECT(clause: JoinClause): JoinStatementWithoutCondition = + select(getKSerializer(), clause, false) + + /** + * Selects distinct rows with a JOIN clause, which ON or USING completes. + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. + */ + @StatementDslMaker + public inline infix fun Relation.SELECT_DISTINCT(clause: JoinClause): JoinStatementWithoutCondition = + select(getKSerializer(), clause, true) + + public fun Relation.select(serializer: KSerializer, clause: JoinClause, isDistinct: Boolean): JoinStatementWithoutCondition { + val container = getSelectStatementGroup() + return Select.select(this, clause, isDistinct, serializer, databaseConnection, container, ::addSelectStatement) + } + + /** + * Selects with NATURAL JOIN (joins on columns with same names). + * + * Example: + * ```kotlin + * val joined = PersonTable SELECT NATURAL_INNER_JOIN(AddressTable) + * ``` + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. + */ + @StatementDslMaker + public inline infix fun Relation.SELECT(clause: NaturalJoinClause): JoinSelectStatement = + select(getKSerializer(), clause, false) + + /** + * Selects distinct rows with a NATURAL or CROSS JOIN clause. + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. + */ + @StatementDslMaker + public inline infix fun Relation.SELECT_DISTINCT(clause: NaturalJoinClause): JoinSelectStatement = + select(getKSerializer(), clause, true) + + public fun Relation.select(serializer: KSerializer, clause: NaturalJoinClause, isDistinct: Boolean): JoinSelectStatement { + val container = getSelectStatementGroup() + val statement = Select.select(this, clause, isDistinct, serializer, databaseConnection, container) + addSelectStatement(statement) + return statement + } + + // ========== CREATE Operations ========== + + /** + * Creates a table from its Table definition. + * + * Example: + * ```kotlin + * CREATE(PersonTable) + * // or + * PersonTable.CREATE() + * ``` + */ + @StatementDslMaker public infix fun CREATE(table: Table) { val statement = Create.createTable(table, databaseConnection) addStatement(statement) @@ -963,11 +1644,32 @@ public class DatabaseScope internal constructor( /** * Creates this table from its definition (extension function variant). */ - @ExperimentalDSLDatabaseAPI @StatementDslMaker @JvmName("create") public fun Table.CREATE(): Unit = CREATE(this) + /** + * Creates [table] unless a table of its name exists, as the SQL `CREATE TABLE IF NOT EXISTS` does, so that a + * migration step can run more than once. An FTS table is created with `CREATE VIRTUAL TABLE IF NOT EXISTS`. + * + * The existing table isn't checked against [table]'s structure: if it differs, it stays as it is. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun CREATE_IF_NOT_EXISTS(table: Table) { + addStatement(Create.createTable(table, databaseConnection, isIfNotExists = true)) + } + + /** + * Creates this table unless a table of its name exists. + * + * @see CREATE_IF_NOT_EXISTS + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + @JvmName("createIfNotExists") + public fun Table.CREATE_IF_NOT_EXISTS(): Unit = CREATE_IF_NOT_EXISTS(this) + /** * Creates an index on the specified columns of this table. * @@ -975,24 +1677,26 @@ public class DatabaseScope internal constructor( * However, they also consume additional storage space and may slow down INSERT, UPDATE, * and DELETE operations. * + * A function of the table's columns makes an expression index, which the queries comparing the same expression + * use, as `lower(email)` for case-insensitive lookups. `WHERE` after it makes a partial index of the rows that + * satisfy a condition. + * * Example: * ```kotlin * database { * UserTable.CREATE_INDEX("idx_user_email", UserTable.email) * UserTable.CREATE_INDEX("idx_user_name_age", UserTable.name, UserTable.age) + * UserTable.CREATE_INDEX("idx_user_lower_email", UserTable.lower(UserTable.email)) * } * ``` * * @param indexName The name of the index to create - * @param columns One or more column elements to include in the index - * @throws IllegalArgumentException if no columns are specified + * @param columns One or more columns, or functions of them, to include in the index + * @throws IllegalArgumentException if no columns are specified, or one reads another table */ - @ExperimentalDSLDatabaseAPI @StatementDslMaker - public fun Table.CREATE_INDEX(indexName: String, vararg columns: ClauseElement<*>) { - val statement = Create.createIndex(this, databaseConnection, indexName, *columns) - addStatement(statement) - } + public fun Table.CREATE_INDEX(indexName: String, vararg columns: ClauseElement<*>): IndexStatement = + indexStatement(indexName, Create.createIndex(this, databaseConnection, indexName, *columns)) /** * Creates a unique index on the specified columns of this table. @@ -1005,18 +1709,100 @@ public class DatabaseScope internal constructor( * database { * UserTable.CREATE_UNIQUE_INDEX("idx_unique_email", UserTable.email) * ProductTable.CREATE_UNIQUE_INDEX("idx_unique_sku", ProductTable.sku) + * UserTable.CREATE_UNIQUE_INDEX("idx_unique_lower_email", UserTable.lower(UserTable.email)) * } * ``` * + * A function of the table's columns makes the values of that expression unique, as `lower(email)` does for emails + * that differ only in case. `WHERE` after it makes the columns unique among the rows that satisfy a condition only, + * as `CREATE_UNIQUE_INDEX("idx_user_email", UserTable.email) WHERE (UserTable.isDeleted IS false)` does for the + * users that aren't deleted. + * * @param indexName The name of the unique index to create - * @param columns One or more column elements to include in the unique index - * @throws IllegalArgumentException if no columns are specified + * @param columns One or more columns, or functions of them, to include in the unique index + * @throws IllegalArgumentException if no columns are specified, or one reads another table + */ + @StatementDslMaker + public fun Table.CREATE_UNIQUE_INDEX(indexName: String, vararg columns: ClauseElement<*>): IndexStatement = + indexStatement(indexName, Create.createUniqueIndex(this, databaseConnection, indexName, *columns)) + + /** + * Creates an index on [columns] of this table unless an index named [indexName] exists, as the SQL + * `CREATE INDEX IF NOT EXISTS` does. + * + * @see CREATE_INDEX + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun Table.CREATE_INDEX_IF_NOT_EXISTS(indexName: String, vararg columns: ClauseElement<*>): IndexStatement = + indexStatement(indexName, Create.createIndex(this, databaseConnection, indexName, *columns, isIfNotExists = true)) + + /** + * Creates a unique index on [columns] of this table unless an index named [indexName] exists, as the SQL + * `CREATE UNIQUE INDEX IF NOT EXISTS` does. + * + * @see CREATE_UNIQUE_INDEX + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun Table.CREATE_UNIQUE_INDEX_IF_NOT_EXISTS(indexName: String, vararg columns: ClauseElement<*>): IndexStatement = + indexStatement(indexName, Create.createUniqueIndex(this, databaseConnection, indexName, *columns, isIfNotExists = true)) + + /** + * Creates an index on [column] and [columns] of this table, each sorted in the order given, as the SQL + * `CREATE INDEX name ON table(column DESC)` does: `CREATE_INDEX("idx_person_age", PersonTable.age to DESC)`. + * A query that sorts in that order, or the reverse, reads the index in order. + * + * @see CREATE_INDEX + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun Table.CREATE_INDEX(indexName: String, column: Pair, OrderByWay>, vararg columns: Pair, OrderByWay>): IndexStatement = + indexStatement(indexName, Create.createIndex(this, databaseConnection, indexName, listOf(column) + columns)) + + /** + * Creates a unique index on [column] and [columns] of this table, each sorted in the order given, as the SQL + * `CREATE UNIQUE INDEX name ON table(column DESC)` does: `CREATE_UNIQUE_INDEX("idx_person_age", PersonTable.age to DESC)`. + * A query that sorts in that order, or the reverse, reads the index in order. + * + * @see CREATE_UNIQUE_INDEX */ @ExperimentalDSLDatabaseAPI @StatementDslMaker - public fun Table.CREATE_UNIQUE_INDEX(indexName: String, vararg columns: ClauseElement<*>) { - val statement = Create.createUniqueIndex(this, databaseConnection, indexName, *columns) + public fun Table.CREATE_UNIQUE_INDEX(indexName: String, column: Pair, OrderByWay>, vararg columns: Pair, OrderByWay>): IndexStatement = + indexStatement(indexName, Create.createUniqueIndex(this, databaseConnection, indexName, listOf(column) + columns)) + + /** + * Creates an index, unless one of that name exists, on [column] and [columns] of this table, each sorted in the order given, as the SQL + * `CREATE INDEX name ON table(column DESC)` does: `CREATE_INDEX_IF_NOT_EXISTS("idx_person_age", PersonTable.age to DESC)`. + * A query that sorts in that order, or the reverse, reads the index in order. + * + * @see CREATE_INDEX + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun Table.CREATE_INDEX_IF_NOT_EXISTS(indexName: String, column: Pair, OrderByWay>, vararg columns: Pair, OrderByWay>): IndexStatement = + indexStatement(indexName, Create.createIndex(this, databaseConnection, indexName, listOf(column) + columns, isIfNotExists = true)) + + /** + * Creates a unique index, unless one of that name exists, on [column] and [columns] of this table, each sorted in the order given, as the SQL + * `CREATE UNIQUE INDEX name ON table(column DESC)` does: `CREATE_UNIQUE_INDEX_IF_NOT_EXISTS("idx_person_age", PersonTable.age to DESC)`. + * A query that sorts in that order, or the reverse, reads the index in order. + * + * @see CREATE_UNIQUE_INDEX + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun Table.CREATE_UNIQUE_INDEX_IF_NOT_EXISTS(indexName: String, column: Pair, OrderByWay>, vararg columns: Pair, OrderByWay>): IndexStatement = + indexStatement(indexName, Create.createUniqueIndex(this, databaseConnection, indexName, listOf(column) + columns, isIfNotExists = true)) + + /** + * Adds [statement], which creates the index named [indexName] of this table, and returns it for a `WHERE` to make a + * partial index. + */ + private fun Table<*>.indexStatement(indexName: String, statement: SingleStatement): IndexStatement { addStatement(statement) + return IndexStatement(this, indexName, databaseConnection, currentContainer, statement) } // ========== DROP Operations ========== @@ -1038,7 +1824,6 @@ public class DatabaseScope internal constructor( * * @param table The table to drop */ - @ExperimentalDSLDatabaseAPI @StatementDslMaker public infix fun DROP(table: Table) { val statement = Drop.drop(table, databaseConnection) @@ -1058,11 +1843,138 @@ public class DatabaseScope internal constructor( * } * ``` */ - @ExperimentalDSLDatabaseAPI @StatementDslMaker @JvmName("drop") public fun Table.DROP(): Unit = DROP(this) + /** + * Drops [table] if it exists, as the SQL `DROP TABLE IF EXISTS` does, so that a migration step can run more than + * once. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun DROP_IF_EXISTS(table: Table) { + addStatement(Drop.drop(table, databaseConnection, isIfExists = true)) + } + + /** + * Drops this table if it exists. + * + * @see DROP_IF_EXISTS + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + @JvmName("dropIfExists") + public fun Table.DROP_IF_EXISTS(): Unit = DROP_IF_EXISTS(this) + + /** + * Drops the index named [indexName], as the SQL `DROP INDEX name` does, such as one that [CREATE_INDEX] or + * [CREATE_UNIQUE_INDEX] created. + * + * Example: + * ```kotlin + * database { + * DROP_INDEX("idx_user_email") + * } + * ``` + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun DROP_INDEX(indexName: String) { + addStatement(Drop.dropIndex(indexName, databaseConnection)) + } + + /** + * Drops the index named [indexName] if it exists, as the SQL `DROP INDEX IF EXISTS` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun DROP_INDEX_IF_EXISTS(indexName: String) { + addStatement(Drop.dropIndex(indexName, databaseConnection, isIfExists = true)) + } + + // ========== CREATE VIEW and DROP VIEW ========== + + /** + * Starts the creation of [view], which `AS` completes with the SELECT that defines it. + * + * Example: + * ```kotlin + * CREATE_VIEW(AdultView) AS (PersonTable SELECT WHERE(PersonTable.age GTE 18)) + * // CREATE VIEW adults(name,age) AS SELECT name,age FROM person WHERE person.age>=18 + * ``` + * + * @return The view to create, which becomes a statement once `AS` gives it its SELECT + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun CREATE_VIEW(view: View): ViewDefinition = ViewDefinition(view) + + /** + * Starts the definition of [view] unless a view of its name exists, as the SQL `CREATE VIEW IF NOT EXISTS` does, + * which [AS] completes with the SELECT that defines it. + * + * The existing view isn't checked against the SELECT: if it differs, it stays as it is. + * + * @see CREATE_VIEW + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun CREATE_VIEW_IF_NOT_EXISTS(view: View): ViewDefinition = ViewDefinition(view, isIfNotExists = true) + + /** + * Creates the view, defined by [select], as the SQL `CREATE VIEW view AS SELECT ...` does. + * + * [select] reads rows of the view's row type, which is checked at compile time, from any tables, through a + * projection, result columns, a join or a compound SELECT. It becomes part of this statement, so it no longer runs + * on its own. As SQLite doesn't let a view take parameters, its values are written into the view's SQL. + * + * @throws IllegalArgumentException if [select] is incomplete, as an aggregate query that needs GROUP BY + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun ViewDefinition.AS(select: SelectStatement) { + select.checkComplete() + select.container removeStatement select + addStatement(Create.createView(view, select, databaseConnection, isIfNotExists)) + } + + /** + * Drops [view], as the SQL `DROP VIEW view` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun DROP(view: View) { + addStatement(Drop.dropView(view, databaseConnection)) + } + + /** + * Drops this view (extension function variant). + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + @JvmName("drop") + public fun View.DROP(): Unit = DROP(this) + + /** + * Drops [view] if it exists, as the SQL `DROP VIEW IF EXISTS` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun DROP_IF_EXISTS(view: View) { + addStatement(Drop.dropView(view, databaseConnection, isIfExists = true)) + } + + /** + * Drops this view if it exists. + * + * @see DROP_IF_EXISTS + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + @JvmName("dropIfExists") + public fun View.DROP_IF_EXISTS(): Unit = DROP_IF_EXISTS(this) + // ========== ALTER Operations ========== /** @@ -1081,7 +1993,6 @@ public class DatabaseScope internal constructor( * * @param column The column definition to add to the table */ - @ExperimentalDSLDatabaseAPI @StatementDslMaker public infix fun Table.ALTER_ADD_COLUMN(column: ClauseElement<*>) { val statement = Alter.addColumn(this, column, databaseConnection) @@ -1100,7 +2011,6 @@ public class DatabaseScope internal constructor( * * @param newTable The new table definition containing the target name */ - @ExperimentalDSLDatabaseAPI @StatementDslMaker public infix fun Table.ALTER_RENAME_TABLE_TO(newTable: Table<*>) { val statement = Alter.renameTable(tableName, newTable, databaseConnection) @@ -1122,7 +2032,6 @@ public class DatabaseScope internal constructor( * @receiver The current name of the table to rename * @param newTable The new table definition containing the target name */ - @ExperimentalDSLDatabaseAPI @StatementDslMaker public infix fun String.ALTER_RENAME_TABLE_TO(newTable: Table<*>) { val statement = Alter.renameTable(this, newTable, databaseConnection) @@ -1142,10 +2051,12 @@ public class DatabaseScope internal constructor( * } * ``` * + * `ALTER TABLE ... RENAME COLUMN` needs SQLite 3.25.0, which Android has from API 30 on. + * * @param oldColumn The current column to rename * @param newColumn The new column definition with the target name */ - @ExperimentalDSLDatabaseAPI + @PlatformDependentSQLiteAPI @StatementDslMaker public fun > Table.RENAME_COLUMN(oldColumn: R, newColumn: R) { val statement = Alter.renameColumn(this, oldColumn.valueName, newColumn, databaseConnection) @@ -1165,10 +2076,12 @@ public class DatabaseScope internal constructor( * } * ``` * + * `ALTER TABLE ... RENAME COLUMN` needs SQLite 3.25.0, which Android has from API 30 on. + * * @param oldColumnName The current name of the column to rename * @param newColumn The new column definition with the target name */ - @ExperimentalDSLDatabaseAPI + @PlatformDependentSQLiteAPI @StatementDslMaker public fun Table.RENAME_COLUMN(oldColumnName: String, newColumn: ClauseElement<*>) { val statement = Alter.renameColumn(this, oldColumnName, newColumn, databaseConnection) @@ -1179,8 +2092,8 @@ public class DatabaseScope internal constructor( * Removes a column from this table. * * **⚠️ WARNING**: This permanently deletes the column and all its data. - * Note: SQLite has limited support for DROP COLUMN (added in version 3.35.0). - * Older SQLite versions may require table recreation to drop columns. + * `ALTER TABLE ... DROP COLUMN` needs SQLite 3.35.0, which Android has from API 34 on. Where SQLite is older, + * rebuild the table without the column, as `withName` describes. * * Example: * ```kotlin @@ -1191,13 +2104,390 @@ public class DatabaseScope internal constructor( * * @param column The column to remove from the table */ - @ExperimentalDSLDatabaseAPI + @PlatformDependentSQLiteAPI @StatementDslMaker public infix fun Table.DROP_COLUMN(column: ClauseElement<*>) { val statement = Alter.dropColumn(this, column, databaseConnection) addStatement(statement) } + // ========== ANALYZE, REINDEX and PRAGMA optimize ========== + + /** + * Gathers the statistics of all tables and indexes, as the SQL `ANALYZE` does: the query planner chooses indexes + * with them, and only reads them when the database is opened. It reads every row, so it takes long for a large + * database; `PRAGMA_OPTIMIZE()` only analyzes what would help. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun ANALYZE() { + addStatement(Maintenance.analyze(null, databaseConnection)) + } + + /** + * Gathers the statistics of [table] and its indexes, as the SQL `ANALYZE table` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun ANALYZE(table: Table<*>) { + addStatement(Maintenance.analyze(table.tableName, databaseConnection)) + } + + /** + * Gathers the statistics of the table or index named [name], as the SQL `ANALYZE name` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun ANALYZE(name: String) { + addStatement(Maintenance.analyze(name, databaseConnection)) + } + + /** + * Rebuilds all indexes, as the SQL `REINDEX` does, such as after a change of a collation the app registers. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun REINDEX() { + addStatement(Maintenance.reindex(null, databaseConnection)) + } + + /** + * Rebuilds the indexes of [table], as the SQL `REINDEX table` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun REINDEX(table: Table<*>) { + addStatement(Maintenance.reindex(table.tableName, databaseConnection)) + } + + /** + * Rebuilds the index named [name], or the indexes of the table named so, as the SQL `REINDEX name` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun REINDEX(name: String) { + addStatement(Maintenance.reindex(name, databaseConnection)) + } + + /** + * Rebuilds the indexes that use [collation], as the SQL `REINDEX collation` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun REINDEX(collation: Collation) { + addStatement(Maintenance.reindex(collation, databaseConnection)) + } + + /** + * Gathers the statistics that would help the query planner, if any, as the SQL `PRAGMA optimize` does. It is quick, + * and SQLite advises to run it before closing a database, or every few hours in a long-lived one. + * + * `PRAGMA optimize` needs SQLite 3.18.0, which Android has from API 26 on. + */ + @PlatformDependentSQLiteAPI + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun PRAGMA_OPTIMIZE() { + addStatement(PRAGMA.optimize(databaseConnection)) + } + + // ========== CREATE TRIGGER and DROP TRIGGER ========== + + /** The trigger being defined, from ON to the end of BEGIN, whose rows `new()` and `old()` read. */ + private var definingTrigger: TriggerDefinition? = null + + /** The body of the trigger being defined, which collects the statements written in BEGIN. */ + private var triggerBody: TriggerBody? = null + + /** + * Creates a trigger named [name], as the SQL `CREATE TRIGGER` does: statements that run whenever a row of a table is + * inserted, deleted or updated, before or after, where a condition holds: + * ```kotlin + * // CREATE TRIGGER person_visits AFTER UPDATE OF visits ON person WHEN NEW.visits>OLD.visits + * // BEGIN UPDATE stats SET visits=(stats.visits + 1); END + * CREATE_TRIGGER("person_visits") AFTER UPDATE_OF(PersonTable.visits) ON PersonTable WHEN (new(PersonTable.visits) GT old(PersonTable.visits)) BEGIN { + * StatsTable UPDATE SET((StatsTable.visits + 1) AS Stats::visits) + * } + * ``` + * `new(column)` and `old(column)` read the row being written, in the condition and the statements: a trigger on + * INSERT has a new row, one on DELETE an old row, and one on UPDATE both. The body can insert, update and delete + * rows, and `RAISE` can stop the statement that fired the trigger. Its values are written into the SQL, as SQLite + * doesn't let a trigger take parameters. + * + * A trigger can't be created on an FTS table, which is a virtual table, nor on a view, as an INSTEAD OF trigger is + * of no use while SQLlin can't write to views. Dropping a table drops its triggers. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun CREATE_TRIGGER(name: String): TriggerName = TriggerName(name, isIfNotExists = false) + + /** + * Creates a trigger named [name] unless one of that name exists, as the SQL `CREATE TRIGGER IF NOT EXISTS` does. + * + * @see CREATE_TRIGGER + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun CREATE_TRIGGER_IF_NOT_EXISTS(name: String): TriggerName = TriggerName(name, isIfNotExists = true) + + /** The event of a trigger that fires when a row is inserted, which has a new row. */ + @ExperimentalDSLDatabaseAPI + @KeyWordDslMaker + public val INSERT: TriggerEvent + get() = TriggerEvent("INSERT", hasNewRow = true, hasOldRow = false) + + /** The event of a trigger that fires when a row is deleted, which has an old row. */ + @ExperimentalDSLDatabaseAPI + @KeyWordDslMaker + public val DELETE: TriggerEvent + get() = TriggerEvent("DELETE", hasNewRow = false, hasOldRow = true) + + /** The event of a trigger that fires when a row is updated, which has a new row and an old row. */ + @ExperimentalDSLDatabaseAPI + @KeyWordDslMaker + public val UPDATE: TriggerEvent + get() = TriggerEvent("UPDATE", hasNewRow = true, hasOldRow = true) + + /** + * The event of a trigger that fires when any of [columns] of a row is updated, as the SQL `UPDATE OF` does. + * + * @throws IllegalArgumentException if there are no columns + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun UPDATE_OF(vararg columns: ClauseElement<*>): TriggerEvent { + require(columns.isNotEmpty()) { "UPDATE OF needs at least one column." } + return TriggerEvent("UPDATE OF ${columns.joinToString(",") { it.valueName }}", hasNewRow = true, hasOldRow = true, columns.toList()) + } + + /** Makes this trigger fire before its event, so that it can stop it with `RAISE`. */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun TriggerName.BEFORE(event: TriggerEvent): TriggerTiming = TriggerTiming(this, "BEFORE", event) + + /** Makes this trigger fire after its event. */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun TriggerName.AFTER(event: TriggerEvent): TriggerTiming = TriggerTiming(this, "AFTER", event) + + /** + * Gives this trigger the [table] whose rows fire it. + * + * @throws IllegalArgumentException if [table] is an FTS table, on which SQLite can't create a trigger, or a column of + * `UPDATE_OF` isn't a column of [table] + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun TriggerTiming.ON(table: Table): TriggerDefinition { + check(triggerBody == null) { "A trigger can't be created in the body of another." } + require(table !is FtsTable<*>) { "SQLite can't create a trigger on '${table.tableName}', which is an FTS table, a virtual table." } + for (column in event.columns) { + require(!column.isFunction && column.table.tableName == table.tableName) { + "The trigger '${trigger.name}' on '${table.tableName}' can only fire on an UPDATE OF its columns, but '${column.valueName}' isn't one." + } + } + return TriggerDefinition(this, table).also { definingTrigger = it } + } + + /** + * Makes this trigger fire only for the rows where [condition] holds, as the SQL `WHEN` does. It can read the rows + * being written with `new()` and `old()`. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun TriggerDefinition.WHEN(condition: SelectCondition): TriggerDefinition { + check(this.condition == null) { "The trigger '${timing.trigger.name}' already has a WHEN." } + return TriggerDefinition(timing, table, condition).also { definingTrigger = it } + } + + /** + * Gives this trigger its statements, which [body] writes as it writes any: INSERT, UPDATE, DELETE and `RAISE`. They + * run when the trigger fires, not when the scope ends, and can read the rows being written with `new()` and `old()`. + * + * @throws IllegalArgumentException if [body] writes no statement, or another one, such as a SELECT + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public infix fun TriggerDefinition.BEGIN(body: () -> Unit) { + val collected = TriggerBody() + definingTrigger = this + triggerBody = collected + try { + body() + } finally { + triggerBody = null + definingTrigger = null + } + val name = timing.trigger.name + require(collected.statements.isNotEmpty()) { "The trigger '$name' needs at least one statement." } + for (statement in collected.statements) { + require(statement is InsertStatement || statement is UpdateDeleteStatement || statement is UpdateStatementWithoutWhereClause<*> || statement is RaiseStatement) { + "The trigger '$name' can only run INSERT, UPDATE, DELETE and RAISE, but not: ${statement.sqlStr}" + } + } + addStatement(Trigger.create(this, collected.statements, databaseConnection)) + } + + /** + * The [column] of the new row of the trigger being defined, `NEW.column`: the row inserted, or a row as it is + * updated to. It can only be used in the WHEN and the body of a trigger on INSERT or UPDATE. + * + * @throws IllegalStateException if no trigger is being defined + * @throws IllegalArgumentException if the trigger has no new row, which SQLite would only report when it fires, or + * [column] isn't a column of its table + */ + @ExperimentalDSLDatabaseAPI + public fun > new(column: E): E = rowColumn(column, NEW_ROW) + + /** + * The [column] of the old row of the trigger being defined, `OLD.column`: the row deleted, or a row as it was before + * it is updated. It can only be used in the WHEN and the body of a trigger on UPDATE or DELETE. + * + * @throws IllegalStateException if no trigger is being defined + * @throws IllegalArgumentException if the trigger has no old row, which SQLite would only report when it fires, or + * [column] isn't a column of its table + */ + @ExperimentalDSLDatabaseAPI + public fun > old(column: E): E = rowColumn(column, OLD_ROW) + + @Suppress("UNCHECKED_CAST") + private fun > rowColumn(column: E, row: String): E { + val definition = checkNotNull(definingTrigger) { "${row.lowercase()}() can only be used in the WHEN or the body of a trigger." } + val event = definition.timing.event + require(if (row == NEW_ROW) event.hasNewRow else event.hasOldRow) { + "The trigger '${definition.timing.trigger.name}' on ${event.sql.substringBefore(' ')} has no $row row, which SQLite would only report when it fires." + } + require(!column.isFunction && column.table.tableName == definition.table.tableName) { + "${row.lowercase()}() takes a column of '${definition.table.tableName}', the table of the trigger '${definition.timing.trigger.name}', but '${column.valueName}' isn't one." + } + return column.rebind(RowRelation(row, definition.table)) as E + } + + /** `RAISE(ABORT, ...)`: stops the statement that fired the trigger and undoes its changes. */ + @ExperimentalDSLDatabaseAPI + @KeyWordDslMaker + public val ABORT: RaiseResolution + get() = RaiseResolution("ABORT") + + /** `RAISE(FAIL, ...)`: stops the statement that fired the trigger, keeping the changes it made before. */ + @ExperimentalDSLDatabaseAPI + @KeyWordDslMaker + public val FAIL: RaiseResolution + get() = RaiseResolution("FAIL") + + /** `RAISE(ROLLBACK, ...)`: stops the statement that fired the trigger and rolls back the transaction. */ + @ExperimentalDSLDatabaseAPI + @KeyWordDslMaker + public val ROLLBACK: RaiseResolution + get() = RaiseResolution("ROLLBACK") + + /** `RAISE(IGNORE)`: skips the row that fired the trigger, and the rest of the trigger. */ + @ExperimentalDSLDatabaseAPI + @KeyWordDslMaker + public val IGNORE: RaiseIgnore + get() = RaiseIgnore() + + /** + * Stops the statement that fired the trigger with [message], which SQLite reports as the error, as the SQL + * `RAISE(resolution, message)` does: `RAISE(ABORT, "age can't be negative")`. It can only be in the body of a + * trigger, where it is usually run where the WHEN of the trigger holds. + * + * @throws IllegalStateException if no trigger is being defined + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun RAISE(resolution: RaiseResolution, message: String) { + addRaise("RAISE(${resolution.sql},${sqlLiteral(message)})") + } + + /** + * Skips the row that fired the trigger, and the rest of the trigger, as the SQL `RAISE(IGNORE)` does. + * + * @throws IllegalStateException if no trigger is being defined + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun RAISE(ignore: RaiseIgnore) { + addRaise("RAISE(IGNORE)") + } + + private fun addRaise(raise: String) { + check(triggerBody != null) { "RAISE can only be in the body of a trigger." } + addStatement(RaiseStatement("SELECT $raise")) + } + + /** + * Drops the trigger named [name], as the SQL `DROP TRIGGER` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun DROP_TRIGGER(name: String) { + addStatement(Trigger.drop(name, isIfExists = false, databaseConnection)) + } + + /** + * Drops the trigger named [name] if it exists, as the SQL `DROP TRIGGER IF EXISTS` does. + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun DROP_TRIGGER_IF_EXISTS(name: String) { + addStatement(Trigger.drop(name, isIfExists = true, databaseConnection)) + } + + // ========== VACUUM ========== + + /** + * Rebuilds the database file, repacking it into the minimum disk space, as the SQL `VACUUM` does: the file doesn't + * shrink when rows are deleted, as SQLite keeps the free pages to reuse them. + * + * VACUUM rewrites the whole database, so it takes long for a large one, and needs free disk space of up to twice + * its size. It may change the rowids of the tables without an INTEGER PRIMARY KEY. + * + * Example: + * ```kotlin + * database { + * LogTable DELETE WHERE (LogTable.time LT expired) + * } + * database { + * VACUUM() + * } + * ``` + * + * @throws IllegalStateException if it is called inside a transaction, where SQLite can't vacuum + */ + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun VACUUM() { + check(!isInTransaction) { "SQLite can't VACUUM inside a transaction." } + addStatement(Vacuum.vacuum(databaseConnection)) + } + + /** + * Writes a vacuumed copy of the database to [file], as the SQL `VACUUM INTO file` does, leaving the database as it + * is: a backup, consistent as of one moment, of a database that is in use. + * + * [file] is a path of the file system, so on Android, where the working directory can't be written, it has to be + * absolute. It must not exist, or SQLite fails. + * + * `VACUUM INTO` needs SQLite 3.27.0, which Android has from API 30 on. + * + * Example: + * ```kotlin + * database { + * VACUUM_INTO("/data/data/com.example/backup/notes.db") + * } + * ``` + * + * @throws IllegalStateException if it is called inside a transaction, where SQLite can't vacuum + */ + @PlatformDependentSQLiteAPI + @ExperimentalDSLDatabaseAPI + @StatementDslMaker + public fun VACUUM_INTO(file: String) { + check(!isInTransaction) { "SQLite can't VACUUM inside a transaction." } + addStatement(Vacuum.vacuumInto(file, databaseConnection)) + } + /** * Enables or disables foreign key constraint enforcement in SQLite. * @@ -1237,7 +2527,6 @@ public class DatabaseScope internal constructor( * @see ForeignKey * @see References */ - @ExperimentalDSLDatabaseAPI @StatementDslMaker public infix fun PRAGMA_FOREIGN_KEYS(flag: Boolean) { val statement = PRAGMA.foreignKeys(flag, databaseConnection) diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/InvalidationTracker.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/InvalidationTracker.kt new file mode 100644 index 00000000..48fbed3f --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/InvalidationTracker.kt @@ -0,0 +1,221 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl + +import com.ctrip.sqllin.driver.DatabaseConnection +import com.ctrip.sqllin.driver.withQuery +import com.ctrip.sqllin.driver.withTransaction +import com.ctrip.sqllin.dsl.sql.statement.Changes +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.StateFlow +import kotlinx.coroutines.flow.update + +/** + * Tracks the changes a [Database] makes to the tables its observed queries read, so that they run again when one of + * them changes. + * + * A table is tracked from the first time a query that reads it is observed. TEMP triggers on its INSERT, UPDATE and + * DELETE add one to its count in a TEMP table, so SQLite itself reports every row the connection changes: those of + * foreign key actions and of `INSERT OR REPLACE` too, and none for an UPDATE or DELETE that matches no rows. A rolled + * back transaction rolls its counts back with it. After a scope that may have written runs, [refresh] reads the counts + * into [versions], which observed queries collect. Counts, rather than a flag that is read and then cleared, can't lose + * a change made between the two. + * + * A view can't have such triggers, so an observed view stands for the tables it reads, which SQLite itself lists: they + * are the ones the program of `EXPLAIN SELECT * FROM view` opens, through views over views and joins alike. A virtual + * table, such as an FTS table, can't have them either, so its changes are counted from the SQLlin statements that write + * to it, INSERT, UPDATE and DELETE: a change made with other SQL isn't seen, and neither is one to a virtual table a + * view reads, as a virtual table isn't opened that way. A rolled back transaction still counts its statements, which + * only means the queries run once more. + * + * TEMP tables and triggers belong to the connection, so changes made through another connection, or by another + * process, aren't seen. A table stays tracked until the connection closes. + * + * @author Yuang Qiao + */ +internal class InvalidationTracker(private val connection: DatabaseConnection) { + + private val trackedTables = MutableStateFlow>(emptySet()) + + /** The tracked tables that are virtual, whose counts [refresh] increments, as they have no triggers. */ + private val virtualTables = MutableStateFlow>(emptySet()) + + /** + * The tables that have triggers, read while virtual tables are tracked: a trigger can write a virtual table, as one + * that keeps an FTS table in step with a table does, which SQLlin's statements don't tell, so a write to any of these + * tables counts as a change of every tracked virtual table. + */ + private val triggeredTables = MutableStateFlow>(emptySet()) + + private val _versions = MutableStateFlow>(emptyMap()) + + /** + * The count of changes to each tracked table. It only grows. + */ + val versions: StateFlow> = _versions + + /** + * Starts tracking the tables that the tables and views named [relations] read, creating their triggers, unless they + * are tracked already. + * + * @return The tables tracked for [relations]: a table stands for itself, and a view for the tables it reads + */ + fun track(relations: Set): Set { + val tables = relations.flatMapTo(LinkedHashSet()) { tablesOf(it) } + val newTables = tables - trackedTables.value + if (newTables.isEmpty()) + return tables + connection.execSQL(CREATE_LOG) + val newVirtualTables = newTables intersect virtualTables() + for (table in newTables) { + connection.execSQL(INSERT_LOG_ROW, arrayOf(table)) + if (table !in newVirtualTables) + createTriggers(table) + } + virtualTables.update { it + newVirtualTables } + if (newVirtualTables.isNotEmpty()) + triggeredTables.value = triggeredTables() + trackedTables.update { it + newTables } + return tables + } + + /** + * Returns the virtual tables of the database, the tables without a root page of their own. + */ + // The triggers of the tracker itself are TEMP ones, which sqlite_master doesn't list + private fun triggeredTables(): Set = + connection.withQuery("SELECT DISTINCT tbl_name FROM sqlite_master WHERE type = 'trigger'") { cursor -> + buildSet { + cursor.forEachRow { add(cursor.getString(0)!!) } + } + } + + private fun virtualTables(): Set = + connection.withQuery("SELECT name FROM sqlite_master WHERE type = 'table' AND rootpage = 0") { cursor -> + buildSet { + cursor.forEachRow { add(cursor.getString(0)!!) } + } + } + + /** + * Returns the tables that the table or view named [relation] reads: the table itself, or the tables of the view. + */ + private fun tablesOf(relation: String): Set { + val type = connection.withQuery("SELECT type FROM sqlite_master WHERE name = ?", arrayOf(relation)) { + if (it.next()) it.getString(0) else null + } + if (type != "view") + return setOf(relation) + // An OpenRead's P2 is the root page of the table or index it opens, and P3 the database, 0 for main + val rootPages = connection.withQuery("EXPLAIN SELECT * FROM ${identifier(relation)}") { cursor -> + val opcode = cursor.getColumnIndex("opcode") + val rootPage = cursor.getColumnIndex("p2") + val database = cursor.getColumnIndex("p3") + buildSet { + cursor.forEachRow { + if (cursor.getString(opcode) == "OpenRead" && cursor.getLong(database) == 0L) + add(cursor.getLong(rootPage)) + } + } + } + if (rootPages.isEmpty()) + return emptySet() + // An index's tbl_name is its table + return connection.withQuery("SELECT DISTINCT tbl_name FROM sqlite_master WHERE rootpage IN (${rootPages.joinToString()})") { cursor -> + buildSet { + cursor.forEachRow { add(cursor.getString(0)!!) } + } + } + } + + /** + * Reads the counts of the tracked tables into [versions], after a scope that made [changes] and wrote rows to + * [writtenTables] ran. + * + * A schema change may have dropped or replaced a tracked table, and its triggers with it, so the triggers are + * created again where they are missing, and every tracked table counts as changed. A table may also have been + * created again as the other kind, virtual or not, or only now, so which tables are virtual is read again too. + */ + fun refresh(changes: Changes, writtenTables: Set) { + val tables = trackedTables.value + if (changes == Changes.NONE || tables.isEmpty()) + return + if (changes == Changes.SCHEMA) { + val virtual = tables intersect virtualTables() + virtualTables.value = virtual + triggeredTables.value = if (virtual.isEmpty()) emptySet() else triggeredTables() + for (table in tables - virtual) + createTriggers(table) + connection.execSQL(COUNT_ALL) + } else { + val virtual = virtualTables.value + val writtenVirtualTables = if (writtenTables.any { it in triggeredTables.value }) virtual else virtual intersect writtenTables + if (writtenVirtualTables.isNotEmpty()) { + val placeholders = writtenVirtualTables.joinToString(",") { "?" } + connection.execSQL("$COUNT_ALL WHERE table_name IN ($placeholders)", writtenVirtualTables.toTypedArray()) + } + } + // On Android, a query outside a transaction may run on a reader connection, which has no TEMP table + val counts = connection.withTransaction { + it.withQuery(SELECT_LOG) { cursor -> + buildMap { + cursor.forEachRow { put(cursor.getString(0)!!, cursor.getLong(1)) } + } + } + } + _versions.update { versions -> + versions + counts.filter { (table, count) -> count > (versions[table] ?: -1L) } + } + } + + /** + * Creates the triggers of [table] unless they exist. A table that doesn't exist, as after DROP, gets them once it + * is created again. + */ + private fun createTriggers(table: String) { + try { + for (operation in OPERATIONS) { + connection.execSQL( + "CREATE TEMP TRIGGER IF NOT EXISTS ${identifier("sqllin_modification_trigger_${table}_$operation")} " + + "AFTER $operation ON ${identifier(table)} BEGIN " + + "UPDATE $LOG SET version = version + 1 WHERE table_name = ${literal(table)}; END" + ) + } + } catch (e: Exception) { + if (!tableExists(table)) + return + throw e + } + } + + private fun tableExists(table: String): Boolean = + connection.withQuery("SELECT count(*) FROM sqlite_master WHERE type = 'table' AND name = ?", arrayOf(table)) { + it.next() && it.getLong(0) > 0 + } + + private companion object { + const val LOG = "sqllin_modification_log" + const val CREATE_LOG = "CREATE TEMP TABLE IF NOT EXISTS $LOG(table_name TEXT PRIMARY KEY NOT NULL, version INTEGER NOT NULL DEFAULT 0)" + const val INSERT_LOG_ROW = "INSERT OR IGNORE INTO $LOG(table_name) VALUES(?)" + const val COUNT_ALL = "UPDATE $LOG SET version = version + 1" + const val SELECT_LOG = "SELECT table_name, version FROM $LOG" + val OPERATIONS = listOf("INSERT", "UPDATE", "DELETE") + + fun identifier(name: String): String = "\"${name.replace("\"", "\"\"")}\"" + fun literal(value: String): String = "'${value.replace("'", "''")}'" + } +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/CreateStatementModifiers.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/CreateStatementModifiers.kt index f53966f2..86bd99e3 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/CreateStatementModifiers.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/CreateStatementModifiers.kt @@ -53,13 +53,17 @@ package com.ctrip.sqllin.dsl.annotation * auto-incrementing strategy that ensures row IDs are never reused. * **Important Note**: This parameter requires a property of type `Long?`, the only kind of key the * database assigns. Setting it to `true` on any other property is a compile-time error. + * @property onConflict What SQLite does with a row whose key is already in the table, as the key's `ON CONFLICT` clause + * says: by default, ABORT, which stops the statement * * @see DBRow * @see CompositePrimaryKey + * @see OnConflict */ @Target(AnnotationTarget.PROPERTY) @Retention(AnnotationRetention.BINARY) -public annotation class PrimaryKey(val autoIncrement: Boolean = false) +@OptIn(ExperimentalDSLDatabaseAPI::class) +public annotation class PrimaryKey(val autoIncrement: Boolean = false, val onConflict: OnConflict = OnConflict.ABORT) /** * Marks a property as a part of a composite primary key for the table. @@ -80,13 +84,17 @@ public annotation class PrimaryKey(val autoIncrement: Boolean = false) * declared `NOT NULL` in the generated table, because SQLite, unlike standard SQL, does not let * `PRIMARY KEY` imply it. * + * @property onConflict What SQLite does with a row whose key is already in the table, as the key's `ON CONFLICT` clause + * says. It can be given on any of the key's properties, which can't give different ones. + * * @see DBRow * @see PrimaryKey - * + * @see OnConflict */ @Target(AnnotationTarget.PROPERTY) @Retention(AnnotationRetention.BINARY) -public annotation class CompositePrimaryKey +@OptIn(ExperimentalDSLDatabaseAPI::class) +public annotation class CompositePrimaryKey(val onConflict: OnConflict = OnConflict.ABORT) /** * Marks a text column to use case-insensitive collation in SQLite. @@ -153,13 +161,18 @@ public annotation class CollateNoCase * - Multiple NULL values are allowed in a UNIQUE column (NULL is not equal to NULL in SQL) * - To prevent NULL values, combine with a non-nullable type: `val email: String` * + * @property onConflict What SQLite does with a row whose value is already in the column, as the constraint's + * `ON CONFLICT` clause says: by default, ABORT, which stops the statement + * * @see DBRow * @see CompositeUnique * @see CollateNoCase + * @see OnConflict */ @Target(AnnotationTarget.PROPERTY) @Retention(AnnotationRetention.BINARY) -public annotation class Unique +@OptIn(ExperimentalDSLDatabaseAPI::class) +public annotation class Unique(val onConflict: OnConflict = OnConflict.ABORT) /** * Marks a property as part of one or more composite UNIQUE constraints. @@ -220,13 +233,18 @@ public annotation class Unique * @property group One or more group numbers (0-based integers) identifying which * composite UNIQUE constraint(s) this property belongs to. Properties sharing * the same group number are combined into a single `UNIQUE(col1, col2, ...)` clause. + * @property onConflict What SQLite does with a row whose values are already in the group's columns, as the constraint's + * `ON CONFLICT` clause says, for each group the property is in. It can be given on any of a group's properties, which + * can't give different ones. * * @see DBRow * @see Unique + * @see OnConflict */ @Target(AnnotationTarget.PROPERTY) @Retention(AnnotationRetention.BINARY) -public annotation class CompositeUnique(vararg val group: Int = [0]) +@OptIn(ExperimentalDSLDatabaseAPI::class) +public annotation class CompositeUnique(vararg val group: Int = [0], val onConflict: OnConflict = OnConflict.ABORT) /** * Defines a table-level foreign key constraint that references another table. @@ -324,6 +342,9 @@ public annotation class CompositeUnique(vararg val group: Int = [0]) * @property tableName The name of the parent table being referenced (cannot be blank) * @property trigger The action to take when the referenced row is deleted or updated * @property constraintName Optional name for the constraint (appears in error messages and schema introspection) + * @property deferred Whether the constraint is checked when the transaction commits, rather than after each statement, + * as the SQL `DEFERRABLE INITIALLY DEFERRED` makes it, so that rows referring to each other, or a child inserted before + * its parent, can be written in one transaction * * @see ForeignKey * @see References @@ -338,6 +359,7 @@ public annotation class ForeignKeyGroup( val tableName: String, val trigger: Trigger = Trigger.NULL, val constraintName: String = "", + val deferred: Boolean = false, ) /** @@ -453,6 +475,9 @@ public annotation class ForeignKeyGroup( * @property foreignKeys Array of column names in the parent table to reference (cannot be empty) * @property trigger The action to take when the referenced row is deleted or updated (defaults to no action) * @property constraintName Optional name for the constraint (useful for error messages and debugging) + * @property deferred Whether the constraint is checked when the transaction commits, rather than after each statement, + * as the SQL `DEFERRABLE INITIALLY DEFERRED` makes it, so that rows referring to each other, or a child inserted before + * its parent, can be written in one transaction * * @see ForeignKeyGroup * @see ForeignKey @@ -467,6 +492,7 @@ public annotation class References( val trigger: Trigger = Trigger.NULL, val constraintName: String = "", vararg val foreignKeys: String, + val deferred: Boolean = false, ) /** @@ -907,4 +933,89 @@ public enum class Trigger { */ @Target(AnnotationTarget.PROPERTY) @Retention(AnnotationRetention.BINARY) -public annotation class Default(val value: String) \ No newline at end of file +public annotation class Default(val value: String) +/** + * Adds a `CHECK` constraint to a table: an SQL expression that every row has to satisfy, checked by SQLite whenever + * an INSERT or UPDATE writes a row, which fails if the expression is false for it. + * + * On a property, it is a constraint of the property's column; on a class annotated with [DBRow], a constraint of the + * table, after its columns. SQLite checks them alike, and both can refer to any column of the table, so a constraint + * that compares columns reads best on the class. They only differ when a column is dropped: the column's own + * constraints are dropped with it, while SQLite refuses to drop a column that another column's constraint or the + * table's refers to. Both can be repeated, and every constraint is checked. + * + * ```kotlin + * @OptIn(ExperimentalDSLDatabaseAPI::class) + * @DBRow + * @Check("endDate IS NULL OR endDate >= startDate", constraintName = "valid_period") + * @Serializable + * data class Membership( + * @PrimaryKey val id: Long?, + * @Check("level IN (0, 1, 2)") val level: Int, + * val startDate: String, + * val endDate: String?, + * ) + * // CREATE TABLE Membership(id INTEGER PRIMARY KEY,level INT NOT NULL CHECK (level IN (0, 1, 2)),startDate TEXT NOT NULL, + * // endDate TEXT,CONSTRAINT valid_period CHECK (endDate IS NULL OR endDate >= startDate)) + * ``` + * + * ### NULL Passes + * A row fails only when the expression is false: when it is NULL, the row passes. So `CHECK (age >= 0)` lets a NULL + * age through, which only a non-null property keeps out. This also catches expressions that look right: `date(day)` + * is NULL for a text that isn't a date, so `CHECK (date(day) = day)` lets every such text through, while + * `CHECK (day IS NULL OR date(day) IS day)` keeps them out. + * + * ### The Expression + * The expression is SQL, written into the CREATE TABLE statement as it is, and so only checked by SQLite when the table + * is created. It names columns by their names in the table, can't contain a subquery, so it can't refer to other + * tables, and can only call deterministic functions: a comparison with the current time such as + * `expires > date('now')` fails when a row is written. + * + * A row that fails the constraint makes its INSERT or UPDATE fail with the driver's exception, and an + * `INSERT_OR_IGNORE` skip the row, while an `INSERT_OR_REPLACE` fails as an INSERT does. SQLite's `ALTER TABLE` can't + * add a constraint to a table, and `ALTER_ADD_COLUMN` adds a column without its constraints: to add one to an existing + * table, rebuild the table, as `withName` describes. + * + * @property expression The SQL expression every row has to satisfy, such as `age BETWEEN 0 AND 150` + * @property constraintName The name of the constraint, which SQLite reports when a row fails it, or empty for none. + * SQLite reports an unnamed constraint by its expression from 3.34 on, but by the table's name before, as on Android + * below API 34, where only the names tell a table's constraints apart. It has to be an SQL identifier, and the names + * of a table's CHECK constraints have to differ + * + * @see DBRow + */ +@ExperimentalDSLDatabaseAPI +@Repeatable +@Target(AnnotationTarget.PROPERTY, AnnotationTarget.CLASS) +@Retention(AnnotationRetention.BINARY) +public annotation class Check(val expression: String, val constraintName: String = "") + +/** + * What SQLite does with a statement that would break a PRIMARY KEY or UNIQUE constraint, as the constraint's + * `ON CONFLICT` clause says, the `onConflict` of [PrimaryKey], [CompositePrimaryKey], [Unique] and [CompositeUnique]: + * ```kotlin + * @Unique(onConflict = OnConflict.REPLACE) val email: String + * // Generated: email TEXT NOT NULL UNIQUE ON CONFLICT REPLACE + * ``` + * The OR of a statement, as in INSERT OR IGNORE, overrides it. A composite key or group gets the one given on any of its + * properties, and properties of a key or group can't give different ones. + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public enum class OnConflict { + /** Stops the statement and rolls back the transaction. */ + ROLLBACK, + + /** Stops the statement and undoes its changes, as SQLite does by default, so that no clause is written. */ + ABORT, + + /** Stops the statement, keeping the changes it made to the rows before. */ + FAIL, + + /** Skips the row that would break the constraint, and goes on with the statement. */ + IGNORE, + + /** Deletes the rows the row conflicts with, and goes on with the statement. */ + REPLACE, +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/DBView.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/DBView.kt new file mode 100644 index 00000000..b62b3882 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/DBView.kt @@ -0,0 +1,44 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.annotation + +/** + * Marks a class as the rows of a SQLite view. + * + * sqllin-processor generates a view object for the annotated class, as it generates a table object for a `@DBRow` + * class: the view of a class `Adult` is the object `AdultView`, with a property for each column, named after the class's + * properties. A SELECT reads the view like a table, and `CREATE_VIEW(AdultView) AS (select)` creates it from a SELECT of + * its row type. A view can't be written to, so its properties take none of the column constraint annotations, such as + * `@PrimaryKey` or `@Unique`. + * + * The class must also be annotated with `@Serializable`. + * + * Example: + * ```kotlin + * @DBView("adults") + * @Serializable + * data class Adult(val name: String, val age: Int) + * ``` + * + * @property viewName The name of the SQLite view. If not specified or empty, the name of the annotated class is used. + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +@Target(AnnotationTarget.CLASS) +@Retention(AnnotationRetention.BINARY) +public annotation class DBView(val viewName: String = "") diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/Fts.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/Fts.kt new file mode 100644 index 00000000..caf58321 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/Fts.kt @@ -0,0 +1,92 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.annotation + +/** + * Makes the table of a `@DBRow` class an FTS4 full-text search table: `CREATE VIRTUAL TABLE name USING fts4(...)`. + * + * The properties of the class are the columns, and have to be `String` or `String?`, as FTS indexes text. The class + * can also have a `Long` or `Long?` property named `rowid` or `docid`, annotated with `@PrimaryKey`, which reads and + * writes the row's id rather than being a column; `Long?` lets SQLite assign it. The other column annotations, such as + * `@Unique`, don't apply to an FTS table. + * + * An FTS table is searched with `MATCH`, and its matches described with `snippet`, `offsets` and `matchinfo`. FTS4 is in + * the SQLite of every platform SQLlin supports, Android's included. It has no built-in ranking: `matchinfo` gives the + * statistics to compute one from. + * + * Example: + * ```kotlin + * @OptIn(ExperimentalDSLDatabaseAPI::class) + * @DBRow("articles") + * @Fts4(tokenizer = FtsTokenizer.PORTER, prefix = [2]) + * @Serializable + * data class Article(@PrimaryKey val rowid: Long?, val title: String, val body: String) + * ``` + * + * @property tokenizer The tokenizer that splits the text into terms + * @property tokenizerArgs Arguments for the tokenizer, such as `"remove_diacritics=2"` for [FtsTokenizer.UNICODE61] + * @property prefix The lengths of the prefixes to index, which make prefix queries such as `kot*` faster + * @property notIndexed The columns whose values are stored but not indexed, so a query doesn't find them + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +@Target(AnnotationTarget.CLASS) +@Retention(AnnotationRetention.BINARY) +public annotation class Fts4( + val tokenizer: FtsTokenizer = FtsTokenizer.SIMPLE, + val tokenizerArgs: Array = [], + val prefix: IntArray = [], + val notIndexed: Array = [], +) + +/** + * Makes the table of a `@DBRow` class an FTS3 full-text search table: `CREATE VIRTUAL TABLE name USING fts3(...)`. + * + * FTS3 is the older version of FTS4, which it reads the same way, without FTS4's options and statistics. Prefer + * [Fts4], unless the table has to be FTS3, such as one an earlier version of an app created. The rules for the + * properties are those of [Fts4]. + * + * @property tokenizer The tokenizer that splits the text into terms + * @property tokenizerArgs Arguments for the tokenizer, such as `"remove_diacritics=2"` for [FtsTokenizer.UNICODE61] + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +@Target(AnnotationTarget.CLASS) +@Retention(AnnotationRetention.BINARY) +public annotation class Fts3( + val tokenizer: FtsTokenizer = FtsTokenizer.SIMPLE, + val tokenizerArgs: Array = [], +) + +/** + * A tokenizer of FTS3 and FTS4, which splits text into terms. + * + * @property sqlName Its name in `CREATE VIRTUAL TABLE` + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public enum class FtsTokenizer(public val sqlName: String) { + /** Splits on spaces and punctuation and folds ASCII to lower case. */ + SIMPLE("simple"), + /** [SIMPLE], then reduces English words to their stems, so that "running" matches "run". */ + PORTER("porter"), + /** Splits by the Unicode properties of the characters and folds them to lower case, removing diacritics by default. */ + UNICODE61("unicode61"), +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/PlatformDependentSQLiteAPI.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/PlatformDependentSQLiteAPI.kt new file mode 100644 index 00000000..4f0dab5b --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/annotation/PlatformDependentSQLiteAPI.kt @@ -0,0 +1,47 @@ +/* + * Copyright (C) 2025 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.annotation + +/** + * Marks the APIs that only work with some SQLite versions or compile-time options, which some platforms SQLlin + * supports don't have. + * + * The SQLite SQLlin runs on depends on the platform: Android's system SQLite, whose version depends on the API level + * (3.9 at API 24, SQLlin's minimum, 3.32 below API 34, and 3.39 from API 34 on) and which lacks options such as the + * math functions; the SQLite of the JVM driver; Apple's system SQLite, whose version depends on the OS version; and on + * Linux and Windows, the SQLite the app links. An API with this annotation fails at runtime where the SQLite lacks what + * it needs, which its documentation describes, so using one has to be accepted, with + * `@OptIn(PlatformDependentSQLiteAPI::class)` or the compiler argument + * `-opt-in=com.ctrip.sqllin.dsl.annotation.PlatformDependentSQLiteAPI`, by an app whose platforms all have it. + * + * @see OptIn + * @see RequiresOptIn + */ +@RequiresOptIn( + message = "This API only works with some SQLite versions or compile-time options, which some platforms don't have, " + + "such as Android's system SQLite at lower API levels. Its documentation describes what it needs: use it only " + + "if all your app's platforms have that.", + level = RequiresOptIn.Level.ERROR +) +@Target( + AnnotationTarget.CLASS, + AnnotationTarget.FUNCTION, + AnnotationTarget.PROPERTY, + AnnotationTarget.TYPEALIAS +) +@Retention(AnnotationRetention.BINARY) +public annotation class PlatformDependentSQLiteAPI diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/DefaultValues.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/DefaultValues.kt new file mode 100644 index 00000000..6a73d0ca --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/DefaultValues.kt @@ -0,0 +1,28 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI + +/** + * The SQL `DEFAULT VALUES` of INSERT, which inserts a row of the default values of all columns, as + * `table INSERT DEFAULT_VALUES` does. + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public class DefaultValues internal constructor() diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/DerivedTable.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/DerivedTable.kt new file mode 100644 index 00000000..d1e0379c --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/DerivedTable.kt @@ -0,0 +1,41 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.sql.statement.SelectStatement +import kotlinx.serialization.KSerializer + +/** + * A derived table: a SELECT written in the FROM clause or a JOIN of another query, as `(SELECT ...) AS name`. + * + * Its rows are those of [view]'s type, and it is named after [view], whose column properties so name its columns, + * though no such view has to exist in the database. Like a view, it can only be read. + * + * @author Yuang Qiao + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +internal class DerivedTable(private val view: View, select: SelectStatement) : Relation(view.tableName) { + + override fun kSerializer(): KSerializer = view.kSerializer() + + override val fromSQL: String = "(${select.sqlStr}) AS $tableName" + + override val fromParameters: List = select.parameters?.toList().orEmpty() + + override val readTables: Set = select.tables +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/From.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/From.kt new file mode 100644 index 00000000..7b6b39e5 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/From.kt @@ -0,0 +1,173 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql + +import kotlinx.serialization.ExperimentalSerializationApi +import kotlinx.serialization.descriptors.SerialDescriptor +import kotlinx.serialization.encoding.CompositeDecoder + +/** + * What a SELECT reads, as its FROM clause writes it: a relation, or the relations of a join. + * + * It finds the columns a SELECT names among its relations, so that the SELECT can check them: a column of one relation + * is written with its relation's name, as other relations may have a column of that name, while a column that a USING + * or a NATURAL join merges is written with its name alone. A relation that an outer join can leave without a matching + * row is nullable, as its columns are NULL in such a row. + * + * @property members The relations, in the order they are joined + * @property sql The FROM clause, without FROM + * @property parameters The parameters of [sql], which a statement binds before those of its other clauses + * @property tables The tables and views the relations read, which an observed query watches + * @property mergedColumns The names of the columns that USING or NATURAL joins merge + * + * @author Yuang Qiao + */ +internal class From private constructor( + val members: List, + val sql: String, + val parameters: List, + val tables: Set, + private val mergedColumns: Set, +) { + + /** + * A relation of a FROM clause, which is [isNullable] if an outer join can leave it without a matching row. + */ + class Member(val relation: Relation<*>, val isNullable: Boolean) { + val columns: SerialDescriptor + get() = relation.kSerializer().descriptor + } + + /** + * A column that a SELECT names, as [find] finds it. + * + * @property descriptor The column's descriptor, of its relation's row type + * @property isNullable Whether the column can be NULL: as it is nullable, or its relation is + * @property relation The relation the column belongs to + * @property sql The column as the SELECT writes it + */ + class Column(val descriptor: SerialDescriptor, val isNullable: Boolean, val relation: Relation<*>, val sql: String) + + val isJoin: Boolean + get() = members.size > 1 + + /** + * The names of the relations an outer join can leave without a matching row. + */ + val nullableTables: Set = members.filterTo(ArrayList()) { it.isNullable }.mapTo(HashSet()) { it.relation.tableName } + + /** + * How a message names what the SELECT reads. + */ + val description: String + get() = if (isJoin) + "the join of ${members.joinToString { "'${it.relation.tableName}'" }}" + else + "table '${members.single().relation.tableName}'" + + /** + * Returns the column named [name] that a SELECT of [projectionName] reads into its property of that name. + * + * @throws IllegalArgumentException if no relation has such a column, or several have one that isn't merged + */ + @OptIn(ExperimentalSerializationApi::class) + fun find(name: String, projectionName: String): Column { + val candidates = members.filter { it.columns.getElementIndex(name) != CompositeDecoder.UNKNOWN_NAME } + require(candidates.isNotEmpty()) { + "Can't select '$projectionName' from $description: its property '$name' isn't a column of ${if (isJoin) "any of them" else "that table"}." + } + val isMerged = name in mergedColumns + require(candidates.size == 1 || isMerged) { + "Can't select '$projectionName' from $description: its property '$name' is a column of ${candidates.joinToString { "'${it.relation.tableName}'" }}, which can't be told apart. Select one of them with AS, as in `${candidates.first().relation.tableName}.$name AS $projectionName::$name`." + } + val member = candidates.first() + val columns = member.columns + val descriptor = columns.getElementDescriptor(columns.getElementIndex(name)) + val sql = if (!isJoin || isMerged) name else "${member.relation.tableName}.$name" + return Column(descriptor, descriptor.isNullable || member.isNullable, member.relation, sql) + } + + /** + * Returns this FROM clause joined with [relation] by [keyword], such as `INNER JOIN`, under [constraint]. + * + * @param isLeftNullable Whether the join can leave the relations before it without a matching row, as RIGHT and + * FULL OUTER JOIN do + * @param isRightNullable Whether the join can leave [relation] without a matching row, as LEFT and FULL OUTER JOIN do + * @param isNatural Whether the join is NATURAL, which merges the columns of the same name + * @throws IllegalArgumentException if [relation] is already joined, which only a derived table of it can be again + */ + fun join( + keyword: String, + relation: Relation<*>, + constraint: Constraint, + isLeftNullable: Boolean, + isRightNullable: Boolean, + isNatural: Boolean, + ): From { + require(members.none { it.relation.tableName == relation.tableName }) { + "Can't join '${relation.tableName}' with $description, which already has it. To join a table with itself, join a derived table of its rows, which gives it another name." + } + val newMembers = members.map { if (isLeftNullable) Member(it.relation, true) else it } + Member(relation, isRightNullable) + val naturalColumns = if (isNatural) { + val names = relation.columnNames() + members.flatMapTo(HashSet()) { it.relation.columnNames() } intersect names + } else { + emptySet() + } + return From( + members = newMembers, + sql = "$sql $keyword ${relation.fromSQL}${constraint.sql}", + parameters = parameters + relation.fromParameters + constraint.parameters, + tables = tables + relation.readTables + constraint.tables, + mergedColumns = mergedColumns + constraint.mergedColumns + naturalColumns, + ) + } + + /** + * The constraint of a join: ON, USING, or none. + * + * @property sql The constraint as the join writes it, after the relation + * @property parameters The parameters of [sql] + * @property tables The tables the subqueries of an ON condition read + * @property mergedColumns The names of the columns USING merges + */ + class Constraint( + val sql: String, + val parameters: List = emptyList(), + val tables: Set = emptySet(), + val mergedColumns: Set = emptySet(), + ) { + companion object { + val NONE = Constraint("") + } + } + + companion object { + + /** + * Returns the FROM clause of [relation] alone. + */ + fun of(relation: Relation<*>): From = + From(listOf(Member(relation, false)), relation.fromSQL, relation.fromParameters, relation.readTables, emptySet()) + + @OptIn(ExperimentalSerializationApi::class) + private fun Relation<*>.columnNames(): Set { + val descriptor = kSerializer().descriptor + return (0 ..< descriptor.elementsCount).mapTo(HashSet()) { descriptor.getElementName(it) } + } + } +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/FtsTable.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/FtsTable.kt new file mode 100644 index 00000000..31d9669b --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/FtsTable.kt @@ -0,0 +1,33 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI + +/** + * An FTS3 or FTS4 full-text search table, which [Table]'s operations write to and read like any table, and which + * `MATCH` searches. + * + * FTS table objects are generated by sqllin-processor for `@DBRow` classes annotated with + * [@Fts4][com.ctrip.sqllin.dsl.annotation.Fts4] or [@Fts3][com.ctrip.sqllin.dsl.annotation.Fts3]. + * + * @param T The type of the table's rows + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public abstract class FtsTable(tableName: String) : Table(tableName) diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/Table.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/Table.kt index d0a70c9b..f25a04dc 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/Table.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/Table.kt @@ -16,8 +16,50 @@ package com.ctrip.sqllin.dsl.sql +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import kotlinx.serialization.KSerializer +/** + * What a SELECT reads: a [Table], or a [View]. + * + * A relation names the table or view and gives the serializer of its rows, which is all a query needs, as a property + * is read from the column of the same name. Only a [Table] can be written to, created with `CREATE` and altered. + * + * @param T The type of the rows + * @property tableName The name of the table or view in SQLite + * + * @author Yuang Qiao + */ +public abstract class Relation internal constructor( + internal val tableName: String, +) { + /** + * Returns the kotlinx.serialization serializer for the entity type. + * + * Used internally for serializing entities to SQL INSERT/UPDATE statements + * and deserializing query results to entity objects. + */ + public abstract fun kSerializer(): KSerializer + + /** + * The relation as written after FROM or JOIN: its name, or the subquery of a derived table with its alias. + */ + internal open val fromSQL: String + get() = tableName + + /** + * The parameters of [fromSQL], which a statement binds before those of its other clauses. + */ + internal open val fromParameters: List + get() = emptyList() + + /** + * The tables and views a query of this relation reads, which an observed query watches. + */ + internal open val readTables: Set + get() = setOf(tableName) +} + /** * Abstract base class representing a SQLite table with type-safe DSL operations. * @@ -50,16 +92,7 @@ import kotlinx.serialization.KSerializer * * @author Yuang Qiao */ -public abstract class Table( - internal val tableName: String, -) { - /** - * Returns the kotlinx.serialization serializer for the entity type. - * - * Used internally for serializing entities to SQL INSERT/UPDATE statements - * and deserializing query results to entity objects. - */ - public abstract fun kSerializer(): KSerializer +public abstract class Table(tableName: String) : Relation(tableName) { /** * Metadata about the table's primary key configuration. @@ -142,30 +175,66 @@ public abstract class Table( * at a table that no longer exists once it's dropped. * * The returned table has no column properties, as those of this table name this table. It is meant for statements - * on the table as a whole: `CREATE`, `INSERT`, `DROP` and `ALTER_RENAME_TABLE_TO`. + * on the table as a whole: `CREATE`, `INSERT`, `DROP` and `ALTER_RENAME_TABLE_TO`. The table of an [FtsTable] is an + * [FtsTable] too, with the same options, which is how an FTS table is rebuilt with other ones. * * @param name The name of the returned table * @return A table with this table's columns, constraints and row type, named [name] */ -public fun Table.withName(name: String): Table = NamedTable(this, name) +@OptIn(ExperimentalDSLDatabaseAPI::class) +public fun Table.withName(name: String): Table { + require(name.isNotBlank()) { "The name of a table can't be blank." } + return if (this is FtsTable) NamedFtsTable(this, name) else NamedTable(this, name) +} /** * A table with the structure of [table] under another name. */ private class NamedTable(private val table: Table, name: String) : Table(name) { - init { - require(name.isNotBlank()) { "The name of a table can't be blank." } - } + override fun kSerializer(): KSerializer = table.kSerializer() + + override val primaryKeyInfo: PrimaryKeyInfo? + get() = table.primaryKeyInfo + + override val createSQL: String = table.createSQLNamed(name) +} + +/** + * An FTS table with the structure of [table] under another name. + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +private class NamedFtsTable(private val table: FtsTable, name: String) : FtsTable(name) { override fun kSerializer(): KSerializer = table.kSerializer() override val primaryKeyInfo: PrimaryKeyInfo? get() = table.primaryKeyInfo - override val createSQL: String = run { - val prefix = "CREATE TABLE ${table.tableName}(" - check(table.createSQL.startsWith(prefix)) { "The CREATE TABLE statement of table '${table.tableName}' doesn't start with '$prefix'." } - "CREATE TABLE $name(${table.createSQL.substring(prefix.length)}" + override val createSQL: String = table.createSQLNamed(name) +} + +/** + * Returns the CREATE statement of this table with `IF NOT EXISTS`, which does nothing if a table of its name exists: + * `CREATE TABLE IF NOT EXISTS name(...)`, or `CREATE VIRTUAL TABLE IF NOT EXISTS name USING ...`. + */ +internal fun Table<*>.createSQLIfNotExists(): String { + for (start in listOf("CREATE TABLE ", "CREATE VIRTUAL TABLE ")) { + if (createSQL.startsWith(start)) + return "${start}IF NOT EXISTS ${createSQL.substring(start.length)}" + } + error("The CREATE statement of table '$tableName' doesn't start with 'CREATE TABLE ' or 'CREATE VIRTUAL TABLE '.") +} + +/** + * Returns the CREATE statement of this table with [name] in place of its own: `CREATE TABLE name(...)`, or + * `CREATE VIRTUAL TABLE name USING ...` for a virtual table. + */ +private fun Table<*>.createSQLNamed(name: String): String { + for ((start, end) in listOf("CREATE TABLE " to "(", "CREATE VIRTUAL TABLE " to " USING ")) { + val prefix = "$start$tableName$end" + if (createSQL.startsWith(prefix)) + return "$start$name$end${createSQL.substring(prefix.length)}" } + error("The CREATE statement of table '$tableName' doesn't start with 'CREATE TABLE $tableName(' or 'CREATE VIRTUAL TABLE $tableName USING '.") } \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/Trigger.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/Trigger.kt new file mode 100644 index 00000000..10cde51d --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/Trigger.kt @@ -0,0 +1,75 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.sql.clause.ClauseElement +import com.ctrip.sqllin.dsl.sql.clause.SelectCondition + +/** + * A trigger that `CREATE_TRIGGER` names, which `BEFORE` or `AFTER` gives the event it fires on: + * ```kotlin + * CREATE_TRIGGER("person_audit") AFTER DELETE ON PersonTable BEGIN { + * AuditTable INSERT listOf(old(PersonTable.id) AS Audit::personId, literal("delete") AS Audit::action) + * } + * ``` + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public class TriggerName internal constructor(internal val name: String, internal val isIfNotExists: Boolean) + +/** + * The event a trigger fires on: `INSERT`, `DELETE`, `UPDATE`, or `UPDATE_OF` columns. A trigger on INSERT has a new + * row, one on DELETE an old row, and one on UPDATE both. + */ +@ExperimentalDSLDatabaseAPI +public class TriggerEvent internal constructor( + internal val sql: String, + internal val hasNewRow: Boolean, + internal val hasOldRow: Boolean, + internal val columns: List> = emptyList(), +) + +/** + * A trigger and when it fires, which `ON` gives its table. + */ +@ExperimentalDSLDatabaseAPI +public class TriggerTiming internal constructor(internal val trigger: TriggerName, internal val timing: String, internal val event: TriggerEvent) + +/** + * A trigger on its table, which `WHEN` can give a condition, and `BEGIN` gives its body. + */ +@ExperimentalDSLDatabaseAPI +public class TriggerDefinition internal constructor( + internal val timing: TriggerTiming, + internal val table: Table<*>, + internal val condition: SelectCondition? = null, +) + +/** + * What `RAISE` does with the statement that fired the trigger: `ABORT` stops it and undoes its changes, `FAIL` stops it + * and keeps the changes it made before, and `ROLLBACK` stops it and rolls back the transaction. + */ +@ExperimentalDSLDatabaseAPI +public class RaiseResolution internal constructor(internal val sql: String) + +/** + * `RAISE(IGNORE)`, which skips the row that fired the trigger, and the rest of the trigger, while the statement goes on. + */ +@ExperimentalDSLDatabaseAPI +public class RaiseIgnore internal constructor() diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/View.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/View.kt new file mode 100644 index 00000000..cfdb0010 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/View.kt @@ -0,0 +1,44 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI + +/** + * A SQLite view, which a SELECT reads like a table. + * + * View objects are generated by sqllin-processor for classes annotated with + * [@DBView][com.ctrip.sqllin.dsl.annotation.DBView]: the view of a class `Adult` is the object `AdultView`, with a + * property for each column. A view is created with `CREATE_VIEW(AdultView) AS (select)` and dropped with + * `DROP(AdultView)`. As it isn't a [Table], it can't be written to, created with `CREATE`, or altered. + * + * @param T The type of the view's rows + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public abstract class View(viewName: String) : Relation(viewName) + +/** + * The view to create, as `CREATE_VIEW(view)` starts. It becomes a statement once `AS` gives it its SELECT. + * + * @param T The type of the view's rows + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public class ViewDefinition internal constructor(internal val view: View, internal val isIfNotExists: Boolean = false) diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/BaseJoinClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/BaseJoinClause.kt index 16bf7385..03db52a0 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/BaseJoinClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/BaseJoinClause.kt @@ -16,8 +16,8 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.sql.Relation import com.ctrip.sqllin.dsl.annotation.StatementDslMaker -import com.ctrip.sqllin.dsl.sql.Table import com.ctrip.sqllin.dsl.sql.statement.JoinSelectStatement import com.ctrip.sqllin.dsl.sql.statement.JoinStatementWithoutCondition @@ -32,16 +32,28 @@ import com.ctrip.sqllin.dsl.sql.statement.JoinStatementWithoutCondition * * @author Yuang Qiao */ -public sealed class BaseJoinClause(private vararg val tables: Table<*>) : SelectClause { +public sealed class BaseJoinClause(private vararg val joinedTables: Relation<*>) : SelectClause { internal abstract val clauseName: String + /** + * The tables and views that the relations this clause joins read, which an observed query watches. + */ + internal val tables: Set + get() = joinedTables.flatMapTo(LinkedHashSet()) { it.readTables } + + /** + * The parameters of the derived tables this clause joins, in their order. + */ + internal val parameters: List + get() = joinedTables.flatMap { it.fromParameters } + final override val clauseStr: String get() = buildString { append(clauseName) - tables.forEachIndexed { index, table -> - append(table.tableName) - if (index < tables.lastIndex) + joinedTables.forEachIndexed { index, table -> + append(table.fromSQL) + if (index < joinedTables.lastIndex) append(',') } } @@ -54,7 +66,7 @@ public sealed class BaseJoinClause(private vararg val tables: Table<*>) : Sel * * @param R The result entity type after JOIN */ -public sealed class NaturalJoinClause(vararg tables: Table<*>) : BaseJoinClause(*tables) +public sealed class NaturalJoinClause(vararg tables: Relation<*>) : BaseJoinClause(*tables) /** * JOIN clause that requires an ON or USING condition. @@ -63,18 +75,42 @@ public sealed class NaturalJoinClause(vararg tables: Table<*>) : BaseJoinClau * * @param R The result entity type after JOIN */ -public sealed class JoinClause(vararg tables: Table<*>) : BaseJoinClause(*tables) +public sealed class JoinClause(vararg tables: Relation<*>) : BaseJoinClause(*tables) +/** + * Completes a join of the old join API with `ON condition`. + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. + */ @Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") @StatementDslMaker public infix fun JoinStatementWithoutCondition.ON(condition: SelectCondition): JoinSelectStatement = convertToJoinSelectStatement(condition) +/** + * Completes a join of the old join API with `USING (column)`. + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. + */ @Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") @StatementDslMaker public inline infix fun JoinStatementWithoutCondition.USING(clauseElement: ClauseElement<*>): JoinSelectStatement = USING(listOf(clauseElement)) +/** + * Completes a join of the old join API with `USING (columns)`. + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. + */ @Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") @StatementDslMaker public infix fun JoinStatementWithoutCondition.USING(clauseElements: Iterable>): JoinSelectStatement = diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Case.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Case.kt new file mode 100644 index 00000000..d5ef990a --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Case.kt @@ -0,0 +1,99 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +@file:Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") + +package com.ctrip.sqllin.dsl.sql.clause + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.annotation.StatementDslMaker +import com.ctrip.sqllin.dsl.sql.compiler.inlineParameters + +/** + * The condition of a branch of `CASE`, which `THEN` gives the branch's value: `WHEN(age LT 18) THEN literal("minor")`. + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public class When internal constructor(internal val condition: SelectCondition) + +/** + * A branch of `CASE`: the value it gives where its condition holds. + */ +@ExperimentalDSLDatabaseAPI +public class CaseBranch> internal constructor(internal val condition: SelectCondition, internal val value: E) + +/** + * Starts a branch of `CASE` with its [condition]. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public fun WHEN(condition: SelectCondition): When = When(condition) + +/** + * Gives this branch of `CASE` its [value]. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun > When.THEN(value: E): CaseBranch = CaseBranch(condition, value) + +/** + * SQL's searched `CASE`: the value of the first of [branches] whose condition holds, or [ELSE] where none does, which is + * NULL if it is omitted. + * ```kotlin + * CASE( + * WHEN(age LT 18) THEN literal("minor"), + * WHEN(age LT 65) THEN literal("adult"), + * ELSE = literal("senior"), + * ) + * // CASE WHEN person.age<18 THEN 'minor' WHEN person.age<65 THEN 'adult' ELSE 'senior' END + * ``` + * The branches and `ELSE` have values of the same type, which `CASE` has. It can be NULL where a value can, or where + * `ELSE` is omitted. The values of the conditions are written into the SQL, as the values of any expression are. + * + * @throws IllegalArgumentException if there are no branches + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +@Suppress("UNCHECKED_CAST") +public fun > CASE(vararg branches: CaseBranch, ELSE: E? = null): E { + require(branches.isNotEmpty()) { "CASE needs at least one branch: WHEN(condition) THEN value." } + val values = branches.map { it.value } + listOfNotNull(ELSE) + val conditions = branches.map { it.condition } + val sql = buildString { + append("CASE") + for (branch in branches) { + append(" WHEN ") + append(inlineParameters(branch.condition.conditionSQL, branch.condition.parameters)) + append(" THEN ") + append(branch.value.sql) + } + if (ELSE != null) { + append(" ELSE ") + append(ELSE.sql) + } + append(" END") + } + val traits = Traits( + isNullable = ELSE == null || values.any { it.isNullable }, + isAggregate = values.any { it.isAggregate }, + isNullOnNoRows = ELSE == null || values.any { it.isNullOnNoRows }, + columnTables = values.flatMapTo(LinkedHashSet()) { it.columnTables }, + tables = values.flatMapTo(LinkedHashSet()) { it.tables } + conditions.flatMap { it.tables }, + isDeterministic = values.all { it.isDeterministic } && conditions.all { it.isDeterministic }, + ) + return branches.first().value.derive(sql, traits) as E +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Cast.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Cast.kt new file mode 100644 index 00000000..2f2f6175 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Cast.kt @@ -0,0 +1,132 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +@file:Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") + +package com.ctrip.sqllin.dsl.sql.clause + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.annotation.FunctionDslMaker +import com.ctrip.sqllin.dsl.annotation.KeyWordDslMaker +import com.ctrip.sqllin.dsl.annotation.StatementDslMaker + +/** + * The types an expression can be converted to with `CAST`, named as SQLlin names the type of a column of that Kotlin + * type when it creates a table: `INT` for `Int`, `BIGINT` for `Long`, `TEXT` for `String` and so on. SQLite converts by + * the type's affinity, so all integer types convert alike, as do `FLOAT` and `DOUBLE`. + * + * There is no `BOOLEAN`, as SQLite's `CAST(5 AS BOOLEAN)` is `5`, not a Boolean, nor `NUMERIC`, whose result is an + * integer or a real number depending on the value. + * + * @param V The Kotlin type of the values the expression is converted to + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public sealed class SQLType(internal val name: String) + +/** + * A number type, which `CAST` converts an expression to: an integer type truncates a real number, as `CAST(3.7 AS INT)` + * is `3`, and reads the leading digits of a string, as `CAST('12abc' AS INT)` is `12`. It doesn't check the range of the + * Kotlin type, so a value that doesn't fit is read back wrong. + */ +@ExperimentalDSLDatabaseAPI +public sealed class NumberType(name: String) : SQLType(name) + +/** `Byte`, the type of a `Byte` column. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object TINYINT : NumberType("TINYINT") + +/** `Short`, the type of a `Short` column. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object SMALLINT : NumberType("SMALLINT") + +/** `Int`, the type of an `Int` column. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object INT : NumberType("INT") + +/** `Long`, the type of a `Long` column. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object BIGINT : NumberType("BIGINT") + +/** `Float`, the type of a `Float` column. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object FLOAT : NumberType("FLOAT") + +/** `Double`, the type of a `Double` column. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object DOUBLE : NumberType("DOUBLE") + +/** `String`, the type of a `String` column: a number converts to its decimal digits, a BLOB to its bytes as text. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object TEXT : SQLType("TEXT") + +/** `ByteArray`, the type of a `ByteArray` column: a string converts to its bytes. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object BLOB : SQLType("BLOB") + +/** + * An expression and the type `CAST` converts it to, as `AS` pairs them in `CAST(price AS INT)`. + */ +@ExperimentalDSLDatabaseAPI +public class Cast> internal constructor(internal val element: ClauseElement<*>, internal val type: T) + +/** + * Pairs this expression with the [type] that `CAST` converts it to: `CAST(price AS INT)`. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun > ClauseElement<*>.AS(type: T): Cast = Cast(this, type) + +/** + * Converts an expression to a number type, SQL's `CAST(x AS type)`, as in `CAST(price AS INT)`, an `Int`. It is NULL + * where the expression is. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun CAST(cast: Cast>): ClauseNumber { + val element = cast.element + return ClauseNumber("CAST(${element.sql} AS ${cast.type.name})", element.table, isFunction = true, isNullable = element.isNullable, isAggregate = element.isAggregate, isNullOnNoRows = element.isNullOnNoRows, columnTables = element.columnTables, tables = element.tables, isDeterministic = element.isDeterministic) +} + +/** + * Converts an expression to a string, SQL's `CAST(x AS TEXT)`, as in `CAST(price AS TEXT)`. It is NULL where the + * expression is. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun CAST(cast: Cast): ClauseString { + val element = cast.element + return ClauseString("CAST(${element.sql} AS TEXT)", element.table, isFunction = true, isNullable = element.isNullable, isAggregate = element.isAggregate, isNullOnNoRows = element.isNullOnNoRows, columnTables = element.columnTables, tables = element.tables, isDeterministic = element.isDeterministic) +} + +/** + * Converts an expression to a BLOB, SQL's `CAST(x AS BLOB)`. It is NULL where the expression is. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun CAST(cast: Cast): ClauseBlob { + val element = cast.element + return ClauseBlob("CAST(${element.sql} AS BLOB)", element.table, isFunction = true, isNullable = element.isNullable, isAggregate = element.isAggregate, isNullOnNoRows = element.isNullOnNoRows, columnTables = element.columnTables, tables = element.tables, isDeterministic = element.isDeterministic) +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseBlob.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseBlob.kt index 7b8eeaca..f149d54e 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseBlob.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseBlob.kt @@ -16,7 +16,7 @@ package com.ctrip.sqllin.dsl.sql.clause -import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.Relation /** * Wrapper for BLOB (Binary Large Object) column/function references in SQL clauses. @@ -44,23 +44,35 @@ import com.ctrip.sqllin.dsl.sql.Table */ public class ClauseBlob internal constructor( valueName: String, - table: Table<*>, + table: Relation<*>, isFunction: Boolean, isNullable: Boolean, isAggregate: Boolean, isNullOnNoRows: Boolean, -) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows) { + columnTables: Set? = null, + tables: Set = emptySet(), + isDeterministic: Boolean = true, + isLiteral: Boolean = false, +) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows, columnTables, tables, isDeterministic, isLiteral) { /** * Creates the element of a column, as the code generated for a table does. * * @param isNullable Whether the column is nullable */ - public constructor(valueName: String, table: Table<*>, isNullable: Boolean) : + public constructor(valueName: String, table: Relation<*>, isNullable: Boolean) : this(valueName, table, isFunction = false, isNullable = isNullable, isAggregate = false, isNullOnNoRows = true) - override fun toAggregate(valueName: String, table: Table<*>): ClauseBlob = - ClauseBlob(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true) + override fun toAggregate(valueName: String, table: Relation<*>): ClauseBlob = + ClauseBlob(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true, columnTables = columnTables, tables = tables, isDeterministic = isDeterministic) + + override fun derive(valueName: String, traits: Traits): ClauseBlob = + ClauseBlob(valueName, table, isFunction = true, isNullable = traits.isNullable, isAggregate = traits.isAggregate, isNullOnNoRows = traits.isNullOnNoRows, columnTables = traits.columnTables, tables = traits.tables, isDeterministic = traits.isDeterministic) + + override fun literalOf(value: ByteArray): ClauseBlob = + ClauseBlob(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + + override fun rebind(relation: Relation<*>): ClauseBlob = ClauseBlob(valueName, relation, isNullable) /** * Creates an equality comparison condition (=). @@ -180,7 +192,7 @@ public class ClauseBlob internal constructor( append('?') } } - return SelectCondition(sql, if (blob == null) null else mutableListOf(blob)) + return condition(sql, if (blob == null) null else mutableListOf(blob)) } private fun appendBlob(symbol: String, blob: ByteArray): SelectCondition { @@ -192,7 +204,7 @@ public class ClauseBlob internal constructor( append(valueName) append(symbol) } - return SelectCondition(sql, mutableListOf(blob)) + return condition(sql, mutableListOf(blob)) } private fun appendClauseBlob(symbol: String, clauseBlob: ClauseBlob): SelectCondition { @@ -203,7 +215,7 @@ public class ClauseBlob internal constructor( append(' ') clauseBlob.appendSQL(this) } - return SelectCondition(sql, null) + return condition(sql, null, clauseBlob) } /** @@ -232,7 +244,7 @@ public class ClauseBlob internal constructor( } append(')') } - return SelectCondition(sql, parameters) + return condition(sql, parameters) } /** @@ -252,7 +264,7 @@ public class ClauseBlob internal constructor( append(valueName) append(" BETWEEN ? AND ?") } - return SelectCondition(sql, mutableListOf(range.first, range.second)) + return condition(sql, mutableListOf(range.first, range.second)) } override fun hashCode(): Int = valueName.hashCode() + table.tableName.hashCode() diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseBoolean.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseBoolean.kt index 497f12ca..fa82c21b 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseBoolean.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseBoolean.kt @@ -16,7 +16,7 @@ package com.ctrip.sqllin.dsl.sql.clause -import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.Relation /** * Wrapper for Boolean column/function references in SQL clauses. @@ -30,23 +30,35 @@ import com.ctrip.sqllin.dsl.sql.Table */ public class ClauseBoolean internal constructor( valueName: String, - table: Table<*>, + table: Relation<*>, isFunction: Boolean, isNullable: Boolean, isAggregate: Boolean, isNullOnNoRows: Boolean, -) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows) { + columnTables: Set? = null, + tables: Set = emptySet(), + isDeterministic: Boolean = true, + isLiteral: Boolean = false, +) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows, columnTables, tables, isDeterministic, isLiteral) { /** * Creates the element of a column, as the code generated for a table does. * * @param isNullable Whether the column is nullable */ - public constructor(valueName: String, table: Table<*>, isNullable: Boolean) : + public constructor(valueName: String, table: Relation<*>, isNullable: Boolean) : this(valueName, table, isFunction = false, isNullable = isNullable, isAggregate = false, isNullOnNoRows = true) - override fun toAggregate(valueName: String, table: Table<*>): ClauseBoolean = - ClauseBoolean(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true) + override fun toAggregate(valueName: String, table: Relation<*>): ClauseBoolean = + ClauseBoolean(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true, columnTables = columnTables, tables = tables, isDeterministic = isDeterministic) + + override fun derive(valueName: String, traits: Traits): ClauseBoolean = + ClauseBoolean(valueName, table, isFunction = true, isNullable = traits.isNullable, isAggregate = traits.isAggregate, isNullOnNoRows = traits.isNullOnNoRows, columnTables = traits.columnTables, tables = traits.tables, isDeterministic = traits.isDeterministic) + + override fun literalOf(value: Boolean): ClauseBoolean = + ClauseBoolean(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + + override fun rebind(relation: Relation<*>): ClauseBoolean = ClauseBoolean(valueName, relation, isNullable) /** * Creates a condition comparing this Boolean column/function to a value. @@ -75,7 +87,7 @@ public class ClauseBoolean internal constructor( } ) } - return SelectCondition(sql, null) + return condition(sql, null) } /** @@ -112,7 +124,7 @@ public class ClauseBoolean internal constructor( } ) } - return SelectCondition(sql, null) + return condition(sql, null) } override fun hashCode(): Int = valueName.hashCode() + table.tableName.hashCode() diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseElement.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseElement.kt index 6c913b88..41df0772 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseElement.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseElement.kt @@ -16,7 +16,7 @@ package com.ctrip.sqllin.dsl.sql.clause -import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.Relation /** * Base class for elements used in SQL clauses. @@ -54,23 +54,65 @@ import com.ctrip.sqllin.dsl.sql.Table * an aggregate query * @property isNullOnNoRows Whether the element is NULL when an aggregate query without GROUP BY matches no rows. Such a * query still returns one row, in which a column, and every aggregate function except `count`, is NULL. + * @property columnTables The names of the tables whose columns the element reads: a column's own table, and the tables + * of a function's arguments, which an outer join can fill with NULL + * @property tables The tables and views that the subqueries in the element read, which an observed query watches + * @property isDeterministic Whether the element always gives the same result for the same row, which an index needs: + * `random()` doesn't + * @property isLiteral Whether the element is a literal on its own, such as `5`, which ORDER BY and GROUP BY would read + * as the number of a result column * * @author Yuang Qiao */ public sealed class ClauseElement( internal val valueName: String, - internal val table: Table<*>, + internal val table: Relation<*>, internal val isFunction: Boolean, internal val isNullable: Boolean, internal val isAggregate: Boolean, internal val isNullOnNoRows: Boolean, + columnTables: Set? = null, + internal val tables: Set = emptySet(), + internal val isDeterministic: Boolean = true, + internal val isLiteral: Boolean = false, ) { + internal val columnTables: Set = columnTables ?: if (isFunction) emptySet() else setOf(table.tableName) + /** * Creates the element of an aggregate function of this element that has values of the same type, as `max` and * `min` do. */ - internal abstract fun toAggregate(valueName: String, table: Table<*>): ClauseElement + internal abstract fun toAggregate(valueName: String, table: Relation<*>): ClauseElement + + /** + * Creates the element of an expression of this element that has values of the same type, as an operator or a + * function does, with [valueName] as its SQL and [traits] as what is known about it. + */ + internal abstract fun derive(valueName: String, traits: Traits): ClauseElement + + /** + * Creates the literal of [value], of the same type as the values of this element, as an operator or a function + * makes of a value given to it. + */ + internal abstract fun literalOf(value: V): ClauseElement + + /** + * Creates the element of this column of another [relation], such as the row a trigger reads. + */ + internal abstract fun rebind(relation: Relation<*>): ClauseElement + + /** + * Creates the condition [sql] of this element, compared with [other] if given, with its [parameters]: it reads the + * tables that the subqueries of the elements read. + */ + internal fun condition(sql: String, parameters: MutableList?, other: ClauseElement<*>? = null): SelectCondition = + SelectCondition( + conditionSQL = sql, + parameters = parameters, + tables = if (other == null) tables else tables + other.tables, + isDeterministic = isDeterministic && other?.isDeterministic != false, + ) /** * Appends this element as SQL to [builder]: a column qualified by its table's name, so that it can be told apart @@ -83,4 +125,11 @@ public sealed class ClauseElement( } builder.append(valueName) } + + /** + * This element as SQL, as [appendSQL] writes it, which a function writes for its argument, so that a column is told + * apart from a column of the same name of another table in a join. + */ + internal val sql: String + get() = buildString { appendSQL(this) } } diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseEnum.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseEnum.kt index 06e446f0..0fe38608 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseEnum.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseEnum.kt @@ -16,7 +16,7 @@ package com.ctrip.sqllin.dsl.sql.clause -import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.Relation /** * Wrapper for enum column references in SQL clauses. @@ -62,23 +62,35 @@ import com.ctrip.sqllin.dsl.sql.Table */ public class ClauseEnum> internal constructor( valueName: String, - table: Table<*>, + table: Relation<*>, isFunction: Boolean, isNullable: Boolean, isAggregate: Boolean, isNullOnNoRows: Boolean, -) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows) { + columnTables: Set? = null, + tables: Set = emptySet(), + isDeterministic: Boolean = true, + isLiteral: Boolean = false, +) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows, columnTables, tables, isDeterministic, isLiteral) { /** * Creates the element of a column, as the code generated for a table does. * * @param isNullable Whether the column is nullable */ - public constructor(valueName: String, table: Table<*>, isNullable: Boolean) : + public constructor(valueName: String, table: Relation<*>, isNullable: Boolean) : this(valueName, table, isFunction = false, isNullable = isNullable, isAggregate = false, isNullOnNoRows = true) - override fun toAggregate(valueName: String, table: Table<*>): ClauseEnum = - ClauseEnum(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true) + override fun toAggregate(valueName: String, table: Relation<*>): ClauseEnum = + ClauseEnum(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true, columnTables = columnTables, tables = tables, isDeterministic = isDeterministic) + + override fun derive(valueName: String, traits: Traits): ClauseEnum = + ClauseEnum(valueName, table, isFunction = true, isNullable = traits.isNullable, isAggregate = traits.isAggregate, isNullOnNoRows = traits.isNullOnNoRows, columnTables = traits.columnTables, tables = traits.tables, isDeterministic = traits.isDeterministic) + + override fun literalOf(value: T): ClauseEnum = + ClauseEnum(expressionLiteral(value.ordinal), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + + override fun rebind(relation: Relation<*>): ClauseEnum = ClauseEnum(valueName, relation, isNullable) /** * Less than (<) comparison using the enum's ordinal value. @@ -218,7 +230,7 @@ public class ClauseEnum> internal constructor( append(valueName) append(symbol) } - return SelectCondition(sql, mutableListOf(entry.ordinal)) + return condition(sql, mutableListOf(entry.ordinal)) } /** @@ -247,7 +259,7 @@ public class ClauseEnum> internal constructor( builder.append('?') mutableListOf(entry.ordinal) } - return SelectCondition(builder.toString(), parameters) + return condition(builder.toString(), parameters) } /** @@ -267,7 +279,7 @@ public class ClauseEnum> internal constructor( append(symbol) clauseEnum.appendSQL(this) } - return SelectCondition(sql, null) + return condition(sql, null, clauseEnum) } override fun hashCode(): Int = valueName.hashCode() + table.tableName.hashCode() diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseNumber.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseNumber.kt index 34a703d9..d9208985 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseNumber.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseNumber.kt @@ -16,7 +16,9 @@ package com.ctrip.sqllin.dsl.sql.clause -import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.sql.Relation +import kotlin.jvm.JvmName /** * Wrapper for numeric column/function references in SQL clauses. @@ -41,23 +43,35 @@ import com.ctrip.sqllin.dsl.sql.Table */ public class ClauseNumber internal constructor( valueName: String, - table: Table<*>, + table: Relation<*>, isFunction: Boolean, isNullable: Boolean, isAggregate: Boolean, isNullOnNoRows: Boolean, -) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows) { + columnTables: Set? = null, + tables: Set = emptySet(), + isDeterministic: Boolean = true, + isLiteral: Boolean = false, +) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows, columnTables, tables, isDeterministic, isLiteral) { /** * Creates the element of a column, as the code generated for a table does. * * @param isNullable Whether the column is nullable */ - public constructor(valueName: String, table: Table<*>, isNullable: Boolean) : + public constructor(valueName: String, table: Relation<*>, isNullable: Boolean) : this(valueName, table, isFunction = false, isNullable = isNullable, isAggregate = false, isNullOnNoRows = true) - override fun toAggregate(valueName: String, table: Table<*>): ClauseNumber = - ClauseNumber(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true) + override fun toAggregate(valueName: String, table: Relation<*>): ClauseNumber = + ClauseNumber(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true, columnTables = columnTables, tables = tables, isDeterministic = isDeterministic) + + override fun derive(valueName: String, traits: Traits): ClauseNumber = + ClauseNumber(valueName, table, isFunction = true, isNullable = traits.isNullable, isAggregate = traits.isAggregate, isNullOnNoRows = traits.isNullOnNoRows, columnTables = traits.columnTables, tables = traits.tables, isDeterministic = traits.isDeterministic) + + override fun literalOf(value: V): ClauseNumber = + ClauseNumber(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + + override fun rebind(relation: Relation<*>): ClauseNumber = ClauseNumber(valueName, relation, isNullable) /** * Less than (<) comparison using parameterized binding. @@ -164,7 +178,7 @@ public class ClauseNumber internal constructor( } append(')') } - return SelectCondition(sql, parameters) + return condition(sql, parameters) } /** @@ -185,7 +199,7 @@ public class ClauseNumber internal constructor( append(valueName) append(" BETWEEN ? AND ?") } - return SelectCondition(sql, mutableListOf(range.first, range.last)) + return condition(sql, mutableListOf(range.first, range.last)) } private fun appendNumber(symbol: String, number: Number): SelectCondition { @@ -197,7 +211,7 @@ public class ClauseNumber internal constructor( append(valueName) append(symbol) } - return SelectCondition(sql, mutableListOf(number)) + return condition(sql, mutableListOf(number)) } private fun appendNullableNumber(notNullSymbol: String, nullSymbol: String, number: Number?): SelectCondition { @@ -215,7 +229,7 @@ public class ClauseNumber internal constructor( builder.append('?') mutableListOf(number) } - return SelectCondition(builder.toString(), parameters) + return condition(builder.toString(), parameters) } private fun appendClauseNumber(symbol: String, clauseNumber: ClauseNumber<*>): SelectCondition { @@ -224,11 +238,263 @@ public class ClauseNumber internal constructor( append(symbol) clauseNumber.appendSQL(this) } - return SelectCondition(sql, null) + return condition(sql, null, clauseNumber) } + // ========== Arithmetic ========== + // + // An operator of two numbers of the same type gives a number of that type, as `visits + 1` of an `Int` column is an + // `Int`; CAST converts a number to another type, to compute with numbers of different types. SQLite computes as + // Kotlin does, except that it turns an integer that overflows into a real number, rather than wrapping it around, + // and that it gives NULL for a division by zero, rather than throwing. + + /** + * Addition: `(this + other)`. + */ + @ExperimentalDSLDatabaseAPI + public operator fun plus(other: ClauseNumber): ClauseNumber = arithmetic("+", other) + + /** + * Addition of a value: `(this + value)`, as in `visits + 1`. + */ + @ExperimentalDSLDatabaseAPI + public operator fun plus(value: V): ClauseNumber = arithmetic("+", literalOf(value)) + + /** + * Subtraction: `(this - other)`. + */ + @ExperimentalDSLDatabaseAPI + public operator fun minus(other: ClauseNumber): ClauseNumber = arithmetic("-", other) + + /** + * Subtraction of a value: `(this - value)`. + */ + @ExperimentalDSLDatabaseAPI + public operator fun minus(value: V): ClauseNumber = arithmetic("-", literalOf(value)) + + /** + * Multiplication: `(this * other)`. + */ + @ExperimentalDSLDatabaseAPI + public operator fun times(other: ClauseNumber): ClauseNumber = arithmetic("*", other) + + /** + * Multiplication by a value: `(this * value)`. + */ + @ExperimentalDSLDatabaseAPI + public operator fun times(value: V): ClauseNumber = arithmetic("*", literalOf(value)) + + /** + * Division: `(this / other)`, which divides integers as Kotlin does, `7 / 2` being `3`. SQLite gives NULL for a + * division by zero, so the result can be NULL. + */ + @ExperimentalDSLDatabaseAPI + public operator fun div(other: ClauseNumber): ClauseNumber = arithmetic("/", other, isNullable = true) + + /** + * Division by a value: `(this / value)`, which can only be NULL where this element can, unless [value] is zero. + */ + @ExperimentalDSLDatabaseAPI + public operator fun div(value: V): ClauseNumber { + val divisor = literalOf(value) + return arithmetic("/", divisor, isNullable = isNullable || divisor.valueName.trim('(', ')').toDouble() == 0.0) + } + + /** + * Negation: `(-this)`. + */ + @ExperimentalDSLDatabaseAPI + public operator fun unaryMinus(): ClauseNumber = derive("(-$sql)", strictTraits(listOf(this))) + + /** + * The remainder of the division of integers, `(this % other)`, which, as in Kotlin, has the sign of this element. + * The remainder operators are only given to integer types, as SQLite converts real numbers to integers for it. + */ + internal fun remainder(other: ClauseElement<*>): ClauseNumber { + val isNonZero = other.isLiteral && other.valueName.trim('(', ')').toDouble() != 0.0 + return arithmetic("%", other, isNullable = isNullable || !isNonZero || other.isNullable) + } + + /** + * A bitwise operator, `(this operator other)`, which is NULL where either is. + */ + internal fun bitwise(operator: String, other: ClauseElement<*>): ClauseNumber = arithmetic(operator, other) + + /** + * The bitwise complement, `(~this)`. + */ + internal fun complement(): ClauseNumber = derive("(~$sql)", strictTraits(listOf(this))) + + private fun arithmetic(operator: String, other: ClauseElement<*>, isNullable: Boolean = this.isNullable || other.isNullable): ClauseNumber = + derive("($sql $operator ${other.sql})", strictTraits(listOf(this, other), isNullable)) + override fun hashCode(): Int = valueName.hashCode() + table.tableName.hashCode() override fun equals(other: Any?): Boolean = (other as? ClauseNumber<*>)?.let { it.valueName == valueName && it.table.tableName == table.tableName } ?: false -} \ No newline at end of file +} + +// The remainder of integers: SQLite converts real numbers to integers for `%`, so `5.5 % 2` is `1.0` there, but `1.5` in +// Kotlin, and the operator is only given to integer types. + +/** The remainder of the division of `Byte`s: `(this % other)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remByte") +public operator fun ClauseNumber.rem(other: ClauseNumber): ClauseNumber = remainder(other) + +/** The remainder of the division by a `Byte`: `(this % value)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remByteValue") +public operator fun ClauseNumber.rem(value: Byte): ClauseNumber = remainder(literalOf(value)) + +/** The remainder of the division of `Short`s: `(this % other)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remShort") +public operator fun ClauseNumber.rem(other: ClauseNumber): ClauseNumber = remainder(other) + +/** The remainder of the division by a `Short`: `(this % value)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remShortValue") +public operator fun ClauseNumber.rem(value: Short): ClauseNumber = remainder(literalOf(value)) + +/** The remainder of the division of `Int`s: `(this % other)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remInt") +public operator fun ClauseNumber.rem(other: ClauseNumber): ClauseNumber = remainder(other) + +/** The remainder of the division by a `Int`: `(this % value)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remIntValue") +public operator fun ClauseNumber.rem(value: Int): ClauseNumber = remainder(literalOf(value)) + +/** The remainder of the division of `Long`s: `(this % other)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remLong") +public operator fun ClauseNumber.rem(other: ClauseNumber): ClauseNumber = remainder(other) + +/** The remainder of the division by a `Long`: `(this % value)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remLongValue") +public operator fun ClauseNumber.rem(value: Long): ClauseNumber = remainder(literalOf(value)) + +/** The remainder of the division of `UByte`s: `(this % other)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remUByte") +public operator fun ClauseNumber.rem(other: ClauseNumber): ClauseNumber = remainder(other) + +/** The remainder of the division by a `UByte`: `(this % value)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remUByteValue") +public operator fun ClauseNumber.rem(value: UByte): ClauseNumber = remainder(literalOf(value)) + +/** The remainder of the division of `UShort`s: `(this % other)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remUShort") +public operator fun ClauseNumber.rem(other: ClauseNumber): ClauseNumber = remainder(other) + +/** The remainder of the division by a `UShort`: `(this % value)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remUShortValue") +public operator fun ClauseNumber.rem(value: UShort): ClauseNumber = remainder(literalOf(value)) + +/** The remainder of the division of `UInt`s: `(this % other)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remUInt") +public operator fun ClauseNumber.rem(other: ClauseNumber): ClauseNumber = remainder(other) + +/** The remainder of the division by a `UInt`: `(this % value)`, NULL for a division by zero. */ +@ExperimentalDSLDatabaseAPI +@JvmName("remUIntValue") +public operator fun ClauseNumber.rem(value: UInt): ClauseNumber = remainder(literalOf(value)) + +// Bitwise operators, named as Kotlin names them, of `Int` and `Long`, as SQLite computes them with 64-bit integers. + +/** Bitwise AND of `Int`s: `(this & other)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("andInt") +public infix fun ClauseNumber.and(other: ClauseNumber): ClauseNumber = bitwise("&", other) + +/** Bitwise AND with a `Int`: `(this & value)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("andIntValue") +public infix fun ClauseNumber.and(value: Int): ClauseNumber = bitwise("&", literalOf(value)) + +/** Bitwise OR of `Int`s: `(this | other)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("orInt") +public infix fun ClauseNumber.or(other: ClauseNumber): ClauseNumber = bitwise("|", other) + +/** Bitwise OR with a `Int`: `(this | value)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("orIntValue") +public infix fun ClauseNumber.or(value: Int): ClauseNumber = bitwise("|", literalOf(value)) + +/** Shift left of a `Int` by [bits]: `(this << bits)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("shlInt") +public infix fun ClauseNumber.shl(bits: ClauseNumber): ClauseNumber = bitwise("<<", bits) + +/** Shift left of a `Int` by the value [bits]: `(this << bits)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("shlIntValue") +public infix fun ClauseNumber.shl(bits: Int): ClauseNumber = bitwise("<<", literal(bits)) + +/** Shift right, keeping the sign, of a `Int` by [bits]: `(this >> bits)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("shrInt") +public infix fun ClauseNumber.shr(bits: ClauseNumber): ClauseNumber = bitwise(">>", bits) + +/** Shift right, keeping the sign, of a `Int` by the value [bits]: `(this >> bits)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("shrIntValue") +public infix fun ClauseNumber.shr(bits: Int): ClauseNumber = bitwise(">>", literal(bits)) + +/** The bitwise complement of a `Int`: `(~this)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("invInt") +public fun ClauseNumber.inv(): ClauseNumber = complement() + +/** Bitwise AND of `Long`s: `(this & other)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("andLong") +public infix fun ClauseNumber.and(other: ClauseNumber): ClauseNumber = bitwise("&", other) + +/** Bitwise AND with a `Long`: `(this & value)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("andLongValue") +public infix fun ClauseNumber.and(value: Long): ClauseNumber = bitwise("&", literalOf(value)) + +/** Bitwise OR of `Long`s: `(this | other)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("orLong") +public infix fun ClauseNumber.or(other: ClauseNumber): ClauseNumber = bitwise("|", other) + +/** Bitwise OR with a `Long`: `(this | value)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("orLongValue") +public infix fun ClauseNumber.or(value: Long): ClauseNumber = bitwise("|", literalOf(value)) + +/** Shift left of a `Long` by [bits]: `(this << bits)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("shlLong") +public infix fun ClauseNumber.shl(bits: ClauseNumber): ClauseNumber = bitwise("<<", bits) + +/** Shift left of a `Long` by the value [bits]: `(this << bits)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("shlLongValue") +public infix fun ClauseNumber.shl(bits: Int): ClauseNumber = bitwise("<<", literal(bits)) + +/** Shift right, keeping the sign, of a `Long` by [bits]: `(this >> bits)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("shrLong") +public infix fun ClauseNumber.shr(bits: ClauseNumber): ClauseNumber = bitwise(">>", bits) + +/** Shift right, keeping the sign, of a `Long` by the value [bits]: `(this >> bits)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("shrLongValue") +public infix fun ClauseNumber.shr(bits: Int): ClauseNumber = bitwise(">>", literal(bits)) + +/** The bitwise complement of a `Long`: `(~this)`. */ +@ExperimentalDSLDatabaseAPI +@JvmName("invLong") +public fun ClauseNumber.inv(): ClauseNumber = complement() diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseString.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseString.kt index 3390c595..e85df6c1 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseString.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ClauseString.kt @@ -16,7 +16,8 @@ package com.ctrip.sqllin.dsl.sql.clause -import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.sql.Relation /** * Wrapper for String column/function references in SQL clauses. @@ -42,23 +43,35 @@ import com.ctrip.sqllin.dsl.sql.Table */ public class ClauseString internal constructor( valueName: String, - table: Table<*>, + table: Relation<*>, isFunction: Boolean, isNullable: Boolean, isAggregate: Boolean, isNullOnNoRows: Boolean, -) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows) { + columnTables: Set? = null, + tables: Set = emptySet(), + isDeterministic: Boolean = true, + isLiteral: Boolean = false, +) : ClauseElement(valueName, table, isFunction, isNullable, isAggregate, isNullOnNoRows, columnTables, tables, isDeterministic, isLiteral) { /** * Creates the element of a column, as the code generated for a table does. * * @param isNullable Whether the column is nullable */ - public constructor(valueName: String, table: Table<*>, isNullable: Boolean) : + public constructor(valueName: String, table: Relation<*>, isNullable: Boolean) : this(valueName, table, isFunction = false, isNullable = isNullable, isAggregate = false, isNullOnNoRows = true) - override fun toAggregate(valueName: String, table: Table<*>): ClauseString = - ClauseString(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true) + override fun toAggregate(valueName: String, table: Relation<*>): ClauseString = + ClauseString(valueName, table, isFunction = true, isNullable = isNullable, isAggregate = true, isNullOnNoRows = true, columnTables = columnTables, tables = tables, isDeterministic = isDeterministic) + + override fun derive(valueName: String, traits: Traits): ClauseString = + ClauseString(valueName, table, isFunction = true, isNullable = traits.isNullable, isAggregate = traits.isAggregate, isNullOnNoRows = traits.isNullOnNoRows, columnTables = traits.columnTables, tables = traits.tables, isDeterministic = traits.isDeterministic) + + override fun literalOf(value: V): ClauseString = + ClauseString(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + + override fun rebind(relation: Relation<*>): ClauseString = ClauseString(valueName, relation, isNullable) /** Equals (=), or IS NULL if value is null */ internal infix fun eq(str: String?): SelectCondition = appendNullableString("=", " IS NULL", str) @@ -142,7 +155,7 @@ public class ClauseString internal constructor( * Wildcards: `%` (any characters), `_` (single character) * Example: `"John%"` matches "John", "Johnson", etc. */ - internal infix fun like(regex: String): SelectCondition = appendRegex(" LIKE ", regex) + internal infix fun like(regex: String): LikeCondition = LikeCondition(regexSQL(" LIKE "), mutableListOf(regex), tables, isDeterministic) /** * GLOB operator - case-sensitive pattern matching. @@ -150,19 +163,16 @@ public class ClauseString internal constructor( * Wildcards: `*` (any characters), `?` (single character) * Example: `"John*"` matches "John", "Johnson", etc. (case-sensitive) */ - internal infix fun glob(regex: String): SelectCondition = appendRegex(" GLOB ", regex) + internal infix fun glob(regex: String): SelectCondition = condition(regexSQL(" GLOB "), mutableListOf(regex)) - private fun appendRegex(symbol: String, regex: String): SelectCondition { - val sql = buildString { - if (!isFunction) { - append(table.tableName) - append('.') - } - append(valueName) - append(symbol) - append('?') + private fun regexSQL(symbol: String): String = buildString { + if (!isFunction) { + append(table.tableName) + append('.') } - return SelectCondition(sql, mutableListOf(regex)) + append(valueName) + append(symbol) + append('?') } private fun appendNullableString(notNullSymbol: String, nullSymbol: String, str: String?): SelectCondition { @@ -180,7 +190,7 @@ public class ClauseString internal constructor( append('?') } } - return SelectCondition(sql, if (str == null) null else mutableListOf(str)) + return condition(sql, if (str == null) null else mutableListOf(str)) } private fun appendString(symbol: String, str: String): SelectCondition { @@ -192,7 +202,7 @@ public class ClauseString internal constructor( append(valueName) append(symbol) } - return SelectCondition(sql, mutableListOf(str)) + return condition(sql, mutableListOf(str)) } private fun appendClauseString(symbol: String, clauseString: ClauseString<*>): SelectCondition { @@ -203,7 +213,7 @@ public class ClauseString internal constructor( append(' ') clauseString.appendSQL(this) } - return SelectCondition(sql, null) + return condition(sql, null, clauseString) } /** @@ -232,7 +242,7 @@ public class ClauseString internal constructor( } append(')') } - return SelectCondition(sql, parameters) + return condition(sql, parameters) } /** @@ -252,7 +262,24 @@ public class ClauseString internal constructor( append(valueName) append(" BETWEEN ? AND ?") } - return SelectCondition(sql, mutableListOf(range.first, range.second)) + return condition(sql, mutableListOf(range.first, range.second)) + } + + /** + * Concatenation, SQL's `||`: `(this || other)`, which is NULL if either is. + */ + @ExperimentalDSLDatabaseAPI + public operator fun plus(other: ClauseString<*>): ClauseString = concat(other) + + /** + * Concatenation of a string, SQL's `||`: `(this || value)`. + */ + @ExperimentalDSLDatabaseAPI + public operator fun plus(value: String): ClauseString = concat(literal(value)) + + private fun concat(other: ClauseString<*>): ClauseString { + val traits = strictTraits(listOf(this, other)) + return ClauseString("($sql || ${other.sql})", table, isFunction = true, isNullable = traits.isNullable, isAggregate = traits.isAggregate, isNullOnNoRows = traits.isNullOnNoRows, columnTables = traits.columnTables, tables = traits.tables, isDeterministic = traits.isDeterministic) } override fun hashCode(): Int = valueName.hashCode() + table.tableName.hashCode() diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Collation.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Collation.kt new file mode 100644 index 00000000..f3faa2a1 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Collation.kt @@ -0,0 +1,55 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +@file:Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") + +package com.ctrip.sqllin.dsl.sql.clause + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.annotation.KeyWordDslMaker +import com.ctrip.sqllin.dsl.annotation.StatementDslMaker + +/** + * The collations SQLite has on every platform, which decide how strings compare and sort. + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public sealed class Collation(internal val name: String) + +/** Compares the bytes of strings, as SQLite does by default. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object BINARY : Collation("BINARY") + +/** Compares strings ignoring the case of the ASCII letters only, so `'A'` equals `'a'`, but `'É'` doesn't equal `'é'`. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object NOCASE : Collation("NOCASE") + +/** Compares strings as BINARY does, ignoring the spaces at their ends. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object RTRIM : Collation("RTRIM") + +/** + * The COLLATE operator: this string compared and sorted by [collation], as in `(name COLLATE NOCASE) EQ "ann"`, which + * matches `'Ann'` too, or in `ORDER_BY((name COLLATE NOCASE) to ASC)`. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseString.COLLATE(collation: Collation): ClauseString = + derive("($sql COLLATE ${collation.name})", traits) diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ConditionClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ConditionClause.kt index 5f54f4b0..f1ecc79e 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ConditionClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ConditionClause.kt @@ -18,6 +18,7 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import com.ctrip.sqllin.dsl.annotation.StatementDslMaker /** @@ -118,9 +119,21 @@ public infix fun ClauseString<*>.NEQ(str: String?): SelectCondition = neq(str) @StatementDslMaker public infix fun ClauseString<*>.NEQ(clauseString: ClauseString<*>): SelectCondition = neq(clauseString) -// SQL LIKE operator +// SQL LIKE operator, which ESCAPE can follow @StatementDslMaker -public infix fun ClauseString<*>.LIKE(regex: String): SelectCondition = like(regex) +public infix fun ClauseString<*>.LIKE(regex: String): LikeCondition = like(regex) + +/** + * Gives the pattern of this LIKE an escape character, before which `%`, `_` and the escape character itself match + * only themselves, as the SQL `LIKE pattern ESCAPE escape` does: + * ```kotlin + * ProductTable SELECT WHERE (ProductTable.name LIKE "100\\%%" ESCAPE '\\') + * // Matches the names that start with "100%" + * ``` + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun LikeCondition.ESCAPE(escape: Char): SelectCondition = escape(escape) // SQL GLOB operator @StatementDslMaker @@ -284,4 +297,70 @@ public infix fun SelectCondition.OR(prediction: SelectCondition): SelectConditio // Condition 'AND' operator @StatementDslMaker -public infix fun SelectCondition.AND(prediction: SelectCondition): SelectCondition = and(prediction) \ No newline at end of file +public infix fun SelectCondition.AND(prediction: SelectCondition): SelectCondition = and(prediction) + +/** + * Negates [condition]: `NOT (condition)`, as in `NOT(EXISTS(select))`, `NOT(age IN listOf(1, 2))` or + * `NOT((age LT 18) OR (age GT 65))`. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public fun NOT(condition: SelectCondition): SelectCondition = + SelectCondition("NOT (${condition.conditionSQL})", condition.parameters, condition.tables, isDeterministic = condition.isDeterministic) + +// Null-safe comparisons of two expressions, SQL's IS and IS NOT: like EQ and NEQ, but NULL equals NULL, so a +// comparison with NULL is true or false rather than NULL. They do what IS DISTINCT FROM does, on every SQLite version. + +/** Whether this number equals [other], NULL equaling NULL: `this IS other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseNumber<*>.IS(other: ClauseNumber<*>): SelectCondition = nullSafe("IS", other) + +/** Whether this number differs from [other], NULL equaling NULL: `this IS NOT other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseNumber<*>.ISNOT(other: ClauseNumber<*>): SelectCondition = nullSafe("IS NOT", other) + +/** Whether this string equals [other], NULL equaling NULL: `this IS other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseString<*>.IS(other: ClauseString<*>): SelectCondition = nullSafe("IS", other) + +/** Whether this string differs from [other], NULL equaling NULL: `this IS NOT other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseString<*>.ISNOT(other: ClauseString<*>): SelectCondition = nullSafe("IS NOT", other) + +/** Whether this BLOB equals [other], NULL equaling NULL: `this IS other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseBlob.IS(other: ClauseBlob): SelectCondition = nullSafe("IS", other) + +/** Whether this BLOB differs from [other], NULL equaling NULL: `this IS NOT other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseBlob.ISNOT(other: ClauseBlob): SelectCondition = nullSafe("IS NOT", other) + +/** Whether this Boolean equals [other], NULL equaling NULL: `this IS other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseBoolean.IS(other: ClauseBoolean): SelectCondition = nullSafe("IS", other) + +/** Whether this Boolean differs from [other], NULL equaling NULL: `this IS NOT other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseBoolean.ISNOT(other: ClauseBoolean): SelectCondition = nullSafe("IS NOT", other) + +/** Whether this enum entry equals [other], NULL equaling NULL: `this IS other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun > ClauseEnum.IS(other: ClauseEnum): SelectCondition = nullSafe("IS", other) + +/** Whether this enum entry differs from [other], NULL equaling NULL: `this IS NOT other`. */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun > ClauseEnum.ISNOT(other: ClauseEnum): SelectCondition = nullSafe("IS NOT", other) + +private fun ClauseElement<*>.nullSafe(operator: String, other: ClauseElement<*>): SelectCondition = + condition("$sql $operator ${other.sql}", null, other) + diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/CrossJoinClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/CrossJoinClause.kt index d1401deb..69b0d894 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/CrossJoinClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/CrossJoinClause.kt @@ -16,8 +16,8 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.sql.Relation import com.ctrip.sqllin.dsl.annotation.StatementDslMaker -import com.ctrip.sqllin.dsl.sql.Table /** * CROSS JOIN clause - returns the Cartesian product of two tables. @@ -32,7 +32,7 @@ import com.ctrip.sqllin.dsl.sql.Table * * @author Yuang Qiao */ -internal class CrossJoinClause(vararg tables: Table<*>) : NaturalJoinClause(*tables) { +internal class CrossJoinClause(vararg tables: Relation<*>) : NaturalJoinClause(*tables) { override val clauseName: String = " CROSS JOIN " } @@ -46,7 +46,12 @@ internal class CrossJoinClause(vararg tables: Table<*>) : NaturalJoinClause()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. */ @Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") @StatementDslMaker -public fun CROSS_JOIN(vararg tables: Table<*>): NaturalJoinClause = CrossJoinClause(*tables) \ No newline at end of file +public fun CROSS_JOIN(vararg tables: Relation<*>): NaturalJoinClause = CrossJoinClause(*tables) \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/DateTimeFunction.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/DateTimeFunction.kt new file mode 100644 index 00000000..ece39703 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/DateTimeFunction.kt @@ -0,0 +1,216 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +@file:Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") + +package com.ctrip.sqllin.dsl.sql.clause + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.annotation.FunctionDslMaker +import com.ctrip.sqllin.dsl.sql.Relation + +/** + * SQLite's date and time functions, which SQLite has on every platform SQLlin supports. + * + * Each takes a time value and modifiers. The time value is a string, such as `"2026-10-07 12:30:00"` or `"now"`, or an + * expression: a string, or a number, which is a Julian day number, or a Unix time with the `"unixepoch"` modifier. The + * modifiers change the time in turn, as `"+1 day"`, `"start of month"` or `"localtime"` do. + * + * A function is NULL for a value or a modifier it can't read, so it can be NULL, except of `"now"` without modifiers. + * The current time, and the `"localtime"` and `"utc"` modifiers, which depend on where the code runs, make it give + * different results over time, so an index can't hold it. + * + * @author Yuang Qiao + */ + +/** + * DATE function - the date, as `YYYY-MM-DD`, of [time], a time string, as `"now"`, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.date(time: String, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("date($arguments)", traits) +} + +/** + * DATE function - the date, as `YYYY-MM-DD`, of [time], a string expression, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.date(time: ClauseString<*>, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("date($arguments)", traits) +} + +/** + * DATE function - the date, as `YYYY-MM-DD`, of [time], a number expression, a Julian day number or, with the `"unixepoch"` modifier, a Unix time, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.date(time: ClauseNumber<*>, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("date($arguments)", traits) +} + +/** + * TIME function - the time, as `HH:MM:SS`, of [time], a time string, as `"now"`, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.time(time: String, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("time($arguments)", traits) +} + +/** + * TIME function - the time, as `HH:MM:SS`, of [time], a string expression, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.time(time: ClauseString<*>, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("time($arguments)", traits) +} + +/** + * TIME function - the time, as `HH:MM:SS`, of [time], a number expression, a Julian day number or, with the `"unixepoch"` modifier, a Unix time, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.time(time: ClauseNumber<*>, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("time($arguments)", traits) +} + +/** + * DATETIME function - the date and time, as `YYYY-MM-DD HH:MM:SS`, of [time], a time string, as `"now"`, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.datetime(time: String, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("datetime($arguments)", traits) +} + +/** + * DATETIME function - the date and time, as `YYYY-MM-DD HH:MM:SS`, of [time], a string expression, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.datetime(time: ClauseString<*>, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("datetime($arguments)", traits) +} + +/** + * DATETIME function - the date and time, as `YYYY-MM-DD HH:MM:SS`, of [time], a number expression, a Julian day number or, with the `"unixepoch"` modifier, a Unix time, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.datetime(time: ClauseNumber<*>, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("datetime($arguments)", traits) +} + +/** + * JULIANDAY function - the Julian day number, as a `Double`, of [time], a time string, as `"now"`, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.julianday(time: String, vararg modifiers: String): ClauseNumber { + val (arguments, traits) = timeArguments(time, modifiers) + return numberExpression("julianday($arguments)", traits) +} + +/** + * JULIANDAY function - the Julian day number, as a `Double`, of [time], a string expression, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.julianday(time: ClauseString<*>, vararg modifiers: String): ClauseNumber { + val (arguments, traits) = timeArguments(time, modifiers) + return numberExpression("julianday($arguments)", traits) +} + +/** + * JULIANDAY function - the Julian day number, as a `Double`, of [time], a number expression, a Julian day number or, with the `"unixepoch"` modifier, a Unix time, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.julianday(time: ClauseNumber<*>, vararg modifiers: String): ClauseNumber { + val (arguments, traits) = timeArguments(time, modifiers) + return numberExpression("julianday($arguments)", traits) +} + +/** + * STRFTIME function - [format] with its specifiers, as `%Y` or `%m`, replaced by the parts of [time], a time string, as `"now"`, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.strftime(format: String, time: String, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("strftime(${expressionLiteral(format)},$arguments)", traits) +} + +/** + * STRFTIME function - [format] with its specifiers, as `%Y` or `%m`, replaced by the parts of [time], a string expression, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.strftime(format: String, time: ClauseString<*>, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("strftime(${expressionLiteral(format)},$arguments)", traits) +} + +/** + * STRFTIME function - [format] with its specifiers, as `%Y` or `%m`, replaced by the parts of [time], a number expression, a Julian day number or, with the `"unixepoch"` modifier, a Unix time, changed by [modifiers]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.strftime(format: String, time: ClauseNumber<*>, vararg modifiers: String): ClauseString { + val (arguments, traits) = timeArguments(time, modifiers) + return stringExpression("strftime(${expressionLiteral(format)},$arguments)", traits) +} + +/** + * The SQL of the arguments of a date and time function, [time] and [modifiers], and the traits of the function. + */ +internal fun timeArguments(time: Any, modifiers: Array): Pair { + val isLocal = modifiers.any { it.trim().lowercase() in LOCAL_MODIFIERS } + val (sql, traits) = if (time is ClauseElement<*>) { + time.sql to strictTraits(listOf(time), isNullable = true, isDeterministic = !isLocal) + } else { + val isNow = (time as String).trim().equals("now", ignoreCase = true) + expressionLiteral(time) to strictTraits( + operands = emptyList(), + isNullable = !isNow || modifiers.isNotEmpty(), + isNullOnNoRows = false, + isDeterministic = !isNow && !isLocal, + ) + } + val arguments = buildString { + append(sql) + for (modifier in modifiers) { + append(',') + append(expressionLiteral(modifier)) + } + } + return arguments to traits +} + +/** The modifiers that convert between the local time and UTC, which depends on where the code runs. */ +private val LOCAL_MODIFIERS = setOf("localtime", "utc") diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Expression.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Expression.kt new file mode 100644 index 00000000..4d470973 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Expression.kt @@ -0,0 +1,104 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.clause + +import com.ctrip.sqllin.dsl.sql.Relation +import com.ctrip.sqllin.dsl.sql.compiler.sqlLiteral +import com.ctrip.sqllin.dsl.sql.compiler.storedValue +import kotlinx.serialization.KSerializer + +/** + * What is known about an expression, which an expression of it derives its own from: whether it can be NULL, in a row + * and when an aggregate query without GROUP BY matches no rows, whether it is an aggregate, the tables of the columns it + * reads and of its subqueries, and whether it always gives the same result for the same row. + * + * An expression carries no parameters: the values in it are written into its SQL as literals, so that it can be part + * of any statement, a view, a trigger and an index included, which SQLite doesn't let take parameters. + * + * @author Yuang Qiao + */ +internal class Traits( + val isNullable: Boolean, + val isAggregate: Boolean, + val isNullOnNoRows: Boolean, + val columnTables: Set, + val tables: Set, + val isDeterministic: Boolean, +) + +internal val ClauseElement<*>.traits: Traits + get() = Traits(isNullable, isAggregate, isNullOnNoRows, columnTables, tables, isDeterministic) + +/** + * The traits of an expression of [operands] that is NULL when any of them is, as an arithmetic operator, or a function + * such as `abs` or `upper`, which also makes it NULL where any of them is when no rows match. [isNullable] is given + * where the expression can be NULL even when its operands aren't, as a division by zero is, and [isNullOnNoRows] where + * it differs, as `hex` is never NULL. [isDeterministic] is false where the function isn't, as `random()` isn't. + */ +internal fun strictTraits( + operands: List>, + isNullable: Boolean = operands.any { it.isNullable }, + isNullOnNoRows: Boolean = operands.any { it.isNullOnNoRows }, + isDeterministic: Boolean = true, +): Traits = Traits( + isNullable = isNullable, + isAggregate = operands.any { it.isAggregate }, + isNullOnNoRows = isNullOnNoRows, + columnTables = operands.flatMapTo(LinkedHashSet()) { it.columnTables }, + tables = operands.flatMapTo(LinkedHashSet()) { it.tables }, + isDeterministic = isDeterministic && operands.all { it.isDeterministic }, +) + +/** + * The relation of the elements that belong to none, such as literals: as it isn't a column, an element writes no + * relation's name, so this one is never written. + */ +internal object ExpressionRelation : Relation("") { + override fun kSerializer(): KSerializer = error("An expression isn't a relation that can be read.") +} + +/** + * The row a trigger reads, `NEW` or `OLD`, of [table], whose columns `new(column)` and `old(column)` give. + */ +internal class RowRelation(name: String, val table: Relation<*>) : Relation(name) { + override fun kSerializer(): KSerializer = error("The row of a trigger isn't a relation that can be read.") +} + +/** + * Writes [value] as the SQL literal of an expression, the number SQLite stores for an unsigned one included. A negative + * number is put in parentheses, as `x - -1` written without spaces, `x--1`, would start a comment. + */ +internal fun expressionLiteral(value: Any): String { + val literal = sqlLiteral(storedValue(value)) + return if (literal.startsWith('-')) "($literal)" else literal +} + +/** The element of a function with `String` values, [sql], that has [traits]. */ +internal fun Relation<*>.stringExpression(sql: String, traits: Traits): ClauseString = + ClauseString(sql, this, isFunction = true, isNullable = traits.isNullable, isAggregate = traits.isAggregate, isNullOnNoRows = traits.isNullOnNoRows, columnTables = traits.columnTables, tables = traits.tables, isDeterministic = traits.isDeterministic) + +/** The element of a function with values of type [V], [sql], that has [traits]. */ +internal fun Relation<*>.numberExpression(sql: String, traits: Traits): ClauseNumber = + ClauseNumber(sql, this, isFunction = true, isNullable = traits.isNullable, isAggregate = traits.isAggregate, isNullOnNoRows = traits.isNullOnNoRows, columnTables = traits.columnTables, tables = traits.tables, isDeterministic = traits.isDeterministic) + +/** The element of a function with BLOB values, [sql], that has [traits]. */ +internal fun Relation<*>.blobExpression(sql: String, traits: Traits): ClauseBlob = + ClauseBlob(sql, this, isFunction = true, isNullable = traits.isNullable, isAggregate = traits.isAggregate, isNullOnNoRows = traits.isNullOnNoRows, columnTables = traits.columnTables, tables = traits.tables, isDeterministic = traits.isDeterministic) + +/** The SQL of [elements], separated by commas. */ +internal fun sqlOf(elements: List>): String = elements.joinToString(",") { it.sql } + diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/FullTextSearch.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/FullTextSearch.kt new file mode 100644 index 00000000..5fe4f5d9 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/FullTextSearch.kt @@ -0,0 +1,112 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +@file:Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") + +package com.ctrip.sqllin.dsl.sql.clause + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.annotation.FunctionDslMaker +import com.ctrip.sqllin.dsl.annotation.StatementDslMaker +import com.ctrip.sqllin.dsl.sql.FtsTable +import com.ctrip.sqllin.dsl.sql.Relation + +/** + * Full-text search of FTS3 and FTS4 tables: the MATCH operator, and the functions that describe a match. + * + * The query is FTS's own language: terms, which all have to be found; `OR` between terms; phrases in double quotes; + * prefixes such as `kot*`; `column:term` for one column; and `NEAR` between terms close to each other. Queries made of + * these work the same on every platform. Beyond them, FTS has two syntaxes: Android's SQLite has the standard one, + * where `-term` leaves out the rows with the term, while the SQLite of the JVM driver and of Apple's platforms has the + * enhanced one, which adds `AND`, `NOT` and parentheses, and reads `-term` as the term itself. On Linux and Windows, + * the syntax is that of the SQLite the app links. + * + * @author Yuang Qiao + */ + +/** + * Finds the rows of this FTS table that match [query], in any column: `table MATCH ?`. + * + * Example: + * ```kotlin + * ArticleTable SELECT WHERE(ArticleTable MATCH "kotlin coroutines") + * ``` + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun FtsTable.MATCH(query: String): SelectCondition = + SelectCondition("$tableName MATCH ?", mutableListOf(query)) + +/** + * Finds the rows whose value in this column of an FTS table matches [query]: `table.column MATCH ?`. + * + * @throws IllegalArgumentException if this column isn't one of an FTS table + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun ClauseString<*>.MATCH(query: String): SelectCondition { + require(!isFunction && table is FtsTable<*>) { "MATCH searches a column of an FTS table, which '$valueName' isn't." } + return condition("${table.tableName}.$valueName MATCH ?", mutableListOf(query)) +} + +/** + * The text around the terms a full-text query matched, with them marked: `snippet(table, start, end, ellipsis, column, + * tokens)`. Only meaningful in a query with MATCH. + * + * @param start The text before each matched term + * @param end The text after each matched term + * @param ellipsis The text where the snippet leaves out text + * @param column The index of the column to take the snippet from, or -1 for the one that matches best + * @param tokens The number of terms in the snippet, at most 64; a negative number is its upper bound, taken from the + * column that matches best + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun FtsTable.snippet( + start: String = "", + end: String = "", + ellipsis: String = "...", + column: Int = -1, + tokens: Int = -15, +): ClauseString = ftsFunction( + "snippet($tableName,${sqlString(start)},${sqlString(end)},${sqlString(ellipsis)},$column,$tokens)", +) + +/** + * Where the terms a full-text query matched are, as a text of numbers, four per match: the column, the term of the + * query, and the byte offset and size of the match. Only meaningful in a query with MATCH. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun FtsTable.offsets(): ClauseString = ftsFunction("offsets($tableName)") + +/** + * The statistics of a full-text match, as a BLOB of 32-bit unsigned integers in the machine's byte order, which a + * ranking of the matches is computed from, as FTS3 and FTS4 have no ranking of their own. Only meaningful in a query + * with MATCH. + * + * @param format The statistics to include, as the letters SQLite's documentation of `matchinfo` lists; the default, + * "pcx", gives the number of phrases and columns, and how often each phrase occurs in each column + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun FtsTable.matchinfo(format: String = "pcx"): ClauseBlob = + ClauseBlob("matchinfo($tableName,${sqlString(format)})", this, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = true, columnTables = setOf(tableName)) + +private fun Relation<*>.ftsFunction(valueName: String): ClauseString = + ClauseString(valueName, this, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = true, columnTables = setOf(tableName)) + +private fun sqlString(string: String): String = "'${string.replace("'", "''")}'" diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Function.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Function.kt index a03e5f09..1acd60a7 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Function.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Function.kt @@ -18,8 +18,9 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import com.ctrip.sqllin.dsl.annotation.FunctionDslMaker -import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.Relation import com.ctrip.sqllin.dsl.sql.X import kotlin.jvm.JvmName @@ -39,16 +40,16 @@ import kotlin.jvm.JvmName */ /** An aggregate function of [element] with values of type [V]: NULL when all its values are, or no rows match. */ -private fun Table<*>.numberAggregate(valueName: String, element: ClauseElement<*>): ClauseNumber = - ClauseNumber(valueName, this, isFunction = true, isNullable = element.isNullable, isAggregate = true, isNullOnNoRows = true) +private fun Relation<*>.numberAggregate(valueName: String, element: ClauseElement<*>): ClauseNumber = + ClauseNumber(valueName, this, isFunction = true, isNullable = element.isNullable, isAggregate = true, isNullOnNoRows = true, columnTables = element.columnTables, tables = element.tables, isDeterministic = element.isDeterministic) /** A scalar function of [element] with values of type [V]: NULL when [element] is. */ -private fun Table<*>.numberFunction(valueName: String, element: ClauseElement<*>): ClauseNumber = - ClauseNumber(valueName, this, isFunction = true, isNullable = element.isNullable, isAggregate = element.isAggregate, isNullOnNoRows = element.isNullOnNoRows) +private fun Relation<*>.numberFunction(valueName: String, element: ClauseElement<*>): ClauseNumber = + ClauseNumber(valueName, this, isFunction = true, isNullable = element.isNullable, isAggregate = element.isAggregate, isNullOnNoRows = element.isNullOnNoRows, columnTables = element.columnTables, tables = element.tables, isDeterministic = element.isDeterministic) /** A scalar function of [element] with `String` values: NULL when [element] is. */ -private fun Table<*>.stringFunction(valueName: String, element: ClauseElement<*>): ClauseString = - ClauseString(valueName, this, isFunction = true, isNullable = element.isNullable, isAggregate = element.isAggregate, isNullOnNoRows = element.isNullOnNoRows) +private fun Relation<*>.stringFunction(valueName: String, element: ClauseElement<*>): ClauseString = + ClauseString(valueName, this, isFunction = true, isNullable = element.isNullable, isAggregate = element.isAggregate, isNullOnNoRows = element.isNullOnNoRows, columnTables = element.columnTables, tables = element.tables, isDeterministic = element.isDeterministic) /** * Writes [string] as a SQL string literal. The only character SQLite escapes in one is `'`, by doubling it, so this @@ -65,8 +66,8 @@ private fun sqlString(string: String): String = "'${string.replace("'", "''")}'" * ``` */ @FunctionDslMaker -public fun Table.count(element: ClauseElement<*>): ClauseNumber = - ClauseNumber("count(${element.valueName})", this, isFunction = true, isNullable = false, isAggregate = true, isNullOnNoRows = false) +public fun Relation.count(element: ClauseElement<*>): ClauseNumber = + ClauseNumber("count(${element.sql})", this, isFunction = true, isNullable = false, isAggregate = true, isNullOnNoRows = false, columnTables = element.columnTables, tables = element.tables, isDeterministic = element.isDeterministic) /** * COUNT(*) aggregate function - counts all rows (including NULLs). @@ -77,15 +78,15 @@ public fun Table.count(element: ClauseElement<*>): ClauseNumber = * ``` */ @FunctionDslMaker -public fun Table.count(x: X): ClauseNumber = +public fun Relation.count(x: X): ClauseNumber = ClauseNumber("count(*)", this, isFunction = true, isNullable = false, isAggregate = true, isNullOnNoRows = false) /** * AVG aggregate function - returns average value, as a `Double`. */ @FunctionDslMaker -public fun Table.avg(element: ClauseElement<*>): ClauseNumber = - numberAggregate("avg(${element.valueName})", element) +public fun Relation.avg(element: ClauseElement<*>): ClauseNumber = + numberAggregate("avg(${element.sql})", element) /** * SUM aggregate function - returns sum of values. @@ -96,61 +97,61 @@ public fun Table.avg(element: ClauseElement<*>): ClauseNumber = */ @FunctionDslMaker @JvmName("sumOfByte") -public fun Table.sum(element: ClauseNumber): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseNumber): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** SUM aggregate function of a `Short` column - returns a `Long`. */ @FunctionDslMaker @JvmName("sumOfShort") -public fun Table.sum(element: ClauseNumber): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseNumber): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** SUM aggregate function of an `Int` column - returns a `Long`. */ @FunctionDslMaker @JvmName("sumOfInt") -public fun Table.sum(element: ClauseNumber): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseNumber): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** SUM aggregate function of a `Long` column - returns a `Long`. */ @FunctionDslMaker @JvmName("sumOfLong") -public fun Table.sum(element: ClauseNumber): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseNumber): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** SUM aggregate function of a `UByte` column - returns a `Long`. */ @FunctionDslMaker @JvmName("sumOfUByte") -public fun Table.sum(element: ClauseNumber): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseNumber): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** SUM aggregate function of a `UShort` column - returns a `Long`. */ @FunctionDslMaker @JvmName("sumOfUShort") -public fun Table.sum(element: ClauseNumber): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseNumber): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** SUM aggregate function of a `UInt` column - returns a `Long`. */ @FunctionDslMaker @JvmName("sumOfUInt") -public fun Table.sum(element: ClauseNumber): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseNumber): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** SUM aggregate function of a `Float` column - returns a `Double`. */ @FunctionDslMaker @JvmName("sumOfFloat") -public fun Table.sum(element: ClauseNumber): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseNumber): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** SUM aggregate function of a `Double` column - returns a `Double`. */ @FunctionDslMaker @JvmName("sumOfDouble") -public fun Table.sum(element: ClauseNumber): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseNumber): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** SUM aggregate function of a Boolean column - returns the number of `true` values, as a `Long`. */ @FunctionDslMaker -public fun Table.sum(element: ClauseBoolean): ClauseNumber = - numberAggregate("sum(${element.valueName})", element) +public fun Relation.sum(element: ClauseBoolean): ClauseNumber = + numberAggregate("sum(${element.sql})", element) /** * MAX aggregate function - returns maximum value, of the same type as [element]: a `max` of an `Int` column is an @@ -158,8 +159,8 @@ public fun Table.sum(element: ClauseBoolean): ClauseNumber = */ @Suppress("UNCHECKED_CAST") @FunctionDslMaker -public fun > Table.max(element: E): E = - element.toAggregate("max(${element.valueName})", this) as E +public fun > Relation.max(element: E): E = + element.toAggregate("max(${element.sql})", this) as E /** * MIN aggregate function - returns minimum value, of the same type as [element]: a `min` of an `Int` column is an @@ -167,8 +168,8 @@ public fun > Table.max(element: E): E = */ @Suppress("UNCHECKED_CAST") @FunctionDslMaker -public fun > Table.min(element: E): E = - element.toAggregate("min(${element.valueName})", this) as E +public fun > Relation.min(element: E): E = + element.toAggregate("min(${element.sql})", this) as E /** * GROUP_CONCAT aggregate function - concatenates all non-NULL values in a group with a separator. @@ -187,15 +188,135 @@ public fun > Table.min(element: E): E = * @return ClauseString representing the concatenated result */ @FunctionDslMaker -public fun Table.group_concat(element: ClauseString<*>, infix: String): ClauseString = - ClauseString("group_concat(${element.valueName},${sqlString(infix)})", this, isFunction = true, isNullable = element.isNullable, isAggregate = true, isNullOnNoRows = true) +public fun Relation.group_concat(element: ClauseString<*>, infix: String): ClauseString = + ClauseString("group_concat(${element.sql},${sqlString(infix)})", this, isFunction = true, isNullable = element.isNullable, isAggregate = true, isNullOnNoRows = true, columnTables = element.columnTables, tables = element.tables, isDeterministic = element.isDeterministic) + +/** + * The argument of an aggregate function that takes only the distinct values of [element], as SQL's `DISTINCT` does + * in `count(DISTINCT author)`. [DISTINCT] makes it, and only the aggregate functions it changes take it: `count`, + * `avg`, `sum` and `group_concat`. + * + * @author Yuang Qiao + */ +public class Distinct> internal constructor(internal val element: E) { + + internal val sql: String + get() = "DISTINCT ${element.sql}" +} + +/** + * Makes an aggregate function take only the distinct values of [element], as SQL's `DISTINCT` does: + * ```kotlin + * BookTable SELECT (count(DISTINCT(BookTable.author)) AS BookStats::authors) + * // SELECT count(DISTINCT book.author) AS authors FROM book + * ``` + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun > DISTINCT(element: E): Distinct = Distinct(element) + +/** + * COUNT aggregate function of the distinct values - counts the distinct non-NULL values of an element, as + * `count(DISTINCT author)` counts the authors. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.count(distinct: Distinct<*>): ClauseNumber = + ClauseNumber("count(${distinct.sql})", this, isFunction = true, isNullable = false, isAggregate = true, isNullOnNoRows = false, columnTables = distinct.element.columnTables, tables = distinct.element.tables, isDeterministic = distinct.element.isDeterministic) + +/** + * AVG aggregate function of the distinct values - returns the average of the distinct values, as a `Double`. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.avg(distinct: Distinct<*>): ClauseNumber = + numberAggregate("avg(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of a `Byte` column - returns a `Long`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctByte") +public fun Relation.sum(distinct: Distinct>): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of a `Short` column - returns a `Long`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctShort") +public fun Relation.sum(distinct: Distinct>): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of an `Int` column - returns a `Long`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctInt") +public fun Relation.sum(distinct: Distinct>): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of a `Long` column - returns a `Long`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctLong") +public fun Relation.sum(distinct: Distinct>): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of an `UByte` column - returns a `Long`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctUByte") +public fun Relation.sum(distinct: Distinct>): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of an `UShort` column - returns a `Long`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctUShort") +public fun Relation.sum(distinct: Distinct>): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of an `UInt` column - returns a `Long`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctUInt") +public fun Relation.sum(distinct: Distinct>): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of a `Float` column - returns a `Double`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctFloat") +public fun Relation.sum(distinct: Distinct>): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of a `Double` column - returns a `Double`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctDouble") +public fun Relation.sum(distinct: Distinct>): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** SUM aggregate function of the distinct values of a Boolean column - returns 1 if any is `true`, as a `Long`. */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +@JvmName("sumOfDistinctBoolean") +public fun Relation.sum(distinct: Distinct): ClauseNumber = + numberAggregate("sum(${distinct.sql})", distinct.element) + +/** + * GROUP_CONCAT aggregate function of the distinct values - concatenates the distinct non-NULL values, separated by + * commas. SQLite allows no other separator here, as an aggregate function with DISTINCT takes only one argument. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.group_concat(distinct: Distinct>): ClauseString = + ClauseString("group_concat(${distinct.sql})", this, isFunction = true, isNullable = distinct.element.isNullable, isAggregate = true, isNullOnNoRows = true, columnTables = distinct.element.columnTables, tables = distinct.element.tables, isDeterministic = distinct.element.isDeterministic) /** * ABS scalar function - returns absolute value, of the same type as [element]. */ @FunctionDslMaker -public fun Table.abs(element: ClauseNumber): ClauseNumber = - numberFunction("abs(${element.valueName})", element) +public fun Relation.abs(element: ClauseNumber): ClauseNumber = + numberFunction("abs(${element.sql})", element) /** * ROUND scalar function - rounds a number to a specified number of decimal places. @@ -214,8 +335,8 @@ public fun Table.abs(element: ClauseNumber): ClauseNumber * @return ClauseNumber representing the rounded value, a `Double` even for an integer column */ @FunctionDslMaker -public fun Table.round(element: ClauseNumber<*>, digits: Int): ClauseNumber = - numberFunction("round(${element.valueName},$digits)", element) +public fun Relation.round(element: ClauseNumber<*>, digits: Int): ClauseNumber = + numberFunction("round(${element.sql},$digits)", element) /** * RANDOM scalar function - returns a pseudo-random integer. @@ -231,49 +352,29 @@ public fun Table.round(element: ClauseNumber<*>, digits: Int): ClauseNumb * @return ClauseNumber representing the random integer */ @FunctionDslMaker -public fun Table.random(): ClauseNumber = - ClauseNumber("random()", this, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false) - -/** - * SIGN scalar function - returns the sign of a number. - * - * Returns -1, 0, or +1 if the argument is negative, zero, or positive respectively. - * If the argument is NULL, then NULL is returned. - * - * Example: - * ```kotlin - * // Get the sign of balance - * SELECT WHERE (sign(Account::balance) EQ 1) - * ``` - * ***It is based on SQLite 3.51.1, disabled it temporarily*** - * - * @param element The numeric value to get the sign of - * @return ClauseNumber representing -1, 0, or 1 - */ -/* @FunctionDslMaker - public fun Table.sign(element: ClauseNumber<*>): ClauseNumber = - numberFunction("sign(${element.valueName})", element) */ +public fun Relation.random(): ClauseNumber = + ClauseNumber("random()", this, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, isDeterministic = false) /** * UPPER scalar function - converts string to uppercase. */ @FunctionDslMaker -public fun Table.upper(element: ClauseString<*>): ClauseString = - stringFunction("upper(${element.valueName})", element) +public fun Relation.upper(element: ClauseString<*>): ClauseString = + stringFunction("upper(${element.sql})", element) /** * LOWER scalar function - converts string to lowercase. */ @FunctionDslMaker -public fun Table.lower(element: ClauseString<*>): ClauseString = - stringFunction("lower(${element.valueName})", element) +public fun Relation.lower(element: ClauseString<*>): ClauseString = + stringFunction("lower(${element.sql})", element) /** * LENGTH scalar function - returns string/blob length in bytes. */ @FunctionDslMaker -public fun Table.length(element: ClauseString<*>): ClauseNumber = - numberFunction("length(${element.valueName})", element) +public fun Relation.length(element: ClauseString<*>): ClauseNumber = + numberFunction("length(${element.sql})", element) /** * LENGTH scalar function - returns the length of a BLOB in bytes. @@ -290,8 +391,8 @@ public fun Table.length(element: ClauseString<*>): ClauseNumber = * @return ClauseNumber representing the length in bytes */ @FunctionDslMaker -public fun Table.length(element: ClauseBlob): ClauseNumber = - numberFunction("length(${element.valueName})", element) +public fun Relation.length(element: ClauseBlob): ClauseNumber = + numberFunction("length(${element.sql})", element) /** * SUBSTR scalar function - extracts a substring from a string. @@ -311,8 +412,8 @@ public fun Table.length(element: ClauseBlob): ClauseNumber = * @return ClauseString representing the extracted substring */ @FunctionDslMaker -public fun Table.substr(element: ClauseString<*>, start: Int, len: Int): ClauseString = - stringFunction("substr(${element.valueName},$start,$len)", element) +public fun Relation.substr(element: ClauseString<*>, start: Int, len: Int): ClauseString = + stringFunction("substr(${element.sql},$start,$len)", element) /** * TRIM scalar function - removes leading and trailing whitespace from a string. @@ -329,8 +430,8 @@ public fun Table.substr(element: ClauseString<*>, start: Int, len: Int): * @return ClauseString with whitespace removed from both ends */ @FunctionDslMaker -public fun Table.trim(element: ClauseString<*>): ClauseString = - stringFunction("trim(${element.valueName})", element) +public fun Relation.trim(element: ClauseString<*>): ClauseString = + stringFunction("trim(${element.sql})", element) /** * LTRIM scalar function - removes leading (left) whitespace from a string. @@ -347,8 +448,8 @@ public fun Table.trim(element: ClauseString<*>): ClauseString = * @return ClauseString with leading whitespace removed */ @FunctionDslMaker -public fun Table.ltrim(element: ClauseString<*>): ClauseString = - stringFunction("ltrim(${element.valueName})", element) +public fun Relation.ltrim(element: ClauseString<*>): ClauseString = + stringFunction("ltrim(${element.sql})", element) /** * RTRIM scalar function - removes trailing (right) whitespace from a string. @@ -365,8 +466,8 @@ public fun Table.ltrim(element: ClauseString<*>): ClauseString = * @return ClauseString with trailing whitespace removed */ @FunctionDslMaker -public fun Table.rtrim(element: ClauseString<*>): ClauseString = - stringFunction("rtrim(${element.valueName})", element) +public fun Relation.rtrim(element: ClauseString<*>): ClauseString = + stringFunction("rtrim(${element.sql})", element) /** * REPLACE scalar function - replaces all occurrences of a substring with another string. @@ -385,8 +486,8 @@ public fun Table.rtrim(element: ClauseString<*>): ClauseString = * @return ClauseString with replacements applied */ @FunctionDslMaker -public fun Table.replace(element: ClauseString<*>, old: String, new: String): ClauseString = - stringFunction("replace(${element.valueName},${sqlString(old)},${sqlString(new)})", element) +public fun Relation.replace(element: ClauseString<*>, old: String, new: String): ClauseString = + stringFunction("replace(${element.sql},${sqlString(old)},${sqlString(new)})", element) /** * INSTR scalar function - finds the first occurrence of a substring. @@ -405,8 +506,8 @@ public fun Table.replace(element: ClauseString<*>, old: String, new: Stri * @return ClauseNumber representing the position (1-indexed) or 0 if not found */ @FunctionDslMaker -public fun Table.instr(element: ClauseString<*>, sub: String): ClauseNumber = - numberFunction("instr(${element.valueName},${sqlString(sub)})", element) +public fun Relation.instr(element: ClauseString<*>, sub: String): ClauseNumber = + numberFunction("instr(${element.sql},${sqlString(sub)})", element) /** * PRINTF scalar function - formats a string according to a format specification. @@ -425,5 +526,234 @@ public fun Table.instr(element: ClauseString<*>, sub: String): ClauseNumb * @return ClauseString with the formatted result */ @FunctionDslMaker -public fun Table.printf(format: String, element: ClauseString<*>): ClauseString = - stringFunction("printf(${sqlString(format)},${element.valueName})", element) \ No newline at end of file +public fun Relation.printf(format: String, element: ClauseString<*>): ClauseString = + stringFunction("printf(${sqlString(format)},${element.sql})", element) + +// ========== More functions ========== +// +// The functions below take expressions as all their arguments, and Kotlin values where a value is given: a value becomes +// a literal of the expression's type, as `coalesce(nickname, "?")` does. + +/** + * COALESCE function - the first of [first], [second] and [others] that isn't NULL, or NULL if they all are, as in + * `coalesce(nickname, name)`. It is NULL only where they all can be. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun > Relation.coalesce(first: E, second: E, vararg others: E): E = + firstNotNull("coalesce", listOf(first, second) + others) + +/** + * COALESCE function with a value - [element], or [value] where it is NULL, as in `coalesce(nickname, "?")`, which is + * never NULL. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun > Relation.coalesce(element: E, value: V): E = + firstNotNull("coalesce", listOf(element, element.literalOf(value))) + +/** + * IFNULL function - [element], or [other] where it is NULL. It is NULL only where both can be. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun > Relation.ifnull(element: E, other: E): E = + firstNotNull("ifnull", listOf(element, other)) + +/** + * IFNULL function with a value - [element], or [value] where it is NULL, which is never NULL. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun > Relation.ifnull(element: E, value: V): E = + firstNotNull("ifnull", listOf(element, element.literalOf(value))) + +/** + * NULLIF function - [element], or NULL where it equals [other], so it can be NULL. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun > Relation.nullif(element: E, other: E): E = + nullIfEqual(element, other) + +/** + * NULLIF function with a value - [element], or NULL where it equals [value], as in `nullif(name, "")`, so it can be + * NULL. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun > Relation.nullif(element: E, value: V): E = + nullIfEqual(element, element.literalOf(value)) + +@Suppress("UNCHECKED_CAST") +private fun > firstNotNull(function: String, elements: List>): E { + val traits = strictTraits( + operands = elements, + isNullable = elements.all { it.isNullable }, + isNullOnNoRows = elements.all { it.isNullOnNoRows }, + ) + return elements.first().derive("$function(${sqlOf(elements)})", traits) as E +} + +@Suppress("UNCHECKED_CAST") +private fun > nullIfEqual(element: ClauseElement<*>, other: ClauseElement<*>): E { + val traits = strictTraits(listOf(element, other), isNullable = true, isNullOnNoRows = element.isNullOnNoRows) + return element.derive("nullif(${element.sql},${other.sql})", traits) as E +} + +/** + * MAX scalar function - the greatest of [first], [second] and [others], of their type, as in `max(price, minimumPrice)`. + * Unlike the aggregate `max` of one argument, it is NULL where any of them is. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun > Relation.max(first: E, second: E, vararg others: E): E = + extreme("max", listOf(first, second) + others) + +/** + * MIN scalar function - the least of [first], [second] and [others], of their type. Unlike the aggregate `min` of one + * argument, it is NULL where any of them is. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun > Relation.min(first: E, second: E, vararg others: E): E = + extreme("min", listOf(first, second) + others) + +@Suppress("UNCHECKED_CAST") +private fun > extreme(function: String, elements: List>): E = + elements.first().derive("$function(${sqlOf(elements)})", strictTraits(elements)) as E + +/** + * HEX function - [element]'s value as upper-case hexadecimal digits: of the bytes of a BLOB, and of the text of another + * value. It is an empty string, not NULL, for NULL. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.hex(element: ClauseElement<*>): ClauseString = + stringExpression("hex(${element.sql})", strictTraits(listOf(element), isNullable = false, isNullOnNoRows = false)) + +/** + * QUOTE function - [element]'s value as an SQL literal, such as `'it''s'`, `X'01'` or `NULL`, so it is never NULL. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.quote(element: ClauseElement<*>): ClauseString = + stringExpression("quote(${element.sql})", strictTraits(listOf(element), isNullable = false, isNullOnNoRows = false)) + +/** + * UNICODE function - the code point of the first character of [element], which is NULL for an empty string. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.unicode(element: ClauseString<*>): ClauseNumber = + numberExpression("unicode(${element.sql})", strictTraits(listOf(element), isNullable = true)) + +/** + * RANDOMBLOB function - a BLOB of [length] random bytes, which is different each time it is computed. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.randomblob(length: Int): ClauseBlob = + blobExpression("randomblob($length)", strictTraits(emptyList(), isNullable = false, isNullOnNoRows = false, isDeterministic = false)) + +/** + * ZEROBLOB function - a BLOB of [length] zero bytes. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.zeroblob(length: Int): ClauseBlob = + blobExpression("zeroblob($length)", strictTraits(emptyList(), isNullable = false, isNullOnNoRows = false)) + +/** + * TRIM function - [element] without the [characters] at its start and end, as in `trim(code, "0")`. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.trim(element: ClauseString<*>, characters: String): ClauseString = + stringExpression("trim(${element.sql},${expressionLiteral(characters)})", strictTraits(listOf(element))) + +/** + * LTRIM function - [element] without the [characters] at its start. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.ltrim(element: ClauseString<*>, characters: String): ClauseString = + stringExpression("ltrim(${element.sql},${expressionLiteral(characters)})", strictTraits(listOf(element))) + +/** + * RTRIM function - [element] without the [characters] at its end. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.rtrim(element: ClauseString<*>, characters: String): ClauseString = + stringExpression("rtrim(${element.sql},${expressionLiteral(characters)})", strictTraits(listOf(element))) + +/** + * SUBSTR function - the characters of [element] from position [start] on, counted from 1. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.substr(element: ClauseString<*>, start: Int): ClauseString = + stringExpression("substr(${element.sql},$start)", strictTraits(listOf(element))) + +/** + * SUBSTR function of expressions - the [length] characters of [element] from position [start] on, counted from 1. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.substr(element: ClauseString<*>, start: ClauseNumber<*>, length: ClauseNumber<*>): ClauseString = + stringExpression("substr(${element.sql},${start.sql},${length.sql})", strictTraits(listOf(element, start, length))) + +/** + * REPLACE function of expressions - [element] with every [old] replaced by [new]. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.replace(element: ClauseString<*>, old: ClauseString<*>, new: ClauseString<*>): ClauseString = + stringExpression("replace(${element.sql},${old.sql},${new.sql})", strictTraits(listOf(element, old, new))) + +/** + * INSTR function of an expression - the position of the first [sub] in [element], counted from 1, or 0 if there is none. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.instr(element: ClauseString<*>, sub: ClauseString<*>): ClauseNumber = + numberExpression("instr(${element.sql},${sub.sql})", strictTraits(listOf(element, sub))) + +/** + * PRINTF function of several values - [format] with its specifiers, as `%d` or `%s`, replaced by [elements] in turn, + * as in `printf("%s: %d", name, age)`. A NULL value is written as an empty string or zero, so it is never NULL. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.printf(format: String, vararg elements: ClauseElement<*>): ClauseString { + val operands = elements.toList() + val arguments = if (operands.isEmpty()) "" else ",${sqlOf(operands)}" + return stringExpression("printf(${expressionLiteral(format)}$arguments)", strictTraits(operands, isNullable = false, isNullOnNoRows = false)) +} + +/** + * ROUND function - [element] rounded to an integer, as a `Double`. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.round(element: ClauseNumber<*>): ClauseNumber = + numberExpression("round(${element.sql})", strictTraits(listOf(element))) + +/** + * TOTAL aggregate function - the sum of [element]'s values as a `Double`, which, unlike `sum`, is `0.0` rather than + * NULL when they are all NULL, or no rows match. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.total(element: ClauseNumber<*>): ClauseNumber = + ClauseNumber("total(${element.sql})", this, isFunction = true, isNullable = false, isAggregate = true, isNullOnNoRows = false, columnTables = element.columnTables, tables = element.tables, isDeterministic = element.isDeterministic) + +/** + * TOTAL aggregate function of the distinct values - the sum of the distinct values as a `Double`, `0.0` when there are + * none. + */ +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.total(distinct: Distinct>): ClauseNumber = + ClauseNumber("total(${distinct.sql})", this, isFunction = true, isNullable = false, isAggregate = true, isNullOnNoRows = false, columnTables = distinct.element.columnTables, tables = distinct.element.tables, isDeterministic = distinct.element.isDeterministic) diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/GroupByClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/GroupByClause.kt index d9626307..374aa6db 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/GroupByClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/GroupByClause.kt @@ -38,17 +38,21 @@ import com.ctrip.sqllin.dsl.sql.statement.WhereSelectStatement * * @author Yuang Qiao */ -public class GroupByClause internal constructor(private val columnNames: Iterable>) : SelectClause { +public class GroupByClause internal constructor(internal val columnNames: Iterable>) : SelectClause { + + init { + requireNoLiteral(columnNames, "GROUP BY") + } override val clauseStr: String get() = buildString { append(" GROUP BY ") val iterator = columnNames.iterator() require(iterator.hasNext()) { "Please provider at least one 'BaseClauseElement' for 'GROUP BY' clause!!!" } - append(iterator.next().valueName) + iterator.next().appendSQL(this) while (iterator.hasNext()) { append(',') - append(iterator.next().valueName) + iterator.next().appendSQL(this) } } } diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/InnerJoinClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/InnerJoinClause.kt index fb7728a5..bba48786 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/InnerJoinClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/InnerJoinClause.kt @@ -18,8 +18,8 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.sql.Relation import com.ctrip.sqllin.dsl.annotation.StatementDslMaker -import com.ctrip.sqllin.dsl.sql.Table /** * INNER JOIN clause - returns rows where there's a match in both tables. @@ -32,7 +32,7 @@ import com.ctrip.sqllin.dsl.sql.Table * @author Yuang Qiao */ internal class InnerJoinClause( - vararg tables: Table<*>, + vararg tables: Relation<*>, ) : JoinClause(*tables) { override val clauseName: String = " JOIN " @@ -46,15 +46,25 @@ internal class InnerJoinClause( * SELECT(user) JOIN (order) ON (user.id EQ order.userId) * SELECT(user) JOIN (order) USING (user.id) * ``` + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. */ @StatementDslMaker -public fun JOIN(vararg tables: Table<*>): JoinClause = InnerJoinClause(*tables) +public fun JOIN(vararg tables: Relation<*>): JoinClause = InnerJoinClause(*tables) /** * Alias for [JOIN] - creates an INNER JOIN clause. + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. */ @StatementDslMaker -public inline fun INNER_JOIN(vararg tables: Table<*>): JoinClause = JOIN(*tables) +public inline fun INNER_JOIN(vararg tables: Relation<*>): JoinClause = JOIN(*tables) /** * NATURAL INNER JOIN - automatically joins on columns with matching names. @@ -67,7 +77,7 @@ public inline fun INNER_JOIN(vararg tables: Table<*>): JoinClause = JOIN( * @author Yuang Qiao */ internal class NaturalInnerJoinClause( - vararg tables: Table<*>, + vararg tables: Relation<*>, ) : NaturalJoinClause(*tables) { override val clauseName: String = " NATURAL JOIN " @@ -80,12 +90,22 @@ internal class NaturalInnerJoinClause( * ```kotlin * SELECT(user) NATURAL_JOIN (profile) // Joins on matching column names * ``` + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. */ @StatementDslMaker -public fun NATURAL_JOIN(vararg tables: Table<*>): NaturalJoinClause = NaturalInnerJoinClause(*tables) +public fun NATURAL_JOIN(vararg tables: Relation<*>): NaturalJoinClause = NaturalInnerJoinClause(*tables) /** * Alias for [NATURAL_JOIN]. + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. */ @StatementDslMaker -public inline fun NATURAL_INNER_JOIN(vararg tables: Table<*>): NaturalJoinClause = NATURAL_JOIN(*tables) \ No newline at end of file +public inline fun NATURAL_INNER_JOIN(vararg tables: Relation<*>): NaturalJoinClause = NATURAL_JOIN(*tables) \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/LeftOuterJoinClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/LeftOuterJoinClause.kt index 951e3041..0cbb7835 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/LeftOuterJoinClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/LeftOuterJoinClause.kt @@ -16,8 +16,8 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.sql.Relation import com.ctrip.sqllin.dsl.annotation.StatementDslMaker -import com.ctrip.sqllin.dsl.sql.Table /** * LEFT OUTER JOIN clause - returns all rows from the left table and matching rows from the right. @@ -31,7 +31,7 @@ import com.ctrip.sqllin.dsl.sql.Table * @author Yuang Qiao */ internal class LeftOuterJoinClause( - vararg tables: Table<*> + vararg tables: Relation<*> ) : JoinClause(*tables) { override val clauseName: String = " LEFT OUTER JOIN " @@ -45,10 +45,15 @@ internal class LeftOuterJoinClause( * SELECT(user) LEFT_OUTER_JOIN (order) ON (user.id EQ order.userId) * // Returns all users, including those without orders * ``` + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. */ @Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") @StatementDslMaker -public fun LEFT_OUTER_JOIN(vararg tables: Table<*>): JoinClause = LeftOuterJoinClause(*tables) +public fun LEFT_OUTER_JOIN(vararg tables: Relation<*>): JoinClause = LeftOuterJoinClause(*tables) /** * NATURAL LEFT OUTER JOIN - automatically joins on matching column names. @@ -62,7 +67,7 @@ public fun LEFT_OUTER_JOIN(vararg tables: Table<*>): JoinClause = LeftOut * @author Yuang Qiao */ internal class NaturalLeftOuterJoinClause( - vararg tables: Table<*> + vararg tables: Relation<*> ) : NaturalJoinClause(*tables) { override val clauseName: String = " NATURAL LEFT OUTER JOIN " @@ -75,7 +80,12 @@ internal class NaturalLeftOuterJoinClause( * ```kotlin * SELECT(user) NATURAL_LEFT_OUTER_JOIN (profile) * ``` + * + * **To be removed in the next version after 2.5.0**: this join API is replaced by joins of relations, such as + * `(FROM(PersonTable) INNER_JOIN BookTable ON (...)) SELECT X()`, which check the type their rows are read into + * against the joined relations, and can join more than two. See + * [JoinedRelation][com.ctrip.sqllin.dsl.sql.clause.JoinedRelation]. */ @Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") @StatementDslMaker -public fun NATURAL_LEFT_OUTER_JOIN(vararg tables: Table<*>): NaturalJoinClause = NaturalLeftOuterJoinClause(*tables) \ No newline at end of file +public fun NATURAL_LEFT_OUTER_JOIN(vararg tables: Relation<*>): NaturalJoinClause = NaturalLeftOuterJoinClause(*tables) \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/LimitClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/LimitClause.kt index 7f961434..bb034c4c 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/LimitClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/LimitClause.kt @@ -18,6 +18,7 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import com.ctrip.sqllin.dsl.annotation.StatementDslMaker import com.ctrip.sqllin.dsl.sql.statement.* @@ -72,6 +73,13 @@ public infix fun JoinSelectStatement.LIMIT(count: Int): LimitSelectStatem container changeLastStatement it } +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun CompoundSelectStatement.LIMIT(count: Int): LimitSelectStatement = + appendToLimit(LimitClause(count)).also { + container changeLastStatement it + } + @StatementDslMaker public infix fun ResultColumnSelectStatement.LIMIT(count: Int): LimitSelectStatement = appendToLimit(LimitClause(count)).also { diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Literal.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Literal.kt new file mode 100644 index 00000000..b97cc76d --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/Literal.kt @@ -0,0 +1,74 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.clause + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI + +/** + * Literal values, as elements of expressions. + * + * A Kotlin value given to an operator or a function becomes a literal by itself, as `1` in `visits + 1`, so `literal` + * is only needed where a value stands alone, such as a value to set a column to, or a branch of `CASE`. The value is + * written into the SQL, which SQLite lets a view, a trigger and an index have, where it allows no parameters, and a + * string is written between single quotes with each one inside doubled, so that it stays a literal. + * + * A literal can't be NULL: leave a column out to set it to NULL, or omit `ELSE` in a `CASE`. + * + * @author Yuang Qiao + */ + +/** A literal number, of the type of [value]: `literal(5)` is a `ClauseNumber`. */ +@ExperimentalDSLDatabaseAPI +public fun literal(value: V): ClauseNumber = ClauseNumber(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + +/** A literal `UByte`, written as the number SQLite stores for it. */ +@ExperimentalDSLDatabaseAPI +public fun literal(value: UByte): ClauseNumber = unsignedLiteral(value) + +/** A literal `UShort`, written as the number SQLite stores for it. */ +@ExperimentalDSLDatabaseAPI +public fun literal(value: UShort): ClauseNumber = unsignedLiteral(value) + +/** A literal `UInt`, written as the number SQLite stores for it. */ +@ExperimentalDSLDatabaseAPI +public fun literal(value: UInt): ClauseNumber = unsignedLiteral(value) + +/** A literal `ULong`, written as the `Long` of the same bits, which SQLite stores for it. */ +@ExperimentalDSLDatabaseAPI +public fun literal(value: ULong): ClauseNumber = unsignedLiteral(value) + +/** A literal string. */ +@ExperimentalDSLDatabaseAPI +public fun literal(value: String): ClauseString = ClauseString(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + +/** A literal `Char`, which SQLite stores as a string of one character. */ +@ExperimentalDSLDatabaseAPI +public fun literal(value: Char): ClauseString = ClauseString(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + +/** A literal Boolean, written as `1` or `0`, as SQLite stores it. */ +@ExperimentalDSLDatabaseAPI +public fun literal(value: Boolean): ClauseBoolean = ClauseBoolean(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + +/** A literal BLOB, written as `X'...'`. */ +@ExperimentalDSLDatabaseAPI +public fun literal(value: ByteArray): ClauseBlob = ClauseBlob(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + +/** A literal enum entry, written as its ordinal, as SQLite stores it. */ +@ExperimentalDSLDatabaseAPI +public fun > literal(value: T): ClauseEnum = ClauseEnum(expressionLiteral(value.ordinal), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) + +private fun unsignedLiteral(value: V): ClauseNumber = ClauseNumber(expressionLiteral(value), ExpressionRelation, isFunction = true, isNullable = false, isAggregate = false, isNullOnNoRows = false, columnTables = emptySet(), isLiteral = true) diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/OrderByClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/OrderByClause.kt index 90d1e69c..a18a3b70 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/OrderByClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/OrderByClause.kt @@ -18,7 +18,9 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import com.ctrip.sqllin.dsl.annotation.KeyWordDslMaker +import com.ctrip.sqllin.dsl.annotation.PlatformDependentSQLiteAPI import com.ctrip.sqllin.dsl.annotation.StatementDslMaker import com.ctrip.sqllin.dsl.sql.statement.* @@ -28,7 +30,7 @@ import com.ctrip.sqllin.dsl.sql.statement.* * Generates SQL in the format: ` ORDER BY column1 ASC, column2 DESC, ...` * * Supports two modes: - * - Explicit direction: `ORDER_BY(user.name to ASC, user.age to DESC)` + * - Explicit direction: `ORDER_BY(user.name to ASC, user.age to DESC)`, which `NULLS FIRST` or `NULLS LAST` can follow * - Default ascending: `ORDER_BY(user.name, user.age)` * * @param T The entity type this clause operates on @@ -37,7 +39,14 @@ import com.ctrip.sqllin.dsl.sql.statement.* */ public sealed interface OrderByClause : SelectClause -internal class CompleteOrderByClause(private val column2WayMap: Map, OrderByWay>) : OrderByClause { +internal class CompleteOrderByClause( + internal val column2WayMap: Map, Ordering>, + private val isQualified: Boolean = true, +) : OrderByClause { + + init { + requireNoLiteral(column2WayMap.keys, "ORDER BY") + } override val clauseStr: String get() { @@ -47,9 +56,9 @@ internal class CompleteOrderByClause(private val column2WayMap: Map(private val column2WayMap: Map, isQualified: Boolean) { + if (isQualified) + element.appendSQL(this) + else + append(element.valueName) +} + +/** + * How ORDER BY sorts an expression: ASC or DESC, which NULLS can follow, as in `ASC NULLS LAST`. + */ +public sealed interface Ordering + +public enum class OrderByWay(internal val str: String) : Ordering { @KeyWordDslMaker ASC("ASC"), @@ -68,61 +94,144 @@ public enum class OrderByWay(internal val str: String) { DESC("DESC") } +/** + * Where NULLS puts the NULL values of an expression that ORDER BY sorts: FIRST or LAST. + */ +@ExperimentalDSLDatabaseAPI +public sealed class NullsPosition(internal val str: String) + +/** `NULLS FIRST`: the NULL values before the others, as SQLite sorts them in ascending order by default. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object FIRST : NullsPosition("FIRST") + +/** `NULLS LAST`: the NULL values after the others, as SQLite sorts them in descending order by default. */ +@ExperimentalDSLDatabaseAPI +@KeyWordDslMaker +public object LAST : NullsPosition("LAST") + +/** + * ASC or DESC with where the NULL values go, as `ASC NULLS LAST` gives. + */ +@ExperimentalDSLDatabaseAPI +public class NullsOrdering internal constructor(internal val way: OrderByWay, internal val position: NullsPosition) : Ordering + +/** + * Puts the NULL values first or last, as the SQL `NULLS FIRST` and `NULLS LAST` do: `ORDER_BY(name to (ASC NULLS LAST))`. + * + * It needs SQLite 3.30.0, which Android has from API 31 on. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun OrderByWay.NULLS(position: NullsPosition): NullsOrdering = NullsOrdering(this, position) + +/** + * Puts the NULL values of this expression first or last, as the SQL `NULLS FIRST` and `NULLS LAST` do, so that it reads + * as SQL: `ORDER_BY(name to ASC NULLS LAST)`. + * + * It needs SQLite 3.30.0, which Android has from API 31 on. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun Pair, OrderByWay>.NULLS(position: NullsPosition): Pair, NullsOrdering> = + first to NullsOrdering(second, position) + +/** This ordering as SQL: `ASC`, or `ASC NULLS LAST`. */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +internal val Ordering.sql: String + get() = when (this) { + is OrderByWay -> str + is NullsOrdering -> "${way.str} NULLS ${position.str}" + } + @StatementDslMaker -public fun ORDER_BY(vararg column2Ways: Pair, OrderByWay>): OrderByClause = +public fun ORDER_BY(vararg column2Ways: Pair, Ordering>): OrderByClause = CompleteOrderByClause(mapOf(*column2Ways)) @StatementDslMaker -public inline infix fun WhereSelectStatement.ORDER_BY(column2Way: Pair, OrderByWay>): OrderBySelectStatement = +public inline infix fun WhereSelectStatement.ORDER_BY(column2Way: Pair, Ordering>): OrderBySelectStatement = ORDER_BY(mapOf(column2Way)) @StatementDslMaker -public infix fun WhereSelectStatement.ORDER_BY(column2WayMap: Map, OrderByWay>): OrderBySelectStatement = +public infix fun WhereSelectStatement.ORDER_BY(column2WayMap: Map, Ordering>): OrderBySelectStatement = appendToOrderBy(CompleteOrderByClause(column2WayMap)).also { container changeLastStatement it } @StatementDslMaker -public inline infix fun HavingSelectStatement.ORDER_BY(column2Way: Pair, OrderByWay>): OrderBySelectStatement = +public inline infix fun HavingSelectStatement.ORDER_BY(column2Way: Pair, Ordering>): OrderBySelectStatement = ORDER_BY(mapOf(column2Way)) @StatementDslMaker -public infix fun HavingSelectStatement.ORDER_BY(column2WayMap: Map, OrderByWay>): OrderBySelectStatement = +public infix fun HavingSelectStatement.ORDER_BY(column2WayMap: Map, Ordering>): OrderBySelectStatement = appendToOrderBy(CompleteOrderByClause(column2WayMap)).also { container changeLastStatement it } @StatementDslMaker -public inline infix fun GroupBySelectStatement.ORDER_BY(column2Way: Pair, OrderByWay>): OrderBySelectStatement = +public inline infix fun GroupBySelectStatement.ORDER_BY(column2Way: Pair, Ordering>): OrderBySelectStatement = ORDER_BY(mapOf(column2Way)) @StatementDslMaker -public infix fun GroupBySelectStatement.ORDER_BY(column2WayMap: Map, OrderByWay>): OrderBySelectStatement = +public infix fun GroupBySelectStatement.ORDER_BY(column2WayMap: Map, Ordering>): OrderBySelectStatement = appendToOrderBy(CompleteOrderByClause(column2WayMap)).also { container changeLastStatement it } @StatementDslMaker -public inline infix fun JoinSelectStatement.ORDER_BY(column2Way: Pair, OrderByWay>): OrderBySelectStatement = +public inline infix fun JoinSelectStatement.ORDER_BY(column2Way: Pair, Ordering>): OrderBySelectStatement = ORDER_BY(mapOf(column2Way)) @StatementDslMaker -public infix fun JoinSelectStatement.ORDER_BY(column2WayMap: Map, OrderByWay>): OrderBySelectStatement = +public infix fun JoinSelectStatement.ORDER_BY(column2WayMap: Map, Ordering>): OrderBySelectStatement = appendToOrderBy(CompleteOrderByClause(column2WayMap)).also { container changeLastStatement it } +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun CompoundSelectStatement.ORDER_BY(column2Way: Pair, Ordering>): OrderBySelectStatement = + ORDER_BY(mapOf(column2Way)) + +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun CompoundSelectStatement.ORDER_BY(column2WayMap: Map, Ordering>): OrderBySelectStatement = + appendToOrderBy(CompleteOrderByClause(column2WayMap, isQualified = false)).also { + container changeLastStatement it + } + +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun CompoundSelectStatement.ORDER_BY(column: ClauseElement<*>): OrderBySelectStatement = + ORDER_BY(listOf(column)) + +@ExperimentalDSLDatabaseAPI @StatementDslMaker -public infix fun ResultColumnSelectStatement.ORDER_BY(column2Way: Pair, OrderByWay>): OrderBySelectStatement = +public infix fun CompoundSelectStatement.ORDER_BY(columns: Iterable>): OrderBySelectStatement = + appendToOrderBy(SimpleOrderByClause(columns, isQualified = false)).also { + container changeLastStatement it + } + +@StatementDslMaker +public infix fun ResultColumnSelectStatement.ORDER_BY(column2Way: Pair, Ordering>): OrderBySelectStatement = ORDER_BY(mapOf(column2Way)) @StatementDslMaker -public infix fun ResultColumnSelectStatement.ORDER_BY(column2WayMap: Map, OrderByWay>): OrderBySelectStatement = +public infix fun ResultColumnSelectStatement.ORDER_BY(column2WayMap: Map, Ordering>): OrderBySelectStatement = appendToOrderBy(CompleteOrderByClause(column2WayMap)).also { container changeLastStatement it } -internal class SimpleOrderByClause(private val columns: Iterable>) : OrderByClause { +internal class SimpleOrderByClause( + internal val columns: Iterable>, + private val isQualified: Boolean = true, +) : OrderByClause { + + init { + requireNoLiteral(columns, "ORDER BY") + } override val clauseStr: String get() { @@ -130,10 +239,10 @@ internal class SimpleOrderByClause(private val columns: Iterable Relation.sign(element: ClauseNumber<*>): ClauseNumber = + numberExpression("sign(${element.sql})", strictTraits(listOf(element))) + +/** + * UNIXEPOCH function - the Unix time, in seconds, of [time], a time string, as `"now"`, changed by [modifiers], as a `Long`. It needs SQLite + * 3.38.0, which Android has from API 34 on. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.unixepoch(time: String, vararg modifiers: String): ClauseNumber { + val (arguments, traits) = timeArguments(time, modifiers) + return numberExpression("unixepoch($arguments)", traits) +} + +/** + * UNIXEPOCH function - the Unix time, in seconds, of [time], a string expression, changed by [modifiers], as a `Long`. It needs SQLite + * 3.38.0, which Android has from API 34 on. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.unixepoch(time: ClauseString<*>, vararg modifiers: String): ClauseNumber { + val (arguments, traits) = timeArguments(time, modifiers) + return numberExpression("unixepoch($arguments)", traits) +} + +/** + * UNIXEPOCH function - the Unix time, in seconds, of [time], a number expression, a Julian day number or, with the `"unixepoch"` modifier, a Unix time, changed by [modifiers], as a `Long`. It needs SQLite + * 3.38.0, which Android has from API 34 on. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.unixepoch(time: ClauseNumber<*>, vararg modifiers: String): ClauseNumber { + val (arguments, traits) = timeArguments(time, modifiers) + return numberExpression("unixepoch($arguments)", traits) +} + +/** + * CEIL math function - the least integer not less than [element], of its type: an integer stays an integer. It needs the math functions, which + * Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.ceil(element: ClauseNumber): ClauseNumber = + element.derive("ceil(${element.sql})", strictTraits(listOf(element))) + +/** + * FLOOR math function - the greatest integer not greater than [element], of its type: an integer stays an integer. It needs the math functions, which + * Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.floor(element: ClauseNumber): ClauseNumber = + element.derive("floor(${element.sql})", strictTraits(listOf(element))) + +/** + * TRUNC math function - [element] without its fractional part, of its type: an integer stays an integer. It needs the math functions, which + * Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.trunc(element: ClauseNumber): ClauseNumber = + element.derive("trunc(${element.sql})", strictTraits(listOf(element))) + +/** + * SQRT math function - the square root of [element], as a `Double`, NULL for a value outside its domain, so it can be NULL. It needs + * the math functions, which Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.sqrt(element: ClauseNumber<*>): ClauseNumber = + numberExpression("sqrt(${element.sql})", strictTraits(listOf(element), isNullable = true)) + +/** + * EXP math function - e raised to [element], as a `Double`, NULL for a value outside its domain, so it can be NULL. It needs + * the math functions, which Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.exp(element: ClauseNumber<*>): ClauseNumber = + numberExpression("exp(${element.sql})", strictTraits(listOf(element), isNullable = true)) + +/** + * LN math function - the natural logarithm of [element], as a `Double`, NULL for a value outside its domain, so it can be NULL. It needs + * the math functions, which Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.ln(element: ClauseNumber<*>): ClauseNumber = + numberExpression("ln(${element.sql})", strictTraits(listOf(element), isNullable = true)) + +/** + * LOG10 math function - the base-10 logarithm of [element], as a `Double`, NULL for a value outside its domain, so it can be NULL. It needs + * the math functions, which Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.log10(element: ClauseNumber<*>): ClauseNumber = + numberExpression("log10(${element.sql})", strictTraits(listOf(element), isNullable = true)) + +/** + * LOG2 math function - the base-2 logarithm of [element], as a `Double`, NULL for a value outside its domain, so it can be NULL. It needs + * the math functions, which Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.log2(element: ClauseNumber<*>): ClauseNumber = + numberExpression("log2(${element.sql})", strictTraits(listOf(element), isNullable = true)) + +/** + * POW math function - [base] raised to [exponent], as a `Double`, which can be NULL. It needs the math functions, which + * Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.pow(base: ClauseNumber<*>, exponent: ClauseNumber<*>): ClauseNumber = + numberExpression("pow(${base.sql},${exponent.sql})", strictTraits(listOf(base, exponent), isNullable = true)) + +/** + * POW math function - [base] raised to the value [exponent], as a `Double`, which can be NULL. It needs the math + * functions, which Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.pow(base: ClauseNumber<*>, exponent: Double): ClauseNumber = + numberExpression("pow(${base.sql},${expressionLiteral(exponent)})", strictTraits(listOf(base), isNullable = true)) + +/** + * MOD math function - the remainder of [dividend] divided by [divisor], as a `Double`, for real numbers too, unlike + * `%`. It is NULL for a division by zero, so it can be NULL. It needs the math functions, which Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.mod(dividend: ClauseNumber<*>, divisor: ClauseNumber<*>): ClauseNumber = + numberExpression("mod(${dividend.sql},${divisor.sql})", strictTraits(listOf(dividend, divisor), isNullable = true)) + +/** + * MOD math function - the remainder of [dividend] divided by the value [divisor], as a `Double`, NULL for a division by + * zero. It needs the math functions, which Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.mod(dividend: ClauseNumber<*>, divisor: Double): ClauseNumber = + numberExpression("mod(${dividend.sql},${expressionLiteral(divisor)})", strictTraits(listOf(dividend), isNullable = true)) + +/** + * PI math function - pi, as a `Double`. It needs the math functions, which Android never has. + */ +@PlatformDependentSQLiteAPI +@ExperimentalDSLDatabaseAPI +@FunctionDslMaker +public fun Relation.pi(): ClauseNumber = + numberExpression("pi()", strictTraits(emptyList(), isNullable = false, isNullOnNoRows = false)) diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/RelationJoin.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/RelationJoin.kt new file mode 100644 index 00000000..d18239ba --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/RelationJoin.kt @@ -0,0 +1,249 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +@file:Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") + +package com.ctrip.sqllin.dsl.sql.clause + +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.annotation.PlatformDependentSQLiteAPI +import com.ctrip.sqllin.dsl.annotation.StatementDslMaker +import com.ctrip.sqllin.dsl.sql.From +import com.ctrip.sqllin.dsl.sql.Relation + +/** + * Joins of relations: tables, views and derived tables, joined before SELECT reads them, as the FROM clause of SQL + * does, so that every SELECT works on them alike. + * + * Example: + * ```kotlin + * @Serializable + * data class PersonBooks(val name: String, val books: Long) + * + * // SELECT person.name AS name,count(book.title) AS books FROM person LEFT OUTER JOIN book ON person.id=book.authorId + * // GROUP BY person.id + * (FROM(PersonTable) LEFT_OUTER_JOIN BookTable ON (PersonTable.id EQ BookTable.authorId)) SELECT listOf( + * PersonTable.name AS PersonBooks::name, + * PersonTable.count(BookTable.title) AS PersonBooks::books, + * ) GROUP_BY PersonTable.id + * ``` + * + * A join reads its rows into the type a SELECT names, as `X()`, `WHERE(...)` or result columns do, and checks it + * against its relations when the statement is built: every property has to name a column of one of them, of a single + * one unless USING or a NATURAL join merges them, as columns of the same name of two relations can't be told apart + * otherwise; and has to be nullable when its column or expression can be NULL, which an outer join makes those of a + * relation it can leave without a matching row, but `count`. SQLlin writes the columns with their relations' names. + * + * A join starts with `FROM(relation)`, and its joins are applied in the order they are written: + * `FROM(a) INNER_JOIN b ON (...) LEFT_OUTER_JOIN c ON (...)`. A relation can only be joined once; to join a table with + * itself, join a derived table of its rows, which gives it another name. + * + * @author Yuang Qiao + */ + +/** + * Relations joined as a FROM clause writes them, such as `person INNER JOIN book ON person.id=book.authorId`, which a + * SELECT reads, or another join extends. `FROM(relation)` starts one. + * + * A SELECT of a join has to name the type it reads rows into, as rows of a join have no type of their own: + * `joined SELECT X()`, `joined SELECT WHERE(...)`, or result columns. + */ +@ExperimentalDSLDatabaseAPI +public class JoinedRelation internal constructor(internal val from: From) + +/** + * A join that still needs its constraint: `ON condition`, or `USING (columns)`. + */ +@ExperimentalDSLDatabaseAPI +public class JoinWithoutCondition internal constructor( + private val from: From, + private val keyword: String, + private val relation: Relation<*>, + private val isLeftNullable: Boolean, + private val isRightNullable: Boolean, +) { + + internal fun on(condition: SelectCondition): JoinedRelation = join( + From.Constraint(" ON ${condition.conditionSQL}", condition.parameters.orEmpty(), condition.tables) + ) + + internal fun using(columns: Iterable>): JoinedRelation { + val names = columns.map { it.valueName } + require(names.isNotEmpty()) { "USING needs at least one column." } + return join(From.Constraint(names.joinToString(",", " USING (", ")"), mergedColumns = names.toSet())) + } + + private fun join(constraint: From.Constraint): JoinedRelation = + JoinedRelation(from.join(keyword, relation, constraint, isLeftNullable, isRightNullable, isNatural = false)) +} + +/** + * Starts a join with [relation], the first relation of a FROM clause, which `INNER_JOIN`, `LEFT_OUTER_JOIN` and the + * other joins extend: `FROM(PersonTable) INNER_JOIN BookTable ON (...)`. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public fun FROM(relation: Relation<*>): JoinedRelation = JoinedRelation(From.of(relation)) + +/** + * Completes this join with `ON condition`: the rows of the relations that satisfy [condition] match. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun JoinWithoutCondition.ON(condition: SelectCondition): JoinedRelation = on(condition) + +/** + * Completes this join with `USING (column)`: the rows whose values of [column], which both relations have, are the same + * match, and the join merges the two columns into one. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun JoinWithoutCondition.USING(column: ClauseElement<*>): JoinedRelation = using(listOf(column)) + +/** + * Completes this join with `USING (columns)`: the rows whose values of [columns], which both relations have, are the + * same match, and the join merges the columns of the same name into one. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun JoinWithoutCondition.USING(columns: Iterable>): JoinedRelation = using(columns) + +/** + * Joins [relation] to this join with `INNER JOIN`: the rows of both that match, which [ON] or [USING] completes. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun JoinedRelation.INNER_JOIN(relation: Relation<*>): JoinWithoutCondition = + from.innerjoin(relation) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +private fun From.innerjoin(relation: Relation<*>): JoinWithoutCondition = + JoinWithoutCondition(this, "INNER JOIN", relation, isLeftNullable = false, isRightNullable = false) + +/** + * Joins [relation] to this join with `LEFT OUTER JOIN`: the rows of both that match, and the rows of the relations + * before it that match none, with NULL in the columns of [relation], which [ON] or [USING] completes. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun JoinedRelation.LEFT_OUTER_JOIN(relation: Relation<*>): JoinWithoutCondition = + from.leftouterjoin(relation) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +private fun From.leftouterjoin(relation: Relation<*>): JoinWithoutCondition = + JoinWithoutCondition(this, "LEFT OUTER JOIN", relation, isLeftNullable = false, isRightNullable = true) + +/** + * Joins [relation] to this join with `RIGHT OUTER JOIN`: the rows of both that match, and the rows of [relation] that + * match none, with NULL in the columns of the relations before it, which [ON] or [USING] completes. + * + * `RIGHT` and `FULL` joins need SQLite 3.39.0, which Android has from API 34 on. + */ +@ExperimentalDSLDatabaseAPI +@PlatformDependentSQLiteAPI +@StatementDslMaker +public infix fun JoinedRelation.RIGHT_OUTER_JOIN(relation: Relation<*>): JoinWithoutCondition = + from.rightouterjoin(relation) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +private fun From.rightouterjoin(relation: Relation<*>): JoinWithoutCondition = + JoinWithoutCondition(this, "RIGHT OUTER JOIN", relation, isLeftNullable = true, isRightNullable = false) + +/** + * Joins [relation] to this join with `FULL OUTER JOIN`: the rows of both that match, and the rows of each that match + * none, with NULL in the columns of the other, which [ON] or [USING] completes. + * + * `RIGHT` and `FULL` joins need SQLite 3.39.0, which Android has from API 34 on. + */ +@ExperimentalDSLDatabaseAPI +@PlatformDependentSQLiteAPI +@StatementDslMaker +public infix fun JoinedRelation.FULL_OUTER_JOIN(relation: Relation<*>): JoinWithoutCondition = + from.fullouterjoin(relation) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +private fun From.fullouterjoin(relation: Relation<*>): JoinWithoutCondition = + JoinWithoutCondition(this, "FULL OUTER JOIN", relation, isLeftNullable = true, isRightNullable = true) + +/** + * Joins [relation] to this join with `CROSS JOIN`: every row of the relations before it with every row of [relation]. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun JoinedRelation.CROSS_JOIN(relation: Relation<*>): JoinedRelation = + from.crossjoin(relation) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +private fun From.crossjoin(relation: Relation<*>): JoinedRelation = + JoinedRelation(join("CROSS JOIN", relation, From.Constraint.NONE, isLeftNullable = false, isRightNullable = false, isNatural = false)) + +/** + * Joins [relation] to this join with `NATURAL JOIN`: the rows of both whose columns of the same names have the same + * values, which it merges. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun JoinedRelation.NATURAL_JOIN(relation: Relation<*>): JoinedRelation = + from.naturaljoin(relation) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +private fun From.naturaljoin(relation: Relation<*>): JoinedRelation = + JoinedRelation(join("NATURAL JOIN", relation, From.Constraint.NONE, isLeftNullable = false, isRightNullable = false, isNatural = true)) + +/** + * Joins [relation] to this join with `NATURAL LEFT OUTER JOIN`: as [NATURAL_JOIN] does, and the rows of the relations + * before it that match none, with NULL in the columns of [relation]. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun JoinedRelation.NATURAL_LEFT_OUTER_JOIN(relation: Relation<*>): JoinedRelation = + from.naturalleftouterjoin(relation) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +private fun From.naturalleftouterjoin(relation: Relation<*>): JoinedRelation = + JoinedRelation(join("NATURAL LEFT OUTER JOIN", relation, From.Constraint.NONE, isLeftNullable = false, isRightNullable = true, isNatural = true)) + +/** + * Joins [relation] to this join with `NATURAL RIGHT OUTER JOIN`: as [NATURAL_JOIN] does, and the rows of [relation] + * that match none, with NULL in the columns of the relations before it. + * + * `RIGHT` and `FULL` joins need SQLite 3.39.0, which Android has from API 34 on. + */ +@ExperimentalDSLDatabaseAPI +@PlatformDependentSQLiteAPI +@StatementDslMaker +public infix fun JoinedRelation.NATURAL_RIGHT_OUTER_JOIN(relation: Relation<*>): JoinedRelation = + from.naturalrightouterjoin(relation) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +private fun From.naturalrightouterjoin(relation: Relation<*>): JoinedRelation = + JoinedRelation(join("NATURAL RIGHT OUTER JOIN", relation, From.Constraint.NONE, isLeftNullable = true, isRightNullable = false, isNatural = true)) + +/** + * Joins [relation] to this join with `NATURAL FULL OUTER JOIN`: as [NATURAL_JOIN] does, and the rows of each that match + * none, with NULL in the columns of the other. + * + * `RIGHT` and `FULL` joins need SQLite 3.39.0, which Android has from API 34 on. + */ +@ExperimentalDSLDatabaseAPI +@PlatformDependentSQLiteAPI +@StatementDslMaker +public infix fun JoinedRelation.NATURAL_FULL_OUTER_JOIN(relation: Relation<*>): JoinedRelation = + from.naturalfullouterjoin(relation) + +@OptIn(ExperimentalDSLDatabaseAPI::class) +private fun From.naturalfullouterjoin(relation: Relation<*>): JoinedRelation = + JoinedRelation(join("NATURAL FULL OUTER JOIN", relation, From.Constraint.NONE, isLeftNullable = true, isRightNullable = true, isNatural = true)) diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ResultColumn.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ResultColumn.kt index ceb5a3f3..4654c4ad 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ResultColumn.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/ResultColumn.kt @@ -19,6 +19,9 @@ package com.ctrip.sqllin.dsl.sql.clause import com.ctrip.sqllin.dsl.annotation.StatementDslMaker +import com.ctrip.sqllin.dsl.sql.Table +import kotlinx.serialization.ExperimentalSerializationApi +import kotlinx.serialization.encoding.CompositeDecoder import kotlin.reflect.KProperty1 /** @@ -36,7 +39,7 @@ import kotlin.reflect.KProperty1 * BookTable { table -> * table SELECT listOf(count(X) AS AuthorStats::books, sum(pages) AS AuthorStats::totalPages) GROUP_BY author * } - * // SELECT author,count(*) AS books,sum(pages) AS totalPages FROM book GROUP BY author + * // SELECT author,count(*) AS books,sum(book.pages) AS totalPages FROM book GROUP BY book.author * ``` * * @param R The result type the expression is selected into @@ -64,3 +67,50 @@ public class ResultColumn internal constructor( @StatementDslMaker public infix fun ClauseElement

.AS(property: KProperty1): ResultColumn = ResultColumn(this, property.name) + +/** The names of the rows a trigger reads, the new one of INSERT and UPDATE, and the old one of UPDATE and DELETE. */ +internal const val NEW_ROW = "NEW" +internal const val OLD_ROW = "OLD" + +/** + * Checks that [bindings], expressions given to properties of the row type of [table] with `AS`, can be written to its + * columns by [statement], and returns the expressions by the names of their columns, in the order given. + * + * Every property has to be serialized under its own name, which names its column, and can't be given two expressions. + * An expression that can be NULL can only be written to a nullable column, can't be an aggregate function, which SQLite + * only computes over the rows of a query, and can only read the columns of [tables], and of the rows of a trigger. + * + * @throws IllegalArgumentException if a binding can't be written + */ +@OptIn(ExperimentalSerializationApi::class) +internal fun writtenColumns(table: Table<*>, bindings: Iterable>, statement: String, tables: Set): Map> { + val descriptor = table.kSerializer().descriptor + val prefix = "Can't $statement '${table.tableName}'" + val columns = LinkedHashMap>() + for (binding in bindings) { + val name = binding.propertyName + val index = descriptor.getElementIndex(name) + require(index != CompositeDecoder.UNKNOWN_NAME) { + "$prefix: its row type doesn't serialize the property '$name' under that name, which names a column." + } + require(name !in columns) { "$prefix: the column '$name' is given more than one expression." } + val element = binding.element + require(!element.isNullable || descriptor.getElementDescriptor(index).isNullable) { + "$prefix: '${element.valueName}' can be NULL, so it can't be written to the column '$name', which isn't nullable." + } + require(!element.isAggregate) { + "$prefix: '${element.valueName}' is an aggregate function, which SQLite only computes over the rows of a query." + } + val otherTable = (element.columnTables - tables - NEW_ROW - OLD_ROW).firstOrNull() + require(otherTable == null) { + if (tables.isEmpty()) + "$prefix: '${element.valueName}' reads table '$otherTable', while the values of INSERT can't read any. Use INSERT with a SELECT." + else + "$prefix: '${element.valueName}' reads table '$otherTable'." + } + columns[name] = element + } + require(columns.isNotEmpty()) { "$prefix with no expressions." } + return columns +} + diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SelectClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SelectClause.kt index f2d6f1aa..3aa6e1a2 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SelectClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SelectClause.kt @@ -38,4 +38,28 @@ package com.ctrip.sqllin.dsl.sql.clause */ public sealed interface SelectClause : Clause { public val clauseStr: String -} \ No newline at end of file +} + +/** + * The tables that the subqueries in the elements of this clause read, which an observed query watches. Those of a + * condition are in the condition. + */ +internal val SelectClause<*>.elementTables: Set + get() = when (this) { + is CompleteOrderByClause<*> -> column2WayMap.keys.flatMapTo(HashSet()) { it.tables } + is SimpleOrderByClause<*> -> columns.flatMapTo(HashSet()) { it.tables } + is GroupByClause<*> -> columnNames.flatMapTo(HashSet()) { it.tables } + else -> emptySet() + } + +/** + * Rejects a literal on its own among [elements] of [clause]: SQLite reads `ORDER BY 2` as the second result column, and + * so `(2)` and `+2` too. + */ +internal fun requireNoLiteral(elements: Iterable>, clause: String) { + val literal = elements.firstOrNull { it.isLiteral } + require(literal == null) { + "$clause can't have the literal ${literal?.valueName} on its own: SQLite would read it as the number of a result column." + } +} + diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SelectCondition.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SelectCondition.kt index 1417f557..fe526e13 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SelectCondition.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SelectCondition.kt @@ -16,6 +16,8 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.sql.operation.parametersOf + /** * Represents a condition expression used in WHERE or HAVING clauses. * @@ -31,43 +33,76 @@ package com.ctrip.sqllin.dsl.sql.clause * * @property conditionSQL The SQL condition expression (may contain ? placeholders) * @property parameters Parameterized query values (String, ByteArray, etc.), or null if none + * @property tables The tables and views that the subqueries of the condition read, which an observed query watches + * @property operator The operator, `AND` or `OR`, that this condition combines others with, or null if it doesn't + * @property isDeterministic Whether the condition always gives the same result for the same row, which the condition + * of a partial index needs * * @author Yuang Qiao */ -public class SelectCondition internal constructor( +public open class SelectCondition internal constructor( internal val conditionSQL: String, internal val parameters: MutableList?, + internal val tables: Set = emptySet(), + private val operator: String? = null, + internal val isDeterministic: Boolean = true, ) { /** * Combines this condition with another using OR. * - * Creates: `(condition1) OR (condition2)` + * Creates: `condition1 OR condition2`, with a condition that combines others with AND in parentheses. */ internal infix fun or(next: SelectCondition): SelectCondition = append("OR", next) /** * Combines this condition with another using AND. * - * Creates: `(condition1) AND (condition2)` + * Creates: `condition1 AND condition2`, with a condition that combines others with OR in parentheses. */ internal infix fun and(next: SelectCondition): SelectCondition = append("AND", next) private fun append(symbol: String, next: SelectCondition): SelectCondition { val sql = buildString { - append(conditionSQL) + appendOperand(this, symbol) append(" $symbol ") - append(next.conditionSQL) + next.appendOperand(this, symbol) } - val combinedParameters = when { - parameters == null && next.parameters != null -> next.parameters - parameters != null && next.parameters == null -> parameters - parameters == null && next.parameters == null -> null - else -> { - parameters!!.addAll(next.parameters!!) - parameters - } + // A new list, as this condition may be kept in a variable and used again, with its own parameters + return SelectCondition(sql, parametersOf(parameters, next.parameters), tables + next.tables, symbol, isDeterministic && next.isDeterministic) + } + + /** + * Appends this condition to [builder] as an operand of [symbol]: in parentheses if it combines others with the + * other operator. The infix AND and OR apply in the order they are written, as Kotlin's infix functions do, while + * SQL gives AND precedence over OR, so `(a OR b) AND c` would otherwise become `a OR (b AND c)`. + */ + private fun appendOperand(builder: StringBuilder, symbol: String) { + if (operator != null && operator != symbol) { + builder.append('(') + builder.append(conditionSQL) + builder.append(')') + } else { + builder.append(conditionSQL) } - return SelectCondition(sql, combinedParameters) } -} \ No newline at end of file +} + +/** + * A LIKE condition, which [ESCAPE] can follow to give its pattern an escape character. + * + * @author Yuang Qiao + */ +public class LikeCondition internal constructor( + conditionSQL: String, + parameters: MutableList, + tables: Set, + isDeterministic: Boolean, +) : SelectCondition(conditionSQL, parameters, tables, isDeterministic = isDeterministic) { + + /** + * This condition with [escape] as the escape character of its pattern: `column LIKE ? ESCAPE ?`. + */ + internal infix fun escape(escape: Char): SelectCondition = + SelectCondition("$conditionSQL ESCAPE ?", parametersOf(parameters, listOf(escape.toString())), tables, isDeterministic = isDeterministic) +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SetClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SetClause.kt index e34fc9b0..72f92a43 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SetClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/SetClause.kt @@ -16,7 +16,9 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import com.ctrip.sqllin.dsl.annotation.StatementDslMaker +import com.ctrip.sqllin.dsl.sql.compiler.storedValue /** * SET clause for UPDATE statements. @@ -53,6 +55,12 @@ public class SetClause : Clause { internal var parameters: MutableList? = null private set + /** + * The expressions given to columns by `SET(listOf(...))`, which UPDATE checks against its table, or null for the + * values assigned in `SET {}`. + */ + internal var assignments: List>? = null + /** * Appends a column assignment to the SET clause using parameterized binding. * @@ -67,7 +75,7 @@ public class SetClause : Clause { val params = parameters ?: ArrayList().also { parameters = it } - params.add(propertyValue) + params.add(storedValue(propertyValue)) } /** @@ -83,4 +91,27 @@ public class SetClause : Clause { @Suppress("DSL_MARKER_APPLIED_TO_WRONG_TARGET") @StatementDslMaker -public inline fun SET(block: SetClause.() -> Unit): SetClause = SetClause().apply(block) \ No newline at end of file +public inline fun SET(block: SetClause.() -> Unit): SetClause = SetClause().apply(block) + +/** + * Sets columns to expressions, as SQL's `SET column = expression` does, such as one computed from the row's own columns: + * ```kotlin + * // UPDATE person SET visits=(person.visits + 1),seen=datetime('now') WHERE person.id=? + * table UPDATE SET(listOf((visits + 1) AS Person::visits, datetime("now") AS Person::seen)) WHERE (id EQ 7) + * ``` + * Each expression is given to a property of the row type with `AS`, which checks its type at compile time. UPDATE checks + * the rest when the statement is built: an expression that can be NULL can only set a nullable column, and it can only + * read the columns of the table it updates, and not be an aggregate function. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public fun SET(assignments: Iterable>): SetClause = + SetClause().apply { this.assignments = assignments.toList() } + +/** + * Sets a column to an expression, as SQL's `SET column = expression` does: `SET((visits + 1) AS Person::visits)`. + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public fun SET(assignment: ResultColumn): SetClause = SET(listOf(assignment)) + diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/WhereClause.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/WhereClause.kt index ca0971d8..8060c742 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/WhereClause.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/clause/WhereClause.kt @@ -18,7 +18,9 @@ package com.ctrip.sqllin.dsl.sql.clause +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import com.ctrip.sqllin.dsl.annotation.StatementDslMaker +import com.ctrip.sqllin.dsl.sql.statement.IndexStatement import com.ctrip.sqllin.dsl.sql.statement.JoinSelectStatement import com.ctrip.sqllin.dsl.sql.statement.ResultColumnSelectStatement import com.ctrip.sqllin.dsl.sql.statement.UpdateDeleteStatement @@ -96,7 +98,22 @@ public infix fun UpdateStatementWithoutWhereClause.WHERE(condition: Selec val statement = UpdateDeleteStatement(buildString { append(sqlStr) append(WhereClause(condition).clauseStr) - }, connection, params) + }, connection, params, table) statementContainer changeLastStatement statement return statement.sqlStr -} \ No newline at end of file +} + +/** + * Makes this index a partial index of the rows that satisfy [condition], as the SQL `CREATE INDEX ... WHERE condition` + * does, such as a unique index of the rows that aren't deleted: + * ```kotlin + * UserTable.CREATE_UNIQUE_INDEX("idx_user_email", UserTable.email) WHERE (UserTable.isDeleted IS false) + * ``` + * Its values are written into the SQL, as SQLite allows no parameters there, nor subqueries. + * + * @throws IllegalArgumentException if [condition] has a subquery, or a function that doesn't always give the same result + * for a row, such as `random()` + */ +@ExperimentalDSLDatabaseAPI +@StatementDslMaker +public infix fun IndexStatement.WHERE(condition: SelectCondition): Unit = where(condition) diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/InlineParameters.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/InlineParameters.kt new file mode 100644 index 00000000..ff123693 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/InlineParameters.kt @@ -0,0 +1,93 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.compiler + +/** + * Returns [sql] with each `?` placeholder replaced by the SQL literal of the corresponding parameter. + * + * SQLite doesn't let a view or a trigger take parameters, so a statement that becomes part of one has to carry its + * values in its SQL. A `?` inside a string literal or a quoted identifier isn't a placeholder, so it is skipped, as in + * the SQL of `replace(name, "?", "")`. + * + * @throws IllegalArgumentException if the placeholders and [parameters] don't match in number, or a parameter has a + * type no literal is written for + */ +internal fun inlineParameters(sql: String, parameters: List?): String { + if (parameters.isNullOrEmpty()) + return sql + val iterator = parameters.iterator() + val builder = StringBuilder(sql.length + parameters.size * 8) + var quote: Char? = null + var index = 0 + while (index < sql.length) { + val char = sql[index] + when { + quote != null -> { + builder.append(char) + if (char == quote) { + // A doubled quote is an escaped one, inside the literal or identifier + if (index + 1 < sql.length && sql[index + 1] == quote) { + builder.append(quote) + index++ + } else { + quote = null + } + } + } + char == '\'' || char == '"' || char == '`' -> { + quote = char + builder.append(char) + } + char == '?' -> { + require(iterator.hasNext()) { "The SQL has more placeholders than the ${parameters.size} parameters: $sql" } + builder.append(sqlLiteral(iterator.next())) + } + else -> builder.append(char) + } + index++ + } + require(!iterator.hasNext()) { "The SQL has fewer placeholders than the ${parameters.size} parameters: $sql" } + return builder.toString() +} + +/** + * Returns [value] as a SQL literal: a string between single quotes, with each one inside doubled; a BLOB as `X'...'`; + * a Boolean as 1 or 0, as SQLite stores it. + */ +internal fun sqlLiteral(value: Any?): String = when (value) { + null -> "NULL" + is String -> "'${value.replace("'", "''")}'" + is Char -> sqlLiteral(value.toString()) + is Boolean -> if (value) "1" else "0" + is Double -> doubleLiteral(value) + is Float -> doubleLiteral(value.toDouble()) + is Byte, is Short, is Int, is Long -> value.toString() + is ByteArray -> value.joinToString(separator = "", prefix = "X'", postfix = "'") { HEX_DIGITS[(it.toInt() shr 4) and 0xF].toString() + HEX_DIGITS[it.toInt() and 0xF] } + else -> throw IllegalArgumentException("Can't write a ${value::class.simpleName} as a SQL literal.") +} + +private const val HEX_DIGITS = "0123456789ABCDEF" + +/** + * SQLite has no literal for NaN, which it stores as NULL, nor for infinity, which a number too large to represent is. + */ +private fun doubleLiteral(value: Double): String = when { + value.isNaN() -> "NULL" + value == Double.POSITIVE_INFINITY -> "9e999" + value == Double.NEGATIVE_INFINITY -> "-9e999" + else -> value.toString() +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/InsertValuesEncoder.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/InsertValuesEncoder.kt index 1774afd8..1a7eb2b6 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/InsertValuesEncoder.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/InsertValuesEncoder.kt @@ -109,14 +109,16 @@ internal class InsertValuesEncoder( override fun encodeNull() = appendAny(null) /** - * Handles inline values (including ByteArray). + * Handles inline values (including ByteArray) and unsigned values. * * ByteArray is treated as an inline value in kotlinx.serialization, so we need to extract - * the actual ByteArray value and append it as a parameter. + * the actual ByteArray value and append it as a parameter. An unsigned value is appended as the + * number it is, as [storedValue] explains, where its serializer would encode its bits as a signed one. */ override fun encodeSerializableValue(serializer: SerializationStrategy, value: T) = - if (value is ByteArray) - appendAny(value) - else - super.encodeSerializableValue(serializer, value) + when (value) { + is ByteArray -> appendAny(value) + is UByte, is UShort, is UInt, is ULong -> appendAny(storedValue(value)) + else -> super.encodeSerializableValue(serializer, value) + } } \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/StoredValue.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/StoredValue.kt new file mode 100644 index 00000000..a61fa301 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/compiler/StoredValue.kt @@ -0,0 +1,33 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.compiler + +/** + * Returns [value] as it is written to the database: an unsigned value as the number it is, and anything else as it is. + * + * SQLite's 64-bit integers hold every `UByte`, `UShort` and `UInt`, so they are stored as their numbers, which SQL + * compares, orders and sums as Kotlin does. Their serializers, and some drivers, would store their bits as a signed + * number instead, so that a `UByte` of 200 read back as 200 but compared as -56. A `ULong` above `Long.MAX_VALUE` fits + * no 64-bit signed integer, so a `ULong` is stored as the `Long` of the same bits. + */ +internal fun storedValue(value: Any?): Any? = when (value) { + is UByte -> value.toLong() + is UShort -> value.toLong() + is UInt -> value.toLong() + is ULong -> value.toLong() + else -> value +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Create.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Create.kt index 0ab6e67d..f78a4bfb 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Create.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Create.kt @@ -17,8 +17,15 @@ package com.ctrip.sqllin.dsl.sql.operation import com.ctrip.sqllin.driver.DatabaseConnection +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.createSQLIfNotExists +import com.ctrip.sqllin.dsl.sql.View import com.ctrip.sqllin.dsl.sql.clause.ClauseElement +import com.ctrip.sqllin.dsl.sql.clause.OrderByWay +import com.ctrip.sqllin.dsl.sql.compiler.appendDBColumnName +import com.ctrip.sqllin.dsl.sql.compiler.inlineParameters +import com.ctrip.sqllin.dsl.sql.statement.SelectStatement import com.ctrip.sqllin.dsl.sql.statement.SingleStatement import com.ctrip.sqllin.dsl.sql.statement.TableStructureStatement @@ -38,6 +45,32 @@ internal object Create : Operation { private const val INDEX = "INDEX " private const val UNIQUE_INDEX = "UNIQUE INDEX " + private const val IF_NOT_EXISTS = "IF NOT EXISTS " + + /** + * Builds a CREATE VIEW statement for [view], defined by [select]. + * + * Generates SQL in the format: `CREATE VIEW view_name(column1, column2, ...) AS SELECT ...`, naming the view's + * columns after the properties of its row type. SQLite doesn't let a view take parameters, so the values of + * [select] are written into the SQL as literals. + * + * @return A [TableStructureStatement] representing the CREATE VIEW operation + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun createView(view: View, select: SelectStatement, connection: DatabaseConnection, isIfNotExists: Boolean = false): SingleStatement { + val sql = buildString { + append(sqlStr) + append("VIEW ") + if (isIfNotExists) + append(IF_NOT_EXISTS) + append(view.tableName) + append('(') + appendDBColumnName(view.kSerializer().descriptor) + append(") AS ") + append(inlineParameters(select.sqlStr, select.parameters)) + } + return TableStructureStatement(sql, connection) + } /** * Builds a CREATE TABLE statement for the given table definition. @@ -46,8 +79,8 @@ internal object Create : Operation { * @param connection Database connection for execution * @return CREATE statement ready for execution */ - fun createTable(table: Table, connection: DatabaseConnection): SingleStatement = - TableStructureStatement(table.createSQL, connection) + fun createTable(table: Table, connection: DatabaseConnection, isIfNotExists: Boolean = false): SingleStatement = + TableStructureStatement(if (isIfNotExists) table.createSQLIfNotExists() else table.createSQL, connection) /** * Builds a CREATE INDEX statement for the specified table and columns. @@ -62,11 +95,17 @@ internal object Create : Operation { * @return CREATE INDEX statement ready for execution * @throws IllegalArgumentException if no columns are specified */ - fun createIndex(table: Table, connection: DatabaseConnection, indexName: String, vararg columns: ClauseElement<*>): SingleStatement { + fun createIndex(table: Table, connection: DatabaseConnection, indexName: String, vararg columns: ClauseElement<*>, isIfNotExists: Boolean = false): SingleStatement { require(columns.isNotEmpty()) { "You must create an index for at least one column." } - return createIndex(INDEX, table, connection, indexName, *columns) + return createIndex(INDEX, table, connection, indexName, columns.map { it to null }, isIfNotExists = isIfNotExists) } + /** + * Builds a CREATE INDEX statement of [columns] in the orders given, as `CREATE INDEX name ON table(age DESC)`. + */ + fun createIndex(table: Table, connection: DatabaseConnection, indexName: String, columns: List, OrderByWay>>, isIfNotExists: Boolean = false): SingleStatement = + createIndex(INDEX, table, connection, indexName, columns, isIfNotExists = isIfNotExists) + /** * Builds a CREATE UNIQUE INDEX statement for the specified table and columns. * @@ -81,11 +120,17 @@ internal object Create : Operation { * @return CREATE UNIQUE INDEX statement ready for execution * @throws IllegalArgumentException if no columns are specified */ - fun createUniqueIndex(table: Table, connection: DatabaseConnection, indexName: String, vararg columns: ClauseElement<*>): SingleStatement { + fun createUniqueIndex(table: Table, connection: DatabaseConnection, indexName: String, vararg columns: ClauseElement<*>, isIfNotExists: Boolean = false): SingleStatement { require(columns.isNotEmpty()) { "You must create an index for at least one column." } - return createIndex(UNIQUE_INDEX, table, connection, indexName, *columns) + return createIndex(UNIQUE_INDEX, table, connection, indexName, columns.map { it to null }, isIfNotExists = isIfNotExists) } + /** + * Builds a CREATE UNIQUE INDEX statement of [columns] in the orders given. + */ + fun createUniqueIndex(table: Table, connection: DatabaseConnection, indexName: String, columns: List, OrderByWay>>, isIfNotExists: Boolean = false): SingleStatement = + createIndex(UNIQUE_INDEX, table, connection, indexName, columns, isIfNotExists = isIfNotExists) + /** * Internal helper function to build CREATE INDEX statements with different prefixes. * @@ -95,31 +140,83 @@ internal object Create : Operation { * @param table Table definition to create the index on * @param connection Database connection for execution * @param indexName Name for the new index - * @param columns One or more columns to include in the index + * @param columns One or more columns to include in the index, with the orders they are sorted in, or null for the + * default, ascending * @return CREATE INDEX statement ready for execution - * @throws IllegalArgumentException if no columns are specified + * @throws IllegalArgumentException if no columns are specified, or one reads another table, or has a subquery, an + * aggregate function or a function that doesn't always give the same result for a row, such as `random()` */ - private fun createIndex(prefix: String, table: Table, connection: DatabaseConnection, indexName: String, vararg columns: ClauseElement<*>): SingleStatement { + private fun createIndex(prefix: String, table: Table, connection: DatabaseConnection, indexName: String, columns: List, OrderByWay?>>, isIfNotExists: Boolean): SingleStatement { val sql = buildString { append(sqlStr) append(prefix) + if (isIfNotExists) + append(IF_NOT_EXISTS) append(indexName) append(" ON ") append(table.tableName) append('(') - val iterator = columns.iterator() - if (!iterator.hasNext()) + if (columns.isEmpty()) throw IllegalArgumentException("You must create an index for at least one column.") - // Extract column name without table prefix (e.g., "book.name" -> "name") - val firstColumn = iterator.next().valueName.substringAfterLast('.') - append(firstColumn) - while (iterator.hasNext()) { - append(',') - val columnName = iterator.next().valueName.substringAfterLast('.') - append(columnName) + columns.forEachIndexed { index, (column, order) -> + if (index > 0) + append(',') + val otherTable = column.columnTables.firstOrNull { it != table.tableName } + require(otherTable == null) { + "The index '$indexName' of table '${table.tableName}' can only hold the columns of that table, but '${column.valueName}' reads table '$otherTable'." + } + // SQLite rejects these too, but a function that isn't deterministic only once the table has rows + require(column.tables.isEmpty()) { + "The index '$indexName' can't hold '${column.valueName}', which has a subquery." + } + require(!column.isAggregate) { + "The index '$indexName' can't hold '${column.valueName}', which is an aggregate function." + } + require(column.isDeterministic) { + "The index '$indexName' can't hold '${column.valueName}', which doesn't always give the same result for a row." + } + append(if (column.isFunction) unqualified(column.valueName, table.tableName) else column.valueName) + if (order != null) { + append(' ') + append(order.str) + } } append(')') } return TableStructureStatement(sql, connection) } + + /** + * Returns the SQL of a function without the name [table] before its columns, outside its string literals, as + * SQLite doesn't allow `.` in the expressions of an index, which can only read the indexed table anyway. + */ + internal fun unqualified(sql: String, table: String): String = buildString { + val prefix = "$table." + var quote: Char? = null + var index = 0 + while (index < sql.length) { + val char = sql[index] + when { + quote != null -> { + // A quote doubled to escape it closes the literal and opens it again + if (char == quote) + quote = null + append(char) + index++ + } + char == '\'' || char == '"' -> { + quote = char + append(char) + index++ + } + sql.startsWith(prefix, index) && (index == 0 || !sql[index - 1].isIdentifierPart()) -> index += prefix.length + else -> { + append(char) + index++ + } + } + } + } + + private fun Char.isIdentifierPart(): Boolean = isLetterOrDigit() || this == '_' } \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Delete.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Delete.kt index 5e17e467..3598ae70 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Delete.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Delete.kt @@ -50,7 +50,7 @@ internal object Delete : Operation { buildBaseDeleteStatement(table) append(whereClause.clauseStr) } - return UpdateDeleteStatement(sql, connection, whereClause.selectCondition.parameters) + return UpdateDeleteStatement(sql, connection, whereClause.selectCondition.parameters, table.tableName) } /** @@ -66,7 +66,7 @@ internal object Delete : Operation { val sql = buildString { buildBaseDeleteStatement(table) } - return UpdateDeleteStatement(sql, connection, null) + return UpdateDeleteStatement(sql, connection, null, table.tableName) } private fun StringBuilder.buildBaseDeleteStatement(table: Table<*>) { diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Drop.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Drop.kt index fc32eb6f..9f82839d 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Drop.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Drop.kt @@ -17,7 +17,9 @@ package com.ctrip.sqllin.dsl.sql.operation import com.ctrip.sqllin.driver.DatabaseConnection +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.View import com.ctrip.sqllin.dsl.sql.statement.SingleStatement import com.ctrip.sqllin.dsl.sql.statement.TableStructureStatement @@ -51,11 +53,32 @@ internal object Drop : Operation { * @param connection The database connection for executing the statement * @return A [TableStructureStatement] representing the DROP TABLE operation */ - fun drop(table: Table<*>, connection: DatabaseConnection): SingleStatement { + fun drop(table: Table<*>, connection: DatabaseConnection, isIfExists: Boolean = false): SingleStatement { val sql = buildString { append(sqlStr) + if (isIfExists) + append(IF_EXISTS) append(table.tableName) } return TableStructureStatement(sql, connection) } -} \ No newline at end of file + + /** + * Creates a DROP VIEW statement for the specified view. + * + * @param view The view to drop + * @param connection The database connection for executing the statement + * @return A [TableStructureStatement] representing the DROP VIEW operation + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun dropView(view: View<*>, connection: DatabaseConnection, isIfExists: Boolean = false): SingleStatement = + TableStructureStatement("DROP VIEW ${if (isIfExists) IF_EXISTS else ""}${view.tableName}", connection) + + /** + * Builds a DROP INDEX statement for the index named [indexName]. + */ + fun dropIndex(indexName: String, connection: DatabaseConnection, isIfExists: Boolean = false): SingleStatement = + TableStructureStatement("DROP INDEX ${if (isIfExists) IF_EXISTS else ""}$indexName", connection) + + private const val IF_EXISTS = "IF EXISTS " +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Insert.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Insert.kt index 21128547..cc3195c6 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Insert.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Insert.kt @@ -21,6 +21,8 @@ import com.ctrip.sqllin.dsl.sql.statement.SingleStatement import com.ctrip.sqllin.dsl.sql.statement.InsertStatement import com.ctrip.sqllin.dsl.sql.statement.SelectStatement import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.clause.ResultColumn +import com.ctrip.sqllin.dsl.sql.clause.writtenColumns import com.ctrip.sqllin.dsl.sql.compiler.appendDBColumnName import com.ctrip.sqllin.dsl.sql.compiler.encodeEntities2InsertValues @@ -59,7 +61,7 @@ internal object Insert : Operation { append(' ') encodeEntities2InsertValues(table, this,entities, parameters, isInsertWithId) } - return InsertStatement(sql, connection, parameters) + return InsertStatement(sql, connection, parameters, table.tableName) } fun insertOrReplace(table: Table, connection: DatabaseConnection, entities: Iterable): SingleStatement { @@ -70,7 +72,7 @@ internal object Insert : Operation { append(' ') encodeEntities2InsertValues(table, this, entities, parameters, isInsertWithId = true) } - return InsertStatement(sql, connection, parameters) + return InsertStatement(sql, connection, parameters, table.tableName) } fun insertOrIgnore(table: Table, connection: DatabaseConnection, entities: Iterable): SingleStatement { @@ -82,7 +84,7 @@ internal object Insert : Operation { // Write the primary key even when the database would assign it, or a conflict on it could never be seen encodeEntities2InsertValues(table, this, entities, parameters, isInsertWithId = true) } - return InsertStatement(sql, connection, parameters) + return InsertStatement(sql, connection, parameters, table.tableName) } /** @@ -112,6 +114,36 @@ internal object Insert : Operation { append(") ") append(select.sqlStr) } - return InsertStatement(sql, connection, select.parameters?.toMutableList()) + return InsertStatement(sql, connection, select.parameters?.toMutableList(), table.tableName) } -} \ No newline at end of file + + /** + * `INSERT INTO table(columns) VALUES (expressions)`, of [values], the expressions given to properties of the row + * type, which can't read any table but the rows of a trigger. The columns left out take their default values, or + * NULL. + */ + fun insert(insert: String, table: Table, connection: DatabaseConnection, values: Iterable>): SingleStatement { + val columns = writtenColumns(table, values, insert.trim().removeSuffix(" INTO"), emptySet()) + val sql = buildString { + append(insert) + append(table.tableName) + append('(') + append(columns.keys.joinToString(",")) + append(") VALUES (") + columns.values.forEachIndexed { index, element -> + if (index > 0) + append(',') + element.appendSQL(this) + } + append(')') + } + return InsertStatement(sql, connection, null, table.tableName) + } + + /** + * `INSERT INTO table DEFAULT VALUES`, which inserts a row of the default values of all columns, or NULL where a + * column has none. + */ + fun insertDefaultValues(insert: String, table: Table<*>, connection: DatabaseConnection): SingleStatement = + InsertStatement("$insert${table.tableName} DEFAULT VALUES", connection, null, table.tableName) +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Maintenance.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Maintenance.kt new file mode 100644 index 00000000..bba4b006 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Maintenance.kt @@ -0,0 +1,42 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.operation + +import com.ctrip.sqllin.driver.DatabaseConnection +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.sql.clause.Collation +import com.ctrip.sqllin.dsl.sql.statement.SingleStatement +import com.ctrip.sqllin.dsl.sql.statement.TableStructureStatement + +/** + * ANALYZE, which gathers the statistics of tables and indexes that the query planner chooses indexes with, and REINDEX, + * which rebuilds indexes. + * + * @author Yuang Qiao + */ +internal object Maintenance { + + fun analyze(name: String?, connection: DatabaseConnection): SingleStatement = + TableStructureStatement(if (name == null) "ANALYZE" else "ANALYZE $name", connection) + + fun reindex(name: String?, connection: DatabaseConnection): SingleStatement = + TableStructureStatement(if (name == null) "REINDEX" else "REINDEX $name", connection) + + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun reindex(collation: Collation, connection: DatabaseConnection): SingleStatement = + TableStructureStatement("REINDEX ${collation.name}", connection) +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/PRAGMA.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/PRAGMA.kt index e8360cc5..b31b5719 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/PRAGMA.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/PRAGMA.kt @@ -88,4 +88,9 @@ internal object PRAGMA : Operation { } return TableStructureStatement(sql, connection) } -} \ No newline at end of file + + /** + * `PRAGMA optimize`, which runs ANALYZE on the tables whose statistics would help the query planner, if any. + */ + fun optimize(connection: DatabaseConnection): SingleStatement = TableStructureStatement("${sqlStr}optimize", connection) +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Select.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Select.kt index 087c4a46..3cca52e5 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Select.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Select.kt @@ -17,7 +17,12 @@ package com.ctrip.sqllin.dsl.sql.operation import com.ctrip.sqllin.driver.DatabaseConnection -import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.sql.From +import com.ctrip.sqllin.dsl.sql.Relation +import com.ctrip.sqllin.dsl.sql.clause.NEW_ROW +import com.ctrip.sqllin.dsl.sql.clause.OLD_ROW +import com.ctrip.sqllin.dsl.sql.clause.elementTables import com.ctrip.sqllin.dsl.sql.clause.* import com.ctrip.sqllin.dsl.sql.compiler.appendDBColumnName import com.ctrip.sqllin.dsl.sql.statement.* @@ -46,15 +51,15 @@ internal object Select : Operation { * @return Statement that can be followed by GROUP BY, ORDER BY, or LIMIT */ fun select( - table: Table<*>, + from: From, clause: WhereClause, isDistinct: Boolean, deserializer: DeserializationStrategy, connection: DatabaseConnection, container: StatementContainer, ): WhereSelectStatement { - checkProjection(table, deserializer) - return WhereSelectStatement(buildSQL(table, clause, isDistinct, deserializer), deserializer, connection, container, clause.selectCondition.parameters, null) + checkProjection(from, deserializer) + return WhereSelectStatement(buildSQL(from, clause, isDistinct, deserializer), deserializer, connection, container, parametersOf(from.parameters, clause.selectCondition.parameters), null, from.tables + clause.selectCondition.tables) } /** @@ -63,15 +68,15 @@ internal object Select : Operation { * @return Statement that can be followed by LIMIT */ fun select( - table: Table<*>, + from: From, clause: OrderByClause, isDistinct: Boolean, deserializer: DeserializationStrategy, connection: DatabaseConnection, container: StatementContainer, ): OrderBySelectStatement { - checkProjection(table, deserializer) - return OrderBySelectStatement(buildSQL(table, clause, isDistinct, deserializer), deserializer, connection, container, null, null) + checkProjection(from, deserializer) + return OrderBySelectStatement(buildSQL(from, clause, isDistinct, deserializer), deserializer, connection, container, parametersOf(from.parameters), null, from.tables + clause.elementTables) } /** @@ -80,15 +85,15 @@ internal object Select : Operation { * @return Statement that can be followed by OFFSET */ fun select( - table: Table<*>, + from: From, clause: LimitClause, isDistinct: Boolean, deserializer: DeserializationStrategy, connection: DatabaseConnection, container: StatementContainer, ): LimitSelectStatement { - checkProjection(table, deserializer) - return LimitSelectStatement(buildSQL(table, clause, isDistinct, deserializer), deserializer, connection, container, null, null) + checkProjection(from, deserializer) + return LimitSelectStatement(buildSQL(from, clause, isDistinct, deserializer), deserializer, connection, container, parametersOf(from.parameters), null, from.tables) } /** @@ -97,15 +102,15 @@ internal object Select : Operation { * @return Statement that can be followed by HAVING or ORDER BY */ fun select( - table: Table<*>, + from: From, clause: GroupByClause, isDistinct: Boolean, deserializer: DeserializationStrategy, connection: DatabaseConnection, container: StatementContainer, ): GroupBySelectStatement { - checkProjection(table, deserializer) - return GroupBySelectStatement(buildSQL(table, clause, isDistinct, deserializer), deserializer, connection, container, null, null) + checkProjection(from, deserializer) + return GroupBySelectStatement(buildSQL(from, clause, isDistinct, deserializer), deserializer, connection, container, parametersOf(from.parameters), null, from.tables + clause.elementTables) } /** @@ -116,14 +121,14 @@ internal object Select : Operation { * @return Statement that can be followed by WHERE, GROUP BY, ORDER BY, or LIMIT */ fun select( - table: Table<*>, + table: Relation<*>, clause: NaturalJoinClause, isDistinct: Boolean, deserializer: DeserializationStrategy, connection: DatabaseConnection, container: StatementContainer, ) : JoinSelectStatement = - JoinSelectStatement(buildSQL(table, clause, isDistinct, deserializer), deserializer, connection, container, null, null) + JoinSelectStatement(buildSQL(From.of(table), clause, isDistinct, deserializer), deserializer, connection, container, parametersOf(table.fromParameters, clause.parameters), null, table.readTables + clause.tables) /** * Builds a SELECT statement with JOIN clause (requires ON or USING). @@ -133,7 +138,7 @@ internal object Select : Operation { * @return Incomplete JOIN statement requiring condition */ fun select( - table: Table<*>, + table: Relation<*>, clause: JoinClause, isDistinct: Boolean, deserializer: DeserializationStrategy, @@ -142,55 +147,74 @@ internal object Select : Operation { addSelectStatement: (SelectStatement) -> Unit ) : JoinStatementWithoutCondition = JoinStatementWithoutCondition( - buildSQL(table, clause, isDistinct, deserializer), + buildSQL(From.of(table), clause, isDistinct, deserializer), deserializer, connection, container, + parametersOf(table.fromParameters, clause.parameters), + table.readTables + clause.tables, addSelectStatement, ) /** - * Checks that [deserializer] can read rows of [table], when it reads them into a type other than the table's - * own row type. + * Checks that [deserializer] can read rows of [from], when it reads them into a type other than the row type of a + * single table. * * The columns a SELECT reads are the element names of [deserializer]'s descriptor, so a projection type has to - * fit the table. Every property must name a column, have that column's type, and be nullable when the column is, - * as a NULL read into a non-null property would quietly become `0` or `""`. A mismatch fails here, while the - * statement is built, rather than in SQLite or not at all. + * fit what the SELECT reads. Every property must name a column, of a single relation unless a join merges them, + * have that column's type, and be nullable when the column can be NULL, as it is nullable or an outer join can + * leave its relation without a matching row, as a NULL read into a non-null property would quietly become `0` or + * `""`. A mismatch fails here, while the statement is built, rather than in SQLite or not at all. * - * @throws IllegalArgumentException if the projection type doesn't fit the table + * @throws IllegalArgumentException if the projection type doesn't fit what the SELECT reads */ @OptIn(ExperimentalSerializationApi::class) - private fun checkProjection(table: Table<*>, deserializer: DeserializationStrategy<*>) { - val columns = table.kSerializer().descriptor + private fun checkProjection(from: From, deserializer: DeserializationStrategy<*>) { val projection = deserializer.descriptor - if (projection == columns) + if (!from.isJoin && projection == from.members.single().columns) return for (index in 0 ..< projection.elementsCount) - checkColumnProperty(table, columns, projection, index) + checkColumnProperty(from, projection, index) } /** - * Checks that property [index] of [projection] can be read from the column of [table] it names, as described by - * [checkProjection]. + * Checks that property [index] of [projection] can be read from the column of [from] it names, as described by + * [checkProjection], and returns that column. */ @OptIn(ExperimentalSerializationApi::class) - private fun checkColumnProperty(table: Table<*>, columns: SerialDescriptor, projection: SerialDescriptor, index: Int) { + private fun checkColumnProperty(from: From, projection: SerialDescriptor, index: Int): From.Column { val projectionName = projection.serialName val name = projection.getElementName(index) - val columnIndex = columns.getElementIndex(name) - require(columnIndex != CompositeDecoder.UNKNOWN_NAME) { - "Can't select '$projectionName' from table '${table.tableName}': its property '$name' isn't a column of that table." - } - val column = columns.getElementDescriptor(columnIndex) + val column = from.find(name, projectionName) val property = projection.getElementDescriptor(index) - val columnType = column.serialName.removeSuffix("?") + val columnType = column.descriptor.serialName.removeSuffix("?") val propertyType = property.serialName.removeSuffix("?") require(propertyType == columnType) { - "Can't select '$projectionName' from table '${table.tableName}': its property '$name' is a $propertyType, but the column holds a $columnType." + "Can't select '$projectionName' from ${from.description}: its property '$name' is a $propertyType, but the column holds a $columnType." } require(property.isNullable || !column.isNullable) { - "Can't select '$projectionName' from table '${table.tableName}': column '$name' is nullable, so property '$name' has to be nullable too." + if (column.descriptor.isNullable) + "Can't select '$projectionName' from ${from.description}: column '$name' is nullable, so property '$name' has to be nullable too." + else + "Can't select '$projectionName' from ${from.description}: column '$name' of '${column.relation.tableName}' is NULL where the outer join finds no matching row, so property '$name' has to be nullable." + } + return column + } + + /** + * Appends the columns that the properties of [projection] name, as [from] writes them: with the names of their + * relations in a join, which [checkProjection] has checked. + */ + @OptIn(ExperimentalSerializationApi::class) + private fun StringBuilder.appendColumns(from: From, projection: SerialDescriptor) { + if (!from.isJoin) { + appendDBColumnName(projection) + return + } + for (index in 0 ..< projection.elementsCount) { + if (index > 0) + append(',') + append(from.find(projection.getElementName(index), projection.serialName).sql) } } @@ -203,15 +227,16 @@ internal object Select : Operation { * * @return Statement that can be followed by WHERE, GROUP BY, ORDER BY, or LIMIT */ + @OptIn(ExperimentalSerializationApi::class) fun select( - table: Table<*>, + from: From, resultColumns: Iterable>, isDistinct: Boolean, deserializer: DeserializationStrategy, connection: DatabaseConnection, container: StatementContainer, ): ResultColumnSelectStatement { - val expressions = checkResultColumns(table, resultColumns, deserializer) + val expressions = checkResultColumns(from, resultColumns, deserializer) val projection = deserializer.descriptor val sql = buildString { append(sqlStr) @@ -221,16 +246,20 @@ internal object Select : Operation { if (index > 0) append(',') val name = projection.getElementName(index) - expressions[name]?.let { - append(it.valueName) + val expression = expressions[name] + if (expression != null) { + expression.appendSQL(this) append(" AS ") + append(name) + } else { + append(if (from.isJoin) from.find(name, projection.serialName).sql else name) } - append(name) } append(" FROM ") - append(table.tableName) + append(from.sql) } - return ResultColumnSelectStatement(sql, deserializer, connection, container, null, ungroupedError(table, expressions, deserializer)) + val tables = from.tables + expressions.values.flatMap { it.tables } + return ResultColumnSelectStatement(sql, deserializer, connection, container, parametersOf(from.parameters), ungroupedError(from, expressions, deserializer), tables) } /** @@ -238,47 +267,54 @@ internal object Select : Operation { * the properties they are selected into. * * Every result column has to name a property that is serialized under its own name, which is how the result is - * read, and no property can be given two expressions. A property given an expression has to be nullable when the - * expression can be NULL in a row, or in a group of GROUP BY. Every other property is read from its column, and - * is checked as [checkProjection] does. Whether a property has to be nullable because the query isn't grouped is - * left to [ungroupedError]. + * read, and no property can be given two expressions. An expression can only read the columns of what the SELECT + * reads, and a property given one has to be nullable when the expression can be NULL in a row, or in a group of + * GROUP BY, which an outer join makes any expression but `count` of a relation it can leave without a matching row. + * Every other property is read from its column, and is checked as [checkProjection] does. Whether a property has to + * be nullable because the query isn't grouped is left to [ungroupedError]. * * @throws IllegalArgumentException if the result columns don't fit the result type */ @OptIn(ExperimentalSerializationApi::class) private fun checkResultColumns( - table: Table<*>, + from: From, resultColumns: Iterable>, deserializer: DeserializationStrategy, ): Map> { val projection = deserializer.descriptor val projectionName = projection.serialName + val relations = from.members.mapTo(HashSet()) { it.relation.tableName } val expressions = HashMap>() for (resultColumn in resultColumns) { val name = resultColumn.propertyName val index = projection.getElementIndex(name) require(index != CompositeDecoder.UNKNOWN_NAME) { - "Can't select '$projectionName' from table '${table.tableName}': it doesn't serialize its property '$name' under that name. The property may be @Transient or computed, or renamed with @SerialName, which a property given an expression with AS can't be." + "Can't select '$projectionName' from ${from.description}: it doesn't serialize its property '$name' under that name. The property may be @Transient or computed, or renamed with @SerialName, which a property given an expression with AS can't be." } require(name !in expressions) { - "Can't select '$projectionName' from table '${table.tableName}': its property '$name' is given more than one expression." + "Can't select '$projectionName' from ${from.description}: its property '$name' is given more than one expression." } val element = resultColumn.element - require(element.table.tableName == table.tableName) { - "Can't select '$projectionName' from table '${table.tableName}': the expression '${element.valueName}' of property '$name' belongs to table '${element.table.tableName}'." + // The rows of a trigger, which only the WHEN and the body of a trigger can read + val otherTable = (element.columnTables - relations - NEW_ROW - OLD_ROW).firstOrNull() + require(otherTable == null) { + "Can't select '$projectionName' from ${from.description}: the expression '${element.valueName}' of property '$name' belongs to table '$otherTable'." + } + val isNullable = projection.getElementDescriptor(index).isNullable + require(!element.isNullable || isNullable) { + "Can't select '$projectionName' from ${from.description}: '${element.valueName}' can be NULL, so property '$name' has to be nullable." } - require(!element.isNullable || projection.getElementDescriptor(index).isNullable) { - "Can't select '$projectionName' from table '${table.tableName}': '${element.valueName}' can be NULL, so property '$name' has to be nullable." + require(isNullable || !element.isNullOnNoRows || element.columnTables.none { it in from.nullableTables }) { + "Can't select '$projectionName' from ${from.description}: '${element.valueName}' is NULL where the outer join finds no matching row, so property '$name' has to be nullable." } expressions[name] = element } require(expressions.isNotEmpty()) { - "Can't select '$projectionName' from table '${table.tableName}' with no result columns. To select columns only, use X<$projectionName>()." + "Can't select '$projectionName' from ${from.description} with no result columns. To select columns only, use X<$projectionName>()." } - val columns = table.kSerializer().descriptor for (index in 0 ..< projection.elementsCount) if (projection.getElementName(index) !in expressions) - checkColumnProperty(table, columns, projection, index) + checkColumnProperty(from, projection, index) return expressions } @@ -291,7 +327,7 @@ internal object Select : Operation { */ @OptIn(ExperimentalSerializationApi::class) private fun ungroupedError( - table: Table<*>, + from: From, expressions: Map>, deserializer: DeserializationStrategy<*>, ): String? { @@ -305,11 +341,11 @@ internal object Select : Operation { if (nonNullProperties.isEmpty()) return null val properties = nonNullProperties.joinToString { "'$it'" } - return "Can't select '${projection.serialName}' from table '${table.tableName}' without GROUP BY: an aggregate query that isn't grouped returns one row even when no rows match, in which $properties would be NULL. Make them nullable, or append GROUP BY." + return "Can't select '${projection.serialName}' from ${from.description} without GROUP BY: an aggregate query that isn't grouped returns one row even when no rows match, in which $properties would be NULL. Make them nullable, or append GROUP BY." } private fun buildSQL( - table: Table<*>, + from: From, clause: SelectClause, isDistinct: Boolean, deserializer: DeserializationStrategy, @@ -317,9 +353,9 @@ internal object Select : Operation { append(sqlStr) if (isDistinct) append("DISTINCT ") - appendDBColumnName(deserializer.descriptor) + appendColumns(from, deserializer.descriptor) append(" FROM ") - append(table.tableName) + append(from.sql) append(clause.clauseStr) } @@ -331,21 +367,76 @@ internal object Select : Operation { * @return Final SELECT statement ready for execution */ fun select( - table: Table<*>, + from: From, isDistinct: Boolean, deserializer: DeserializationStrategy, connection: DatabaseConnection, container: StatementContainer, ): FinalSelectStatement { - checkProjection(table, deserializer) + checkProjection(from, deserializer) val sql = buildString { append(sqlStr) if (isDistinct) append("DISTINCT ") - appendDBColumnName(deserializer.descriptor) + appendColumns(from, deserializer.descriptor) append(" FROM ") - append(table.tableName) + append(from.sql) + } + return FinalSelectStatement(sql, deserializer, connection, container, parametersOf(from.parameters), null, from.tables, isSimple = true) + } + + /** + * Builds a compound SELECT statement: [left] and [right] combined with [operator], which is ` UNION `, + * ` UNION ALL `, ` INTERSECT ` or ` EXCEPT `. + * + * SQLite evaluates a compound from left to right, and only allows ORDER BY and LIMIT after its last member, where + * they apply to the whole compound. So a member is written as a subquery, `SELECT * FROM (...)`, when it would + * otherwise mean something else: when it ends with ORDER BY or LIMIT, which then apply to it alone, and when it + * is itself a compound on the right, as in `a UNION (b UNION ALL c)`, which then is combined as a whole. A + * compound on the left needs no subquery, as SQLite evaluates it first anyway. + * + * @return Statement that can be followed by another compound operator, ORDER BY or LIMIT + */ + @OptIn(ExperimentalDSLDatabaseAPI::class) + fun compound( + left: SelectStatement, + operator: String, + right: SelectStatement, + container: StatementContainer, + ): CompoundSelectStatement { + val sql = buildString { + appendMember(left, isRight = false) + append(operator) + appendMember(right, isRight = true) + } + val parameters = ArrayList().apply { + left.parameters?.let { addAll(it) } + right.parameters?.let { addAll(it) } } - return FinalSelectStatement(sql, deserializer, connection, container, null, null) + return CompoundSelectStatement(sql, left.deserializer, left.connection, container, parameters.ifEmpty { null }, left.tables + right.tables) } -} \ No newline at end of file + + @OptIn(ExperimentalDSLDatabaseAPI::class) + private fun StringBuilder.appendMember(member: SelectStatement<*>, isRight: Boolean) { + val isSubquery = when (member) { + is CompoundSelectStatement -> isRight + is OrderBySelectStatement, is LimitSelectStatement -> true + is FinalSelectStatement -> !member.isSimple + else -> false + } + if (isSubquery) { + append("SELECT * FROM (") + append(member.sqlStr) + append(')') + } else { + append(member.sqlStr) + } + } +} + +/** + * Returns the parameters of a statement made of parts with [parameters], in their order: a new list, so that a list of a + * part, such as a derived table's, is never changed, or null for none. + */ +internal fun parametersOf(vararg parameters: List?): MutableList? = + parameters.flatMapTo(ArrayList()) { it.orEmpty() }.ifEmpty { null } diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Trigger.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Trigger.kt new file mode 100644 index 00000000..c47f223f --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Trigger.kt @@ -0,0 +1,69 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.operation + +import com.ctrip.sqllin.driver.DatabaseConnection +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.sql.TriggerDefinition +import com.ctrip.sqllin.dsl.sql.compiler.inlineParameters +import com.ctrip.sqllin.dsl.sql.statement.SingleStatement +import com.ctrip.sqllin.dsl.sql.statement.TableStructureStatement + +/** + * CREATE TRIGGER and DROP TRIGGER operations. + * + * SQLite doesn't let a trigger take parameters, so the values of its condition and of the statements of its body are + * written into its SQL as literals. + * + * @author Yuang Qiao + */ +@OptIn(ExperimentalDSLDatabaseAPI::class) +internal object Trigger : Operation { + + override val sqlStr: String + get() = "CREATE TRIGGER " + + fun create(definition: TriggerDefinition, body: List, connection: DatabaseConnection): SingleStatement { + val timing = definition.timing + val sql = buildString { + append(sqlStr) + if (timing.trigger.isIfNotExists) + append("IF NOT EXISTS ") + append(timing.trigger.name) + append(' ') + append(timing.timing) + append(' ') + append(timing.event.sql) + append(" ON ") + append(definition.table.tableName) + definition.condition?.let { + append(" WHEN ") + append(inlineParameters(it.conditionSQL, it.parameters)) + } + append(" BEGIN ") + for (statement in body) { + append(inlineParameters(statement.sqlStr, statement.parameters)) + append("; ") + } + append("END") + } + return TableStructureStatement(sql, connection) + } + + fun drop(name: String, isIfExists: Boolean, connection: DatabaseConnection): SingleStatement = + TableStructureStatement("DROP TRIGGER ${if (isIfExists) "IF EXISTS " else ""}$name", connection) +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Update.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Update.kt index 10844468..263cc22b 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Update.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Update.kt @@ -20,6 +20,7 @@ import com.ctrip.sqllin.driver.DatabaseConnection import com.ctrip.sqllin.dsl.sql.statement.StatementContainer import com.ctrip.sqllin.dsl.sql.Table import com.ctrip.sqllin.dsl.sql.clause.SetClause +import com.ctrip.sqllin.dsl.sql.clause.writtenColumns import com.ctrip.sqllin.dsl.sql.statement.UpdateStatementWithoutWhereClause /** @@ -53,12 +54,23 @@ internal object Update : Operation { container: StatementContainer, clause: SetClause, ): UpdateStatementWithoutWhereClause { + val assignments = clause.assignments val sql = buildString { append(sqlStr) append(table.tableName) append(" SET ") - append(clause.finalize()) + if (assignments == null) { + append(clause.finalize()) + } else { + writtenColumns(table, assignments, "UPDATE", setOf(table.tableName)).entries.forEachIndexed { index, (column, element) -> + if (index > 0) + append(',') + append(column) + append('=') + element.appendSQL(this) + } + } } - return UpdateStatementWithoutWhereClause(sql, container, connection, clause.parameters) + return UpdateStatementWithoutWhereClause(sql, container, connection, clause.parameters, table.tableName) } } \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Vacuum.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Vacuum.kt new file mode 100644 index 00000000..b10e1025 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/operation/Vacuum.kt @@ -0,0 +1,38 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.operation + +import com.ctrip.sqllin.driver.DatabaseConnection +import com.ctrip.sqllin.dsl.sql.compiler.sqlLiteral +import com.ctrip.sqllin.dsl.sql.statement.SingleStatement +import com.ctrip.sqllin.dsl.sql.statement.TableStructureStatement + +/** + * VACUUM operation, which rebuilds the database file, or writes a vacuumed copy of it to another file. + * + * @author Yuang Qiao + */ +internal object Vacuum : Operation { + + override val sqlStr: String + get() = "VACUUM" + + fun vacuum(connection: DatabaseConnection): SingleStatement = TableStructureStatement(sqlStr, connection) + + fun vacuumInto(file: String, connection: DatabaseConnection): SingleStatement = + TableStructureStatement("$sqlStr INTO ${sqlLiteral(file)}", connection) +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/Changes.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/Changes.kt new file mode 100644 index 00000000..3bb0e467 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/Changes.kt @@ -0,0 +1,53 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.statement + +/** + * What the statements of a scope may change when they run, from least to most. + * + * @author Yuang Qiao + */ +internal enum class Changes { + + /** Nothing: the statements only read. */ + NONE, + + /** The rows of tables: INSERT, UPDATE or DELETE. */ + ROWS, + + /** The tables themselves: CREATE, DROP, ALTER or PRAGMA, which may also drop or replace their triggers. */ + SCHEMA; + + companion object { + + fun of(statement: SingleStatement): Changes = when (statement) { + is SelectStatement<*>, is RaiseStatement -> NONE + is TableStructureStatement -> SCHEMA + is InsertStatement, is UpdateDeleteStatement, is UpdateStatementWithoutWhereClause<*> -> ROWS + } + + /** + * The table [statement] writes rows to, or null if it writes none. + */ + fun writtenTable(statement: SingleStatement): String? = when (statement) { + is InsertStatement -> statement.table + is UpdateDeleteStatement -> statement.table + is UpdateStatementWithoutWhereClause<*> -> statement.table + is SelectStatement<*>, is TableStructureStatement, is RaiseStatement -> null + } + } +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/DatabaseExecuteEngine.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/DatabaseExecuteEngine.kt index 995a35f6..373b51ef 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/DatabaseExecuteEngine.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/DatabaseExecuteEngine.kt @@ -42,10 +42,32 @@ internal class DatabaseExecuteEngine( throw IllegalStateException("Current statement can't append clause.") } + override fun replaceStatement(statement: SingleStatement, newStatement: SingleStatement) { + val index = statementList.indexOf(statement) + check(index >= 0) { "The statement to replace isn't one of the scope's." } + statementList[index] = newStatement + } + infix fun addStatement(statement: ExecutableStatement) { statementList.add(statement) } + /** + * What the statements may change when they run, so that the queries observing the tables can be refreshed. + */ + val changes: Changes + get() = statementList.fold(Changes.NONE) { changes, statement -> + maxOf(changes, if (statement is TransactionStatementsGroup) statement.changes else Changes.of(statement as SingleStatement)) + } + + /** + * The tables the statements write rows to, as INSERT, UPDATE and DELETE name them. + */ + val writtenTables: Set + get() = statementList.flatMapTo(LinkedHashSet()) { + if (it is TransactionStatementsGroup) it.writtenTables else Changes.writtenTable(it as SingleStatement)?.let(::setOf).orEmpty() + } + override infix fun removeStatement(statement: SingleStatement) { statementList.remove(statement) } diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/IndexStatement.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/IndexStatement.kt new file mode 100644 index 00000000..7bddf173 --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/IndexStatement.kt @@ -0,0 +1,59 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.statement + +import com.ctrip.sqllin.driver.DatabaseConnection +import com.ctrip.sqllin.dsl.sql.Table +import com.ctrip.sqllin.dsl.sql.clause.SelectCondition +import com.ctrip.sqllin.dsl.sql.compiler.inlineParameters +import com.ctrip.sqllin.dsl.sql.operation.Create + +/** + * A CREATE INDEX statement, which `WHERE` makes a partial index: an index of the rows that satisfy a condition. + * + * @author Yuang Qiao + */ +public class IndexStatement internal constructor( + private val table: Table<*>, + private val indexName: String, + private val connection: DatabaseConnection, + private val container: StatementContainer, + private val statement: SingleStatement, +) { + + private var hasWhere = false + + /** + * Makes the index a partial index of the rows that satisfy [condition]: `CREATE INDEX ... WHERE condition`. + * + * SQLite allows neither parameters nor subqueries there, and a partial index can only read its table, so the + * values of [condition] are written into the SQL as literals, and its columns without the table's name. + */ + internal infix fun where(condition: SelectCondition) { + check(!hasWhere) { "The index '$indexName' already has a WHERE." } + require(condition.tables.isEmpty()) { + "The WHERE of index '$indexName' can't have a subquery, which SQLite doesn't allow in a partial index." + } + // SQLite rejects it too, but only once the table has rows + require(condition.isDeterministic) { + "The WHERE of index '$indexName' can't have a function that doesn't always give the same result for a row." + } + val where = inlineParameters(Create.unqualified(condition.conditionSQL, table.tableName), condition.parameters) + container.replaceStatement(statement, TableStructureStatement("${statement.sqlStr} WHERE $where", connection)) + hasWhere = true + } +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/JoinStatementWithoutCondition.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/JoinStatementWithoutCondition.kt index b137d8b5..9c8ffd0e 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/JoinStatementWithoutCondition.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/JoinStatementWithoutCondition.kt @@ -19,6 +19,7 @@ package com.ctrip.sqllin.dsl.sql.statement import com.ctrip.sqllin.driver.DatabaseConnection import com.ctrip.sqllin.dsl.sql.clause.ClauseElement import com.ctrip.sqllin.dsl.sql.clause.SelectCondition +import com.ctrip.sqllin.dsl.sql.operation.parametersOf import kotlinx.serialization.DeserializationStrategy /** @@ -41,6 +42,8 @@ public class JoinStatementWithoutCondition internal constructor( private val deserializer: DeserializationStrategy, private val connection: DatabaseConnection, private val container: StatementContainer, + private val parameters: MutableList?, + private val tables: Set, private val addSelectStatement: (SelectStatement) -> Unit ) { /** @@ -65,7 +68,7 @@ public class JoinStatementWithoutCondition internal constructor( } append(')') } - val joinStatement = JoinSelectStatement(sql, deserializer, connection, container, null, null) + val joinStatement = JoinSelectStatement(sql, deserializer, connection, container, parameters, null, tables) addSelectStatement(joinStatement) return joinStatement } @@ -85,7 +88,7 @@ public class JoinStatementWithoutCondition internal constructor( append(" ON ") append(condition.conditionSQL) } - val joinStatement = JoinSelectStatement(sql, deserializer, connection, container, condition.parameters, null) + val joinStatement = JoinSelectStatement(sql, deserializer, connection, container, parametersOf(parameters, condition.parameters), null, tables + condition.tables) addSelectStatement(joinStatement) return joinStatement } diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/OtherStatement.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/OtherStatement.kt index 4822c357..b671abd2 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/OtherStatement.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/OtherStatement.kt @@ -40,6 +40,7 @@ public class UpdateStatementWithoutWhereClause internal constructor( internal val statementContainer: StatementContainer, internal val connection: DatabaseConnection, override val parameters: MutableList?, + internal val table: String, ) : SingleStatement(preSQLStr) { public override fun execute(): Unit = connection.executeUpdateDelete(sqlStr, params) } @@ -56,6 +57,7 @@ public class UpdateDeleteStatement internal constructor( sqlStr: String, private val connection: DatabaseConnection, override val parameters: MutableList?, + internal val table: String, ) : SingleStatement(sqlStr) { public override fun execute(): Unit = connection.executeUpdateDelete(sqlStr, params) } @@ -72,6 +74,7 @@ public class InsertStatement internal constructor( sqlStr: String, private val connection: DatabaseConnection, override val parameters: MutableList?, + internal val table: String, ) : SingleStatement(sqlStr) { public override fun execute(): Unit = connection.executeInsert(sqlStr, params) } diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/SelectStatement.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/SelectStatement.kt index 0ecbeea0..ee129c99 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/SelectStatement.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/SelectStatement.kt @@ -18,8 +18,11 @@ package com.ctrip.sqllin.dsl.sql.statement import com.ctrip.sqllin.driver.CommonCursor import com.ctrip.sqllin.driver.DatabaseConnection +import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI +import com.ctrip.sqllin.dsl.sql.clause.elementTables import com.ctrip.sqllin.dsl.sql.clause.* import com.ctrip.sqllin.dsl.sql.compiler.QueryDecoder +import com.ctrip.sqllin.dsl.sql.operation.parametersOf import kotlinx.serialization.DeserializationStrategy import kotlin.concurrent.Volatile @@ -42,6 +45,7 @@ import kotlin.concurrent.Volatile * aggregate query whose result type has non-null properties that can only be relied on in a group: without GROUP BY, * an aggregate query returns one row even when no rows match, in which they are NULL. GROUP BY clears it, and * [checkComplete] reports it. + * @property tables The names of the tables the statement reads, which an observed query watches for changes * * @author Yuang Qiao */ @@ -52,6 +56,7 @@ public sealed class SelectStatement( internal val container: StatementContainer, final override val parameters: MutableList?, internal val ungroupedError: String?, + internal val tables: Set, ) : SingleStatement(sqlStr) { @Volatile @@ -115,16 +120,17 @@ public class WhereSelectStatement internal constructor( container: StatementContainer, parameters: MutableList?, ungroupedError: String?, -) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError) { + tables: Set, +) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError, tables) { internal infix fun appendToLimit(clause: LimitClause): LimitSelectStatement = - LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables) internal infix fun appendToOrderBy(clause: OrderByClause): OrderBySelectStatement = - OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables + clause.elementTables) internal infix fun appendToGroupBy(clause: GroupByClause): GroupBySelectStatement = - GroupBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, null) + GroupBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, null, tables + clause.elementTables) } /** @@ -145,19 +151,20 @@ public class ResultColumnSelectStatement internal constructor( container: StatementContainer, parameters: MutableList?, ungroupedError: String?, -) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError) { + tables: Set, +) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError, tables) { internal infix fun appendToWhere(clause: WhereClause): WhereSelectStatement = - WhereSelectStatement(buildSQL(clause), deserializer, connection, container, clause.selectCondition.parameters, ungroupedError) + WhereSelectStatement(buildSQL(clause), deserializer, connection, container, parametersOf(parameters, clause.selectCondition.parameters), ungroupedError, tables + clause.selectCondition.tables) internal infix fun appendToLimit(clause: LimitClause): LimitSelectStatement = - LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables) internal infix fun appendToOrderBy(clause: OrderByClause): OrderBySelectStatement = - OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables + clause.elementTables) internal infix fun appendToGroupBy(clause: GroupByClause): GroupBySelectStatement = - GroupBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, null) + GroupBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, null, tables + clause.elementTables) } /** @@ -178,26 +185,20 @@ public class JoinSelectStatement internal constructor( container: StatementContainer, parameters: MutableList?, ungroupedError: String?, -) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError) { + tables: Set, +) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError, tables) { - internal infix fun appendToWhere(clause: WhereClause): WhereSelectStatement { - val clauseParams = clause.selectCondition.parameters - val params = parameters?.also { - clauseParams?.let { p -> - it.addAll(p) - } - } ?: clauseParams - return WhereSelectStatement(buildSQL(clause), deserializer, connection, container, params, ungroupedError) - } + internal infix fun appendToWhere(clause: WhereClause): WhereSelectStatement = + WhereSelectStatement(buildSQL(clause), deserializer, connection, container, parametersOf(parameters, clause.selectCondition.parameters), ungroupedError, tables + clause.selectCondition.tables) internal infix fun appendToLimit(clause: LimitClause): LimitSelectStatement = - LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables) internal infix fun appendToOrderBy(clause: OrderByClause): OrderBySelectStatement = - OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables + clause.elementTables) internal infix fun appendToGroupBy(clause: GroupByClause): GroupBySelectStatement = - GroupBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, null) + GroupBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, null, tables + clause.elementTables) } /** @@ -216,20 +217,14 @@ public class GroupBySelectStatement internal constructor( container: StatementContainer, parameters: MutableList?, ungroupedError: String?, -) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError) { + tables: Set, +) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError, tables) { internal infix fun appendToOrderBy(clause: OrderByClause): OrderBySelectStatement = - OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables + clause.elementTables) - internal infix fun appendToHaving(clause: HavingClause): HavingSelectStatement { - val clauseParams = clause.selectCondition.parameters - val params = parameters?.also { - clauseParams?.let { p -> - it.addAll(p) - } - } ?: clauseParams - return HavingSelectStatement(buildSQL(clause), deserializer, connection, container, params, ungroupedError) - } + internal infix fun appendToHaving(clause: HavingClause): HavingSelectStatement = + HavingSelectStatement(buildSQL(clause), deserializer, connection, container, parametersOf(parameters, clause.selectCondition.parameters), ungroupedError, tables + clause.selectCondition.tables) } /** @@ -248,13 +243,14 @@ public class HavingSelectStatement internal constructor( container: StatementContainer, parameters: MutableList?, ungroupedError: String?, -) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError) { + tables: Set, +) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError, tables) { internal infix fun appendToOrderBy(clause: OrderByClause): OrderBySelectStatement = - OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables + clause.elementTables) internal infix fun appendToLimit(clause: LimitClause): LimitSelectStatement = - LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables) } /** @@ -272,10 +268,11 @@ public class OrderBySelectStatement internal constructor( container: StatementContainer, parameters: MutableList?, ungroupedError: String?, -) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError) { + tables: Set, +) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError, tables) { internal infix fun appendToLimit(clause: LimitClause): LimitSelectStatement = - LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables) } /** @@ -293,10 +290,11 @@ public class LimitSelectStatement internal constructor( container: StatementContainer, parameters: MutableList?, ungroupedError: String?, -) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError) { + tables: Set, +) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError, tables) { internal infix fun appendToFinal(clause: OffsetClause): FinalSelectStatement = - FinalSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError) + FinalSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, ungroupedError, tables, isSimple = false) } /** @@ -305,6 +303,9 @@ public class LimitSelectStatement internal constructor( * This is the terminal state in the SELECT statement hierarchy - no further clauses can be added. * The statement is ready for execution. * + * @property isSimple Whether the statement is a single SELECT without LIMIT, as `SELECT X` builds, rather than one + * with OFFSET, or a compound built by the deprecated `UNION {}` block + * * @author Yuang Qiao */ public class FinalSelectStatement internal constructor( @@ -314,4 +315,36 @@ public class FinalSelectStatement internal constructor( container: StatementContainer, parameters: MutableList?, ungroupedError: String?, -) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError) \ No newline at end of file + tables: Set, + internal val isSimple: Boolean, +) : SelectStatement(sqlStr, deserializer, connection, container, parameters, ungroupedError, tables) + +/** + * Compound SELECT statement: SELECTs combined with UNION, UNION ALL, INTERSECT or EXCEPT, as + * `(select1) UNION (select2)` builds. + * + * Can be followed by: + * - UNION, UNION ALL, INTERSECT or EXCEPT, with another SELECT + * - ORDER BY + * - LIMIT + * + * ORDER BY and LIMIT apply to the whole compound, as in SQL. + * + * @author Yuang Qiao + */ +@ExperimentalDSLDatabaseAPI +public class CompoundSelectStatement internal constructor( + sqlStr: String, + deserializer: DeserializationStrategy, + connection: DatabaseConnection, + container: StatementContainer, + parameters: MutableList?, + tables: Set, +) : SelectStatement(sqlStr, deserializer, connection, container, parameters, null, tables) { + + internal infix fun appendToOrderBy(clause: OrderByClause): OrderBySelectStatement = + OrderBySelectStatement(buildSQL(clause), deserializer, connection, container, parameters, null, tables + clause.elementTables) + + internal infix fun appendToLimit(clause: LimitClause): LimitSelectStatement = + LimitSelectStatement(buildSQL(clause), deserializer, connection, container, parameters, null, tables) +} \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/StatementContainer.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/StatementContainer.kt index 4765ae03..adc7a500 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/StatementContainer.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/StatementContainer.kt @@ -44,4 +44,9 @@ internal interface StatementContainer { * Removes [statement] if this container holds it, wherever it is. */ infix fun removeStatement(statement: SingleStatement) + + /** + * Replaces [statement] with [newStatement], where [statement] is, as when a clause completes it. + */ + fun replaceStatement(statement: SingleStatement, newStatement: SingleStatement) } \ No newline at end of file diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/TransactionStatementsGroup.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/TransactionStatementsGroup.kt index 068b4826..f790db92 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/TransactionStatementsGroup.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/TransactionStatementsGroup.kt @@ -42,10 +42,32 @@ internal class TransactionStatementsGroup( statementList.add(statement) } + /** + * What the statements of the transaction may change when they run. + * + * @see DatabaseExecuteEngine.changes + */ + val changes: Changes + get() = statementList.fold(Changes.NONE) { changes, statement -> maxOf(changes, Changes.of(statement)) } + + /** + * The tables the statements of the transaction write rows to. + * + * @see DatabaseExecuteEngine.writtenTables + */ + val writtenTables: Set + get() = statementList.mapNotNullTo(LinkedHashSet()) { Changes.writtenTable(it) } + override infix fun removeStatement(statement: SingleStatement) { statementList.remove(statement) } + override fun replaceStatement(statement: SingleStatement, newStatement: SingleStatement) { + val index = statementList.indexOf(statement) + check(index >= 0) { "The statement to replace isn't one of the transaction's." } + statementList[index] = newStatement + } + /** * Checks that every statement of the transaction is complete, before any of them runs. * diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/TriggerBody.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/TriggerBody.kt new file mode 100644 index 00000000..c628f9bd --- /dev/null +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/TriggerBody.kt @@ -0,0 +1,58 @@ +/* + * Copyright (C) 2026 Ctrip.com. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.ctrip.sqllin.dsl.sql.statement + +/** + * Collects the statements of the body of a trigger, which run when the trigger fires rather than when the scope ends: + * they become the SQL of `CREATE TRIGGER`. + * + * @author Yuang Qiao + */ +internal class TriggerBody : StatementContainer { + + val statements = ArrayDeque() + + infix fun addStatement(statement: SingleStatement) { + statements.add(statement) + } + + override infix fun changeLastStatement(statement: SingleStatement) { + if (statements.lastOrNull() is UpdateStatementWithoutWhereClause<*> || statements.lastOrNull() is SelectStatement<*>) { + statements.removeLast() + statements.add(statement) + } else + throw IllegalStateException("Current statement can't append clause.") + } + + override infix fun removeStatement(statement: SingleStatement) { + statements.remove(statement) + } + + override fun replaceStatement(statement: SingleStatement, newStatement: SingleStatement) { + val index = statements.indexOf(statement) + check(index >= 0) { "The statement to replace isn't one of the trigger's." } + statements[index] = newStatement + } +} + +/** + * `SELECT RAISE(...)`, which only a trigger runs: it stops the statement that fired the trigger, or skips its row. + */ +internal class RaiseStatement(sqlStr: String) : SingleStatement(sqlStr) { + override val parameters: MutableList? = null + override fun execute(): Unit = error("RAISE only runs in a trigger.") +} diff --git a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/UnionSelectStatementGroup.kt b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/UnionSelectStatementGroup.kt index a256a491..606dbd87 100644 --- a/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/UnionSelectStatementGroup.kt +++ b/sqllin-dsl/src/commonMain/kotlin/com/ctrip/sqllin/dsl/sql/statement/UnionSelectStatementGroup.kt @@ -39,6 +39,9 @@ internal class UnionSelectStatementGroup : StatementContainer { statementList.remove(statement) } + override fun replaceStatement(statement: SingleStatement, newStatement: SingleStatement): Unit = + error("A UNION block only holds SELECT statements, which aren't replaced.") + /** * Combines all accumulated SELECT statements into a single UNION query. * @@ -75,6 +78,8 @@ internal class UnionSelectStatementGroup : StatementContainer { container = container, parameters, ungroupedError = null, + tables = statementList.flatMapTo(LinkedHashSet()) { it.tables }, + isSimple = false, ) } } diff --git a/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ClauseProcessor.kt b/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ClauseProcessor.kt index c51bf1b5..87a5a684 100644 --- a/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ClauseProcessor.kt +++ b/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ClauseProcessor.kt @@ -44,12 +44,17 @@ import java.io.OutputStreamWriter * - NOT NULL constraints * - UNIQUE constraints (single and composite) * - COLLATE NOCASE for case-insensitive text columns + * - CHECK constraints of columns and of the table, optionally named * - **Type support**: * - All Kotlin primitive types and unsigned variants * - String, Char, Boolean, ByteArray * - Enum classes (stored as integers) * - Typealiases of supported types * + * A `@DBRow` class annotated with [@Fts4][com.ctrip.sqllin.dsl.annotation.Fts4] or + * [@Fts3][com.ctrip.sqllin.dsl.annotation.Fts3] gets an `FtsTable` object instead, whose statement is a + * `CREATE VIRTUAL TABLE`, and a class annotated with [@DBView][com.ctrip.sqllin.dsl.annotation.DBView] a `View` object. + * * ### Performance Optimization * CREATE TABLE statements are generated at **compile-time** rather than runtime, * eliminating the overhead of runtime reflection and string building during table creation. @@ -61,6 +66,7 @@ import java.io.OutputStreamWriter * @see com.ctrip.sqllin.dsl.annotation.Unique * @see com.ctrip.sqllin.dsl.annotation.CompositeUnique * @see com.ctrip.sqllin.dsl.annotation.CollateNoCase + * @see com.ctrip.sqllin.dsl.annotation.Check */ class ClauseProcessor( private val environment: SymbolProcessorEnvironment, @@ -71,8 +77,31 @@ class ClauseProcessor( */ private companion object { const val ANNOTATION_DATABASE_ROW_NAME = "com.ctrip.sqllin.dsl.annotation.DBRow" + const val ANNOTATION_DATABASE_VIEW_NAME = "com.ctrip.sqllin.dsl.annotation.DBView" + const val ANNOTATION_PACKAGE = "com.ctrip.sqllin.dsl.annotation" + const val ANNOTATION_FTS4 = "$ANNOTATION_PACKAGE.Fts4" + const val ANNOTATION_FTS3 = "$ANNOTATION_PACKAGE.Fts3" + const val ANNOTATION_PRIMARY_KEY = "$ANNOTATION_PACKAGE.PrimaryKey" + const val ANNOTATION_CHECK = "$ANNOTATION_PACKAGE.Check" + val SQL_IDENTIFIER = Regex("[A-Za-z_][A-Za-z0-9_]*") + + /** The names SQLite gives the rowid; an FTS table also calls it docid. */ + val ROW_ID_NAMES = setOf("rowid", "oid", "_rowid_", "docid") + + /** The annotations that declare constraints, which only a table's columns can have. */ + val CONSTRAINT_ANNOTATIONS = listOf( + "PrimaryKey", "CompositePrimaryKey", "CollateNoCase", "Unique", "CompositeUnique", + "ForeignKeyGroup", "References", "ForeignKey", "Default", "Check", + ).map { "$ANNOTATION_PACKAGE.$it" }.toSet() const val ANNOTATION_SERIALIZABLE = "kotlinx.serialization.Serializable" const val ANNOTATION_TRANSIENT = "kotlinx.serialization.Transient" + + /** + * The getter of a column's property in `SET {}` only returns a placeholder, as `0` or `""`, so reading it, as + * in `age = age + 1`, would set the column to a value computed from the placeholder. It is an error to read it. + */ + const val SET_CLAUSE_GETTER_DEPRECATION = + " @Deprecated(\"A column in SET {} can only be assigned: reading it gives a placeholder, not the column's value. To set a column to an expression of columns, use SET(listOf(expression AS Row::property)).\", level = DeprecationLevel.ERROR)\n" } /** @@ -84,9 +113,9 @@ class ClauseProcessor( @Suppress("UNCHECKED_CAST") override fun process(resolver: Resolver): List { val allDBRowClasses = resolver.getSymbolsWithAnnotation(ANNOTATION_DATABASE_ROW_NAME) - val invalidateDBRowClasses = allDBRowClasses.filter { !it.validate() }.toList() + val invalidateDBRowClasses = allDBRowClasses.filter { !it.validate(enableNewFeatures = false) }.toList() - val validateDBRowClasses = allDBRowClasses.filter { it.validate() } as Sequence + val validateDBRowClasses = allDBRowClasses.filter { it.validate(enableNewFeatures = false) } as Sequence val serializableType = resolver.getClassDeclarationByName(ANNOTATION_SERIALIZABLE)!!.asStarProjectedType() for (classDeclaration in validateDBRowClasses) { @@ -142,6 +171,15 @@ class ClauseProcessor( if (unsupportedProperties.isNotEmpty()) continue + val ftsAnnotations = classDeclaration.annotations.filter { it.qualifiedName == ANNOTATION_FTS4 || it.qualifiedName == ANNOTATION_FTS3 }.toList() + if (ftsAnnotations.isNotEmpty()) { + processFtsTable(classDeclaration, ftsAnnotations, propertyList, tableName as String, visibilityModifier) + continue + } + + if (!validateChecks(classDeclaration, propertyList)) + continue + val outputStream = environment.codeGenerator.createNewFile( dependencies = classDeclaration.containingFile?.let { Dependencies(true, it) } ?: Dependencies(true), packageName = packageName, @@ -191,6 +229,9 @@ class ClauseProcessor( columnConstraintParser.parseProperty(this, property, propertyName, isNotNull) + // Before @References, whose constraint name SQLite would give the CHECK constraints after it + appendColumnChecks(this, property) + // Handle @Reference and @ForeignKey foreignKeyParser.parseColumnAnnotations(createSQLBuilder, property.annotations, propertyName, isNotNull) @@ -205,22 +246,381 @@ class ClauseProcessor( writer.write(" @ColumnNameDslMaker\n") writer.write(" var SetClause<$className>.$propertyName: ${property.typeName}") writer.write(if (isNotNull) "\n" else "?\n") + writer.write(SET_CLAUSE_GETTER_DEPRECATION) writer.write(" get() = ${getSetClauseGetterValue(property)}\n") writer.write(" set(value) = ${appendFunction(elementName, property, isNotNull)}\n\n") } columnConstraintParser.generateCodeForPrimaryKey(writer, createSQLBuilder) foreignKeyParser.generateCodeForForeignKey(createSQLBuilder) + for (check in classDeclaration.checkConstraints()) { + createSQLBuilder.append(',') + check.appendTo(createSQLBuilder) + } createSQLBuilder.append(')') - writer.write(" override val createSQL = \"$createSQLBuilder\"\n") + // A CHECK expression or a DEFAULT value can hold characters that a Kotlin string literal escapes + writer.write(" override val createSQL = \"${kotlinStringContent(createSQLBuilder.toString())}\"\n") writer.write("}\n") } } - return invalidateDBRowClasses + // @Fts4 and @Fts3 only describe the table of a @DBRow class, without which nothing would be generated for them + for (ftsAnnotationName in listOf(ANNOTATION_FTS4, ANNOTATION_FTS3)) { + resolver.getSymbolsWithAnnotation(ftsAnnotationName) + .filter { symbol -> symbol.annotations.none { it.qualifiedName == ANNOTATION_DATABASE_ROW_NAME } } + .forEach { + val annotationName = ftsAnnotationName.substringAfterLast('.') + environment.logger.error("The class annotated with '@$annotationName' must also be annotated with '@DBRow', which makes it a table.", it) + } + } + return invalidateDBRowClasses + processViews(resolver) } + /** + * Generates the table object of a `@DBRow` class annotated with [@Fts4][com.ctrip.sqllin.dsl.annotation.Fts4] or + * [@Fts3][com.ctrip.sqllin.dsl.annotation.Fts3]: an `FtsTable`, created with `CREATE VIRTUAL TABLE`. + * + * Its columns are the class's `String` properties, as FTS indexes text, without types or constraints, which an FTS + * table doesn't have. A `@PrimaryKey` property of type `Long` named `rowid` or `docid` isn't a column but the row's + * id, which `Long?` leaves for SQLite to assign, as for a table's `INTEGER PRIMARY KEY`. + */ + private fun processFtsTable( + classDeclaration: KSClassDeclaration, + ftsAnnotations: List, + propertyList: List, + tableName: String, + visibilityModifier: String, + ) { + val className = classDeclaration.simpleName.asString() + var isValid = true + fun error(message: String, symbol: KSNode) { + environment.logger.error(message, symbol) + isValid = false + } + + if (ftsAnnotations.size > 1) + error("The class '$className' is annotated with both '@Fts3' and '@Fts4'; its FTS table can only be one of them.", classDeclaration) + val ftsAnnotation = ftsAnnotations.first() + val isFts4 = ftsAnnotation.qualifiedName == ANNOTATION_FTS4 + classDeclaration.annotations.filter { it.qualifiedName in CONSTRAINT_ANNOTATIONS }.forEach { + error("The FTS table '$className' can't be annotated with '@${it.shortName.asString()}': an FTS table has no constraints.", classDeclaration) + } + + var rowIdProperty: KSPropertyDeclaration? = null + val columns = ArrayList() + for (property in propertyList) { + val propertyName = property.simpleName.asString() + val constraints = property.annotations.filter { it.qualifiedName in CONSTRAINT_ANNOTATIONS }.toList() + val primaryKey = constraints.find { it.qualifiedName == ANNOTATION_PRIMARY_KEY } + if (propertyName.lowercase() in ROW_ID_NAMES) { + val rules = "The rowid of an FTS table is read and written through a @PrimaryKey property of type Long or Long?, named 'rowid' or 'docid'" + when { + propertyName != "rowid" && propertyName != "docid" -> + error("The property '$propertyName' of FTS table '$className' is named as the rowid. $rules.", property) + primaryKey == null -> + error("The property '$propertyName' of FTS table '$className' has to be annotated with '@PrimaryKey'. $rules.", property) + resolvedTypeName(property) != FullNameCache.LONG -> + error("The property '$propertyName' of FTS table '$className' has the type '${property.type.resolve()}'. $rules.", property) + primaryKey.arguments.any { it.name?.asString() == "autoIncrement" && it.value == true } -> + error("The rowid of FTS table '$className' can't be AUTOINCREMENT, which only a table's INTEGER PRIMARY KEY can be.", property) + rowIdProperty != null -> + error("The FTS table '$className' has two rowid properties, '${rowIdProperty.simpleName.asString()}' and '$propertyName'.", property) + } + rowIdProperty = property + (constraints - primaryKey).filterNotNull().forEach { + error("The rowid '$propertyName' of FTS table '$className' can't be annotated with '@${it.shortName.asString()}': an FTS table has no constraints.", property) + } + continue + } + if (primaryKey != null) + error("The property '$propertyName' of FTS table '$className' can't be its primary key, which is its rowid. To read and write the rowid, give the class a @PrimaryKey property of type Long or Long? named 'rowid' or 'docid'.", property) + (constraints - primaryKey).filterNotNull().forEach { + error("The property '$propertyName' of FTS table '$className' can't be annotated with '@${it.shortName.asString()}': an FTS table's columns have no constraints.", property) + } + if (resolvedTypeName(property) != FullNameCache.STRING) + error("The property '$propertyName' of FTS table '$className' has the type '${property.type.resolve()}', but FTS indexes text, so its columns are String or String?.", property) + columns.add(propertyName) + } + if (columns.isEmpty()) + error("The FTS table '$className' has no columns. Its columns are its String or String? properties.", classDeclaration) + + val arguments = ftsAnnotation.arguments.associate { it.name?.asString() to it.value } + val options = ArrayList() + val tokenizer = when (val value = arguments["tokenizer"]) { + is KSClassDeclaration -> value.simpleName.asString() + is KSType -> value.declaration.simpleName.asString() + null -> "SIMPLE" + else -> value.toString().substringAfterLast('.') + }.lowercase() + val tokenizerArgs = (arguments["tokenizerArgs"] as? List<*>).orEmpty().map { it.toString() } + if (tokenizer != "simple" || tokenizerArgs.isNotEmpty()) + options.add((listOf(tokenizer) + tokenizerArgs.map { "\"${it.replace("\"", "\"\"")}\"" }).joinToString(" ", "tokenize=")) + if (isFts4) { + val prefix = (arguments["prefix"] as? List<*>).orEmpty().map { it as Int } + if (prefix.any { it <= 0 }) + error("The prefix lengths of FTS table '$className' have to be positive, but they are ${prefix.joinToString()}.", classDeclaration) + if (prefix.isNotEmpty()) + options.add("prefix=\"${prefix.joinToString(",")}\"") + for (column in (arguments["notIndexed"] as? List<*>).orEmpty().map { it.toString() }) { + if (column !in columns) + error("The column '$column' that FTS table '$className' doesn't index isn't one of its columns, ${columns.joinToString { "'$it'" }}.", classDeclaration) + options.add("notindexed=$column") + } + } + if (!isValid) + return + + val createSQL = (columns + options).joinToString(",", "CREATE VIRTUAL TABLE $tableName USING ${if (isFts4) "fts4" else "fts3"}(", ")") + val packageName = classDeclaration.packageName.asString() + val objectName = "${className}Table" + val outputStream = environment.codeGenerator.createNewFile( + dependencies = classDeclaration.containingFile?.let { Dependencies(true, it) } ?: Dependencies(true), + packageName = packageName, + fileName = objectName, + ) + OutputStreamWriter(outputStream).use { writer -> + writer.write("package $packageName\n\n") + writer.write("import com.ctrip.sqllin.dsl.annotation.ColumnNameDslMaker\n") + writer.write("import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI\n") + writer.write("import com.ctrip.sqllin.dsl.sql.clause.ClauseNumber\n") + writer.write("import com.ctrip.sqllin.dsl.sql.clause.ClauseString\n") + writer.write("import com.ctrip.sqllin.dsl.sql.clause.SetClause\n") + writer.write("import com.ctrip.sqllin.dsl.sql.FtsTable\n") + writer.write("import com.ctrip.sqllin.dsl.sql.PrimaryKeyInfo\n\n") + // As for table objects, the column properties carry @ColumnNameDslMaker for IntelliJ IDEA's DSL + // highlighting. FtsTable is experimental, and the table object opts in so that its users don't have to. + writer.write("@Suppress(\"DSL_MARKER_APPLIED_TO_WRONG_TARGET\")\n") + writer.write("@OptIn(ExperimentalDSLDatabaseAPI::class)\n") + writer.write("${visibilityModifier}object $objectName : FtsTable<$className>(\"$tableName\") {\n\n") + writer.write(" override fun kSerializer() = $className.serializer()\n\n") + writer.write(" inline operator fun invoke(block: $objectName.(table: $objectName) -> R): R = this.block(this)\n\n") + propertyList.forEachIndexed { index, property -> + val propertyName = property.simpleName.asString() + val elementName = "$className.serializer().descriptor.getElementName($index)" + val isNotNull = property.type.resolve().nullability == Nullability.NOT_NULL + writer.write(" @ColumnNameDslMaker\n") + writer.write(" val $propertyName\n") + writer.write(" get() = ${checkNotNull(getClauseElementTypeStr(property))}($elementName, this, ${!isNotNull})\n\n") + writer.write(" @ColumnNameDslMaker\n") + writer.write(" var SetClause<$className>.$propertyName: ${property.typeName}${if (isNotNull) "" else "?"}\n") + writer.write(SET_CLAUSE_GETTER_DEPRECATION) + writer.write(" get() = ${getSetClauseGetterValue(property)}\n") + writer.write(" set(value) = ${appendFunction(elementName, property, isNotNull)}\n\n") + } + val rowId = rowIdProperty + if (rowId == null) { + writer.write(" override val primaryKeyInfo = null\n\n") + } else { + writer.write(" override val primaryKeyInfo = PrimaryKeyInfo(\n") + writer.write(" primaryKeyName = \"${rowId.simpleName.asString()}\",\n") + writer.write(" isAutomaticIncrement = false,\n") + writer.write(" isGeneratedByDatabase = ${rowId.type.resolve().nullability != Nullability.NOT_NULL},\n") + writer.write(" compositePrimaryKeys = null,\n") + writer.write(" )\n\n") + } + writer.write(" override val createSQL = \"${kotlinStringContent(createSQL)}\"\n") + writer.write("}\n") + } + } + + /** + * A `CHECK` constraint, read from a [@Check][com.ctrip.sqllin.dsl.annotation.Check] annotation. + */ + private class CheckConstraint(val expression: String, val constraintName: String) { + + /** + * Appends `CHECK (expression)` to [builder], after `CONSTRAINT name ` when the constraint is named. + */ + fun appendTo(builder: StringBuilder) { + if (constraintName.isNotEmpty()) { + builder.append("CONSTRAINT ") + builder.append(constraintName) + builder.append(' ') + } + builder.append("CHECK (") + builder.append(expression) + builder.append(')') + } + } + + /** + * Returns the `CHECK` constraints that [@Check][com.ctrip.sqllin.dsl.annotation.Check] annotations declare on + * this class or property, in their order. + */ + private fun KSAnnotated.checkConstraints(): List = + annotations.filter { it.qualifiedName == ANNOTATION_CHECK }.map { annotation -> + val arguments = annotation.arguments.associate { it.name?.asString() to it.value } + CheckConstraint( + expression = (arguments["expression"] as? String).orEmpty().trim(), + constraintName = (arguments["constraintName"] as? String).orEmpty().trim(), + ) + }.toList() + + /** + * Appends the `CHECK` constraints of [property]'s column to [createSQLBuilder]: the unnamed ones first, as SQLite + * gives a column's constraint the name of the named one before it, and would report it under that name. + */ + private fun appendColumnChecks(createSQLBuilder: StringBuilder, property: KSPropertyDeclaration) { + for (check in property.checkConstraints().sortedBy { it.constraintName.isNotEmpty() }) { + createSQLBuilder.append(' ') + check.appendTo(createSQLBuilder) + } + } + + /** + * Reports the [@Check][com.ctrip.sqllin.dsl.annotation.Check] annotations of a `@DBRow` class and its properties + * that would make an invalid table: without an expression, with a name that isn't an SQL identifier, or with the + * name of another of the table's CHECK constraints, which would make SQLite's report of a failed one ambiguous. + * + * @return Whether all of them are valid + */ + private fun validateChecks(classDeclaration: KSClassDeclaration, propertyList: List): Boolean { + val className = classDeclaration.simpleName.asString() + val names = HashSet() + var isValid = true + for (symbol in listOf(classDeclaration) + propertyList) { + for (check in symbol.checkConstraints()) { + val name = check.constraintName + val message = when { + check.expression.isEmpty() -> "A @Check of '$className' has no expression." + name.isNotEmpty() && !SQL_IDENTIFIER.matches(name) -> + "The constraint name '$name' of a @Check of '$className' isn't an SQL identifier: it can only contain letters, digits and underscores, and can't start with a digit." + name.isNotEmpty() && !names.add(name) -> + "The table of '$className' has two CHECK constraints named '$name'. Their names have to differ, as SQLite reports the name of the one a row fails." + else -> continue + } + environment.logger.error(message, symbol) + isValid = false + } + } + return isValid + } + + /** + * Returns the qualified name of a property's type, without nullability, through a type alias. + */ + private fun resolvedTypeName(property: KSPropertyDeclaration): String? = when (val declaration = property.type.resolve().declaration) { + is KSTypeAlias -> declaration.type.resolve().declaration.typeName + else -> declaration.typeName + } + + /** + * Escapes [string] to be the content of a Kotlin string literal. + */ + private fun kotlinStringContent(string: String): String = + string.replace("\\", "\\\\").replace("\"", "\\\"").replace("$", "\\$") + + /** + * Processes all [@DBView][com.ctrip.sqllin.dsl.annotation.DBView] annotated classes and generates their view + * objects: like a table object, with a column property per serialized property, but without SET properties, + * constraints or a CREATE statement, as a view is created from a SELECT and can't be written to. + * + * @return The symbols that couldn't be processed yet + */ + @Suppress("UNCHECKED_CAST") + private fun processViews(resolver: Resolver): List { + val allViewClasses = resolver.getSymbolsWithAnnotation(ANNOTATION_DATABASE_VIEW_NAME) + val invalidViewClasses = allViewClasses.filter { !it.validate(enableNewFeatures = false) }.toList() + val serializableType = resolver.getClassDeclarationByName(ANNOTATION_SERIALIZABLE)!!.asStarProjectedType() + val transientName = resolver.getClassDeclarationByName(ANNOTATION_TRANSIENT)!!.asStarProjectedType() + + for (classDeclaration in allViewClasses.filter { it.validate(enableNewFeatures = false) } as Sequence) { + val className = classDeclaration.simpleName.asString() + if (classDeclaration.annotations.all { !it.annotationType.resolve().isAssignableFrom(serializableType) }) { + environment.logger.error("The class annotated with '@DBView' must be annotated with '@Serializable', but '$className' isn't.", classDeclaration) + continue + } + if (classDeclaration.annotations.any { it.qualifiedName == ANNOTATION_DATABASE_ROW_NAME }) { + environment.logger.error("The class '$className' is annotated with both '@DBRow' and '@DBView'; it can only be a table or a view.", classDeclaration) + continue + } + val visibility = classDeclaration.getVisibility() + if (visibility != Visibility.PUBLIC && visibility != Visibility.INTERNAL) { + environment.logger.error( + "The class annotated with '@DBView' must be 'public' or 'internal', but '$className' is '${visibility.name.lowercase()}'.", + classDeclaration, + ) + continue + } + val visibilityModifier = if (visibility == Visibility.INTERNAL) "internal " else "" + + // The same properties as a table's: those the serializer writes, in its order + val propertyList = classDeclaration.getAllProperties().filter { property -> + property.hasBackingField && + !property.annotations.any { ksAnnotation -> ksAnnotation.annotationType.resolve().isAssignableFrom(transientName) } + }.toList() + + var isValid = true + classDeclaration.annotations.filter { it.qualifiedName in CONSTRAINT_ANNOTATIONS }.forEach { + environment.logger.error("The view '$className' can't be annotated with '@${it.shortName.asString()}': a view has no constraints.", classDeclaration) + isValid = false + } + for (property in propertyList) { + val propertyName = property.simpleName.asString() + property.annotations.filter { it.qualifiedName in CONSTRAINT_ANNOTATIONS }.forEach { + environment.logger.error( + "The property '$propertyName' of view '$className' can't be annotated with '@${it.shortName.asString()}': a view's columns have no constraints.", + property, + ) + isValid = false + } + if (getClauseElementTypeStr(property) == null) { + environment.logger.error( + "The property '$propertyName' of '@DBView' class '$className' has the type '${property.type.resolve()}', which no column can hold. " + + "Supported types are Byte, Short, Int, Long, Float, Double and their unsigned variants, Boolean, Char, String, ByteArray, enum classes, " + + "and type aliases of these. To leave the property out of the view, annotate it with @kotlinx.serialization.Transient.", + property, + ) + isValid = false + } + } + if (!isValid) + continue + + val packageName = classDeclaration.packageName.asString() + val objectName = "${className}View" + val viewName = classDeclaration.annotations.find { it.qualifiedName == ANNOTATION_DATABASE_VIEW_NAME } + ?.arguments?.firstOrNull()?.value?.takeIf { (it as? String)?.isNotBlank() == true } ?: className + + val outputStream = environment.codeGenerator.createNewFile( + dependencies = classDeclaration.containingFile?.let { Dependencies(true, it) } ?: Dependencies(true), + packageName = packageName, + fileName = objectName, + ) + OutputStreamWriter(outputStream).use { writer -> + writer.write("package $packageName\n\n") + writer.write("import com.ctrip.sqllin.dsl.annotation.ColumnNameDslMaker\n") + writer.write("import com.ctrip.sqllin.dsl.annotation.ExperimentalDSLDatabaseAPI\n") + writer.write("import com.ctrip.sqllin.dsl.sql.clause.ClauseBlob\n") + writer.write("import com.ctrip.sqllin.dsl.sql.clause.ClauseBoolean\n") + writer.write("import com.ctrip.sqllin.dsl.sql.clause.ClauseEnum\n") + writer.write("import com.ctrip.sqllin.dsl.sql.clause.ClauseNumber\n") + writer.write("import com.ctrip.sqllin.dsl.sql.clause.ClauseString\n") + writer.write("import com.ctrip.sqllin.dsl.sql.View\n\n") + // As for table objects, the column properties carry @ColumnNameDslMaker for IntelliJ IDEA's DSL + // highlighting. View is experimental, and the view object opts in so that its users don't have to. + writer.write("@Suppress(\"DSL_MARKER_APPLIED_TO_WRONG_TARGET\")\n") + writer.write("@OptIn(ExperimentalDSLDatabaseAPI::class)\n") + writer.write("${visibilityModifier}object $objectName : View<$className>(\"$viewName\") {\n\n") + writer.write(" override fun kSerializer() = $className.serializer()\n\n") + writer.write(" inline operator fun invoke(block: $objectName.(view: $objectName) -> R): R = this.block(this)\n\n") + propertyList.forEachIndexed { index, property -> + val clauseElementTypeName = checkNotNull(getClauseElementTypeStr(property)) // Rejected above + val isNotNull = property.type.resolve().nullability == Nullability.NOT_NULL + writer.write(" @ColumnNameDslMaker\n") + writer.write(" val ${property.simpleName.asString()}\n") + writer.write(" get() = $clauseElementTypeName($className.serializer().descriptor.getElementName($index), this, ${!isNotNull})\n\n") + } + writer.write("}\n") + } + } + return invalidViewClasses + } + + private val KSAnnotation.qualifiedName: String? + get() = annotationType.resolve().declaration.qualifiedName?.asString() + /** * Maps a property's Kotlin type to the corresponding clause element type name. * diff --git a/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ColumnConstraintParser.kt b/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ColumnConstraintParser.kt index db37295c..54251ee8 100644 --- a/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ColumnConstraintParser.kt +++ b/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ColumnConstraintParser.kt @@ -19,8 +19,10 @@ package com.ctrip.sqllin.processor import com.google.devtools.ksp.getClassDeclarationByName import com.google.devtools.ksp.processing.Resolver import com.google.devtools.ksp.symbol.ClassKind +import com.google.devtools.ksp.symbol.KSAnnotation import com.google.devtools.ksp.symbol.KSClassDeclaration import com.google.devtools.ksp.symbol.KSPropertyDeclaration +import com.google.devtools.ksp.symbol.KSType import com.google.devtools.ksp.symbol.KSTypeAlias import java.io.Writer @@ -116,6 +118,12 @@ class ColumnConstraintParser(resolver: Resolver) { // Track composite unique constraints: group number → list of column names private val compositeUniqueColumns = HashMap>() + /** The conflict resolutions other than the default that the properties of the composite primary key give. */ + private val compositePrimaryKeyConflicts = LinkedHashSet() + + /** The conflict resolutions other than the default that the properties of each composite unique group give. */ + private val compositeUniqueConflicts = HashMap>() + /** * Parses property annotations and appends corresponding SQLite constraints to the CREATE TABLE statement. * @@ -227,10 +235,10 @@ class ColumnConstraintParser(resolver: Resolver) { isGeneratedByDatabase = isRowIdAlias && !isNotNull append(" PRIMARY KEY") + val primaryKey = property.annotation(ANNOTATION_PRIMARY_KEY) + primaryKey?.onConflict()?.let { append(" ON CONFLICT $it") } - isAutomaticIncrement = property.annotations.find { - it.annotationType.resolve().declaration.qualifiedName?.asString() == ANNOTATION_PRIMARY_KEY - }?.arguments?.firstOrNull()?.value as? Boolean ?: false + isAutomaticIncrement = primaryKey?.arguments?.firstOrNull { it.name?.asString() == "autoIncrement" }?.value as? Boolean ?: false if (isAutomaticIncrement) { check(isGeneratedByDatabase) { PROMPT_AUTO_INCREMENT_REQUIRES_NULLABLE_LONG } append(" AUTOINCREMENT") @@ -239,6 +247,7 @@ class ColumnConstraintParser(resolver: Resolver) { // Handle @CompositePrimaryKey - collect for table-level constraint check(isNotNull) { PROMPT_PRIMARY_KEY_MUST_NOT_NULL } compositePrimaryKeys.add(propertyName) + property.annotation(ANNOTATION_COMPOSITE_PRIMARY_KEY)?.onConflict()?.let { compositePrimaryKeyConflicts.add(it) } } // A rowid alias is the only column SQLite itself keeps from being NULL. On a rowid table, PRIMARY KEY @@ -254,8 +263,10 @@ class ColumnConstraintParser(resolver: Resolver) { } // Handle @Unique annotation - single column uniqueness - if (annotationKSType.any { it.isAssignableFrom(uniqueAnnotationName) }) + if (annotationKSType.any { it.isAssignableFrom(uniqueAnnotationName) }) { append(" UNIQUE") + property.annotation(ANNOTATION_UNIQUE)?.onConflict()?.let { append(" ON CONFLICT $it") } + } // Handle @CompositeUnique annotation - collect for table-level constraint val compositeUniqueAnnotation = property.annotations @@ -272,11 +283,14 @@ class ColumnConstraintParser(resolver: Resolver) { it.value as? List ?: listOf(0) } // Add this property to each specified group + val onConflict = onConflict() list.forEach { group -> val groupList = compositeUniqueColumns[group] ?: ArrayList().also { gl -> compositeUniqueColumns[group] = gl } groupList.add(propertyName) + if (onConflict != null) + compositeUniqueConflicts.getOrPut(group) { LinkedHashSet() }.add(onConflict) } } } @@ -392,10 +406,11 @@ class ColumnConstraintParser(resolver: Resolver) { append(it[i]) } append(')') + appendConflict(compositePrimaryKeyConflicts, "the composite primary key") } // Add composite unique constraints for each group - compositeUniqueColumns.values.forEach { + compositeUniqueColumns.forEach { (group, it) -> if (it.isEmpty()) return@forEach append(",UNIQUE(") @@ -405,10 +420,40 @@ class ColumnConstraintParser(resolver: Resolver) { append(it[i]) } append(')') + appendConflict(compositeUniqueConflicts[group].orEmpty(), "composite unique group `$group`") } } } + /** + * Appends the `ON CONFLICT` clause of a composite constraint, whose properties give [conflicts], other than the + * default ABORT, which needs no clause. + */ + private fun StringBuilder.appendConflict(conflicts: Set, constraint: String) { + check(conflicts.size <= 1) { + "The properties of $constraint give different conflict resolutions: ${conflicts.joinToString()}. Give one, on any of them." + } + conflicts.firstOrNull()?.let { append(" ON CONFLICT $it") } + } + + /** The annotation of this property named [name], or null. */ + private fun KSPropertyDeclaration.annotation(name: String): KSAnnotation? = + annotations.find { it.annotationType.resolve().declaration.qualifiedName?.asString() == name } + + /** + * The conflict resolution that the `onConflict` of this annotation gives, or null for ABORT, SQLite's default, which + * needs no `ON CONFLICT` clause. + */ + private fun KSAnnotation.onConflict(): String? { + val value = arguments.firstOrNull { it.name?.asString() == "onConflict" }?.value + val name = when (value) { + is KSClassDeclaration -> value.takeIf { it.classKind == ClassKind.ENUM_ENTRY }?.simpleName?.asString() + is KSType -> value.declaration.simpleName.asString() + else -> null + } + return name?.takeIf { it != "ABORT" } + } + /** * Determines the SQLite type declaration for a given property. * diff --git a/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ForeignKeyParser.kt b/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ForeignKeyParser.kt index abf8614f..1903731b 100644 --- a/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ForeignKeyParser.kt +++ b/sqllin-processor/src/main/kotlin/com/ctrip/sqllin/processor/ForeignKeyParser.kt @@ -79,6 +79,9 @@ class ForeignKeyParser { companion object { const val ANNOTATION_GROUP = "com.ctrip.sqllin.dsl.annotation.ForeignKeyGroup" const val ANNOTATION_REFERENCES = "com.ctrip.sqllin.dsl.annotation.References" + + /** Checks a foreign key when the transaction commits, rather than after each statement. */ + const val DEFERRED = " DEFERRABLE INITIALLY DEFERRED" const val ANNOTATION_FOREIGN_KEY = "com.ctrip.sqllin.dsl.annotation.ForeignKey" const val ANNOTATION_DEFAULT = "com.ctrip.sqllin.dsl.annotation.Default" } @@ -123,6 +126,7 @@ class ForeignKeyParser { var triggerEnumName = "" var triggerSQL = "" var constraintName = "" + var isDeferred = false annotation.arguments.forEach { argument -> when (argument.name?.asString()) { "group" -> group = argument.value as Int @@ -139,6 +143,7 @@ class ForeignKeyParser { } } "constraintName" -> constraintName = argument.value as String + "deferred" -> isDeferred = argument.value as? Boolean ?: false } } @@ -152,6 +157,7 @@ class ForeignKeyParser { triggerEnumName = triggerEnumName, triggerSQL = triggerSQL, constraintName = constraintName, + isDeferred = isDeferred, columns = ArrayList(), references = ArrayList(), ) @@ -226,6 +232,7 @@ class ForeignKeyParser { } } "constraintName" -> columnReferenceEntity.constraintName = argument.value as String + "deferred" -> columnReferenceEntity.isDeferred = argument.value as? Boolean ?: false "foreignKeys" -> { columnReferenceEntity.foreignKeys = (argument.value as? List)?.filter { it.isNotBlank() } ?: throw IllegalArgumentException("The parameter `foreignKeys` can't be null.") @@ -306,6 +313,8 @@ class ForeignKeyParser { append(' ') append(it.triggerSQL) } + if (it.isDeferred) + append(DEFERRED) } } } @@ -387,6 +396,8 @@ class ForeignKeyParser { append(' ') append(entity.triggerSQL) } + if (entity.isDeferred) + append(DEFERRED) } } } @@ -410,6 +421,7 @@ class ForeignKeyParser { val triggerEnumName: String, val triggerSQL: String, val constraintName: String, + val isDeferred: Boolean, val columns: MutableList, val references: MutableList, ) @@ -419,5 +431,6 @@ class ForeignKeyParser { var triggerSQL: String = "", var constraintName: String = "", var foreignKeys: List = emptyList(), + var isDeferred: Boolean = false, ) } \ No newline at end of file