diff --git a/CHANGELOG.md b/CHANGELOG.md index c6188ff..8f85df5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -40,6 +40,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 * a backfilled window always attaches with writers held at the parent ([#89](https://github.com/bedrock-python/pg-partsmith/issues/89)) ([b1260c1](https://github.com/bedrock-python/pg-partsmith/commit/b1260c1a8bcdd6388c3836d57eb548f8fe5959bb)) +## 1.9.0 — an expired partition stays detached for a week by default + +A policy that named no drop — every flat config (`create_ahead_count`, `retention_count`), +every bare `TablePartitionConfig(...)` and every YAML table without `drop:` — got +`DropAfter()` with no grace, so an expired partition was detached and dropped in the same +run. A wrong `retention_count` or granularity lost the data on the first scheduled run, with +no window to notice ([#106](https://github.com/bedrock-python/pg-partsmith/issues/106)). + +**What changes:** when the policy names no drop it is now `DropAfter(grace=timedelta(days=7))`. +An expired partition stays detached for a week, where it can be inspected or re-attached, +and the first run after that drops it. Disk is freed a week later than before. + +**What stays:** an explicit `DropAfter()` keeps its zero grace, so a configuration that asked +for the immediate drop keeps it. `DropNever()` is unchanged. + ## 1.8.3 — a refused detach no longer starts the drop grace A `DETACH … CONCURRENTLY` writes the orphan marker, with its `detached-at` instant, in a diff --git a/docs/agents.md b/docs/agents.md index 57f9ce0..454e10c 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -358,9 +358,10 @@ lifecycle units — partitions directly under the root — never once per leaf o (recorded as an issue under `continue_on_error=True`). 7. **`create_ahead_count` and `retention_count` include the current period.** `create_ahead_count=3` means this month and the next two. -8. **The default lifecycle drops.** `DropAfter()` has a zero grace, so an expired partition - is detached and dropped in the same run. Choose `DropAfter(grace=…)` or `DropNever()` - consciously. +8. **The default lifecycle drops a week after the detach.** A policy that names no `drop` + keeps an expired partition detached for seven days, where it can be inspected or + re-attached, then drops it. `DropAfter()` (zero grace) drops in the same run as the + detach; `DropNever()` leaves detached tables to someone else. Choose consciously. 9. **`lifecycle` is meaningless for a scheme with no progression level** — a root `HashPartitioning` or a static `ListPartitioning` has a fixed set of partitions, and the policy is ignored rather than obeyed. diff --git a/docs/concepts/lifecycle.md b/docs/concepts/lifecycle.md index 16ec4ca..c67432e 100644 --- a/docs/concepts/lifecycle.md +++ b/docs/concepts/lifecycle.md @@ -138,8 +138,8 @@ referencing rows are gone. See [Handle foreign keys](../guide/foreign-keys.md). | Rule | Effect | |---|---| -| `DropAfter()` (default: no grace) | dropped in the same run as the detach | -| `DropAfter(grace=timedelta(days=7))` | kept detached for a week, then dropped. The detach instant is recorded on the table's marker; an orphan marked by an older version, or adopted, has no instant and is treated as past its grace | +| `DropAfter()` (no grace) | dropped in the same run as the detach | +| `DropAfter(grace=timedelta(days=7))` (the default when the policy names no drop) | kept detached for a week, then dropped. The detach instant is recorded on the table's marker; an orphan marked by an older version, or adopted, has no instant and is treated as past its grace | | `DropAfter(grace=…, when=predicate)` | dropped only once the grace has passed *and* the predicate holds — "not while bigger than 150 GB on a weekday" is a `Callback` over the size fact | | `DropNever()` | detached partitions are left alone; something else owns the drop — an archive pipeline, a DBA. They are never re-attached, and a wanted window whose name such a table holds is reported (`name_unusable`), not recreated over it | diff --git a/docs/getting-started/first-table.md b/docs/getting-started/first-table.md index bfce24f..9d7321f 100644 --- a/docs/getting-started/first-table.md +++ b/docs/getting-started/first-table.md @@ -241,14 +241,13 @@ plan for public.events at 2027-09-15T03:00:00+00:00 CREATE public.events__2027_11 (create_ahead) DETACH public.events__2026_08 (retention_expired) DETACH public.events__2026_09 (retention_expired) - DROP public.events__2026_08 (follows_detach) - DROP public.events__2026_09 (follows_detach) ``` -Two things to notice. Retirement is a **detach** followed by a **drop**, two separate -statements: the partition leaves the parent first, and only a table that is detached is -ever dropped. And `events__2026_10` survives: the twelve newest months, counted back from -September 2027, are October 2026 through September 2027. +Two things to notice. Retirement is a **detach**, and the **drop** comes later: the +partition leaves the parent first, and only a table that is detached is ever dropped — +by default a week after its detach (see step 9). And `events__2026_10` survives: the +twelve newest months, counted back from September 2027, are October 2026 through +September 2027. After `apply()`: @@ -261,14 +260,17 @@ events partitioned table PARTITION BY RANGE (created_at) ``` (Only three months were ever created between the two runs, which is why the table looks -sparse. A daily tick fills the calendar continuously.) +sparse. A daily tick fills the calendar continuously. The two detached months are still in +the database, outside the parent, until their week is up.) -## 9. A week between detach and drop +## 9. Between detach and drop -Dropping in the same run as the detach is the default. Most teams want a pause — to -export the partition, to be able to put it back. That is the `drop` half of a lifecycle -policy. The flat fields you used so far are sugar for a scheme and a policy; spell the -policy out to change it: +A detached partition waits a week before it is dropped — a pause to export it, or to put +it back if the retention was wrong. That is the `drop` half of a lifecycle policy, and the +flat fields you used so far leave it at its default. The flat fields are sugar for a +scheme and a policy; spell the policy out to choose the pause yourself (`DropAfter()` +drops in the same run as the detach, `DropNever()` leaves detached tables to someone +else): ```python from datetime import timedelta @@ -325,8 +327,8 @@ meantime, a marked table whose month is wanted is *re-attached* rather than recr ## What you have learned - A configuration is a **scheme** (monthly partitions of `created_at`) plus a **lifecycle - policy** (three ahead, twelve kept, drop a week after detach). The flat fields are sugar - for both. + policy** (three ahead, twelve kept, drop a week after detach — the default). The flat + fields are sugar for both. - `plan()` is read-only and explains itself; `apply()` runs the plan under a lock. - Retirement is detach, then drop; grace periods live on the marker. - Findings are the planner telling you what it chose not to do, and why. diff --git a/docs/guide/cli.md b/docs/guide/cli.md index ed69e05..5303bb4 100644 --- a/docs/guide/cli.md +++ b/docs/guide/cli.md @@ -174,14 +174,11 @@ pg-partsmith plan -c partitions.yaml --locks plan for public.events at 2026-09-01T02:15:00+00:00 CREATE public.events__2026_10 (create_ahead) DETACH public.events__2025_08 (retention_expired) size=1073741824 rows~4021553 - DROP public.events__2025_08 (follows_detach) locks: CREATE public.events__2026_10 ACCESS SHARE on the template during CREATE; SHARE UPDATE EXCLUSIVE on the parent and ACCESS EXCLUSIVE on the new partition (and on a DEFAULT sibling) during ATTACH -- EXCLUSIVE on the parent instead, which holds off writes, when rows of the window are taken out of a DEFAULT sibling in the attach's transaction; SHARE ROW EXCLUSIVE on every table referencing the parent through a foreign key DETACH public.events__2025_08 (outside a transaction block) SHARE UPDATE EXCLUSIVE on the parent (CONCURRENTLY), ACCESS EXCLUSIVE on the partition and, in the second transaction, on every table referencing the parent through a foreign key; ... - DROP public.events__2025_08 - ACCESS EXCLUSIVE on the dropped table only ``` The heaviest lock each operation takes, and on what, as measured on PostgreSQL 15 and 17 diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 1d45dee..eec414b 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -10,7 +10,7 @@ Types are the accepted inputs; defaults are what you get when the field is omitt | `schema` | `str` | `None` | Schema of the parent table; read back as `db_schema`. Recommended. | | `table_name` | `str` | required | The parent table, lowercase, at most 63 bytes minus the scheme's longest suffix. | | `scheme` | `RangePartitioning \| ListPartitioning \| HashPartitioning` | from the flat fields | The root level and everything below it. | -| `lifecycle` | `LifecyclePolicy` | `CreateAhead(count=6)`, `KeepNewest(count=12)`, `DetachMode.AUTO`, `DropAfter()` | When partitions of the progression level are created, detached and dropped. | +| `lifecycle` | `LifecyclePolicy` | `CreateAhead(count=6)`, `KeepNewest(count=12)`, `DetachMode.AUTO`, `DropAfter(grace=timedelta(days=7))` | When partitions of the progression level are created, detached and dropped. | | `leaves` | `LocalLeaves \| ForeignLeaves` | `LocalLeaves()` | What kind of relation the leaves are. | Flat fields, accepted instead of `scheme` / `lifecycle` for a time-partitioned RANGE root: @@ -103,7 +103,7 @@ literals. | `creation` | `CreateAhead \| CreateUntil \| CreateNextIf` | `CreateAhead(count=6)` | which windows must exist ahead of the cursor | | `retention` | any predicate | `KeepNewest(count=12)` | when a window behind the cursor has expired | | `detach` | `DetachMode` | `AUTO` | `AUTO`, `CONCURRENT`, `BLOCKING` | -| `drop` | `DropAfter \| DropNever` | `DropAfter()` | what happens after the detach | +| `drop` | `DropAfter \| DropNever` | `DropAfter(grace=timedelta(days=7))` | what happens after the detach; `DropAfter()` drops in the same run | Creation rules: `CreateAhead(count ≥ 1)`; `CreateUntil(position)` — a `datetime`, an `int`, or their string forms; `CreateNextIf(when)` — a predicate over the newest diff --git a/pg_partsmith/constants.py b/pg_partsmith/constants.py index 09dae94..3bf057b 100644 --- a/pg_partsmith/constants.py +++ b/pg_partsmith/constants.py @@ -3,6 +3,8 @@ # Default configuration values. DEFAULT_CREATE_AHEAD_COUNT = 6 DEFAULT_RETENTION_COUNT = 12 +# How long an expired partition stays detached, and can be re-attached, when the policy names no drop. +DEFAULT_DROP_GRACE_DAYS = 7 # PostgreSQL limits. MAX_IDENTIFIER_LENGTH = 63 diff --git a/pg_partsmith/entities.py b/pg_partsmith/entities.py index 9f6654b..17475e7 100644 --- a/pg_partsmith/entities.py +++ b/pg_partsmith/entities.py @@ -238,7 +238,9 @@ class TablePartitionConfig(BaseModel): ) The ordinary time-partitioned table keeps its flat spelling, which is sugar - for exactly that:: + for exactly that. Neither names a drop, so an expired partition stays + detached for a week before it is dropped; pass ``drop=`` in a + ``LifecyclePolicy`` to choose otherwise:: TablePartitionConfig( table_name="events", diff --git a/pg_partsmith/lifecycle.py b/pg_partsmith/lifecycle.py index 9b6502a..5beb189 100644 --- a/pg_partsmith/lifecycle.py +++ b/pg_partsmith/lifecycle.py @@ -25,7 +25,7 @@ from pydantic import BaseModel, ConfigDict, Field, field_validator from .boundaries import Axis, RangeBoundaries, Window -from .constants import DEFAULT_CREATE_AHEAD_COUNT, DEFAULT_RETENTION_COUNT +from .constants import DEFAULT_CREATE_AHEAD_COUNT, DEFAULT_DROP_GRACE_DAYS, DEFAULT_RETENTION_COUNT from .topology import FactKind, PartitionFacts, PartitionNode from .types import NonNegativeInt, PositiveInt @@ -776,7 +776,10 @@ class LifecyclePolicy(BaseModel): creation: Which windows ahead of the cursor must exist. retention: When a window behind the cursor has expired. detach: How an expired partition is detached. - drop: What happens to it afterwards. + drop: What happens to it afterwards. Unless the policy says otherwise, it + stays detached for a week, where it can still be inspected or + re-attached, and is dropped after that. ``DropAfter()`` drops in the + same run as the detach; ``DropNever()`` leaves it alone. """ model_config = ConfigDict(frozen=True, extra="forbid") @@ -784,7 +787,7 @@ class LifecyclePolicy(BaseModel): creation: CreationPolicy = Field(default_factory=CreateAhead) retention: RetentionPolicy = Field(default_factory=KeepNewest) detach: DetachMode = DetachMode.AUTO - drop: DropPolicy = Field(default_factory=DropAfter) + drop: DropPolicy = Field(default_factory=lambda: DropAfter(grace=timedelta(days=DEFAULT_DROP_GRACE_DAYS))) @property def required_facts(self) -> frozenset[FactKind]: diff --git a/tests/e2e/support.py b/tests/e2e/support.py index 59c050b..d090ba3 100644 --- a/tests/e2e/support.py +++ b/tests/e2e/support.py @@ -181,15 +181,22 @@ def write_document( dsn: str | None = None, hooks: Mapping[str, Any] | None = None, ) -> Path: - """One table, monthly, two partitions ahead: the document the guides start from, as JSON.""" + """One table, monthly, two partitions ahead, as JSON. + + Expired partitions are dropped in the run that detaches them, as these tests + expect; without ``drop`` a detached partition would wait a week. + """ payload: dict[str, Any] = { "tables": [ { "table_name": table, "partition_column": "created_at", "granularity": "month", - "create_ahead_count": 2, - "retention_count": retention, + "lifecycle": { + "creation": {"kind": "create_ahead", "count": 2}, + "retention": {"kind": "keep_newest", "count": retention}, + "drop": {"kind": "drop_after", "grace": "P0D"}, + }, } ] } diff --git a/tests/integration/aio/builder.py b/tests/integration/aio/builder.py index da8c4d5..d2b57bc 100644 --- a/tests/integration/aio/builder.py +++ b/tests/integration/aio/builder.py @@ -17,6 +17,7 @@ from pg_partsmith.entities import PartitionGranularity, Period, TablePartitionConfig from pg_partsmith.lifecycle import DetachMode from pg_partsmith.topology import RangeBounds +from tests.integration.nested_support import drops_at_once if TYPE_CHECKING: from sqlalchemy.ext.asyncio import AsyncEngine @@ -99,7 +100,7 @@ async def build(self) -> PartitioningTestContext: metadata = PostgresMetadataProvider(self._engine) locks = PostgresAdvisoryLockManager(self._engine) - config = TablePartitionConfig( + config = drops_at_once(TablePartitionConfig)( table_name=self._table_name, partition_column="created_at", granularity=self._granularity, diff --git a/tests/integration/aio/test_cli.py b/tests/integration/aio/test_cli.py index 1f3eabd..0a0ff45 100644 --- a/tests/integration/aio/test_cli.py +++ b/tests/integration/aio/test_cli.py @@ -46,8 +46,11 @@ def _document(tmp_path: Path, table: str, dsn: str, *, retention: int = 12, hook "schema": schema or None, "partition_column": "created_at", "granularity": "month", - "create_ahead_count": 2, - "retention_count": retention, + "lifecycle": { + "creation": {"kind": "create_ahead", "count": 2}, + "retention": {"kind": "keep_newest", "count": retention}, + "drop": {"kind": "drop_after", "grace": "P0D"}, + }, } ], } diff --git a/tests/integration/aio/test_sliding_list.py b/tests/integration/aio/test_sliding_list.py index 8e76b4c..a068d82 100644 --- a/tests/integration/aio/test_sliding_list.py +++ b/tests/integration/aio/test_sliding_list.py @@ -14,7 +14,7 @@ from pg_partsmith.aio.metadata import PostgresMetadataProvider from pg_partsmith.aio.repositories import PostgresPartitionRepository -from pg_partsmith.lifecycle import CreateNextIf, DetachMode, KeepNewest, LifecyclePolicy, SqlPredicate +from pg_partsmith.lifecycle import CreateNextIf, DetachMode, DropAfter, KeepNewest, LifecyclePolicy, SqlPredicate from pg_partsmith.plan import FindingReason, Reason from pg_partsmith.topology import ListBounds from tests.integration.aio.support import ( @@ -210,4 +210,5 @@ def test__sliding_list_config__is_state_driven() -> None: assert config.lifecycle == LifecyclePolicy( creation=CreateNextIf(when=SqlPredicate(sql="SELECT count(*) >= 3 FROM {partition}")), retention=KeepNewest(count=2), + drop=DropAfter(), ) diff --git a/tests/integration/aio/test_timezone_semantics.py b/tests/integration/aio/test_timezone_semantics.py index 8b9be03..e6ce2f8 100644 --- a/tests/integration/aio/test_timezone_semantics.py +++ b/tests/integration/aio/test_timezone_semantics.py @@ -20,6 +20,7 @@ from pg_partsmith.entities import PartitionGranularity, Period, TablePartitionConfig from pg_partsmith.topology import RangeBounds from pg_partsmith.utils import qualify +from tests.integration.nested_support import drops_at_once pytestmark = pytest.mark.integration @@ -81,6 +82,7 @@ async def _partition_bound_expr(engine: AsyncEngine, partition: str) -> str: return str(expr) +@drops_at_once def _moscow_config(table_relname: str, *, retention: int = 12) -> TablePartitionConfig: return TablePartitionConfig( schema="public", diff --git a/tests/integration/nested_support.py b/tests/integration/nested_support.py index 442d481..1f55db1 100644 --- a/tests/integration/nested_support.py +++ b/tests/integration/nested_support.py @@ -6,7 +6,9 @@ from __future__ import annotations +import functools import re +from collections.abc import Callable from typing import TYPE_CHECKING, Any from sqlalchemy import event @@ -19,7 +21,15 @@ UUIDv7BoundaryCodec, ) from pg_partsmith.entities import PartitionGranularity, TablePartitionConfig -from pg_partsmith.lifecycle import CreateAhead, CreateNextIf, KeepBehind, KeepNewest, LifecyclePolicy, SqlPredicate +from pg_partsmith.lifecycle import ( + CreateAhead, + CreateNextIf, + DropAfter, + KeepBehind, + KeepNewest, + LifecyclePolicy, + SqlPredicate, +) from pg_partsmith.partition_bounds import parse_range_boundaries from pg_partsmith.scheme import HashPartitioning, ListGroup, ListPartitioning, RangePartitioning from pg_partsmith.utils import orphan_table_comment @@ -294,6 +304,26 @@ # ── Config builders ───────────────────────────────────────────────────────────── +def drops_at_once(build: Callable[..., TablePartitionConfig]) -> Callable[..., TablePartitionConfig]: + """Keep the immediate drop these suites were written against. + + Since 1.9.0 a policy that names no drop keeps a detached partition for a week. + The suites here are about detaching and dropping, not about the grace, so the + builders spell out ``DropAfter()``; a test about the grace passes ``drop=`` + in its own ``lifecycle``, which is left as it is. + """ + + @functools.wraps(build) + def builder(*args: Any, **kwargs: Any) -> TablePartitionConfig: + config = build(*args, **kwargs) + if "drop" in config.lifecycle.model_fields_set: + return config + return config.model_copy(update={"lifecycle": config.lifecycle.model_copy(update={"drop": DropAfter()})}) + + return builder + + +@drops_at_once def nested_config( table_name: str, *, @@ -321,6 +351,7 @@ def nested_config( ) +@drops_at_once def list_config( table_name: str, *, @@ -347,6 +378,7 @@ def list_config( ) +@drops_at_once def hash_root_config( table_name: str, *, @@ -361,11 +393,13 @@ def hash_root_config( ) +@drops_at_once def tasks_config(table_name: str, *, modulus: int = 8) -> TablePartitionConfig: """Build the Hatchet-style root HASH configuration over ``task_id``.""" return TablePartitionConfig(table_name=table_name, scheme=HashPartitioning(key="task_id", modulus=modulus)) +@drops_at_once def list_root_config( table_name: str, *, @@ -383,6 +417,7 @@ def list_root_config( ) +@drops_at_once def tiered_config(table_name: str, *, create_ahead: int = 2, retention: int = 2) -> TablePartitionConfig: """Build a LIST(tier) → RANGE(created_at) configuration: a monthly lifecycle inside each group.""" return TablePartitionConfig( @@ -398,6 +433,7 @@ def tiered_config(table_name: str, *, create_ahead: int = 2, retention: int = 2) ) +@drops_at_once def queue_config( table_name: str, *, @@ -414,6 +450,7 @@ def queue_config( ) +@drops_at_once def sliding_list_config( table_name: str, *, @@ -432,6 +469,7 @@ def sliding_list_config( ) +@drops_at_once def glitchtip_config( table_name: str, *, @@ -451,6 +489,7 @@ def glitchtip_config( ) +@drops_at_once def composite_config( table_name: str, *, @@ -469,6 +508,7 @@ def composite_config( ) +@drops_at_once def nullable_composite_config( table_name: str, *, @@ -479,6 +519,7 @@ def nullable_composite_config( return composite_config(table_name, create_ahead=create_ahead, retention=retention) +@drops_at_once def flat_config(table_name: str, *, create_ahead: int = 1, retention: int = 12) -> TablePartitionConfig: """Build the classic one-leaf-per-week configuration.""" return TablePartitionConfig( @@ -490,6 +531,7 @@ def flat_config(table_name: str, *, create_ahead: int = 1, retention: int = 12) ) +@drops_at_once def monthly_config( table_name: str, *, diff --git a/tests/integration/sync/builder.py b/tests/integration/sync/builder.py index fccd04c..a978ff7 100644 --- a/tests/integration/sync/builder.py +++ b/tests/integration/sync/builder.py @@ -17,6 +17,7 @@ from pg_partsmith.sync.repositories import PostgresPartitionRepository from pg_partsmith.sync.service import PartitionLifecycleService from pg_partsmith.topology import RangeBounds +from tests.integration.nested_support import drops_at_once if TYPE_CHECKING: from sqlalchemy import Engine @@ -99,7 +100,7 @@ def build(self) -> PartitioningTestContext: metadata = PostgresMetadataProvider(self._engine) locks = PostgresAdvisoryLockManager(self._engine) - config = TablePartitionConfig( + config = drops_at_once(TablePartitionConfig)( table_name=self._table_name, partition_column="created_at", granularity=self._granularity, diff --git a/tests/integration/sync/test_sliding_list.py b/tests/integration/sync/test_sliding_list.py index 98c68ac..d0ad541 100644 --- a/tests/integration/sync/test_sliding_list.py +++ b/tests/integration/sync/test_sliding_list.py @@ -11,7 +11,7 @@ import pytest -from pg_partsmith.lifecycle import CreateNextIf, DetachMode, KeepNewest, LifecyclePolicy, SqlPredicate +from pg_partsmith.lifecycle import CreateNextIf, DetachMode, DropAfter, KeepNewest, LifecyclePolicy, SqlPredicate from pg_partsmith.plan import FindingReason, Reason from pg_partsmith.sync.metadata import PostgresMetadataProvider from pg_partsmith.sync.repositories import PostgresPartitionRepository @@ -206,4 +206,5 @@ def test__sliding_list_config__is_state_driven() -> None: assert config.lifecycle == LifecyclePolicy( creation=CreateNextIf(when=SqlPredicate(sql="SELECT count(*) >= 3 FROM {partition}")), retention=KeepNewest(count=2), + drop=DropAfter(), ) diff --git a/tests/integration/sync/test_timezone_semantics.py b/tests/integration/sync/test_timezone_semantics.py index e4ce4f1..0c57ac5 100644 --- a/tests/integration/sync/test_timezone_semantics.py +++ b/tests/integration/sync/test_timezone_semantics.py @@ -19,6 +19,7 @@ from pg_partsmith.sync.service import PartitionLifecycleService from pg_partsmith.topology import RangeBounds from pg_partsmith.utils import qualify +from tests.integration.nested_support import drops_at_once pytestmark = pytest.mark.integration @@ -80,6 +81,7 @@ def _partition_bound_expr(engine: Engine, partition: str) -> str: return str(expr) +@drops_at_once def _moscow_config(table_relname: str, *, retention: int = 12) -> TablePartitionConfig: return TablePartitionConfig( schema="public", diff --git a/tests/unit/sync/test_service.py b/tests/unit/sync/test_service.py index 7b65cd5..3b8d22c 100644 --- a/tests/unit/sync/test_service.py +++ b/tests/unit/sync/test_service.py @@ -26,6 +26,7 @@ from pg_partsmith.lifecycle import ( CreateAhead, DetachMode, + DropAfter, ExpireIf, KeepBehind, LifecyclePolicy, @@ -126,6 +127,11 @@ def config() -> TablePartitionConfig: ) +def _dropping_at_once(config: TablePartitionConfig) -> TablePartitionConfig: + """The fixture's policy, but dropping in the same run as the detach.""" + return config.model_copy(update={"lifecycle": config.lifecycle.model_copy(update={"drop": DropAfter()})}) + + def _nested_config() -> TablePartitionConfig: return TablePartitionConfig( table_name="events", @@ -585,6 +591,7 @@ def test__maintain__end_to_end__returns_counters_and_the_plan_it_executed( service: PartitionLifecycleService, repo: MagicMock, metadata: MagicMock, config: TablePartitionConfig ) -> None: # Arrange -- January is expired, February current-ish, March missing; an orphan is past its grace + config = _dropping_at_once(config) metadata.get_actual_tree.return_value = _tree( _range_child("events__2024_01", "2024-01-01", "2024-02-01", oid=1), _range_child("events__2024_02", "2024-02-01", "2024-03-01", oid=2), @@ -629,6 +636,7 @@ def test__maintain__continue_on_error__isolates_a_failed_create_and_still_prunes service: PartitionLifecycleService, repo: MagicMock, metadata: MagicMock, config: TablePartitionConfig ) -> None: # Arrange + config = _dropping_at_once(config) metadata.get_actual_tree.return_value = _tree(_range_child("events__2024_01", "2024-01-01", "2024-02-01", oid=1)) metadata.get_relation_oid.return_value = 1 metadata.is_partition_attached.return_value = True diff --git a/tests/unit/test_entities.py b/tests/unit/test_entities.py index 5316e40..c0024de 100644 --- a/tests/unit/test_entities.py +++ b/tests/unit/test_entities.py @@ -137,7 +137,7 @@ def test__config__neither_scheme_nor_partition_column__rejected() -> None: TablePartitionConfig(table_name="events") -def test__config__flat_defaults__six_ahead_twelve_kept_and_dropped_at_once() -> None: +def test__config__flat_defaults__six_ahead_twelve_kept_and_dropped_after_a_week() -> None: # Arrange / Act config = _flat() @@ -148,7 +148,7 @@ def test__config__flat_defaults__six_ahead_twelve_kept_and_dropped_at_once() -> creation=CreateAhead(count=6), retention=KeepNewest(count=12), detach=DetachMode.AUTO, - drop=DropAfter(grace=timedelta(0)), + drop=DropAfter(grace=timedelta(days=7)), ) diff --git a/tests/unit/test_lifecycle.py b/tests/unit/test_lifecycle.py index 1ca37e4..f695db3 100644 --- a/tests/unit/test_lifecycle.py +++ b/tests/unit/test_lifecycle.py @@ -813,7 +813,7 @@ def test__detach_mode__values__spell_the_three_modes() -> None: # ── LifecyclePolicy ───────────────────────────────────────────────────────────── -def test__lifecycle_policy__defaults__create_ahead_keep_newest_auto_detach_immediate_drop() -> None: +def test__lifecycle_policy__defaults__create_ahead_keep_newest_auto_detach_drop_after_a_week() -> None: # Arrange / Act policy = LifecyclePolicy() @@ -821,7 +821,7 @@ def test__lifecycle_policy__defaults__create_ahead_keep_newest_auto_detach_immed assert policy.creation == CreateAhead(count=DEFAULT_CREATE_AHEAD_COUNT) assert policy.retention == KeepNewest(count=DEFAULT_RETENTION_COUNT) assert policy.detach is DetachMode.AUTO - assert policy.drop == DropAfter() + assert policy.drop == DropAfter(grace=timedelta(days=7)) assert policy.required_facts == frozenset() assert policy.sql_predicates == () assert policy.needs_facts is False diff --git a/tests/unit/test_service.py b/tests/unit/test_service.py index 7f61ab1..604b2f7 100644 --- a/tests/unit/test_service.py +++ b/tests/unit/test_service.py @@ -28,6 +28,7 @@ from pg_partsmith.lifecycle import ( CreateAhead, DetachMode, + DropAfter, ExpireIf, KeepBehind, LifecyclePolicy, @@ -126,6 +127,11 @@ def config() -> TablePartitionConfig: ) +def _dropping_at_once(config: TablePartitionConfig) -> TablePartitionConfig: + """The fixture's policy, but dropping in the same run as the detach.""" + return config.model_copy(update={"lifecycle": config.lifecycle.model_copy(update={"drop": DropAfter()})}) + + def _nested_config() -> TablePartitionConfig: return TablePartitionConfig( table_name="events", @@ -591,6 +597,7 @@ async def test__maintain__end_to_end__returns_counters_and_the_plan_it_executed( service: PartitionLifecycleService, repo: MagicMock, metadata: MagicMock, config: TablePartitionConfig ) -> None: # Arrange -- January is expired, February current-ish, March missing; an orphan is past its grace + config = _dropping_at_once(config) metadata.get_actual_tree.return_value = _tree( _range_child("events__2024_01", "2024-01-01", "2024-02-01", oid=1), _range_child("events__2024_02", "2024-02-01", "2024-03-01", oid=2), @@ -635,6 +642,7 @@ async def test__maintain__continue_on_error__isolates_a_failed_create_and_still_ service: PartitionLifecycleService, repo: MagicMock, metadata: MagicMock, config: TablePartitionConfig ) -> None: # Arrange + config = _dropping_at_once(config) metadata.get_actual_tree.return_value = _tree(_range_child("events__2024_01", "2024-01-01", "2024-02-01", oid=1)) metadata.get_relation_oid.return_value = 1 metadata.is_partition_attached.return_value = True