Support traditional retention policies #677

Activity

1. dosubot

   added

   enhancementNew feature or requestNew feature or request

   on Dec 9, 2025
2. TomHutter commented on Feb 20, 2026

   

   TomHutter

   on Feb 20, 2026

   More actions

   Suggestion:

   apiVersion: postgresql.cnpg.io/v1
   kind: ScheduledBackup
   metadata:
     name: hourly-backup
   spec:
     pluginConfiguration:
       name: barman-cloud.cloudnative-pg.io
       parameters:
         retentionPolicy: 1d

   apiVersion: postgresql.cnpg.io/v1
   kind: ScheduledBackup
   metadata:
     name: daily-backup
   spec:
     pluginConfiguration:
       name: barman-cloud.cloudnative-pg.io
       parameters:
         retentionPolicy: 14d

   Same goes for backups:

   apiVersion: postgresql.cnpg.io/v1
   kind: Backup
   metadata:
     name: on-demand-backup
   spec:
     pluginConfiguration:
       name: barman-cloud.cloudnative-pg.io
       parameters:
         retentionPolicy: 14d
3. akirilltikhonov commented on May 4, 2026

   

   akirilltikhonov

   on May 4, 2026

   Last edited by akirilltikhonov

   More actions

   Hi Barman Cloud plugin maintainers,

   I would like to ask if there is any plan on the horizon to implement this feature? For us, and I think for most of the prod teams, this feature would be very beneficial and crucial, as it is a very common scenario to have different retention policies for daily, weekly, monthly, and so on backups.

   Moreover, as far I can see (and @geekmush also mentioned it), there is no even temporal workaround for "traditional rentention policies" because CNPG + Barman are designed to always have 1 ObjectStore per cluster, all parameters like retention policies and tags are set up on ObjectStore level, and, thus, they will be applied for all backups (daily, weekly, monthly), which makes usage of several ScheduledBackup a bit worthless: if I have daily backup, there is no sense to have weekly.

   Some related issues:

   • Backup to different object store #611
   • [Feature]: Support Backup to Multiple S3 Destinations cloudnative-pg#8757 (comment)
   • Multiple S3 Locations for backups cloudnative-pg#10349 (comment)

   P.S.
   I also add that it would be useful to set up not only renentionPolicy, but also tags on ScheduledBackup level, because it allows managing lifecycle rules on the storage level (for example, for moving backup files in S3 from STANDARD to DEEP_ACRHIVE storage class after N days). Something like:

   kind: ScheduledBackup
   metadata:
     name: daily-backup
   spec:
     pluginConfiguration:
       name: barman-cloud.cloudnative-pg.io
       parameters:
         retentionPolicy: 14d
         tags:
             type: "daily"
   

   👍React with 👍3Reacted by Yusuf Tarık Günaydın, borisroman and Loris
4. olexiyb commented on Aug 21, 2026

   

   olexiyb

   on Aug 21, 2026

   More actions

   Some evidence that may narrow this down: for the count-based half of the request, the underlying tool already supports it — only the CRD schema rejects the syntax.

   barman-cloud-backup-delete (barman 3.x, as shipped in the plugin sidecar image) accepts both policy forms:

   -r RETENTION_POLICY, --retention-policy RETENTION_POLICY
                         If specified, delete all backups eligible for deletion
                         according to the supplied retention policy. Syntax:
                         REDUNDANCY value | RECOVERY WINDOW OF value {DAYS |
                         WEEKS | MONTHS}
   

   But ObjectStore.spec.retentionPolicy is constrained to the recovery-window form only:

   {"pattern": "^[1-9][0-9]*[dwm]$", "type": "string"}

   So REDUNDANCY 1 is unrepresentable through the API even though the binary underneath handles it.

   I verified it works end to end against S3-compatible storage (Garage), running the exact command the plugin would, with --dry-run against a catalog of three daily base backups:

   $ barman-cloud-backup-delete --endpoint-url <s3-compatible> \
       --retention-policy "REDUNDANCY 1" --dry-run \
       s3://<bucket>/<path> <server>
   
   Skipping deletion of objects ['.../base/20260819T000001/data.tar.gz', '.../base/20260819T000001/backup.info'] due to --dry-run option
   Skipping deletion of objects ['.../base/20260820T000000/data.tar.gz', '.../base/20260820T000000/backup.info'] due to --dry-run option
   Skipping deletion of objects ['.../wals/000000330000004B/...', ... 10 segments ...] due to --dry-run option
   

   It correctly identified the two older base backups plus the WAL no longer required, and retained the most recent backup.

   Environment: plugin v0.14.0 (chart 0.7.1), CloudNativePG 1.30.0, PostgreSQL 17.5, S3-compatible object storage.

   Why it matters beyond the long-retention case in the original report: a recovery window cannot express a small number of copies. barman keeps the oldest base backup needed to reach the start of the window, so Nd with a daily schedule settles at roughly N+1 base backups. 1d is therefore the practical floor at two copies — there is no way to ask for one, which is a reasonable ask for ephemeral/dev clusters where the data is reproducible and object storage is the constraint.

   Today the only workaround is to leave retentionPolicy unset and run barman-cloud-backup-delete -r "REDUNDANCY N" from a separate CronJob, which means reimplementing credential plumbing outside the operator and diverging from how every other cluster is configured.

   If it helps, allowing ^([1-9][0-9]*[dwm]|REDUNDANCY [1-9][0-9]*)$ and passing the value straight through to -r would cover the count-based case with no new logic in the plugin, since the string is already forwarded to a flag that parses both forms. That would not address the grandfather-father-son tiering in the original post, which is a genuinely larger change — but it would close the simpler half.

   Happy to test a build if that would be useful.
5. abd-ulbasit commented on Oct 17, 2026

   

   abd-ulbasit

   on Oct 17, 2026

   More actions

   Hi, I'd like to pick up the count-based part of this (REDUNDANCY N in ObjectStore.spec.retentionPolicy). Out of scope: GFS tiers and per-Backup/ScheduledBackup parameters (related: #611, which covers per-Backup parameters).

   One note on the earlier analysis: widening the CRD pattern on its own isn't enough. DeleteBackupsByPolicy runs the value through ParsePolicy in cloudnative-pg/barman-cloud, and that function only returns RECOVERY WINDOW OF ..., so REDUNDANCY 3 would fail with "not a valid policy".

   Proposed approach:

   1. barman-cloud: make ParsePolicy also accept ^REDUNDANCY [1-9][0-9]*$ and pass it through unchanged, with tests.
   2. plugin: bump the library, widen the pattern to ^([1-9][0-9]*[dwm]|REDUNDANCY [1-9][0-9]*)$ (0 stays rejected, since barman would treat it as "delete everything"), regenerate the CRD, manifest and API docs, add a CRD pattern test, and document the semantics in retention.md (only DONE backups count, WAL cleanup follows the oldest kept backup).
   3. Optionally, an e2e test with the existing RustFS harness that takes N+2 backups and checks that N remain.

   Alternatively, the plugin could build the barman-cloud-backup-delete call itself, which needs no library bump (so no wait for a release and no unrelated CRD regen from newer library fields).

   Before I write code, three things I'd like your call on:

   • Is REDUNDANCY N in the existing string field OK, or would you prefer a structured field?
   • Library-first as above (and if so, can it still make 0.7.0, chore(main): release 0.7.0 barman-cloud#303?), or plugin-local so it doesn't wait for a barman-cloud release?
   • Is a count-based option wanted on its own, independently of the GFS design, or should it wait for that? Should this issue stay open for GFS afterwards?

   Smaller details (e2e placement, --minimum-redundancy) can be settled in review. If that works for you, I'm happy to be assigned. I use AI assistance and will disclose it with Assisted-by: per the AI policy. Commits will be signed off. @leonardoce @mnencia, since you wrote and merged the original retention support, does this direction look right to you?