Repository navigation
Show deprovisioning event and date in the audit log - #635
kayjoosten wants to merge 10 commits into
Conversation
|
Update after code review: the initial version wrote the `deprovisioned` entry but `AuditLogRepository::createSecondFactorSearchQuery()` filters against an event allowlist that didn't include `IdentityForgottenEvent`, so it was silently dropped from the RA audit log query — the feature was a no-op. Fixed in the latest commit, along with a corrected (previously backwards) code comment and a test that now mocks `findByIdentityId()` the way the real repository behaves (returning the freshly flushed entry, not omitting it). Also worth calling out explicitly: `stepup:event:replay` is not idempotent (new UUID per replayed entry), so the retroactive backfill described above should only be run once per environment. |
johanib
left a comment
There was a problem hiding this comment.
Nice, focused change — the ordering rationale holds up: AuditLogRepository::save() flushes, so the re-query in applyIdentityForgottenEvent() genuinely picks up the new entry. Ran AuditLogProjectorTest in an isolated worktree, 4/4 green.
One blocker before this can go out, plus two points worth a decision:
- Backfill is not idempotent — the
stepup:event:replayinstruction in the description will duplicate entries on a second run, or when run after new deprovisionings have already been projected live. - The chosen ordering intentionally erases the deprovisioning actor's name; worth confirming that is what the issue reporter wants.
IdentityRestoredEventhas no counterpart action, so the log can end on "deprovisioned" for an identity that is actually live again.
Details inline.
|
Based on the title, I expected this to be easy 😅 One thing I don't understand yet: Why does middleware have a Lets discuss this PR before continuing. |
|
Take a good look at why there are 2 types of replay. Also we need to take a look at how this will work with mega big data. As far as we know atm the replay is not doable in production. possible thoughts.
|
Why is this change needed? Prior to this change, deprovisioning an identity via the lifecycle API left no trace in the audit log. AuditLogProjector explicitly skipped writing an entry for IdentityForgottenEvent, so RA(A)s had no way to see that, or when, an identity had been deprovisioned. How does it address the issue? This change maps IdentityForgottenEvent to a new 'deprovisioned' audit log action and reorders AuditLogProjector::handle() so the deprovisioning entry is persisted before the identity's other audit log entries are anonymized; otherwise the newly created entry would be wiped immediately after. A projector test covers both the new entry and the existing anonymization behavior. Provide links to any relevant tickets, articles or other resources OpenConext/Stepup-RA#423
Why is this change needed? IdentityForgottenEvent was mapped to a new 'deprovisioned' audit log action and persisted, but AuditLogRepository::createSecondFactorSearchQuery() filters entries against an event allowlist that never included it, so the new entry was written but silently filtered out of the RA audit log page. Separately, the comment justifying the projector's insert-before-anonymise ordering had the mechanism backwards, and the projector test mocked findByIdentityId() to omit the just-inserted entry, so it couldn't have caught either issue. How does it address the issue? Adds IdentityForgottenEvent::class to the allowlist so the entry is actually returned to the RA UI. Corrects the ordering comment: inserting before anonymising means the new entry is included in the same anonymisation pass as the identity's other entries (its actor name gets wiped like everything else), not preserved as the old comment claimed. Updates the test to mock findByIdentityId() the way the real repository behaves (returning the freshly flushed entry alongside the pre-existing one), and asserts the new entry's actor name is anonymised too. Provide links to any relevant tickets, articles or other resources OpenConext/Stepup-RA#423
Deprovisioning (IdentityForgottenEvent) was added to the audit log after the fact, so identities forgotten before that change have no 'deprovisioned' entry. The retroactive backfill cannot go through stepup:event:replay + AuditLogProjector: replaying IdentityForgottenEvent also re-runs applyIdentityForgottenEvent(), which anonymises every current audit log entry for the identity - and an identity forgotten, then restored (UpdateIdentityCommand calls Identity::restore()), then active again would have its live audit log scrubbed. stepup:audit-log:backfill-deprovisioned reads IdentityForgottenEvents from the event store and inserts only the missing entries, without any anonymisation: - AuditLogRepository::hasDeprovisionedEntry() keys on identity + event + recordedOn (second precision; a restore has to happen between two forgets so they cannot share a second), and an in-run key guards against re-inserting within a single batch - so the command is idempotent and safe to run more than once. - Entries are persisted in batches of 500 with the entity manager cleared between them, keeping memory flat over a large event set. - A confirmation prompt (bypass with --force or --no-interaction) warns to disable the lifecycle API first: the check and insert are not atomic with AuditLogProjector, so a deprovisioning projected live during the run could otherwise be inserted twice. - --dry-run reports what it would create.
c60a592 to
3ad99dd
Compare
|
Why a new command instead of the existing replay commands:
|
Deprovisioning entries were only added to the audit log going forward, so identities forgotten before this change has been deployed have no 'deprovisioned' entry. Add an Unreleased/Deployment action required section pointing operators at the stepup:audit-log:backfill-deprovisioned command, following the same convention already used for #628.
johanib
left a comment
There was a problem hiding this comment.
Went through it again. The approach is solid 🥇
Still some comments to clarify and cleanup. Then it should be ready to merge.
The collision scenario is covered nicely with the reverting transaction.
|
|
||
| switch (true) { | ||
| case $event instanceof IdentityForgottenEvent: | ||
| // Don't insert the IdentityForgottenEvent into the audit log, as we'd remove it immediately afterwards. |
There was a problem hiding this comment.
I was wondering why this was suddenly no longer the case. Turns out, in the 2015, the deprovision deleted everything. Then, in 2022, that was replaced by anonymisation.
| } | ||
|
|
||
| $entityManager->flush(); | ||
| $entityManager->clear(); |
Why is this change needed? Deprovisioned second factor audit log entries had stopped appearing in the RA audit log API. A later refactor that introduced playhead based deduplication accidentally dropped IdentityForgottenEvent from AuditLogRepository::$secondFactorEvents, silently excluding every deprovisioning entry from the query again. How does it address the issue? Re-add IdentityForgottenEvent::class to the allowlist and add a reflection based regression test asserting it stays there, so this cannot regress silently a second time.
Why is this change needed? Audit log entry ids were only deduplicated for deprovisioning events, using a deprovisioning specific id scheme and a special case in the projector. Every event type can be deduplicated the same way, using the source event's stream id and playhead. How does it address the issue? Replace AuditLogEntry::deprovisionedEntryIdFor() with a generic idFor(DomainMessage) and apply it, together with an existence check, to every event in AuditLogProjector::applyAuditableEvent(). Also move the recordedOn computation inline at its point of use. Update the projector test to match the generalized behaviour.
Why is this change needed? The backfill command's docblock and confirmation prompt overstated the operational requirement to disable the whole application during the run, and did not explain what happens if a deprovisioning event is processed concurrently with the backfill. How does it address the issue? Rewrite the docblock and confirmation prompt to state that only the lifecycle (deprovisioning) API needs to be disabled, and describe the actual failure mode: a primary key collision rolls back the whole transaction, so the command exits safely and can simply be re-run. Switch to the generalized AuditLogEntry::idFor(). Update the command test accordingly.
Why is this change needed?
Prior to this change, deprovisioning an identity via the lifecycle API left no trace in the audit log.
AuditLogProjectorexplicitly skipped writing an entry forIdentityForgottenEvent, so RA(A)s had no way to see that, or when, an identity had been deprovisioned.How does it address the issue?
Maps
IdentityForgottenEventto a newdeprovisionedaudit log action, and records that entry before the identity's audit trail is anonymized. The actor name is still intentionally anonymized, consistent with every other audit-log entry for a forgotten identity.Every audit log entry's id is now derived deterministically from its source event's stream id and playhead (
AuditLogEntry::idFor()), rather than a random uuid4.AuditLogProjector::applyAuditableEvent()skips creating an entry when one with that id already exists, so projecting or replaying the same event twice is a no-op for every event type, not onlyIdentityForgottenEvent. The backfill command below uses the same id scheme, so it is idempotent with the live projector.IdentityForgottenEventwas also restored toAuditLogRepository::$secondFactorEvents, so these newdeprovisionedentries are actually returned by the RA audit-log API and not silently filtered out.The RA-side rendering of this new action (translation for "Deprovisioned") is in OpenConext/Stepup-RA#531 — the audit log template there already renders any action generically, so no other RA change was needed.
Retroactive backfill
This PR ships a dedicated one-shot backfill command for historical
IdentityForgottenEvents:Rollout/runbook notes:
deprovisionedrowsstepup:event:replayfor this backfillFollow-up
IdentityRestoredEventaudit-log handling stays out of scope for this PR and is tracked separately in #637.Provide links to any relevant tickets, articles or other resources
Closes OpenConext/Stepup-RA#423
Test plan
php vendor/bin/phpunit -c ci/qa/phpunit.xml --filter AuditLogProjectorTestphp vendor/bin/phpunit -c ci/qa/phpunit.xml --filter AuditLogRepositoryTestphp vendor/bin/phpunit -c ci/qa/phpunit.xml src/Surfnet/StepupMiddleware/MiddlewareBundle/Tests/Console/Command/BackfillDeprovisionedAuditLogEntriesCommandTest.phpphp vendor/bin/phpunit --configuration=ci/qa/phpunit.xml(full unit + integration suite)php vendor/bin/phpunit --configuration=ci/qa/phpunit.xml --testsuite=database./ci/qa/phpcsphp vendor/bin/phpstan analyse --memory-limit=-1 --no-ansi -c ./ci/qa/phpstan.neonbin/console stepup:audit-log:backfill-deprovisionedin an environment with already-deprovisioned identities and confirmdeprovisionedentries appear with correct date/time