Skip to content

UK local surface before the Budget build: high-value homes OBR leg, per-vehicle fuel flag, Northern Ireland energy share and housekeeping (#1278) - #1306

Merged
juaristi22 merged 25 commits into
mainfrom
uk-local-surface-1278-b
Oct 18, 2026
Merged

juaristi22 merged 25 commits into
mainfrom
uk-local-surface-1278-b

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Second PR for #1278: the items that are actionable now and small enough to land before the frozen pre-Budget build.

Base.

Part of #1278.

What changes (one commit each, then the review fixes and #1307's)

A. The high-value homes benchmarks gate reads the OBR costing facts (bug).

  • Problem. The gate's OBR leg looked for an older row shape (observed_measure.source_concept, layout.record_set_spec_id), so it reported awaiting_vendored_facts on every build, and the derived-stock level question (Place high-value homes where the Land Registry sees them (#1240) #1270 review, finding 4) had no OBR comparison.
  • Row selection. The leg now selects rows by concept, England, fiscal-year period type, year and basis, and needs every band and the total once each. Each count must be finite and non-negative. Otherwise it reports no_matching_rows, incomplete_rows or malformed_rows, and a missing or non-numeric value, or a field that is not a mapping, is reported rather than raised, since the leg runs after the solve.
  • The share. The non-owner-occupied share (40 per cent) comes from the note's own row, and the leg names the year it is stated for.
  • What it reports. The owner-occupied sensitivity (99,000 for 2028-29, static), the frame's England total over it, and the derived stock's private-sector count over Table 1.1's total.
    • No owner-occupied ratio. The stock build sets the owner-occupied share from the same OBR share, so that ratio would only repeat this one.
  • Disclosure. The shipped band counts withhold the total when it rests on fewer than ten distinct source households. They withhold further bands until the withheld ones rest together on ten, so the total cannot give back their sum. The regional shares and their total follow the same rule.
  • Unchanged. The leg stays informational: pass or fail does not depend on it.
  • Cache. The calibrated gate node keys on the high-value homes code through its parameters (terminal_code), hashed once per process, so a resumed store recomputes the evidence. The full-gates kernel's hash, which UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301) #1307's preflight kernel inherits, leaves that code out, because the dense solve keys on the preflight.
    • What that buys. An edit to high_value_home_gates.py never re-solves. high_value_homes.py also feeds the target kernels, so an edit there still re-solves: a disclosure fix between two builds that must reuse the solve belongs in high_value_home_gates.py alone.

B. The high-value homes evidence names its scope.

C. Registers track #1278.

G. The disclosure floor's study conditions are recorded. This answers #796's open #609 question.

  • Conditions. No study the build reads sets a minimum above CD171's ten:
    • the FRS 2024-25 (SN 9563) adds nothing to the End User Licence;
    • the SPI 2022-23 tape (SN 9422) adds two conditions: no record attribution, and analyses go to HMRC ten working days before distribution;
    • LCFS, ETB, WAS and NTS add nothing.
  • Where. The note lives in uk_runtime.receipt_counts.
  • The baselines tool. The weighted-integrity baselines tool no longer asks about a threshold of 30, anywhere: its docstring, its default's comment, the --sdc-minimum-count help and the caller obligations it writes into every receipt. Those obligations now carry the SPI tape's two conditions.

J1. Zero-emission cars are drawn per vehicle.

  • The draw. Each of a household's vehicles is zero-emission at the VEH1103 battery-electric and fuel-cell share. A household with n vehicles is therefore a fuel buyer at 1 - (1 - rate) ** n, against the same identity-keyed uniform. One-vehicle households keep their flag.
  • Rule name. rate_rule becomes per_car_zero_emission_share in the stage spec, the rendered roster and the sources schema.
  • Receipt. The fuel-flag receipt adds two shares: the flagged share among vehicle households, and the share of vehicles in fuel-free households.
    • The flag is per household, so a zero-emission car beside a petrol or diesel one keeps its household's fuel, and the second share sits below the per-car share.
    • The shares on a side difference into the households with and without a vehicle, crossed with fuel or none. So every share on that side is "<10" when any non-empty group rests on one to nine distinct positive-weight sources: the frame's source households, the donor's diaries.
  • Regenerated. The coverage manifest and the H2 spine fixture.
  • Not yet. Who holds the cars by income stays random; a later PR conditions it.

E. The rowwise solve takes its review date.

  • Change. prepare_uk_full_solve, the doctrine solve and the rotated holdout accept now and pass it to the binding review. Builds keep today's date.
  • Why. Seventeen engine-free tests reviewed the committed register on today's date, so they would have failed from 2027-01-15. They now review on its latest approval (COMMITTED_BINDING_NOW), as a build would on that day. Nothing is asserted at import.
  • New tests.
    • A solve and every holdout fold refuse a date past the windows, and all five folds receive the date.
    • test_no_committed_local_adjudication_is_about_to_lapse fails fourteen days before the earliest local expiry, as the uprating holds' clock does.
  • Follow-up. The uk.full.problem kernel and the holdout kernel should take the build's review date.

H. Division tails. The LCFS stage's report-only division capture adds each division's top-1% and top-10% spend shares, on the frame and the donor.

  • The floor. It counts the distinct source households that spend anything, since the frame stacks copies of one. A share is "<10" when one to nine of them sit:
    • in its top group;
    • outside it, since the division's total ships beside the shares;
    • or between it and a shipped smaller fraction's group.
  • Copies. A frame that marks copies without naming their sources is refused.
  • What it answers. The next build says whether the over-draw sits in the tail.

D. The parity register is re-pinned to uk-data a43e8c5 (from 8629dbb).

I. Northern Ireland's energy share comes from DESNZ.

  • Factor. Great Britain's part of the Energy Trends UK level is now its share of DESNZ's sub-national domestic consumption with Northern Ireland's row added (all domestic meters, 2024). So each factor is the UK level over the UK sub-national total.
  • Departs from the plan. The plan of record subtracted Northern Ireland's row from the Energy Trends total. The sub-national statistics run about 9% (electricity) and 12% (gas) above Energy Trends, so subtracting would take Northern Ireland out at a higher basis than Great Britain's part. The share keeps both nations on one basis (María's ruling of 2026-10-17).
  • Periods. Northern Ireland's electricity runs on financial years from April and its gas on Northern Ireland gas years, both labelled 2024. DESNZ calls them not directly comparable.
    • The basis text, the receipt and the signed desnz_subnational_consumption_basis adjudication and census fence say so. María signed off one added sentence on 2026-10-17, with dates unchanged.
    • The census lists DESNZ's two Northern Ireland publications as sources.
  • Resource. desnz_local_energy_facts.json vendors the two Northern Ireland rows in place of the four LFS household totals; its 697 meter rows are unchanged.
  • Retired. The desnz_northern_ireland_interim_share adjudication and its census fence.
  • Receipts. The local compile-parity receipt's 694 kWh rows (348 electricity, 346 gas) move by one factor per fuel: 0.99901 for electricity, 1.01582 for gas. So do the same cells in the local membership report. Nothing else in either file moves.

Review fixes (on top of the item commits)

From the second review:

  • R8, items J1 and H: the receipts' floors on groups and spenders, unnamed copies refused, and the stage wiring tested.
  • R9, items A and B: every unreadable OBR row reported, terminal_code hashed once per process with its scope stated, and two comments corrected.
  • R10, item D: the fragment's file count.

Remaining fixes from the third review of #1307

  • N1. The two below-floor region rows are worded as an expectation from their published stock, not a count of frame records, in the small-cell rule and its fragment. The local membership, which copies the rule into 601 rationales, is regenerated; nothing else in it moves.

  • L6. The re-pin procedure says to regenerate the receipts one --surface at a time with nothing else running, and when the local-selector cut used here holds.

  • T1. The publisher's --preflight-only verdict and printed pointer carry the contract's warnings. The tag of a release published past them records how many, never their text. Nothing refuses.

  • T2. The sign-off drafter accepts digits only in citations of a fixed list of repositories, refuses a citation next to a noun a figure counts or belongs to, accepts review anchors and file URLs, bounds dates to a year of the drafting date, and accepts digits only in listed logins.

  • T3. The disclosure walk catches:

    • order statistics above a figure, figure maps and pandas split tables;
    • any field holding an area code, in any case;
    • numbers with thousands separators, and the missing figure names.

    The release pre-flight reads files as the contract does and walks the incumbent files too, and the tests now drive both ends.

  • T4. The runbook says how to retry build 1, and the full-build doc says the contract warns.

Regenerated

  • uk/uk_local_target_census.json (B, I).
  • uk/release_input_coverage_manifest.json and the H2 spine fixture (J1).
  • uk/uk_data_target_inventory.json and uk/uk_data_target_parity.json (D).
  • uk/desnz_local_energy_facts.json (I).
  • uk/ledger_compile_parity_local_incumbent_2025_signed_differences.json and uk/local_target_reference_membership.json (I), on abf0921. uk/local_target_references.json regenerates byte-identical.
    • How. Both generators hold the whole feed in memory, so they ran on a cut of the pinned artifact (digest checked against the pin): the constituency and local-authority rows matching a local contract selector.
    • Why the cut is complete. No local reference reads operands outside its own selector, so the cut holds every candidate. Every row other than the 694 energy cells came out byte-identical to Re-pin the UK Chronicle feed to abf0921 (#1300) #1302's full-feed regeneration.

Merge order with the open PRs

Rulings owed

Not run locally

  • The feed-gated tests. They need the whole abf0921 feed in memory: the vendored-resource round trip, the English region council-tax sum, and test_committed_surfaces_regenerate_from_pinned_feed[local].
    • That last test regenerates the local references and membership from the whole feed. Here they came from the cut described under Regenerated. It reproduced Re-pin the UK Chronicle feed to abf0921 (#1300) #1302's whole-feed output on every untouched row and, after the rebase, the committed union byte for byte.
  • The full compile on abf0921. It belongs to the frozen build.

Verification (local)

  • Touched surfaces: 103 test files covering every touched surface, the engine lane and the release tooling included, with 4,921 tests.
    • Results. 4,903 passed, none failed and 18 were skipped.
    • Every skip is environment-gated: the pinned feed or national artifact absent (14), the licensed LCFS/ETB tabs, the NZ artifact, the uk-data tree and the rebuilt OA ladder.
    • The uk-data tree re-hash. Run separately with MICROCOSM_UK_DATA_TREE on a43e8c5: the parity file's 14 tests passed.
  • Checks: tools/ci_test_plan.py verify and tools/graph_acceptance_burndown.py --verify are ok. ruff check and ruff format --check are clean on the 33 edited Python files.
  • Generators: census_uk_local_targets.py --check, census_uk_data_target_parity.py --check and build_uk_release_input_coverage_manifest.py --check all report current.

Disclosure

Commit messages, changelog fragments and this body carry no survey-derived figures, only:

  • published figures (OBR, DESNZ, UKDS conditions);
  • counts of tests, register entries, rows and areas.

H's fragment names, without figures, the divisions that drew above their donor.

🤖 Generated with Claude Code

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Independent adversarial review of this PR at cf3aae4

Run with Claude Code on 16–17 October 2026 at cf3aae4: seven independent review lenses (correctness, method, generated artifacts, tests, claims against the diff, disclosure, a second fresh-eyes pass), then one to three skeptics per finding who tried to refute it; (k/n) is how many agreed. No builds and no feed-gated tests were run. The head has not moved since the review.

Keep this in draft; it should not merge yet. Item by item, the code mostly holds up: the targeted tests pass, the generators report current, and the vendored Northern Ireland rows match the abf0921 feed byte for byte. Two things block it:

Blocking

Should fix

  • The local membership report still pins the retired LFS factor on 694 DESNZ holds. CONFIRMED (2/2 medium). packages/microcosm-build/src/microcosm/build/uk/local_target_reference_membership.json:147782 (E06000001 factor); its resolved_value is at :497033.

    • The report carries factor 0.916500968101422 on 348 electricity holds and 0.875963199869941 on 346 gas holds, with resolved values computed at those factors. The generator at head would write 0.915590 / 0.889821.
    • It was last regenerated in cb45632 (Re-pin the UK Chronicle feed to abf0921 (#1300) #1302) and is missing from the body's "Regenerated" list. test_uk_target_references.py:109-182 (test_committed_surfaces_regenerate_from_pinned_feed[local]) sha256-compares it, so that test will fail the first time it runs with the feed. CI has no feed.
    • No build numbers are wrong: the compile re-applies the uprating applier itself. local_target_references.json holds no values and does not change.
    • Fix: after the NI ruling, regenerate the membership in the same pass as the receipt and list it under "Regenerated".
  • Item G says the baselines tool no longer asks about a threshold of 30, but only the docstring changed. CONFIRMED (2/2 medium).

    • The 30-case advice remains in three places in tools/measure_uk_weighted_integrity_baselines.py:
      • :327-330: caller_obligations, which is written into every receipt the tool produces.
      • :61-63: the constant comment.
      • :95-96: the --sdc-minimum-count help text.
    • c5a0a6f changed only docstring lines 23-28. The body (item G's "Tool" bullet), the commit message and changelog.d/1278-disclosure-floor-study-conditions.changed.md all say the tool no longer asks.
    • The receipt's caller_obligations also lack the SN 9422 HMRC condition that item G records elsewhere.
    • Fix: bring all three places in line with the docstring, and add the HMRC condition to caller_obligations.
  • High-value homes band suppression never withholds the total, and it adds a complementary suppression only when exactly one band is suppressed. Item A makes the OBR leg live and repeats the total twice. CONFIRMED (2/2 medium). packages/microcosm-build/src/microcosm/build/uk_runtime/high_value_home_gates.py:127-154 (_suppress_small_cells, line 141); the new OBR fields are at :286-312 and the gate details at :680-692.

    • Synthetic probes show both gaps:
      • One band on 5 sources, the rest empty: the band ships as None, but the total (500.0) equals it.
      • Bands on 40/4/3/0 sources: the two thin bands are suppressed, and the total minus the shipped bands gives back their combined count, which rests on 7 sources.
    • frame_over_all_properties_ratio['total'] and frame_over_owner_occupied_sensitivity multiply back to the total, because their denominators are public OBR figures.
    • Regional shares use the same helper and sum to 1.
    • The terminal gate report is copied into the release dir as terminal_gates.json (tools/assemble_uk_release_dir.py:450).
    • The logic predates this PR (ee7f1bb, Place high-value homes where the Land Registry sees them (#1240) #1270). The test at tests/engine_free/uk/test_uk_high_value_home_gates.py:121-128 deliberately pins the leaky case. Whether a real England build hits it is unverified.
    • Fix:
      • Withhold the total when it rests on fewer than 10 distinct sources.
      • When the suppressed bands together rest on fewer than 10 distinct sources, also suppress the next-smallest band or the total.
      • Apply the same rule to regional shares, and flip the test to expect total=None.
      • Check the frozen build's high-value homes evidence before any gate report goes public.

Minor

Low:

  • The re-pin to uk-data a43e8c5 misses uk-data#529's local validation-only switch. CONFIRMED (2/2, both low; parts refuted, see below). packages/microcosm-build/src/microcosm/build/uk/uk_data_target_parity.json:336-349 and data_target_parity.py:272-282.

    • uk-data#529 made local hmrc/employment_income/count validation-only. It did this in create_datasets.py, which is outside the inventory's fixed scope.
    • Microcosm still binds that count: the census marks it bound_in_code, under fence hmrc_spi_frame_model_proxy, which runs to 2027-01-14. The new national_ons_lfs_employment fence quotes the same part-year reason without mentioning uk-data#529.
    • The plan of record for UK local surface, consumption and high-value homes: remaining items (consolidation) #1278 PR 2 lists this as a ruling owed, but no public issue or PR body records it.
    • Fix: add a uk-data#529 note to local_contract.hmrc.employment_income.count and list the owed ruling in the body.
  • The SPI tape's HMRC pre-distribution condition is recorded but nothing enforces or tracks it. CONFIRMED (2/2 low). packages/microcosm-build/src/microcosm/build/uk_runtime/receipt_counts.py:12-19.

    • git grep 'working days' finds only the docstring and the changelog fragment. No gate, marker, caller obligation, checklist item or follow-up refers to it.
    • Whether frame outputs (such as item H's tails) count as "analyses of the tape" is María's SPI 9422 ruling, still owed.
    • The gap predates the PR.
    • Fix: confirm the condition's wording, settle the SPI 9422 ruling, add the condition to caller_obligations, and list the ruling as a follow-up.
  • The division-tail "<10" floor counts frame rows, not distinct sources. CONFIRMED (1/1). packages/microcosm-build/src/microcosm/build/uk_runtime/lcfs_consumption.py:795-822 (line 817).

    • high_value_home_support splits each selected owner household into 6 copies (high_value_home_support.py:75, 228-301), and spi_support_channel stacks synthetic copies.
    • A synthetic repro: 2 sources split into 6 copies each print an exact 0.5013 share, where the unsplit frame prints "<10".
    • No real-data UK path runs the stage on a small frame.
    • Fix: pass source_household_id into the tail and count distinct sources.
  • The top-1% and top-10% shares can be differenced. CONFIRMED (1/1). lcfs_consumption.py:825-846.

    • A synthetic skewed-weights case recovers the exact weighted spend of the 2 households between the groups, and of a single household in a variant. Realistic lognormal weights leave gaps of 435-548 households.
    • Evidence runs copy division_capture whole, so both shares would be committed together.
    • Fix: withhold the top-10% share when the two group sizes differ by 1 to 9, and add a skewed-weights test.
  • The OBR leg's row selection is under-specified. CONFIRMED (1/1). high_value_home_gates.py:175, :187-198, :201-226, :284, :302.

    • Silent cases:
      • A missing total gives status available with a sensitivity of 0.0.
      • Rows relabelled as calendar_year are accepted.
      • obr_period='2030' uses the FY2028 share without warning.
    • Cases that raise inside the terminal phase (graph_terminal.py:873-879), after the solve: per-year share rows, or an unknown England band.
    • The vendoring row counts and this PR's own tests catch most bad re-vendors before a build runs.
    • Fix: select by period type, geography and basis; require every band plus the total; report a status instead of raising.
  • COMMITTED_BINDING_NOW asserts at import. CONFIRMED (1/1). test_support/microcosm_build/uk_local_rowwise.py:21-40.

    • A partial renewal breaks it: six entries renewed on 2027-01-05, with the high-value homes fence (expired 2026-12-31) left dormant. The real build review passes on that register, but the module raises AssertionError at import.
    • There is also an off-by-one: the assert uses a strict <, while an entry stays in force on its expiry date.
    • 7 test modules fail collection (test_uk_graph_terminal imports the helper inside one function). With no --continue-on-collection-errors, one collection error interrupts the whole pytest run.
    • Fix: compute the date from the families the fixtures actually bind, or make it a lazy fixture.
  • Item E removes CI's only signal that the local adjudications are about to lapse. CONFIRMED (1/1). Same file, :40.

    • With the review clock forced to 2027-01-20, 66 tests in the five touched files still pass. A control run shows a build is refused on that date.
    • The DESNZ fences lapse 2026-12-15, the high-value homes fence 2026-12-31, and six others 2027-01-14. Only the engine-uk test_uk_full_target_graph.py still goes red, from 2027-01-15 and for the census fence only.
    • Fix: add test_no_committed_local_adjudication_is_about_to_lapse, following test_uk_uprating_holds.py:263.
  • DESNZ's "not directly comparable" caveat for NI has no fence, and the family's sources leave out the NI publications. CONFIRMED (1/1). packages/microcosm-build/src/microcosm/build/uk_runtime/local_target_census.py:266-276; fence rules at :960-1021; stale note at :726-727.

    • Retiring the interim also removed the family's only NI fence. The three fences that remain (basis, meters, stage share) say nothing about NI, and require_adjudicated_uk_local_binding reviews only those (local_rowwise.py:813-840).
    • Fix: add an NI comparability sentence to the basis fence before its 2026-12-15 renewal, list the NI publications as family sources, and fix the electricity source note.
  • The module docstring still describes the retired LFS interim. CONFIRMED (1/1). domestic_energy_targets.py:13-20.

    • The function docstrings at :193-199 and :246-251 are correct, so the module docstring contradicts them.
    • Fix: rewrite the module docstring, and update the comment at local_target_census.py:133.
  • Commit D says "No existing concern's classification changes", but credibility_local_council_tax_outliers changes. CONFIRMED (1/1). changelog.d/1278-uk-data-parity-a43e8c5.changed.md:1; data_target_parity.py:591-597.

    • The guard moves from routed / validation_check to ported_local_declared / compile_contract. The same text lists it a few lines later as a corrected verdict, so it contradicts itself.
    • Fix: write "the re-extraction changes no classification; one verdict is corrected by hand".
  • J1's commit message says "The expected zero-emission share of cars stays the VEH1103 share", which holds only for the hidden per-car draw. CONFIRMED (1/1; the company-car sub-claim is unsubstantiated). lcfs_consumption.py:2904-2907; fuel zeroing at :618-621.

    • The frame stores only a household fuel flag, so a zero-emission car in a mixed household keeps the household's fuel spend.
    • Synthetic run with a 56/34/10 split of 1/2/3+ cars: 2.24% of vehicle households are fuel-free and 1.49% of cars sit in them, against z = 3.85% (the old rule gave about 3.84%).
    • Fix: reword the message, and optionally record the fuel-free share of cars in the receipt.
  • The parity register calls pension-age Housing Benefit "ported" and hides a definitional difference. CONFIRMED (1/1). data_target_parity.py:247-252 (line 251).

    • Microcosm's bindings filter only on eldest_adult_age >= 66. uk-data also excludes units on IS, income-based JSA or income-related ESA.
    • policyengine-uk has housing_benefit_pension_age_regulations_apply in both 2.132.6 and 2.134.5, and it encodes the uk-data rule. policyengine-uk marks eldest_adult_age as deprecated.
    • Fix: bind that variable, or state the difference in the concern's evidence.
  • The owner-occupied ratio equals the private-sector ratio by construction. CONFIRMED (1/1). high_value_home_gates.py:323-337; tools/build_uk_high_value_home_stock.py:895-908.

    • The stock build scales the owner-occupied count to exactly 60% of the private-sector count, using the same OBR share. On the committed facts both ratios are 1.1709173, so the second one is not an independent check.
    • Fix: say so in the body, the changelog and a code comment, or drop the ratio.
  • The body's Disclosure section says "Nothing is survey-derived", which is too broad. CONFIRMED (1/1).

    • changelog.d/1278-division-capture-tail-shares.added.md:1 and the 65dd810 message name which divisions drew above the LCFS donor. That is a qualitative, national survey result, not a breach.
    • Fix: write "no survey-derived figures".
  • Item A's hash change ties the dense solve's cache key to the report-only high-value homes gates module. CONFIRMED (1/1). packages/microcosm-build/src/microcosm/build/uk_runtime/graph_terminal.py:717-747 (line 741).

    • The preflight node (:1045-1061) runs the same kernel and feeds the dense solve's key. A verifier reproduced it: a comment-only edit to high_value_home_gates.py changes UKFullGateKernel.implementation_hash.
    • Consequence: any later fix to that module during the build window (for example for 1306-05, the band suppression, or 1306-12, the OBR row selection) re-solves the dense calibration.
    • Fix: keep the high-value homes gates module out of the preflight key. In UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301) #1307's split, override the hash in UKFullPreflightGateKernel; otherwise use a terminal-only kernel ref. At minimum, say it in the body.

Nit:

Cross-PR

Disclosure check

  • The disclosure lens found the PR's own outward text clean. It checked all 9 commit messages, all 9 changelog fragments, the body, the new parity text, the exclusions (a word-level diff shows only the tracking id changed), the holds, the vendored DESNZ NI rows (OGL admin data), the selector notes, the census text, the manifest and fixture, and the test fixtures (synthetic, or published OBR figures). The only figures are OBR-published figures, UKDS study conditions, and counts.
  • Latent floors, shown only with synthetic data:
    • 1306-05: the high-value homes total and differencing.
    • 1306-10: distinct-source counting in the division tails.
    • 1306-11: differencing the two tail shares.
    • 1306-25: the fuel-flag share.
    • Of these, 1306-05's output goes to the release dir (terminal_gates.json). Whether a real England build triggers it is unverified.
  • The HMRC condition (1306-09) is a gap in process, not a figure in this PR; its scope waits on María's SPI 9422 ruling.
  • The PR's self-certification (1306-22) is worded too broadly, but it is not a breach.

Tests and tools run

Checked and found sound

  • A: row selection matches the vendored Table 1.1 rows (static 2028: 71k/54k/25k/15k, total 165k; 40% share; 99,000 sensitivity). Per-band ratios use the suppressed shipped bands. The kernel hash now covers both high-value homes modules.
  • B: the scope dict reaches the evidence summary and both gates' details, and the census text was regenerated.
  • C: exactly 22 entries move to UK local surface, consumption and high-value homes: remaining items (consolidation) #1278. No approved_on or expires_on changed, and no signature or digest covers these registers.
  • D:
    • The register follows its own rules: two concerns may share dwp_housing_benefit, the cited target ids exist, and the status counts move as the body says.
    • Four of the five obr.py citations are exact.
    • The Band D and working-age HB verdicts match uk-data.
    • abf0921 has no LFS employee rows, so ons_labour_market is correctly blocked.
  • E: now reaches the binding review from all three entry points. Builds still default to today, so expiry stays fail-closed.
  • G: no floor value changed.
  • H:
    • weighted_top_share picks the smallest group that reaches the fraction and handles ties correctly.
    • The donor side has one row per LCFS household, so its floor is sound.
    • H adds no new crash path.
  • I:
    • The two NI rows match the feed, the 697 meter rows are unchanged, and _one_value fails closed on a missing or bad NI row.
    • Only local DESNZ kWh references move.
    • No leftover callers or constants remain.
  • J1: 1 − z^n is computed against the same keyed uniforms, a one-vehicle household reproduces the old threshold, and the rename is consistent across the YAML, the stage manifest, the schema and the H2 fixture.
  • Committed hashes: no committed file pins the base sha256 of any changed resource.

Refuted during verification

Not covered

  • The microcosm-graph H2 uk_spine parity test. It runs in the engine lane, which this review did not set up for this PR. The JSON was compared with the live graph, excluding numerical_dependencies.
  • A full local compile on abf0921 with item I. Builds were out of scope, so the release-blocking refusal is inferred from the gate code and the receipt comparison, not run.
  • G's study-condition claims (SN 9563, 9603, 7215, 5340, 8856, 9422) were not re-read against the UKDS catalogue.
  • The feed-gated vendoring round-trip test and the English region council-tax sum test.
  • Whether a real England build hits 1306-05's suppression gaps. No gate output from a real build was available.
  • Outside the diff: whether UK spine: SLC plan-2 borrower stocks at a third of SLC's — the FRS flag captures repayers, not the liable stock #868's closure makes the slc.borrowers.plan_2_* exclusions (expiring 2026-11-03) stale.

🤖 Generated with Claude Code

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Every finding of the review above is addressed below. The branch is rebased onto main after #1302's merge, and the PR now targets main.

History. The nine item commits are replayed on main. Only two messages change: J1 and D, which the review flagged. The fixes sit in seven commits on top, one per item, so each finding's change can be read on its own:

  • R1: A and B
  • R2: C
  • R3: G
  • R4: J1 and H
  • R5: E
  • R6: D
  • R7: I

Blocking

Should fix

  • The membership report. Regenerated in the same pass (R7).

    • What moves. Only the same 694 cells: their hold factors and resolved values.
    • Unchanged. local_target_references.json is byte-identical.
    • Body. The report is now listed under Regenerated.
  • Item G's tool text. R3 brings all three places in line with the docstring: the default's comment, the --sdc-minimum-count help and the caller obligations. The obligations also carry the SPI tape's two conditions.

  • High-value homes band suppression. R1 makes three changes:

    • The total. It is withheld when it rests on fewer than ten distinct source households.
    • The withheld bands together. Further bands are withheld until the withheld ones rest together on ten, so the total less the shipped bands cannot give back their sum.
    • Regions. The regional shares (which sum to one) follow the same rule, and so does their shipped frame total.

    The ratios read the shipped counts, so they go with them. The toy-pool test now expects the total withheld. New tests cover the review's two probes: a single band on five sources, and bands on 40, 4 and 3 sources.

    The frozen build's high-value homes evidence still gets a disclosure check before any gate report goes public.

Minor

  • uk-data#529's validation-only switch. R6 adds a note to local_contract.hmrc.employment_income.count: uk-data#529 made the count validation-only in create_datasets.py, outside the inventory's scope, and Microcosm still binds it under hmrc_spi_frame_model_proxy. The LFS concern points to it. The body lists the ruling owed on UK: track the defects uk-data is fixing that microcosm still has #1095.
  • The SPI tape's HMRC condition. It is now in the baselines tool's caller obligations (R3). Whether frame outputs count as analyses of the tape is María's SPI 9422 ruling, listed in the body as owed.
  • The tail floor counted rows. It now counts distinct source households (R4). The stage passes source_household_id, which the SPI support channel writes before the LCFS stage runs.
  • Differencing the two tail shares. Both fractions now come from one ordering (R4). A top-10% share is "<10" when its households beyond a shipped top 1% rest on one to nine sources. A skewed-weights test covers it.
  • The OBR leg's row selection. R1 selects by concept, England, fiscal-year period type, year and basis, and requires every band plus the total, once each.
    • Otherwise. The leg reports no_matching_rows, incomplete_rows or malformed_rows and never raises.
    • The share. It is read for the leg's year where the rows state one, else the note's one year (2028-29), and the leg names that year.
    • A share it cannot read. The leg reports why and keeps the band comparison.
  • COMMITTED_BINDING_NOW asserted at import. R5 drops it.
    • The date. COMMITTED_BINDING_NOW is still the register's latest approval.
    • What the review checks. On that date it is a build's review: a lapsed fence a fixture binds fails, and a dormant one does not.
  • CI's signal that the local adjudications lapse. R5 adds test_no_committed_local_adjudication_is_about_to_lapse. Following the holds' clock, it fails fourteen days before the earliest local expiry.
  • Northern Ireland's comparability caveat. R7 adds one sentence on Northern Ireland's periods and comparability to the desnz_subnational_consumption_basis adjudication and its census fence. María signed off the rewording on 17 October; the dates are unchanged.
    • Sources. DESNZ's two Northern Ireland publications are now family sources, verified 17 October.
    • The electricity note. It now says Northern Ireland is published separately and enters the factor only.
  • The module docstring. Rewritten in R7, along with the census's verified-on comment.
  • Commit D's "no classification changes". The fragment now says the re-extraction changes no classification and one is corrected by hand. D's commit message is reworded the same way.
  • J1's message. Reworded: the draw keeps the share per car, but the frame stores one flag per household. R4 adds the share of vehicles in fuel-free households to the receipt, beside the per-car share.
  • Pension-age Housing Benefit called "ported". R6 states the difference in the concern's evidence. uk-data applies the pension-age regulations, which policyengine-uk 2.132.6 encodes as housing_benefit_pension_age_regulations_apply. Microcosm's four rows filter on the deprecated eldest_adult_age alone. Re-binding moves national targets, so it goes with UK spine: housing benefit £2.7bn against OBR £12.1bn — pensioner and supported-accommodation HB missing after the UC migration #867 (PR 4) rather than here, and the body says so.
  • The owner-occupied ratio equals the private-sector one. Dropped (R1), with a comment saying why: the stock build sets the owner-occupied share from the same OBR share.
  • The Disclosure section's wording. The body now says "no survey-derived figures".
  • Item A's hash and the dense solve's key. R1 moves the high-value homes code out of the full-gates kernel's hash. The calibrated node now carries that code's source hash in its parameters (terminal_code), and the kernel checks it before running.

Nits

Cross-PR

Verification

Local, since this is the first CI run on main. See the body for the counts:

  • the touched-surface funnel;
  • the generator checks;
  • ci_test_plan.py verify and graph_acceptance_burndown.py --verify;
  • ruff.

The receipt and membership were regenerated on a cut of the pinned abf0921 artifact (digest checked against the pin), not the whole feed: both generators hold the whole file in memory. The cut keeps the constituency and local-authority rows that match a local contract selector. No local reference reads operands outside its own selector, so the cut holds every candidate. Every row other than the 694 energy cells came out byte-identical to the committed full-feed outputs.

🤖 Generated with Claude Code

juaristi22 added a commit that referenced this pull request Oct 17, 2026
On main after #1302: 25,327 compiled cells. The 18 Scottish BRMA rows are
ledger-only by construction, and no signed value moves. The UC drift
rationales now read their fixture/ours bands off the rows, and the support
floor's deferred cells carry their own signed rationale. The release-blocking
preflight gate uk_ledger_compile_parity_local_incumbent_2025 reads this
receipt, so whichever of #1305, #1306, #1307 and #1308 lands last
regenerates it on the combined tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
The energy module's docstring said consumption is calendar 2024 kWh. Since
the abf0921 re-pin the gas rows are DESNZ's gas year 2024, which the
sub-national lookups pair with calendar 2024 (SUBNATIONAL_PERIOD_TYPES);
electricity stays on calendar years. Docstring only.

The other wording the re-review names in this module (the module docstring's
account of the Northern Ireland interim, UK_ENERGY_LEVEL_BASIS and the
interim text in the household-share receipt) is left as it is: the two
stacked changes that retire the interim, #1305 and #1306, rewrite those same
lines, and UK_ENERGY_LEVEL_BASIS is stamped into each kWh cell's metadata, so
it moves with their regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Re-review of this PR at 567d383

Run with Claude Code on 17 October 2026, after the rebase onto main (post #1302), the seven fix commits (R1–R7) and the response. Three independent reviewers re-checked every first-round finding and every claim in the response, attacked the new code with synthetic probes, checked the new public text against the UK Data Service rules, and simulated the merges with #1305, #1307 and #1308. No builds and no feed-gated tests were run.

Verdict

Every first-round blocker and should-fix item is fixed, and the reviewers verified each one in code. CI is 10/10 green at this head.

  • Regenerated receipt. It moves exactly the 694 DESNZ kWh rows, on ledger_value only, at uniform per-fuel factors of 0.99901 and 1.01582. Every other row is byte-identical to main's.
  • Membership. It moves the same 694 cells' hold factors and resolved values. The references are byte-identical, and the gate replay passes with 0 failures.
  • The feed cut was complete for this head. No local reference reads facts outside its own selector.
  • terminal_code is sound. It composes with UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301) #1307's live head.

What is left is low.

Low

  • The fuel-flag share is floored on the wrong side.
    • Where: lcfs_consumption.py:2561-2575.
    • flagged_share_among_vehicle_households is floored on the flagged (large) count. So with 1–9 fuel-free vehicle households it ships, and 1 − it gives back their share, which fuel_free_share_of_vehicles withholds.
    • The raw with_vehicles_share − flagged_share gives the fuel-free share of all households.
    • The floors count rows, including zero-weight rows and copies.
    • This only matters on narrowed or sampled runs.
    • Fix: pass _source_households in, floor both vehicle shares on the smaller of the flagged and fuel-free distinct positive-weight source counts, floor or drop the raw pair, and correct the "<10" wording in the body, the J1 fragment and the docstring.
  • The top-10% share's complement is not floored.
    • Where: :836-847, :981-995.
    • Each division's frame and donor totals ship beside the shares, so total × (1 − s10) is the spend outside the top 10%.
    • The floor counts top-set rows, zero spenders included. So when 1–9 households outside the top 10% spend anything, the share still ships. A synthetic probe recovered 5 spenders' spend exactly.
    • It reaches committed files only through docs/evidence/*/scripts/stage_dev_run.py.
    • Fix: also withhold when the nonzero contributors outside the top set number 1–9.
  • The distinct-source fix is untested and fails open.
    • No test passes sources through division_capture or the stage, so deleting either wiring line leaves every test green.
    • _source_households returns None when the column is missing, and the tails then count rows silently.
    • The real spine does carry source_household_id by stage 24.
    • Fix: a test with copies, and fail closed when copy columns exist without the id.
  • terminal_code re-reads its source files at run time (graph_terminal.py:785).
    • Editing or checking out high_value_home_gates.py or high_value_homes.py in the build's checkout during a long build aborts uk.full.gates.calibrated after the solve. A restart re-runs only the gates.
    • Memoise the hash per process, or say so in the runbook.
  • Merging with UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301) #1307 needs one hand resolution.
  • terminal_code's scope is narrower than the docstring says.
  • "Never raises" isn't quite true for the OBR leg. A matching row with a null, missing or non-numeric value raises, and so does a non-mapping period or dimensions (high_value_home_gates.py:173, :195, :213, :243). NaN or negative counts pass as "available". Low risk, since the rows come from the vendoring tool.

Nits

Regeneration note for #1305

The feed cut used for R7 (constituency and local-authority rows matching a local selector) was complete at this head. #1305 adds 18 ons.rent.private_rent_brma references at geography level statistical_scope, so whoever regenerates after #1305 must widen the cut.

First-round dispositions

  • Fixed: every first-round finding not listed below as partly fixed (c01 to c03 included), and the cross-PR items on this PR's side.
  • Partly fixed:
    • 1306-09 (pending the SPI 9422 ruling);
    • -12 ("never raises");
    • -25 (the fuel-flag floor).

Cross-PR

Disclosure check

These parts of the new public text are clean:

  • R1–R7's messages and the reworded J1 and D messages;
  • the nine fragments, the body and the response;
  • the 694 changed rows of the receipt and membership (DESNZ times a factor; no text);
  • the synthetic test probes.

Tests and tools run

  • At 567d383: 672 tests, 0 failures, 7 skips, plus 158 more, 0 failures.
  • tools/ci_test_plan.py verify, the census and parity --checks, the coverage manifest --check and tools/graph_acceptance_burndown.py --verify are all current or ok. Ruff is clean on the 23 edited files.
  • Gate replay: 0 failures.
  • CI: 10/10 green, engine-uk included.

🤖 Generated with Claude Code

juaristi22 and others added 19 commits October 17, 2026 21:50
The benchmarks gate's OBR leg looked for observed_measure.source_concept
and layout.record_set_spec_id, but uk/obr_hvcts_costing_facts.json
carries a top-level concept and unit, layout.record_set_id, and the band
and behaviour basis as dimensions. Every row was skipped, so the leg
reported awaiting_vendored_facts on every build. It now selects rows by
concept, fiscal year, behaviour basis and England, and takes the band
from property_value_band (the group-by value for the total). A matching
row with an unknown band or unit, or a repeated band, is refused.

The non-owner-occupied share comes from the note's own row (40 percent),
not from a stock-facts parameter that was never set. So the leg now
reports the owner-occupied sensitivity, the derived stock's ratio to it,
and the frame's England total beside it. Rows that match no requested
year or basis report no_matching_rows. The leg stays informational. The
full-gates kernel now hashes both high-value modules, so a resumed store
recomputes the evidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The high-value homes family and its two report-only gates measure
owner-occupiers' main residences. Second homes, let property and homes
owned by companies or non-residents fall outside it, and the OBR costing
note puts the properties in scope that are not owner-occupied at 40 per
cent. Until the coverage decision of #1240 fix 6, every reading of these
figures now names that scope: HIGH_VALUE_HOME_SCOPE travels in the
evidence summary and in the details of both gates, and the family's
census description says the same (uk_local_target_census.json
regenerated). The target labels and the gates.json notes stay as they
are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three consumption-tax measure exclusions (the two ONS energy rows and
obr.fuel_duties_cars) still named #1207 in their tracking field and in
their retire condition. Nineteen uprating holds named #1123 (16) or #1207
(3). Both issues closed into #1278, so every one now tracks #1278
(María's ruling of 2026-10-16). approved_on, expires_on and the
adjudications are unchanged, so nothing is re-signed. The SLC hold keeps
#868.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#796 had asked since #609 whether the SPI usage agreement or the FRS terms
set a minimum cell of 30 rather than 10. Neither does. The UK Data Service
catalogue entries for the FRS 2024-25 (SN 9563) and the SPI 2022-23
Public Use Tape (SN 9422) were read on 2026-10-16. Those for the LCFS,
ETB, WAS and NTS studies were read on 2026-10-14. Only the SPI tape adds
conditions to the End User Licence: no attempt to attribute a record,
and analyses of the tape go to HMRC at least ten working days before
distribution or publication. CD171's working floor of ten stands.

The receipt-count module, the floor's canonical statement, now records
this. The weighted-integrity baselines tool no longer tells its reader to
check for a threshold of 30. No floor value changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fuel-buyer flag drew one Bernoulli per vehicle-owning household at
the fuel-buyer share of licensed cars (one minus the battery-electric
and fuel-cell share of DfT VEH1103). So a household with several
vehicles was all one or the other, and multi-car households were flagged
as zero-emission as often as one-car households.

Each of a household's vehicles is now zero-emission at that share,
independently. A household with n vehicles buys fuel unless all n are
zero-emission: it is flagged at 1 - (1 - rate) ** n, against the same
identity-keyed uniform. One-vehicle households keep their flag. Each
car is drawn zero-emission at the VEH1103 share, but the frame stores
one flag per household, so a zero-emission car beside a petrol or diesel
one keeps its household's fuel spend. Who holds those cars by income is
still random; a later change conditions it.

The declaration's rate_rule becomes per_car_zero_emission_share in the
stage spec, the rendered stage roster and the sources schema, and the
stage notes say so. The fuel-flag receipt adds the flagged share among
vehicle households. The release input coverage manifest and the H2
spine fixture are regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1278)

prepare_uk_full_solve called require_adjudicated_uk_local_binding with no
clock, so every engine-free test that solves a bound local surface
reviewed the committed register on today's date. The six local
adjudications renewed under #1278 lapse on 2027-01-14. Seventeen tests
would have warned from 2027-01-07 and failed from 2027-01-15, whatever
the code did: rowwise, target-weight wiring, holdout and CGT observation
period.

prepare_uk_full_solve, solve_uk_rowwise_weights_under_doctrine and
rotated_uk_local_holdout take an optional now and pass it on; builds and
the graph kernels keep reviewing on today's date. The tests review on
COMMITTED_BINDING_NOW, the register's latest approval, asserted to fall
before its earliest expiry, as the national graph tests' review date
does. An entry outside its window still fails them. Two new tests check
that a solve and every holdout fold refuse a date past the windows, and
the solve's receipt records the date it reviewed on.

Checked by forcing the default review date to 2027-01-20: the 17 tests
fail before this change and none after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The report-only division capture already gave each COICOP division's
frame total beside its donor's. Furnishings and health on the 2023-24
diary, and recreation and education on the 2024-25 one, drew above the
donor, but the receipt could not say whether the excess came from a few
heavy spenders or from the body of the distribution.

Each division now also records the share of its spend held by the top
1% and the top 10% of households, on the frame at prior weights and on
the donor at its design weights, and the frame's share over the
donor's. Households are taken by spend until they carry the fraction of
the weight. A share carried by fewer than ten households is "<10", as
every receipt count is. Nothing is gated. The stage's code changes, so
the spine re-keys; the H2 fixture and the coverage manifest do not move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The register still compared Microcosm with uk-data as it stood on 1
September (8629dbb). uk-data's main is now a43e8c5 (15 October), 136
commits on. Re-extracted there, the inventory has 57 modules. obr.py,
hmrc_salary_sacrifice.py and tests/test_target_registry.py changed, and
three sources are new. The re-extraction changes no existing concern's
classification; one is corrected by hand (below).

New concerns:
- dwp_housing_benefit.py (uk-data#490) splits DWP's Housing Benefit
  line by age. The pension-age half is ported: Microcosm binds
  dwp.hb.amount_pension_age and the pension-age caseloads. The
  working-age half, net of supported and temporary accommodation, is
  routed to #867 and fenced against binding the full line.
- dwp_pension_credit.py (uk-data#510) is ported: obr.pension_credit
  (stamped Great Britain by #1302) and the dwp.pension_credit rows.
- ons_labour_market.py (uk-data#529) is a blocked source: the feed has
  no LFS employment series.

The five obr.py line citations follow the functions that moved. Two
verdicts are corrected. Band D entered uk-data's LA loss only as the
multiplier in England's housing/council_tax_net proxy, never as a
column. The council-tax outlier guard is a ported compile contract: each
authority's cells compile from its own row, and the local compile-parity
preflight pins every one. uk-data#482 and #484 are still open; a second
re-pin follows when they merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local-authority energy rows are levelled onto the LCFS stage's Energy
Trends level, a United Kingdom total. DESNZ's sub-national figures cover
Great Britain, so Great Britain's part of the level was taken at its
share of UK households (ONS LFS), an interim fenced until Chronicle
carried Northern Ireland's consumption. abf0921 (#1302) brought those
rows, and María ruled on 16 October that the change stacked on the
re-pin reads the share from them.

Great Britain's part of the level is now its share of DESNZ's
sub-national domestic consumption with Northern Ireland's row added (all
domestic meters, 2024), so each fuel's factor is the UK level over the
UK sub-national total. Northern Ireland's electricity runs on financial
years from April and its gas on its own gas years. Both labelled 2024,
they pair with Great Britain's 2024 as Great Britain's gas year pairs
with its calendar-year electricity. DESNZ calls the Northern Ireland
figures not directly comparable with Great Britain's, and the basis text
and the receipt say so.

desnz_local_energy_facts.json now vendors the two Northern Ireland rows
in place of the four LFS household totals; its 697 meter rows are
unchanged. The resource was cut with the tool's own vendor_resource from
a grep superset of the pinned artifact's lines, under #1302's pin. The
desnz_northern_ireland_interim_share adjudication and its census fence
retire, and the two energy contract rows' notes say how Northern Ireland
is taken out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1278)

Review of microcosm#1306, items A and B.

The band counts suppressed thin bands but always shipped the total, and
added a complementary band only when exactly one was suppressed. The
total is now withheld when it rests on fewer than ten distinct source
households, and further bands are withheld until the withheld ones rest
together on ten, so the total less the shipped bands cannot give back
their sum. The regional shares, which sum to one, and their frame total
follow the same rule. The ratios read the shipped counts.

The OBR leg selects rows by concept, England, fiscal-year period type,
year and basis, and needs every band and the total once. Otherwise it
reports no_matching_rows, incomplete_rows or malformed_rows rather than
raising in the terminal phase after the solve. It reads the share for
its own year where the rows state one, else the note's one year, and
names that year. The owner-occupied ratio is dropped: the stock build
sets the owner-occupied share of England's 2m+ stock from the same OBR
share, so it repeated the private-sector ratio.

The full-gates kernel no longer hashes the high-value homes modules. The
source preflight runs that kernel and the dense solve keys on the
preflight, so an edit to report-only code re-solved. The calibrated node
carries their source hash as terminal_code instead, checked before the
kernel runs.

The coverage decision is cited as tracked on #1278, which #1240 closed
into: the scope note, its comment, the census family text and B's
fragment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of microcosm#1306, item C. The salary-sacrifice reliefs' open
ruling and the fuel-duty cars roll-forward still named closed #1123 in
their reasons, though their tracking fields already said #1278; only the
two reasons change.

The two plan-2 student-loan borrower exclusions tracked microcosm#868,
which closed on 2026-10-16. María ruled on 2026-10-17 that they move to
#1278: their tracking fields change and their adjudications record the
move. The plan-2 and plan-5 uprating hold keeps #868. Approvals and
windows are unchanged, so nothing is re-signed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of microcosm#1306, item G. Only the tool's docstring had changed:
its default's comment, the --sdc-minimum-count help and the caller
obligations written into every receipt still asked whether a study
requires 30. All three now say that no study the build reads sets a
minimum above ten. The obligations add the SPI 2022-23 tape's two
conditions: no attempt to attribute a record, and material analysing it
goes to HMRC at least ten working days before distribution.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rencing (#1278)

Review of microcosm#1306, items J1 and H.

The division tails' "<10" floor counted frame rows, but the frame stacks
copies of one source household (the SPI support channel, the high-value
home split). It now counts distinct source_household_id values, which
the stage passes in; the donor keeps one row per LCFS household. Both
fractions now come from one ordering. The top 1% leads the top 10%, so
their difference is the spend of the households between them, and a
top-10% share whose households beyond a shipped top 1% rest on one to
nine sources is withheld.

The fuel-flag receipt's share among vehicle households now carries the
floor. It adds the share of vehicles in fuel-free households: the flag
is per household, so a zero-emission car beside a petrol or diesel one
keeps its household's fuel, and that share sits below the per-car share
the draw uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

Review of microcosm#1306, item E.

COMMITTED_BINDING_NOW asserted at import that the register's latest
approval preceded its earliest expiry. A partial renewal (six entries
renewed, a dormant fence left to lapse) would have broken the collection
of every module importing it, though a build's review passes on that
register; the strict comparison was also a day early. It now asserts
nothing: on that date the review is a build's, so a lapsed fence a
fixture binds fails and a dormant one does not.

Pinning the date took away CI's warning that the adjudications lapse.
test_no_committed_local_adjudication_is_about_to_lapse fails fourteen
days before the earliest local expiry, as the uprating holds' clock
does. A holdout test shows all five folds receive the date.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1278)

Review of microcosm#1306, item D.

- Pension-age Housing Benefit stays ported, with the difference stated.
  uk-data applies the pension-age regulations, leaving out units on
  Universal Credit, Income Support, income-based JSA or income-related
  ESA; policyengine-uk 2.132.6 encodes them as
  housing_benefit_pension_age_regulations_apply. Microcosm's four rows
  filter on the deprecated eldest_adult_age alone. Re-binding goes with
  #867.
- The local employment-income count notes that uk-data#529 made it
  validation-only in create_datasets.py, outside the inventory's scope.
  Microcosm still binds it, and whether to follow is a ruling owed on
  #1095. The LFS concern points to it.
- The TV-licence citation reads obr.py:620 (_parse_tv_licence) and
  668-684 (the static rows before get_targets) at a43e8c5.
- D's fragment says the re-extraction changes no classification and one
  is corrected by hand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

Review of microcosm#1306, item I.

The local compile-parity receipt, which the release-blocking preflight
reads, still carried the retired household-share factor on its 694 kWh
rows. Regenerated on abf0921 at this head, exactly those rows move (348
electricity, 346 gas, ledger_value only), each by its fuel's new factor
over the interim's: 0.99901 and 1.01582. The local membership report
moves on the same cells' holds and resolved values. Everything else in
both files, and the local references, is byte-identical to #1302's
full-feed regeneration.

Both generators hold the whole feed in memory, which this machine cannot.
They ran on a cut of the pinned artifact, digest checked against the pin:
the constituency and local-authority rows matching a local contract
selector. No local reference reads operands outside its own selector, so
the cut holds every candidate, as the unchanged rows confirm.

María signed off a rewording of the desnz_subnational_consumption_basis
adjudication and its census fence on 2026-10-17. One sentence on
Northern Ireland's periods and comparability is added, with dates
unchanged, so the caveat stays under an adjudication now the interim's
fence is gone. The census also changes:
- DESNZ's two Northern Ireland publications are family sources,
  verified 2026-10-17.
- The electricity note says Northern Ireland is published separately.
- The module docstring and the verified-on comment no longer describe
  the interim.

I's fragment says why the share stands over the plan's subtraction
(María's ruling of 2026-10-17): the sub-national statistics run about
9% (electricity) and 12% (gas) above Energy Trends, and the share keeps
both nations on that one basis. The re-pin history entry points here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pies (#1278)

Second review of microcosm#1306, items J1 and H.

The fuel-flag receipt floored its share among vehicle households on the
flagged count, so with one to nine fuel-free vehicle households it
shipped and one less it gave back their share; the raw pair differenced
into the same group. Every share on a side now differences into the
households with and without a vehicle, crossed with fuel or none, so
each is "<10" when any non-empty group rests on one to nine distinct
positive-weight sources: the frame's source households, the donor's
diaries.

The division tails counted top-group rows, zero spenders included, and
never floored the complement, though the division's total ships beside
the shares. The floors now count sources that spend anything, and a
share is withheld when one to nine of them sit in its top group, outside
it, or between it and a shipped smaller fraction's group.

The stage computes the source households once and hands them to both
receipts; a frame that marks copies (the SPI support channel's clones,
the high-value home split) without a source_household_id is refused, so
a floor never falls back to rows. A stage test checks both receipts get
them, and division_capture is tested with copies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…process (#1278)

Second review of microcosm#1306, items A and B.

The OBR leg still raised on a matching row with a missing or non-numeric
value, or a period or dimensions field that is not a mapping, and passed
NaN or negative counts as available. Such a row is now reported: a count
must be finite and non-negative, a field that is not a mapping reads as
empty, and an unreadable share leaves the band comparison standing.

uk_terminal_only_code_hash is memoised, so an edit to the high-value
homes files while a build runs cannot fail the calibrated gates after the
solve. Its docstring now says what it buys: an edit to
high_value_home_gates.py never re-solves, but high_value_homes.py also
feeds the target kernels, so an edit there still does. The hash test
covers the preflight kernel #1307 added, which inherits the calibrated
kernel's hash, and the memoisation.

Two comments are corrected: the regional split withholds regions until
the withheld ones rest together on the floor, and its threshold follows
the derived stock while the bands follow OBR's. The evidence summary,
not the gate details, carries the frame total.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

Second review of microcosm#1306, item D. The fragment counted
tests/test_target_registry.py among three changed source files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 6 commits October 17, 2026 22:11
…#1301)

Third review of microcosm#1307. "At about one household weight each" in
the high-value homes small-cell rule could read as a measured count of
frame records. The rule and the changelog fragment now say the rows'
published stock is about one average household weight each at the ruled
150,000 households, an expectation from the stock and not a count of
frame records. The local membership, which copies the rule's reason into
601 rationales, is regenerated; nothing else in it moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Third review of microcosm#1307. The re-pin procedure now tells whoever
regenerates the receipts to run one --surface at a time with nothing
else running, since the tool holds every parsed fact and a run over both
surfaces peaked past a 24 GiB machine on abf0921. It describes the
local-selector cut microcosm#1306 used on a smaller machine, and when
that cut holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Third review of microcosm#1307. The disclosure walk's warnings went to
stderr only: --preflight-only printed {"valid": true, "published":
false} without them, and nothing in the published tag recorded that a
release went out past them.

prepare_release now returns its contract warnings, and --preflight-only
adds them to its verdict, as a real publication adds them to its
printed pointer; a clean verdict is unchanged. The tag of a release
published past warnings says how many there were, never their text, and
no uploaded file carries them. Nothing refuses or asks for an
acknowledgement: the walk warns and never refuses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

Third review of microcosm#1307. The drafter's free-text check accepted
figures dressed as citations (a household and weight beside microcosm
references, uk-data references counting rows, references to made-up
PolicyEngine repositories, counts after anchors, far-off dates and
logins carrying a number), and refused legitimate ones (a lower-case
owner, review anchors, file URLs with digits in the path).

Digits may now stand only in a citation of a repository on a fixed list,
matched in any case: an issue reference, an issue or pull request URL
optionally anchored on a comment or review with GitHub's ten-digit id,
or a file URL. A citation running into a word, or right next to a noun
a figure counts or belongs to, is refused, and an anchor stands only in
a URL. ISO dates must lie within a year of the drafting date. Logins
with digits must be on a fixed list. The sign-off fragment #1307 added
now describes these rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Third review of microcosm#1307.

The disclosure walk now builds its figure names and extreme stems from
one set, so it adds distinct_sources, sample_size, num_rows, row_count,
kish_ess and the source-household names, and catches
min_distinct_source_households again. It also catches:
- an order statistic above a figure ({"min": {"rows": ...}});
- figure maps by or per a level (rows_by_area);
- area_code and geography_id in any case, and any field holding an area
  code (lad_code, geography);
- pandas split tables;
- numbers written with thousands separators.
Region, nation and UK codes still name no area, and the near misses the
second review found (max_excess and its kin) stay unflagged, with tests.
On the real run outputs on this machine the old and new walks flag the
same paths.

The release pre-flight now reads files as the contract does, refusing
NaN and reporting a nesting too deep to walk, and also walks the opt-in
incumbent files, which ship byte for byte (--incumbent-manifest). Tests
drive --candidate-dir to its warnings with a figure planted in every
walked file, the signed report included, and the assembler with a
finding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Third review of microcosm#1307. The runbook now says how to retry build
1: into a new --out, never the failed attempt's, with the same review
date and graph store, and with the failed attempt's Logbook row as the
predecessor, since a second attempt's row in the same spool makes the
drafter, --verify and the assembler refuse the run directory. It also
describes the publisher's verdict, the drafter's citation rules and the
pre-flight's walk as they now stand. The full-build doc says the dense
release contract warns about, rather than refuses, a shipped JSON
document carrying a per-area support figure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-local-surface-1278-b branch from 567d383 to 0f764b1 Compare October 17, 2026 21:53
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Every item of the re-review above is addressed. The branch is rebased onto main after #1307's merge (ff3f348). The first 16 commits are replayed unchanged apart from the four conflicts the re-review named, resolved as it described:

  • The census text. The family keeps both high-value homes sentences, and the census is regenerated.
  • graph_terminal.py. The calibrated node takes {**gate_params, "phase": "terminal", "terminal_code": uk_terminal_only_code_hash()}.
  • test_uk_graph_terminal.py. Both sides' tests are kept.

The receipt and membership auto-merged to the union, 27,249 rows. A regeneration on the combined tree, from a re-cut of the pinned artifact identical to R7's, reproduced the committed receipt, references and membership byte for byte.

Low

  • The fuel-flag floor (R8).

    • Groups. Every share on a side differences into the households with and without a vehicle, crossed with fuel or none. So each share is "<10" when any non-empty group rests on one to nine distinct positive-weight sources: the frame's source households, the donor's diaries.
    • The raw pair. It is floored the same way.
    • Wording. The docstring and J1's fragment say so.
    • Tests. A ten-plus case, a thin-group case on both sides, and copies counted as sources.
  • The top-10% share's complement (R8). The tail floors now count sources that spend anything, so zero rows lend a group nothing. A share is withheld when one to nine spenders sit:

    • in its top group;
    • outside it;
    • or between it and a shipped smaller fraction's group.

    A test covers the five-spenders probe and a top group padded with zero rows.

  • The distinct-source fix (R8).

    • Wiring. The stage computes the source households once and passes them to both receipts. A stage test on the synthetic frame checks that both get them, and division_capture is tested with copies.
    • Failing closed. A frame that marks copies without a source_household_id is refused: the SPI support channel's household_support_clone_index, and the high-value split's household_is_high_value_support_copy or high_value_support_copy_index.
  • terminal_code re-reading its files (R9). The hash is memoised per process, so editing the files during a build cannot fail the calibrated gates after the solve.

  • The merge with UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301) #1307. Resolved as above.

    • Dropped. The "move into the terminal-only kernel's hash" suggestion is gone from the body: the preflight kernel inherits that hash.
    • The hash test. It now covers the preflight kernel too.
  • terminal_code's scope (R9). The docstring, A's fragment and the body now say that only high_value_home_gates.py is solve-free. high_value_homes.py also feeds the target kernels, so a disclosure fix between two builds that must reuse the solve belongs in high_value_home_gates.py alone.

  • "Never raises" (R9). These are now reported as malformed_rows or incomplete_rows:

    • a missing, non-numeric, NaN or negative count;
    • a period, dimensions, layout or geography field that is not a mapping;
    • a row that is not a mapping.

    An unreadable share leaves the band comparison standing. Tests cover each case.

Nits

With #1307's remaining fixes

This PR also carries the remaining fixes from the third look at #1307:

  • the wording of the two below-floor region rows, with the local membership regenerated (N1);
  • the receipt-regeneration memory guidance in the re-pin procedure (L6);
  • the release-tooling items: the verdict JSON, the free-text allowlist, the walk's shapes, the pre-flight gaps, the build-1 retry guidance and the docs wording. See the body.

Regeneration note for #1305

The re-pin procedure now says when the local-selector cut holds, and to widen it for references at other levels, such as #1305's BRMA rows at statistical_scope.

🤖 Generated with Claude Code

@juaristi22
juaristi22 marked this pull request as ready for review October 18, 2026 00:22
@juaristi22
juaristi22 merged commit 58b0f24 into main Oct 18, 2026
10 checks passed
juaristi22 added a commit that referenced this pull request Oct 18, 2026
On main after #1307 and #1306: 27,267 rows, 23,004 of them compiled. The 18
Scottish BRMA rows are ledger-only by construction, and no signed value
moves. The 40 cells of the support floor's two authorities, E06000053 and
E09000001, carry the floor's own signed rationale; nothing else changes.
The release-blocking preflight gate
uk_ledger_compile_parity_local_incumbent_2025 reads this receipt, so
whichever of #1305 and #1308 lands second regenerates it on the combined
tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 19, 2026
On main after #1307 and #1306: 27,267 rows, 23,004 of them compiled. The 18
Scottish BRMA rows are ledger-only by construction, and no signed value
moves. The 40 cells of the support floor's two authorities, E06000053 and
E09000001, carry the floor's own signed rationale; nothing else changes.
The release-blocking preflight gate
uk_ledger_compile_parity_local_incumbent_2025 reads this receipt, so
whichever of #1305 and #1308 lands second regenerates it on the combined
tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 19, 2026
On main after #1307 and #1306: 27,267 rows, 23,004 of them compiled. The 18
Scottish BRMA rows are ledger-only by construction, and no signed value
moves. The 40 cells of the support floor's two authorities, E06000053 and
E09000001, carry the floor's own signed rationale; nothing else changes.
The release-blocking preflight gate
uk_ledger_compile_parity_local_incumbent_2025 reads this receipt, so
whichever of #1305 and #1308 lands second regenerates it on the combined
tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UK spine: SLC plan-2 borrower stocks at a third of SLC's — the FRS flag captures repayers, not the liable stock

1 participant