Skip to content

test: widen the tier guard to catch a SAMPLE_ROOT join via a module-level tuple - #2427

Merged
JarryShaw merged 1 commit into
mainfrom
fix/2426-tier-guard-sample-root-join
Oct 29, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/2426-tier-guard-sample-root-join

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Please follow the guide below

What is the purpose of your pull request?

  • fix
  • feat
  • perf
  • refactor
  • test
  • docs
  • ci
  • release
  • chore

Description

Closes #2426. tests/_tiers.py's audit_module only saw a generated-capture
read as a literal argument to sample_path(...). It missed the indirection
tests/foundation/reassembly/test_datagram_deferred_pickle_2417_unit.py
used on PR #2421's first revision: a module-level CAPTURES tuple read in a
for loop whose body joined SAMPLE_ROOT / name directly, never calling
sample_path() at all. CI's unit-tests leg genuinely failed on that
revision, on every affected Python interpreter, because the capture was not
on disk; it passed only locally, where make_samples.py had already been
run. What this guard's own static pass missed was the violation itself, not
a CI run.

sample_root_join_calls() follows a SAMPLE_ROOT join back to either a
module-level name bound to a single string, or a for loop's target over a
module-level (or inline) tuple/list of strings; audit_module() merges its
findings with the existing sample_path_calls() ones, in source order. It
stays deliberately narrow — the module docstring already documents why a
blanket ".pcap literal" rule was rejected, and this shape doesn't revisit
that: a whole-suite sweep after the change still reports zero findings.

New AuditTests cases in tests/test_tier_guard.py pin both the loop and
direct-join shapes being flagged, and that a committed capture, a
try/except-handled read, and a for loop over sorted(CAPTURES) (a
module-level tuple wrapped in a call, rather than read bare) stay unflagged.
Verified fail-without-fix by reverting the tests/_tiers.py change: the
three "is flagged" tests then fail (0 findings instead of 1-2, plus an
AttributeError for the new function).

@JarryShaw JarryShaw added test Pull requests that add or correct tests (test: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 29, 2026
@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 29, 2026
@github-actions github-actions Bot added this to the PyPCAPKit v1.5 milestone Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Coverage: 91.73% (unit tier, Python 3.15, cb573a16e, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 2 95.16%
pcapkit/const 19609 879 2620 866 92.13%
pcapkit/corekit 2596 74 818 37 96.51%
pcapkit/dumpkit 282 3 102 3 98.44%
pcapkit/foundation 3046 24 1062 26 98.78%
pcapkit/interface 138 4 52 3 96.32%
pcapkit/protocols 17592 193 4678 179 98.32%
pcapkit/toolkit 924 36 334 10 95.55%
pcapkit/utilities 502 7 144 5 98.14%
pcapkit/vendor 4449 1961 1020 167 52.64%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 29, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Review: NEEDS CHANGES at 37c75737f (Sonnet). The code fix itself is correct — no regressions found — but two things need fixing before merge:

  1. False claim in prose: the docstring/commit/PR body say the original fix(foundation): Deferred.__init__'s proto type is too narrow for transport-keyed reassembly #2417 module "ran green with no generated capture on disk" under the old guard. Not true — CI actually failed on that revision across 7 legs (Python 3.11-3.15, pypy3.11, required-checks), and the reviewer's own fresh-tree reproduction confirms it fails without captures. The old guard simply never flagged it at review time (a static-analysis miss), which is different from "ran green." Correct the wording in all four places (tests/_tiers.py docstring, tests/test_tier_guard.py, the commit message, the PR body).
  2. A test that can't fail: test_a_computed_iterable_joined_onto_sample_root_is_left_to_a_human sorts a function parameter, so it passes regardless of whether the resolver widens to follow sorted() — without the fix it only fails via an unrelated AttributeError, not by genuinely testing the "left unresolved" claim. Fix by using a module-level CAPTURES with sorted(CAPTURES) instead, so the test actually exercises what it claims to.

Also flagged, non-blocking: two inaccuracies in the docstring ("the one unit-tier module that touches SAMPLE_ROOT" — test_capture_tracking.py also does; and a counted number, "16", which the project avoids in prose per the no-counted-numbers convention) — worth a drive-by fix in the same pass since you're already touching that docstring. A handful of still-missed shapes (annotated tuples, for name, n in PAIRS, comprehensions, self.CAPTURES) are fine to leave for a future round — the "deliberately narrow" design is sound.

…evel tuple

The AST audit in tests/_tiers.py only recognised a generated-capture read as a
literal argument to tests._support.sample_path(...). It missed the
indirection PR #2421's first revision hit: a module-level CAPTURES tuple read
in a for loop whose body joined SAMPLE_ROOT / name directly, bypassing
sample_path(). The guard's own static pass never flagged it, which is the
defect this fixes -- CI's unit-tests leg did fail on that revision, on every
affected Python interpreter, because the capture genuinely was not on disk;
it only passed locally where make_samples.py had already been run.

- tests/_tiers.py: add sample_root_join_calls(), which follows a SAMPLE_ROOT
  join back to a module-level name bound to a single string, or to a for
  loop's target over a module-level (or inline) tuple/list of strings.
  audit_module() now merges these findings with the existing
  sample_path_calls() ones, in source order. Deliberately narrow, per the
  module docstring's existing rationale for rejecting a blanket ".pcap
  literal" rule: no new false positive across the real suite (verified).
- tests/test_tier_guard.py: new AuditTests cases pin the loop and direct-join
  shapes being flagged, and that a committed capture, a try/except-handled
  read, and a for loop over sorted(CAPTURES) (a module-level tuple wrapped in
  a call, rather than read bare) stay unflagged.

Verified: reverting the tests/_tiers.py change makes the three new
"is flagged" tests fail (0 findings where 1-2 are expected, plus an
AttributeError for the new function); restoring it makes them pass. Full
tests/test_tier_guard.py + test_tier_guard_xdist.py +
test_capture_tracking.py suite green (156 passed, 3 skipped), and a
whole-suite audit_module() sweep over every unit-tier module on disk still
reports zero findings. mypy and pylint report no new findings against either
file.

Closes #2426
@JarryShaw
JarryShaw force-pushed the fix/2426-tier-guard-sample-root-join branch from 37c7573 to cb573a1 Compare October 29, 2026 02:21
@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 29, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Review: GOOD TO GO at cb573a16e. Verified directly (main thread — three consecutive dispatch attempts for this round were compromised by a lingering fork left over from an unrelated earlier task; traced and now contained, see #2426's sibling threads for the detail).

Both requested fixes confirmed:

  1. Prose now correctly says the guard "never flagged the violation" rather than the false "ran green" claim — CI genuinely failed on the fix(foundation): Deferred.__init__'s proto type is too narrow for transport-keyed reassembly #2417 revision, confirmed in the docstring's rewording.
  2. test_a_computed_iterable_joined_onto_sample_root_is_left_to_a_human now uses a module-level CAPTURES tuple wrapped in sorted(CAPTURES), so it genuinely discriminates whether the resolver follows a computed iterable — not a function parameter that would pass either way.

tests/test_tier_guard.py + test_capture_tracking.py: 145 passed, 2507 subtests. A whole-repo audit_module() sweep, correctly scoped via is_unit_tier() (my first attempt over-broadly included tests/integration/, which is exempt by design — corrected), gives 0 findings across all 610 real unit-tier modules. mypy/isort findings on both touched files are identical to main's pre-existing set (confirmed by diffing against a main worktree) — nothing new introduced.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 29, 2026
@JarryShaw
JarryShaw merged commit 98f6000 into main Oct 29, 2026
33 checks passed
@JarryShaw
JarryShaw deleted the fix/2426-tier-guard-sample-root-join branch October 29, 2026 03:28
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 29, 2026
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Pull requests that add or correct tests (test: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

test: tier guard misses capture reads via a module-level CAPTURES tuple

1 participant