Skip to content

fix(foundation): absorb the pyshark crash a reaped tshark causes on Python 3.9–3.12 - #2642

Open
JarryShaw wants to merge 1 commit into
mainfrom
fix/2633-pyshark-reap
Open

JarryShaw wants to merge 1 commit into
mainfrom
fix/2633-pyshark-reap

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • fix — a bug fix

Description of your pull request and other information

On Python 3.9–3.12, extract(engine='pyshark') raised a spurious TSharkCrashException: … retcode: 255. The cause is CPython gh-87744: asyncio's kill() goes through Popen.poll(), which reaps an already-exited tshark, so asyncio reports 255. The owner chose option 1 on #2633, so PyShark.close() now absorbs the crash only under a strict version of that option's condition:

  • Python 3.9–3.12;
  • a TSharkCrashException, matched by class;
  • the capture has reached EOF;
  • asyncio reports 255 while the reaping Popen holds the real status, 0.

It then logs at DEBUG, clears the stale processes and finishes pyshark's close.

Requiring the Popen's 0 is stricter than "255 at EOF". pyshark sets EOF after a mid-stream crash too, and a truncated capture with a real exit 14 would otherwise be absorbed; it still raises. One residual case is documented: a genuine exit 255 that the watcher reaps first reads as 0. asyncio's own Unknown child process warning still prints, but the extraction succeeds.

Evidence:

  • A Sonnet cross-review reproduced the fix with real tshark: 3.12 went from 2/100 to 0/100 (14/20 to 0/20 with the kill widened), and 3.9 and 3.11 behave the same way.
  • Every condition was mutation-tested. The fix has no effect on 3.13.

Closes #2633

@JarryShaw JarryShaw added this to the PyPCAPKit v1.6 milestone Nov 2, 2026
@JarryShaw JarryShaw added fix Pull requests that fix a defect (fix: subject prefix) review: good-to-go Cross-review at the current head says ready; CI state is separate labels Nov 2, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at be29223b0. A Sonnet cross-review read CPython 3.8 to 3.12's Popen.send_signal/_internal_poll and asyncio's _process_exited, which confirmed that the reaping Popen keeps the real exit status. It reproduced the fix with real tshark on 3.9, 3.11 and 3.12. It killed every single-condition mutant, and showed that a real failure (exit 14 at EOF) still raises 30 out of 30 times. After the fix absorbs a crash, there is no hang, no leaked task and no change to the hooks.

@JarryShaw JarryShaw moved this to In review in PyPCAPKit Nov 2, 2026
@JarryShaw
JarryShaw force-pushed the fix/2633-pyshark-reap branch from be29223 to 06ae67a Compare November 2, 2026 07:21
@JarryShaw

Copy link
Copy Markdown
Owner Author

Fix round: be29223b0 → 06ae67a50, same base 18544c3c9.

  • Red Project tests: test_the_stubs_cover_every_import_of_them found no stub for pyshark.capture.capture. The new typings/pyshark/capture/capture.pyi declares TSharkCrashException, the one name the engine imports from it. With the stub removed, that test fails on exactly this case.

Verdict stays good to go.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: good-to-go Cross-review at the current head says ready; CI state is separate labels Nov 2, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at 06ae67a50: CI on Python 3.15 found a regression.

A fix round is in progress. CI on this head is cancelled until then.

…o lost (#2633)

- On Python 3.9-3.12 asyncio's Process.kill polls through Popen.send_signal,
  reaping a tshark that exited before its SIGCHLD was handled; the child
  watcher reports 255 and pyshark's close raises TSharkCrashException
  (CPython gh-87744, fixed in 3.13).
- PyShark.close logs that crash at DEBUG instead, only when the capture
  reached EOF and every listed tshark reports 255 to asyncio but 0 to the
  Popen that reaped it. A real failure, a 255 the Popen also read, or any
  other error still raises. The capture is emptied so its __del__ is quiet.
- Stub pyshark.capture.capture, and list the 3.9 floor as UNRELATED in the
  RFC-constants sweep. New unit tests pair each absorbed crash with one that
  still raises, and lose real statuses.

3.12, 4x100 natural + 40 widened extractions: 97 crashes before, 0 after.
@JarryShaw
JarryShaw force-pushed the fix/2633-pyshark-reap branch from 06ae67a to b3346a3 Compare November 2, 2026 08:20
@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Nov 2, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Fix round and rebase: 06ae67a50 → b3346a37c, now on 16ccc4373. The interdiff is one entry in the constants sweep test, and the gate itself is unchanged.

  • The sweep has no special case for version gates, so a sys.version_info comparison containing 9 always hits the shared constant RESOLUTION_PLACES. The entry goes in UNRELATED, next to PyPCAPFile's (3, 12) ceiling, following that precedent.
  • I checked it myself: the sweep passes on the new head (14 passed), and the worker showed it fails without the entry.

GOOD TO GO at b3346a37c.

@JarryShaw

Copy link
Copy Markdown
Owner Author

The red Python 3.15 leg is main's, not this PR's: its only failure is the re-export test, which #2641 fixes. The constants sweep passes. The verdict stays good to go.

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Coverage: 94.08% -- tests failure, so this is partial (unit tier, Python 3.15, b3346a37c, Unit Tests run failure)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 2 95.16%
pcapkit/const 20561 177 2832 158 98.53%
pcapkit/corekit 2697 74 854 37 96.65%
pcapkit/dumpkit 281 3 100 3 98.43%
pcapkit/foundation 3495 27 1216 32 98.75%
pcapkit/interface 138 4 52 3 96.32%
pcapkit/protocols 21416 218 5650 196 98.46%
pcapkit/toolkit 961 28 342 10 96.32%
pcapkit/utilities 512 7 144 5 98.17%
pcapkit/vendor 5256 2260 1198 194 53.61%

Per-file detail: the coverage-html artifact of this run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Pull requests that fix a defect (fix: subject prefix) review: good-to-go Cross-review at the current head says ready; CI state is separate

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

fix(foundation): the pyshark engine raises a spurious TSharkCrashException on Python 3.9–3.12

1 participant