From a9db8a4c71d71cf9487ec3d19a254175b09329dd Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Fri, 30 Oct 2026 21:57:09 -0400 Subject: [PATCH 01/11] chore(docs): centralise changelog entries for unreleased versions Pull requests for user-visible changes each adding a bullet to the unreleased version's changelog file conflict on every merge. CONTRIBUTING.md now says the maintainer writes those entries centrally, and that such PRs leave the file out of the diff. The PR template's changelog box and the generator that annotates it follow the same rule. Refs #2449 --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- CONTRIBUTING.md | 12 +++++++++--- tests/project/test_python_versions_update_1660.py | 4 +++- util/update_python_versions.py | 2 +- 4 files changed, 14 insertions(+), 6 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index f6ff3f0bb1..ed810c099a 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -7,7 +7,7 @@ - [ ] [Searched](https://github.com/JarryShaw/PyPCAPKit/pulls?q=is%3Apr) for similar pull requests - [ ] Followed the [coding style](https://github.com/JarryShaw/PyPCAPKit/blob/main/CONTRIBUTING.md#coding-style) (`make pylint`, `make mypy`, `make isort`) - [ ] `make test` passes, and a test case covers the change -- [ ] Added a [changelog entry](https://github.com/JarryShaw/PyPCAPKit/blob/main/CONTRIBUTING.md#changelog-entries) under `docs/source/changelog/` and regenerated `CHANGELOG.md`, if the change is user-visible +- [ ] Described the user-visible change below and left its [changelog entry](https://github.com/JarryShaw/PyPCAPKit/blob/main/CONTRIBUTING.md#changelog-entries) to the maintainer, keeping `docs/source/changelog/` and `CHANGELOG.md` out of the diff, if the change is user-visible ### What is the purpose of your *pull request*? diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 24d47a992b..91c421a11f 100755 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -25,7 +25,8 @@ A rough outline: - Create a topic branch, usually from `main`. - Make commits of logical units, and add a test case if the change fixes a bug or adds functionality. - Run the tests and make sure they pass (see below). -- Add a changelog entry if the change is user-visible (see below). +- If the change is user-visible, describe it in the pull request and leave its changelog entry to + the maintainer (see below). - Make sure your commit messages are in the proper format (see below). - Push your changes to a topic branch in your fork of the repository. - Submit a pull request to the repo. @@ -58,10 +59,15 @@ shapes of it at collection time and say what to do; `tests/_tiers.py` has the re Entries live in `docs/source/changelog/.rst`, one file per version, listed newest-first in the toctree of `docs/source/changelog.rst`. That tree is the single source of the project's history. +**Pull requests for user-visible changes do not add their own bullet.** While a version is +unreleased, its entry is written centrally by the maintainer once a change is ready to merge or has +merged. Pull requests that each edit the same file conflict on every merge, so leave +`docs/source/changelog/.rst` and `CHANGELOG.md` out of the diff. + **`CHANGELOG.md` is generated — never edit it by hand.** `util/changelog_md.py` derives it from the newest entry alone, because its consumers (the `Create Release` workflow's release body and the -source distribution) read Markdown rather than reStructuredText. Add your bullet to the `.rst` -entry, then regenerate: +source distribution) read Markdown rather than reStructuredText. The maintainer adds the bullet to +the `.rst` entry, then regenerates: ```shell python util/changelog_md.py # rewrite CHANGELOG.md from the newest entry diff --git a/tests/project/test_python_versions_update_1660.py b/tests/project/test_python_versions_update_1660.py index 7e42138870..9a86e3911a 100644 --- a/tests/project/test_python_versions_update_1660.py +++ b/tests/project/test_python_versions_update_1660.py @@ -876,7 +876,9 @@ def test_the_body_lists_what_the_owner_does_by_hand(self) -> None: body = module.fill_template((ROOT / module.TEMPLATE).read_text(encoding='utf-8'), description, 'failure') self.assertIn('- [x] `ci` — workflows or build tooling', body) self.assertEqual(body.count('- [x]'), 1) - self.assertIn('regenerated `CHANGELOG.md`, if the change is user-visible — N/A — centralised in #657', body) + self.assertIn('- [ ] Described the user-visible change below and left its [changelog entry]', body) + self.assertIn('keeping `docs/source/changelog/` and `CHANGELOG.md` out of the diff, ' + 'if the change is user-visible — N/A — centralised in #657', body) self.assertIn('### Description of your *pull request* and other information', body) self.assertNotIn('  Explanation of your', body) self.assertIn(f'| `3.11` | supported | KTLO | {plan["changes"][0]["reason"]} |', body) diff --git a/util/update_python_versions.py b/util/update_python_versions.py index f654da8321..52a65635fe 100644 --- a/util/update_python_versions.py +++ b/util/update_python_versions.py @@ -1298,7 +1298,7 @@ def fill_template(template: 'str', description: 'str', audit_outcome: 'str') -> text = re.sub(r'(?m)^- \[ \] `ci`', '- [x] `ci`', template) audit = {'success': 'passed', 'failure': 'failed'}.get(audit_outcome, audit_outcome) text = re.sub(r'(?m)^(- \[ \] `make test` passes.*)$', rf'\1 -- not run; the audit {audit} (below)', text) - text = re.sub(r'(?m)^(- \[ \] Added a \[changelog entry\].*)$', r'\1 — N/A — centralised in #657', text) + text = re.sub(r'(?m)^(- \[ \] Described the user-visible change.*)$', r'\1 — N/A — centralised in #657', text) placeholder = re.search(r'(?m)^  Explanation of your \*pull request\*.*$', text) if placeholder is None: return text.rstrip('\n') + '\n\n' + description + '\n' From db01d1b2f6e9213d5df1af941725b956facf83e5 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 01:34:39 -0500 Subject: [PATCH 02/11] docs(changelog): consolidate the 1.6.0 entries (#2449) - add entries for the user-visible changes merged since 1.5.1 that had none: LinuxSLL, ICMP/ICMPv6, IGMP, TLS/DTLS, SSLv2, DTLS reassembly, IP_Pseudo, the IPv6 extension-header walk and the IPv6_Ext id() and membership fixes - add the Radiotap entry from #2509 - drop the FRAD, Netlink and GRE entries' stale "remain Raw" and "tracked separately" notes, and cite each entry's own PR and issue - record the centralised-changelog rule this PR introduces - regenerate CHANGELOG.md; util/changelog_md.py --check passes --- CHANGELOG.md | 32 ++++- docs/source/changelog/1.6.0.rst | 206 +++++++++++++++++++++++++------- 2 files changed, 191 insertions(+), 47 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d88b4c1dad..81155f3efb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,12 +1,36 @@ ## 1.6.0 -- unreleased -Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. +Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, and stricter handling of a malformed IPv6 extension header. -- **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)). `LinuxSLL`/ `LinuxSLL1`/ `LinuxSLL2` dispatch to it for both the `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` cases -- the same GRE framing either way, the "IP"/"IP6" distinction being only which outer IP version the tunnel carries -- instead of falling back to `Raw`. The Protocol Type field dispatches onward as an EtherType, through the same family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL already share. One of [#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)'s five `arphrd_type` follow-ups to get a dissector (`NETLINK` and `FRAD` are tracked separately on [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507) and [#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)); `IEEE80211_RADIOTAP` remains `Raw`. +- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)). The hardware types are `ARPHRD`, generated from a hand-maintained table of `` in `pcapkit.vendor.linux.arphrd.ARPHRD` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). -- **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's own kernel/userspace IPC transport (the kernel's own `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)). `LinuxSLL`/ `LinuxSLL1`/ `LinuxSLL2` dispatch to it for the `ARPHRD_NETLINK` case instead of falling back to `Raw`. The byte order the header was written in is ambiguous -- Netlink is kernel/userspace IPC, not wire format -- and is guessed the way Wireshark's own netlink dissector does, by `nlmsg_byteorder()`. Per-family message payloads (routing table entries and the like) and multiple messages packed into one capture are out of scope for this first cut; every message's payload is kept as `Raw`, unconditionally. One of [#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)'s five `arphrd_type` follow-ups to get a dissector (`FRAD` is tracked separately on [#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)); `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` remain `Raw`. +- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`. -- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)). `LinuxSLL`/ `LinuxSLL1`/ `LinuxSLL2` dispatch to it for the `ARPHRD_FRAD` case instead of falling back to `Raw`; the direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others. The first of [#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)'s five `arphrd_type` follow-ups to get a dissector; the other four (`NETLINK`, `IPGRE`/`IP6GRE`, `IEEE80211_RADIOTAP`) remain `Raw`. +- **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's kernel/userspace IPC transport (the kernel's `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)), which `LinuxSLL` dispatches to for `ARPHRD_NETLINK` ([#2500](https://github.com/JarryShaw/PyPCAPKit/issues/2500), [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507)). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; `nlmsg_byteorder()` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: its per-family payload, and any further message in the same capture, are kept as `Raw`. + +- **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)). The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. + +- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded, and the 802.11 MAC frame behind them, which has no dissector in this package yet, as `Raw`. + +- **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. The checksum is recorded, never verified or computed ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470)). ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. + +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3` on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type and length ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's `id()` is its own name alone ([#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. + +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). + +- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a record TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`. Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. + +- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in: given to the `DTLS` dissector as `reassembly=`, with the 4-tuple as `flow=`, it lets the completing fragment's message decode, for the first record of each datagram only. `Extractor` does not drive it ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496)). + +- **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk, raises `ProtocolError`. It is not a layer on the wire. + +- **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. + +- **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). + +- **Fixed** -- selecting `'IPv6_Ext'` matched none of the dedicated extension headers: `IPv6_Ext.id()` now lists its direct concrete subclasses -- HOPOPT, IPv6_Route, IPv6_Frag, ESP, AH, IPv6_Opts, MH and HIP -- each of which keeps its own name alone, as AH and ESP do under IPsec ([#2462](https://github.com/JarryShaw/PyPCAPKit/issues/2462), [#2464](https://github.com/JarryShaw/PyPCAPKit/pull/2464)). + +- **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). --- diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 2b5b603f2a..9d75e50f25 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -3,52 +3,172 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had -no dissector at all. +no dissector at all. Alongside it come the Linux cooked-capture headers and a +dissector for each ``arphrd_type`` they do not read as an EtherType, the IP +pseudo-header, and stricter handling of a malformed IPv6 extension header. + +* **Added** -- :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1` and + :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2`, dissectors for the + Linux "cooked" capture headers ``tcpdump -i any`` writes, under + ``LINKTYPE_LINUX_SLL`` (113) and ``LINKTYPE_LINUX_SLL2`` (276), on a + :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` family base; PCAP and + PCAP-NG frames of either link type dispatch to them (:issue:`2438`, + :pr:`2443`). ``'LinuxSLL'`` selects either version, and each version's own + name selects that version alone. ``protocol_type`` dispatches onward as an + EtherType, except under the ``arphrd_type`` values that carry none -- + ``FRAD``, ``NETLINK``, ``IPGRE``/``IP6GRE`` and ``IEEE80211_RADIOTAP`` -- + each of which has a dissector of its own, below (:issue:`2460`). The + hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated + from a hand-maintained table of ```` in + :class:`pcapkit.vendor.linux.arphrd.ARPHRD` (:issue:`2495`, :pr:`2497`). + +* **Added** -- :class:`~pcapkit.protocols.link.frad.FRAD`, a dissector for Frame + Relay's multiprotocol encapsulation (:rfc:`2427`, obsoleting :rfc:`1490`), + which :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for + ``ARPHRD_FRAD`` (:pr:`2499`). The direct NLPID and SNAP-encapsulated forms + both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every + Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or + SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. + +* **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for + the fixed 16-octet ``nlmsghdr`` framing of Linux's kernel/userspace IPC + transport (the kernel's ``netlink(7)`` man page; + :rfc:`3549#section-2.3.2`), which + :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for + ``ARPHRD_NETLINK`` (:issue:`2500`, :pr:`2507`). Netlink is IPC rather than a + wire format, so the header's byte order is ambiguous; + :func:`~pcapkit.protocols.link.netlink.nlmsg_byteorder` guesses it the way + Wireshark's netlink dissector does. Only the first message's header is + dissected: its per-family payload, and any further message in the same + capture, are kept as :class:`~pcapkit.protocols.misc.raw.Raw`. * **Added** -- :class:`~pcapkit.protocols.link.gre.GRE`, a dissector for the Generic Routing Encapsulation base header (:rfc:`2784`) plus the optional Key - and Sequence Number extensions (:rfc:`2890`). - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`/ - :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1`/ - :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2` dispatch to it for both - the ``ARPHRD_IPGRE`` and ``ARPHRD_IP6GRE`` cases -- the same GRE framing either - way, the "IP"/"IP6" distinction being only which outer IP version the tunnel - carries -- instead of falling back to - :class:`~pcapkit.protocols.misc.raw.Raw`. The Protocol Type field dispatches - onward as an EtherType, through the same family registry + and Sequence Number extensions (:rfc:`2890`), which + :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for both + ``ARPHRD_IPGRE`` and ``ARPHRD_IP6GRE`` -- the same GRE framing either way, + the two differing only in the tunnel's outer IP version (:issue:`2501`, + :pr:`2508`). The Protocol Type field dispatches onward as an EtherType, + through the family registry (:attr:`Link.__proto__ `) - Ethernet, VLAN and LinuxSLL already share. One of :issue:`2460`'s five - ``arphrd_type`` follow-ups to get a dissector (``NETLINK`` and ``FRAD`` are - tracked separately on :pr:`2507` and :pr:`2499`); ``IEEE80211_RADIOTAP`` - remains :class:`~pcapkit.protocols.misc.raw.Raw`. + Ethernet, VLAN and LinuxSLL share. A non-zero Version, :rfc:`2637`'s + Enhanced GRE, is kept as :class:`~pcapkit.protocols.misc.raw.Raw` rather than + misread. -* **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for - the fixed 16-octet ``nlmsghdr`` framing of Linux's own kernel/userspace IPC - transport (the kernel's own ``netlink(7)`` man page; :rfc:`3549#section-2.3.2`). - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`/ - :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1`/ - :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2` dispatch to it for the - ``ARPHRD_NETLINK`` case instead of falling back to - :class:`~pcapkit.protocols.misc.raw.Raw`. The byte order the header was written - in is ambiguous -- Netlink is kernel/userspace IPC, not wire format -- and is - guessed the way Wireshark's own netlink dissector does, by - :func:`~pcapkit.protocols.link.netlink.nlmsg_byteorder`. Per-family message - payloads (routing table entries and the like) and multiple messages packed into - one capture are out of scope for this first cut; every message's payload is kept - as :class:`~pcapkit.protocols.misc.raw.Raw`, unconditionally. One of - :issue:`2460`'s five ``arphrd_type`` follow-ups to get a dissector (``FRAD`` is - tracked separately on :pr:`2499`); ``IPGRE``/``IP6GRE`` and - ``IEEE80211_RADIOTAP`` remain :class:`~pcapkit.protocols.misc.raw.Raw`. +* **Added** -- :class:`~pcapkit.protocols.link.radiotap.Radiotap`, a dissector + for the radiotap header () capture tools prepend to + an 802.11 frame, which :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` + dispatches to for ``ARPHRD_IEEE80211_RADIOTAP`` (:issue:`2502`, :pr:`2509`). + It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) + and records every present-flags word, following bit 31 from one word to the + next. The optional fields those words select are kept whole rather than + decoded, and the 802.11 MAC frame behind them, which has no dissector in this + package yet, as :class:`~pcapkit.protocols.misc.raw.Raw`. -* **Added** -- :class:`~pcapkit.protocols.link.frad.FRAD`, a dissector for Frame - Relay's multiprotocol encapsulation (:rfc:`2427`, obsoleting :rfc:`1490`). - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`/ - :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1`/ - :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2` dispatch to it for the - ``ARPHRD_FRAD`` case instead of falling back to - :class:`~pcapkit.protocols.misc.raw.Raw`; the direct NLPID and SNAP-encapsulated - forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every - Ethernet/VLAN/LinuxSLL frame shares) ARP among others. The first of :issue:`2460`'s - five ``arphrd_type`` follow-ups to get a dissector; the other four - (``NETLINK``, ``IPGRE``/``IP6GRE``, ``IEEE80211_RADIOTAP``) remain - :class:`~pcapkit.protocols.misc.raw.Raw`. +* **Added** -- :class:`~pcapkit.protocols.internet.icmpv4.ICMPv4` (:rfc:`792`) + and :class:`~pcapkit.protocols.internet.icmpv6.ICMPv6` (:rfc:`4443`), on an + abstract :class:`~pcapkit.protocols.internet.icmp.ICMP` base, for IP + protocols 1 and 58, which were kept as + :class:`~pcapkit.protocols.misc.raw.Raw` (:issue:`2436`, :pr:`2446`); a + protochain names ``ICMPv6`` where it named ``IPv6_ICMP``. Echo, Destination + Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and + ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD + included, keeps its body raw. The checksum is recorded, never verified or + computed (:issue:`2470`). ICMPv4 takes the plain ``ICMP`` of :rfc:`792` as + its ``id()`` and alias. + :func:`~pcapkit.foundation.registry.protocols.register_icmp_type` registers a + message-body parser. The type and code registries are + :mod:`pcapkit.const.icmp` and :mod:`pcapkit.const.icmpv6`, crawled from IANA. + +* **Added** -- IGMPv1 (:rfc:`1112`), IGMPv2 (:rfc:`2236`) and IGMPv3 + (:rfc:`3376`), as :class:`IGMPv1 `, + :class:`IGMPv2 ` and + :class:`IGMPv3 ` on an abstract + :class:`~pcapkit.protocols.internet.igmp.IGMP` base bound at IP protocol 2, + which picks the version from the message type and length (:issue:`2439`, + :issue:`2448`, :pr:`2444`, :pr:`2465`). An IGMPv3 Query carries its source + list and a Membership Report its group records, and an IGMPv3 Query's Max + Resp Code and QQIC are decoded from their floating-point form. Any other + message, a 9- to 11-octet Query included, is kept as + :class:`~pcapkit.protocols.misc.raw.Raw`. IGMPv1 takes the plain ``IGMP`` of + :rfc:`1112` as its ``id()`` and alias, and IGMPv2's ``id()`` is its own name + alone (:pr:`2506`). The registries are :mod:`pcapkit.const.igmp`: the types + crawled from IANA, the group record types from a hand-maintained table. + +* **Added** -- :class:`~pcapkit.protocols.application.tls.TLS` (SSLv3 and TLS + 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.DTLS` + (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract + :class:`~pcapkit.protocols.application.ssl.SSL` base (:issue:`2437`, + :pr:`2441`). Each reads every version of its record layer and reports it + through ``alias``, as :class:`~pcapkit.protocols.link.arp.ARP` does by + operation, telling 1.3 apart by its ``supported_versions`` extension; + ``id()`` stays ``('TLS',)`` or ``('DTLS',)``. A cleartext handshake record is + decoded -- ClientHello and ServerHello down to their cipher suites, + compression methods and extension types, any other message at type and + length -- as are an alert and ChangeCipherSpec, while encrypted records stay + opaque octets. :class:`~pcapkit.protocols.application.tls.SSLv3` exists for + explicit construction only. A record split across TCP segments is not + reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also + reads and builds DTLS 1.3's unified-header records + (:rfc:`9147#section-4.1`), except one carrying a connection ID, which stays + Raw as an :rfc:`9146` ``tls12_cid`` record does (:issue:`2467`, + :pr:`2494`). The content and handshake types are :mod:`pcapkit.const.tls`, + crawled from IANA (:issue:`2495`, :pr:`2503`). + +* **Added** -- :class:`~pcapkit.protocols.application.sslv2.SSLv2`, a dissector + for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares + nothing with TLS's (:issue:`2457`, :pr:`2466`). It is bound to no port: on + TCP 443 a record TLS rejects is tried as SSLv2, and kept only as a complete + CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else + stays :class:`~pcapkit.protocols.misc.raw.Raw`. Its constants are + :mod:`pcapkit.const.sslv2`, from a hand-maintained table. + +* **Added** -- :class:`pcapkit.foundation.reassembly.dtls.DTLS`, exported as + ``DTLS_Reassembly``, which reassembles a DTLS handshake message fragmented + across records (:rfc:`6347#section-4.2.3`), keyed by the UDP 4-tuple and the + epoch, accepting overlapping fragments and holding a buffer until it + completes (:issue:`2440`, :pr:`2451`). It is opt-in: given to the + :class:`~pcapkit.protocols.application.dtls.DTLS` dissector as + ``reassembly=``, with the 4-tuple as ``flow=``, it lets the completing + fragment's message decode, for the first record of each datagram only. + :class:`~pcapkit.foundation.extraction.Extractor` does not drive it + (:issue:`2496`). + +* **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, + the IPv4 (:rfc:`793`) or IPv6 (:rfc:`8200#section-8.1`) pseudo-header an + upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit + values, or both (:issue:`2471`, :pr:`2479`). From a layer it takes the final + destination of a source route or of a Routing header with segments left, a + Home Address option's address as the IPv6 source, and the upper-layer + length, from UDP's Length or a jumbogram's Jumbo Payload Length where those + apply; a fragment of a larger packet, or a header it cannot walk, raises + :exc:`~pcapkit.utilities.exceptions.ProtocolError`. It is not a layer on the + wire. + +* **Changed** -- **a breaking change to** the IPv6 extension-header walk: a + known extension header whose dedicated parser raises is now the upper layer, + parsed once as :class:`~pcapkit.protocols.misc.raw.Raw`, and the walk stops + there, as :rfc:`8200#section-4` has a node discard such a packet + (:issue:`2477`, :pr:`2480`). It used to fall back to the generic + :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` and walk on; + ``IPv6_Ext`` now parses only Shim6, which has no dedicated parser, and + ``IPv6.__generic_ext_codes__``, the headers that fell back, is removed. + +* **Changed** -- a pull request no longer adds its own changelog bullet: an + unreleased version's entries are written centrally by the maintainer, and + ``CONTRIBUTING.md`` and the pull request template say so (:issue:`2449`, + :pr:`2483`). + +* **Fixed** -- selecting ``'IPv6_Ext'`` matched none of the dedicated + extension headers: + :meth:`IPv6_Ext.id() ` now + lists its direct concrete subclasses -- HOPOPT, IPv6_Route, IPv6_Frag, ESP, + AH, IPv6_Opts, MH and HIP -- each of which keeps its own name alone, as AH + and ESP do under IPsec (:issue:`2462`, :pr:`2464`). + +* **Fixed** -- a membership test such as ``'ESP' in layer`` on an extension + header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, + IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised :exc:`AttributeError`. + The layer now holds :class:`~pcapkit.protocols.misc.null.NoPayload`, the + upper layer belonging to the enclosing IPv6 header (:issue:`2484`, + :issue:`2487`, :pr:`2486`, :pr:`2488`). From 4b93dc819dcdc837f416f73d059197eca67b1370 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 02:22:17 -0500 Subject: [PATCH 03/11] docs(changelog): fold the IGMP id() fix and review wording into 1.6.0 (#2449) - IGMP: add IGMPv3.id() ('IGMPv3',) and the base's ('IGMP', 'IGMPv2', 'IGMPv3') from #2518; version selection is by message type, except a Membership Query, split by length and, at 8 octets, by Max Resp Time. - LinuxSLL: ARPHRD names only the values its crawler's table lists; any other, ARPHRD_ETHER included, reads as Unassigned. - IPv6 extension-header walk: the library no longer passes IPv6_Ext.read() an error=, so IPv6_Ext.error is None unless a caller supplies one; the Raw layer's info.error holds the failure. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 6 +++--- docs/source/changelog/1.6.0.rst | 38 ++++++++++++++++++++++----------- 2 files changed, 28 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 81155f3efb..58cccd4ef9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, and stricter handling of a malformed IPv6 extension header. -- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)). The hardware types are `ARPHRD`, generated from a hand-maintained table of `` in `pcapkit.vendor.linux.arphrd.ARPHRD` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). +- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). - **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`. @@ -14,7 +14,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. The checksum is recorded, never verified or computed ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470)). ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3` on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type and length ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's `id()` is its own name alone ([#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3` on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. - **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). @@ -24,7 +24,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk, raises `ProtocolError`. It is not a layer on the wire. -- **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. +- **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. - **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 9d75e50f25..1260398f0b 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -18,9 +18,11 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. EtherType, except under the ``arphrd_type`` values that carry none -- ``FRAD``, ``NETLINK``, ``IPGRE``/``IP6GRE`` and ``IEEE80211_RADIOTAP`` -- each of which has a dissector of its own, below (:issue:`2460`). The - hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated - from a hand-maintained table of ```` in - :class:`pcapkit.vendor.linux.arphrd.ARPHRD` (:issue:`2495`, :pr:`2497`). + hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated by + :class:`pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It + names only the ```` values that table lists, so any other, + ``ARPHRD_ETHER`` included, reads as ``Unassigned`` (:issue:`2495`, + :pr:`2497`). * **Added** -- :class:`~pcapkit.protocols.link.frad.FRAD`, a dissector for Frame Relay's multiprotocol encapsulation (:rfc:`2427`, obsoleting :rfc:`1490`), @@ -85,15 +87,19 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. :class:`IGMPv2 ` and :class:`IGMPv3 ` on an abstract :class:`~pcapkit.protocols.internet.igmp.IGMP` base bound at IP protocol 2, - which picks the version from the message type and length (:issue:`2439`, - :issue:`2448`, :pr:`2444`, :pr:`2465`). An IGMPv3 Query carries its source - list and a Membership Report its group records, and an IGMPv3 Query's Max - Resp Code and QQIC are decoded from their floating-point form. Any other - message, a 9- to 11-octet Query included, is kept as - :class:`~pcapkit.protocols.misc.raw.Raw`. IGMPv1 takes the plain ``IGMP`` of - :rfc:`1112` as its ``id()`` and alias, and IGMPv2's ``id()`` is its own name - alone (:pr:`2506`). The registries are :mod:`pcapkit.const.igmp`: the types - crawled from IANA, the group record types from a hand-maintained table. + which picks the version from the message type alone except for a Membership + Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 + otherwise, one of 12 or more IGMPv3 (:issue:`2439`, :issue:`2448`, + :pr:`2444`, :pr:`2465`). An IGMPv3 Query carries its source list and a + Membership Report its group records, and an IGMPv3 Query's Max Resp Code and + QQIC are decoded from their floating-point form. Any other message, a 9- to + 11-octet Query included, is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. + IGMPv1 takes the plain ``IGMP`` of :rfc:`1112` as its ``id()`` and alias, and + IGMPv2's and IGMPv3's ``id()`` are their own names alone, ``('IGMPv2',)`` and + ``('IGMPv3',)``; the base's is ``('IGMP', 'IGMPv2', 'IGMPv3')``, each + member's canonical name (:issue:`2512`, :pr:`2506`, :pr:`2518`). The + registries are :mod:`pcapkit.const.igmp`: the types crawled from IANA, the + group record types from a hand-maintained table. * **Added** -- :class:`~pcapkit.protocols.application.tls.TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.DTLS` @@ -152,7 +158,13 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. (:issue:`2477`, :pr:`2480`). It used to fall back to the generic :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` and walk on; ``IPv6_Ext`` now parses only Shim6, which has no dedicated parser, and - ``IPv6.__generic_ext_codes__``, the headers that fell back, is removed. + ``IPv6.__generic_ext_codes__``, the headers that fell back, is removed. The + library no longer passes + :meth:`IPv6_Ext.read() ` + an ``error=``, so + :attr:`IPv6_Ext.error ` + is :data:`None` unless a caller supplies one; the ``Raw`` layer's + ``info.error`` holds the failure instead. * **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and From ccedecf79d6984ca7a7de531aff11b4ed9b4183d Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 12:35:45 -0500 Subject: [PATCH 04/11] docs(changelog): fold main's latest merge wave into 1.6.0 (#2449) - Added: one's-complement checksum verification and computation (#2543), and the IPX and SCTP registration wrappers (#2521). - Folded into unreleased entries: LinuxSLL/FRAD/Netlink registries (#2527), the IGMP class names (#2542), TLS record chaining and the SSLv2 trial's warnings (#2537), DTLS reassembly via Extractor and toolkit (#2528), and IP_Pseudo's walk at any depth (#2543). ICMP drops its stale checksum line. - Breaking: make() checksums, DRARP's module (#2540), IPv6 overlapping fragments (#2569), TCP_GAP_LIMIT replacing TCP.__window__ (#2579), and Extractor/extract's positional dtls (#2528). - Changed: member order (#2490), conventions (#2542, #2521, #2570), CI -n logical (#2572). Fixed: AppType.get (#2568), trial warnings (#2519, #2577). - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 46 ++++++-- docs/source/changelog/1.6.0.rst | 186 +++++++++++++++++++++++++++----- 2 files changed, 194 insertions(+), 38 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 58cccd4ef9..7240a87efb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,33 +1,59 @@ ## 1.6.0 -- unreleased -Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, and stricter handling of a malformed IPv6 extension header. +Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, verified and computed one's-complement checksums, and stricter handling of a malformed IPv6 extension header. -- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). +- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). -- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`. +- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). -- **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's kernel/userspace IPC transport (the kernel's `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)), which `LinuxSLL` dispatches to for `ARPHRD_NETLINK` ([#2500](https://github.com/JarryShaw/PyPCAPKit/issues/2500), [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507)). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; `nlmsg_byteorder()` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: its per-family payload, and any further message in the same capture, are kept as `Raw`. +- **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's kernel/userspace IPC transport (the kernel's `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)), which `LinuxSLL` dispatches to for `ARPHRD_NETLINK` ([#2500](https://github.com/JarryShaw/PyPCAPKit/issues/2500), [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507)). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; `nlmsg_byteorder()` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: what follows it, its per-family payload and any further message in the same capture, is kept as `Raw` unless `register_netlink()` registers a dissector for its `nlmsg_type` ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). - **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)). The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. - **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded, and the 802.11 MAC frame behind them, which has no dissector in this package yet, as `Raw`. -- **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. The checksum is recorded, never verified or computed ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470)). ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. +- **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3` on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). -- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a record TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`. Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. +- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. -- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in: given to the `DTLS` dissector as `reassembly=`, with the 4-tuple as `flow=`, it lets the completing fragment's message decode, for the first record of each datagram only. `Extractor` does not drive it ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496)). +- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). -- **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk, raises `ProtocolError`. It is not a layer on the wire. +- **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. + +- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.protocols.misc.checksum.checksum` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. + +- **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. +- **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. + +- **Changed** -- **a breaking change to** `DRARP`: it moves from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp`, with no re-export at the old path, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540)). `pcapkit.DRARP`, `pcapkit.protocols.DRARP` and a parsed RARP's `DRARP` alias are unchanged. + +- **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as `Completion.OVERLAP`, as [RFC 8200 Section 4.5](https://datatracker.ietf.org/doc/html/rfc8200#section-4.5) requires; 1.5.x overwrote the octets it overlapped, as IPv4 still does under [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) ([#2547](https://github.com/JarryShaw/PyPCAPKit/issues/2547), [#2569](https://github.com/JarryShaw/PyPCAPKit/pull/2569)). `Buffer` gains a required `extent` field. + +- **Changed** -- **a breaking change to** TCP reassembly's gap cap: `pcapkit.foundation.reassembly.tcp.TCP.__window__` is removed with no alias, and the cap is `TCP_GAP_LIMIT`, set by `PCAPKIT_TCP_GAP_LIMIT`, 16 MiB by default as before and `-1` for no limit ([#2546](https://github.com/JarryShaw/PyPCAPKit/issues/2546), [#2579](https://github.com/JarryShaw/PyPCAPKit/pull/2579)). It is a safety limit, read each time `TCP.reassembly` runs, and `-1` or a value above the default warns with `ConfigWarning`. + +- **Changed** -- **a breaking change to** `Extractor` and `extract()`: their `dtls` parameter, above, follows `tcp`, so a positional argument from `buffer_size` on moves one place along, and `ReassemblyManager` takes a required `dtls` ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). + +- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541)). + +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). + +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). + - **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). +- **Fixed** -- `AppType.get` takes the `default` `EnumLookup.get` advertises, a registered member's value that it never mints, and a miss raises `EnumValueError`, still a `ValueError`, where it raised a builtin one ([#2545](https://github.com/JarryShaw/PyPCAPKit/issues/2545), [#2568](https://github.com/JarryShaw/PyPCAPKit/pull/2568)). + +- **Fixed** -- under `beholder`, a next layer captured at full length whose parse raised, such as a malformed TCP header, reported its warnings before being replaced with `Raw`. Its trial now holds them however much was captured and drops them with the layer, so a kept layer's warnings follow those of the layers it carries ([#2516](https://github.com/JarryShaw/PyPCAPKit/issues/2516), [#2519](https://github.com/JarryShaw/PyPCAPKit/pull/2519)). + +- **Fixed** -- the PyPCAPFile engine's header checks reported the warnings of a trial parse they rejected. The extension-header trial now drops them as under `beholder`, and the IPv4 and TCP header check holds its trial's and discards them when it raises ([#2526](https://github.com/JarryShaw/PyPCAPKit/issues/2526), [#2577](https://github.com/JarryShaw/PyPCAPKit/pull/2577)). + - **Fixed** -- selecting `'IPv6_Ext'` matched none of the dedicated extension headers: `IPv6_Ext.id()` now lists its direct concrete subclasses -- HOPOPT, IPv6_Route, IPv6_Frag, ESP, AH, IPv6_Opts, MH and HIP -- each of which keeps its own name alone, as AH and ESP do under IPsec ([#2462](https://github.com/JarryShaw/PyPCAPKit/issues/2462), [#2464](https://github.com/JarryShaw/PyPCAPKit/pull/2464)). - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 1260398f0b..8fde3ebe0f 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -5,7 +5,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each ``arphrd_type`` they do not read as an EtherType, the IP -pseudo-header, and stricter handling of a malformed IPv6 extension header. +pseudo-header, verified and computed one's-complement checksums, and stricter +handling of a malformed IPv6 extension header. * **Added** -- :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1` and :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2`, dissectors for the @@ -17,8 +18,11 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. name selects that version alone. ``protocol_type`` dispatches onward as an EtherType, except under the ``arphrd_type`` values that carry none -- ``FRAD``, ``NETLINK``, ``IPGRE``/``IP6GRE`` and ``IEEE80211_RADIOTAP`` -- - each of which has a dissector of its own, below (:issue:`2460`). The - hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated by + each of which has a dissector of its own, below (:issue:`2460`), reached + through ``LinuxSLL``'s own registry keyed by ``arphrd_type``, which + :func:`~pcapkit.foundation.registry.protocols.register_arphrd` extends + (:issue:`2510`, :pr:`2527`). The hardware types are + :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated by :class:`pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the ```` values that table lists, so any other, ``ARPHRD_ETHER`` included, reads as ``Unassigned`` (:issue:`2495`, @@ -30,7 +34,9 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. ``ARPHRD_FRAD`` (:pr:`2499`). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or - SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. + SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, and + :func:`~pcapkit.foundation.registry.protocols.register_frad` registers a + dissector for a further NLPID (:pr:`2527`). * **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for the fixed 16-octet ``nlmsghdr`` framing of Linux's kernel/userspace IPC @@ -41,8 +47,10 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. wire format, so the header's byte order is ambiguous; :func:`~pcapkit.protocols.link.netlink.nlmsg_byteorder` guesses it the way Wireshark's netlink dissector does. Only the first message's header is - dissected: its per-family payload, and any further message in the same - capture, are kept as :class:`~pcapkit.protocols.misc.raw.Raw`. + dissected: what follows it, its per-family payload and any further message + in the same capture, is kept as :class:`~pcapkit.protocols.misc.raw.Raw` + unless :func:`~pcapkit.foundation.registry.protocols.register_netlink` + registers a dissector for its ``nlmsg_type`` (:pr:`2527`). * **Added** -- :class:`~pcapkit.protocols.link.gre.GRE`, a dissector for the Generic Routing Encapsulation base header (:rfc:`2784`) plus the optional Key @@ -75,17 +83,17 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. protochain names ``ICMPv6`` where it named ``IPv6_ICMP``. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD - included, keeps its body raw. The checksum is recorded, never verified or - computed (:issue:`2470`). ICMPv4 takes the plain ``ICMP`` of :rfc:`792` as - its ``id()`` and alias. + included, keeps its body raw. ICMPv4 takes the plain ``ICMP`` of :rfc:`792` + as its ``id()`` and alias. :func:`~pcapkit.foundation.registry.protocols.register_icmp_type` registers a message-body parser. The type and code registries are :mod:`pcapkit.const.icmp` and :mod:`pcapkit.const.icmpv6`, crawled from IANA. * **Added** -- IGMPv1 (:rfc:`1112`), IGMPv2 (:rfc:`2236`) and IGMPv3 - (:rfc:`3376`), as :class:`IGMPv1 `, - :class:`IGMPv2 ` and - :class:`IGMPv3 ` on an abstract + (:rfc:`3376`), as :class:`~pcapkit.protocols.internet.igmpv1.IGMPv1`, + :class:`~pcapkit.protocols.internet.igmpv2.IGMPv2` and + :class:`~pcapkit.protocols.internet.igmpv3.IGMPv3`, each named after its + module (:issue:`2529`, :pr:`2542`), on an abstract :class:`~pcapkit.protocols.internet.igmp.IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 @@ -112,9 +120,13 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay - opaque octets. :class:`~pcapkit.protocols.application.tls.SSLv3` exists for - explicit construction only. A record split across TCP segments is not - reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also + opaque octets. Each whole TLS record in a TCP segment is a layer of its own, + the previous record's payload, so each alias names its own version -- + ``TCP:TLS 1.3:TLS 1.2`` -- and one that fails to decode is + :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment + (:issue:`2517`, :pr:`2537`). :class:`~pcapkit.protocols.application.tls.SSLv3` + exists for explicit construction only. A record split across TCP segments is + not reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also reads and builds DTLS 1.3's unified-header records (:rfc:`9147#section-4.1`), except one carrying a connection ID, which stays Raw as an :rfc:`9146` ``tls12_cid`` record does (:issue:`2467`, @@ -124,21 +136,25 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. * **Added** -- :class:`~pcapkit.protocols.application.sslv2.SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's (:issue:`2457`, :pr:`2466`). It is bound to no port: on - TCP 443 a record TLS rejects is tried as SSLv2, and kept only as a complete - CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else - stays :class:`~pcapkit.protocols.misc.raw.Raw`. Its constants are - :mod:`pcapkit.const.sslv2`, from a hand-maintained table. + TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and + kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields + agree, so anything else stays :class:`~pcapkit.protocols.misc.raw.Raw`; a + kept one drops the rejected TLS trial's warnings (:issue:`2525`, + :pr:`2537`). Its constants are :mod:`pcapkit.const.sslv2`, from a + hand-maintained table. * **Added** -- :class:`pcapkit.foundation.reassembly.dtls.DTLS`, exported as ``DTLS_Reassembly``, which reassembles a DTLS handshake message fragmented across records (:rfc:`6347#section-4.2.3`), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it - completes (:issue:`2440`, :pr:`2451`). It is opt-in: given to the - :class:`~pcapkit.protocols.application.dtls.DTLS` dissector as - ``reassembly=``, with the 4-tuple as ``flow=``, it lets the completing - fragment's message decode, for the first record of each datagram only. - :class:`~pcapkit.foundation.extraction.Extractor` does not drive it - (:issue:`2496`). + completes (:issue:`2440`, :pr:`2451`). It is opt-in, as ``dtls=True`` with + ``reassembly=True`` to :class:`~pcapkit.foundation.extraction.Extractor` or + :func:`~pcapkit.interface.core.extract`, and runs on the default engine + alone, fed by :func:`~pcapkit.toolkit.pcap.dtls_reassembly` and its + :mod:`pcapkit.toolkit.pcapng` counterpart from the first record of each + datagram only; any other engine turns it off with a warning. + :func:`~pcapkit.foundation.analysis.dtls.dtls_handshake` decodes a + reassembled message (:issue:`2496`, :pr:`2528`). * **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, the IPv4 (:rfc:`793`) or IPv6 (:rfc:`8200#section-8.1`) pseudo-header an @@ -147,9 +163,40 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those - apply; a fragment of a larger packet, or a header it cannot walk, raises - :exc:`~pcapkit.utilities.exceptions.ProtocolError`. It is not a layer on the - wire. + apply; a fragment of a larger packet, or a header it cannot walk at any + depth of the chain, raises :exc:`~pcapkit.utilities.exceptions.ProtocolError` + (:issue:`2513`, :pr:`2543`). It is not a layer on the wire. + +* **Added** -- verification and computation of the one's-complement Internet + checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, + in :mod:`pcapkit.protocols.misc.checksum.checksum` (:issue:`2470`, + :issue:`2472`, :issue:`2473`, :issue:`2474`, :issue:`2475`, :pr:`2543`). + Parsing verifies only under ``PCAPKIT_CHECKSUM_VERIFY``, off by default, + recording a :class:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus` + as ``checksum_status`` and warning on a mismatch with + :exc:`~pcapkit.utilities.warnings.ChecksumWarning`, a + :exc:`~pcapkit.utilities.warnings.ProtocolWarning` a filter can silence + alone. Construction computes a checksum the caller leaves out, unless + ``PCAPKIT_CHECKSUM_COMPUTE`` is off. An IPv4 or IPv6 layer verifies and + computes its upper layer's checksum over the + :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo` it builds, and + ``make()`` of TCP, UDP and ICMPv6 also takes a ``pseudo``. What checksum + offload leaves on the sending host is ``NOT_COMPUTED`` rather than a + mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers + octets the fragment lacks, is ``NO_PSEUDO_HEADER``. + +* **Added** -- :func:`~pcapkit.foundation.registry.protocols.register_ipx`, + :func:`~pcapkit.foundation.registry.protocols.register_loopback`, + :func:`~pcapkit.foundation.registry.protocols.register_sctp_chunk`, + :func:`~pcapkit.foundation.registry.protocols.register_sctp_parameter` and + :func:`~pcapkit.foundation.registry.protocols.register_sctp_cause`, wrapping + IPX's packet-type map, Loopback's address-family map and SCTP's chunk, + chunk-parameter and error-cause parser maps with the validation the other + wrappers carry, and the + :meth:`IPX.register ` and + :meth:`Loopback.register ` + classmethods the IPX and Loopback wrappers call (:issue:`2511`, + :pr:`2521`). * **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, @@ -166,11 +213,94 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. is :data:`None` unless a caller supplies one; the ``Raw`` layer's ``info.error`` holds the failure instead. +* **Changed** -- **a breaking change to** construction: a checksum left out of + ``make()`` is computed rather than zero, so the octets built change + (:issue:`2471`, :pr:`2543`). :class:`~pcapkit.protocols.internet.ipv4.IPv4` + computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP + payload built without one, and + :class:`~pcapkit.protocols.application.ospf.OSPF` that of an OSPFv2 packet. + A checksum given is written as given, and turning + ``PCAPKIT_CHECKSUM_COMPUTE`` off restores zero. + +* **Changed** -- **a breaking change to** + :class:`~pcapkit.protocols.application.drarp.DRARP`: it moves from + :mod:`pcapkit.protocols.application.rarp` to + :mod:`pcapkit.protocols.application.drarp`, with no re-export at the old + path, as a module holds one protocol class (:issue:`2529`, :pr:`2540`). + ``pcapkit.DRARP``, ``pcapkit.protocols.DRARP`` and a parsed RARP's + ``DRARP`` alias are unchanged. + +* **Changed** -- **a breaking change to** IPv6 reassembly: a fragment + overlapping another of its datagram, other than an exact duplicate, abandons + the datagram, reported as + :attr:`Completion.OVERLAP `, + as :rfc:`8200#section-4.5` requires; 1.5.x overwrote the octets it overlapped, + as IPv4 still does under :rfc:`791` (:issue:`2547`, :pr:`2569`). + :class:`~pcapkit.foundation.reassembly.data.ip.Buffer` gains a required + ``extent`` field. + +* **Changed** -- **a breaking change to** TCP reassembly's gap cap: + ``pcapkit.foundation.reassembly.tcp.TCP.__window__`` is removed with no + alias, and the cap is :data:`~pcapkit.utilities.config.TCP_GAP_LIMIT`, set + by ``PCAPKIT_TCP_GAP_LIMIT``, 16 MiB by default as before and ``-1`` for no + limit (:issue:`2546`, :pr:`2579`). It is a safety limit, read each time + :meth:`TCP.reassembly ` + runs, and ``-1`` or a value above the default warns with + :exc:`~pcapkit.utilities.warnings.ConfigWarning`. + +* **Changed** -- **a breaking change to** + :class:`~pcapkit.foundation.extraction.Extractor` and + :func:`~pcapkit.interface.core.extract`: their ``dtls`` parameter, above, + follows ``tcp``, so a positional argument from ``buffer_size`` on moves one + place along, and :class:`~pcapkit.foundation.reassembly.ReassemblyManager` + takes a required ``dtls`` (:issue:`2496`, :pr:`2528`). + +* **Changed** -- protocol class bodies are reordered to the member-order + convention, with no change in behaviour, and a project test guards the + order (:issue:`2490`, :pr:`2522`, :pr:`2523`, :pr:`2538`, :pr:`2539`, + :pr:`2541`). + +* **Changed** -- the house conventions record that a protocol class takes its + module's name, one to a module (:issue:`2529`, :pr:`2542`); that a + docstring or prose block carries one ``versionadded`` per version, without + a body that only names the thing added (:issue:`2530`, :pr:`2542`); and that + each class-level registry map has a wrapper in + :mod:`pcapkit.foundation.registry` (:issue:`2511`, :pr:`2521`). ``const`` is + a label added alongside a commit type rather than a type, in the pull + request template and ``CONTRIBUTING.md`` too, and a span the transport + registries assign to a service keeps its bare name, the mint criterion's + exception to code-suffixing (:issue:`2549`, :issue:`2550`, :pr:`2570`). + +* **Changed** -- the CPython unit-test legs run ``pytest -n logical``, a worker + per logical CPU, where ``-n auto`` counted physical cores and ran 2 workers + on the 4-CPU runner; the weekly CI timing report reads the latest Python's + leg too (:issue:`2571`, :pr:`2572`). + * **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and ``CONTRIBUTING.md`` and the pull request template say so (:issue:`2449`, :pr:`2483`). +* **Fixed** -- :meth:`AppType.get ` + takes the ``default`` + :meth:`EnumLookup.get ` advertises, a + registered member's value that it never mints, and a miss raises + :exc:`~pcapkit.utilities.exceptions.EnumValueError`, still a + :exc:`ValueError`, where it raised a builtin one (:issue:`2545`, + :pr:`2568`). + +* **Fixed** -- under ``beholder``, a next layer captured at full length whose + parse raised, such as a malformed TCP header, reported its warnings before + being replaced with :class:`~pcapkit.protocols.misc.raw.Raw`. Its trial now + holds them however much was captured and drops them with the layer, so a + kept layer's warnings follow those of the layers it carries (:issue:`2516`, + :pr:`2519`). + +* **Fixed** -- the PyPCAPFile engine's header checks reported the warnings of + a trial parse they rejected. The extension-header trial now drops them as + under ``beholder``, and the IPv4 and TCP header check holds its trial's and + discards them when it raises (:issue:`2526`, :pr:`2577`). + * **Fixed** -- selecting ``'IPv6_Ext'`` matched none of the dedicated extension headers: :meth:`IPv6_Ext.id() ` now From 521f75040fe4834710a89fb5ef3f0557eeabfec8 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 14:05:31 -0500 Subject: [PATCH 05/11] docs(changelog): fold HTTP dispatch, docs pages and registry conventions into 1.6.0 (#2449) - Breaking: HTTP dispatches to HTTPv1/HTTPv2 on read, make and from_data, and so does a transport payload, whose data key moves from http to httpv1/httpv2; the members and their schema and data classes are renamed from httpv1.HTTP/httpv2.HTTP with no re-export; id() takes the RFC names; unrecognised transport payloads are Raw with no error (#2544). - Changed: conventions record the dispatching base (#2544) and the case-fold exceptions (#2583); a project test guards every shipped module's documentation entry (#2584). Summary names the HTTP change. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 8 +++-- docs/source/changelog/1.6.0.rst | 56 +++++++++++++++++++++++++++++---- 2 files changed, 56 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7240a87efb..c840e26bbf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ ## 1.6.0 -- unreleased -Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, verified and computed one's-complement checksums, and stricter handling of a malformed IPv6 extension header. +Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, verified and computed one's-complement checksums, stricter handling of a malformed IPv6 extension header, and HTTP dispatched to its HTTP/1 and HTTP/2 classes under the RFC names. - **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). @@ -34,6 +34,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `DRARP`: it moves from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp`, with no re-export at the old path, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540)). `pcapkit.DRARP`, `pcapkit.protocols.DRARP` and a parsed RARP's `DRARP` alias are unchanged. +- **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`. + - **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as `Completion.OVERLAP`, as [RFC 8200 Section 4.5](https://datatracker.ietf.org/doc/html/rfc8200#section-4.5) requires; 1.5.x overwrote the octets it overlapped, as IPv4 still does under [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) ([#2547](https://github.com/JarryShaw/PyPCAPKit/issues/2547), [#2569](https://github.com/JarryShaw/PyPCAPKit/pull/2569)). `Buffer` gains a required `extent` field. - **Changed** -- **a breaking change to** TCP reassembly's gap cap: `pcapkit.foundation.reassembly.tcp.TCP.__window__` is removed with no alias, and the cap is `TCP_GAP_LIMIT`, set by `PCAPKIT_TCP_GAP_LIMIT`, 16 MiB by default as before and `-1` for no limit ([#2546](https://github.com/JarryShaw/PyPCAPKit/issues/2546), [#2579](https://github.com/JarryShaw/PyPCAPKit/pull/2579)). It is a safety limit, read each time `TCP.reassembly` runs, and `-1` or a value above the default warns with `ConfigWarning`. @@ -42,7 +44,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). + +- **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). - **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 8fde3ebe0f..bd41bf3277 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -5,8 +5,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each ``arphrd_type`` they do not read as an EtherType, the IP -pseudo-header, verified and computed one's-complement checksums, and stricter -handling of a malformed IPv6 extension header. +pseudo-header, verified and computed one's-complement checksums, stricter +handling of a malformed IPv6 extension header, and HTTP dispatched to its +HTTP/1 and HTTP/2 classes under the RFC names. * **Added** -- :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1` and :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2`, dissectors for the @@ -230,6 +231,34 @@ handling of a malformed IPv6 extension header. ``pcapkit.DRARP``, ``pcapkit.protocols.DRARP`` and a parsed RARP's ``DRARP`` alias are unchanged. +* **Changed** -- **a breaking change to** HTTP: parsing octets through + :class:`~pcapkit.protocols.application.http.HTTP`, building it from fields + and its ``from_data`` give the + :class:`~pcapkit.protocols.application.httpv1.HTTPv1` or + :class:`~pcapkit.protocols.application.httpv2.HTTPv2` member, never an + instance of the base, and so does a TCP or UDP payload on an HTTP port, whose + data sits under ``httpv1`` or ``httpv2`` where it sat under ``http`` + (:issue:`2524`, :pr:`2544`). The members are renamed after their modules, + ``httpv1.HTTP`` and ``httpv2.HTTP`` becoming ``HTTPv1`` and ``HTTPv2`` along + with their schema and data classes, with no re-export at the old paths + (:issue:`2529`); ``pcapkit.HTTPv1``, ``pcapkit.HTTPv2`` and the package + exports of those names are unchanged. ``id()`` takes the RFC names, + ``('HTTP/1',)`` and ``('HTTP/2',)``, and the base's lists both, where they + were ``('HTTP', 'HTTPv1')``, ``('HTTP', 'HTTPv2')`` and + ``('HTTP', 'HTTPv1', 'HTTPv2')``. So a layer no longer compares equal to + ``'HTTP'``, ``'HTTPv1'`` or ``'HTTPv2'``; ``'HTTP'`` still selects either + version, and each version's own name, ``'HTTP/1'`` or ``'HTTPv1'`` say, + selects it alone, where any HTTP layer matched ``'HTTPv1'`` and ``'HTTPv2'`` + both. On the transport path a payload neither version recognises, a bare + connection preface included, is :class:`~pcapkit.protocols.misc.raw.Raw` + with no ``error``, where it carried ``'HTTP: unknown version'`` or + ``'HTTP/2: connection preface with no frame'``; a direct ``HTTP(...)`` on it + still raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`. One that + passes for an HTTP/2 frame header but ``HTTPv2`` refuses carries that + parser's own error on either path, where it was ``'HTTP: unknown version'``. + ``HTTPv2`` itself now parses a payload opening with the connection preface, + kept as ``preface``, and its ``make()`` takes ``preface``. + * **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as @@ -262,14 +291,29 @@ handling of a malformed IPv6 extension header. * **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module (:issue:`2529`, :pr:`2542`); that a - docstring or prose block carries one ``versionadded`` per version, without - a body that only names the thing added (:issue:`2530`, :pr:`2542`); and that - each class-level registry map has a wrapper in + versioned family's base dispatches on every path, so parsing, building and + ``from_data`` give a member rather than the base (:issue:`2524`, + :pr:`2544`); that a docstring or prose block carries one ``versionadded`` + per version, without a body that only names the thing added (:issue:`2530`, + :pr:`2542`); and that each class-level registry map has a wrapper in :mod:`pcapkit.foundation.registry` (:issue:`2511`, :pr:`2521`). ``const`` is a label added alongside a commit type rather than a type, in the pull request template and ``CONTRIBUTING.md`` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's - exception to code-suffixing (:issue:`2549`, :issue:`2550`, :pr:`2570`). + exception to code-suffixing (:issue:`2549`, :issue:`2550`, :pr:`2570`). The + registry conventions record the exceptions to their case-folding rules -- + the crawlers that re-case the names they generate, the ``Method`` + constructor's fold, and ``Command.get`` and ``Method.get`` answering a name + miss with an unregistered member -- and the folds that live outside any + registry (:issue:`2551`, :issue:`2552`, :issue:`2553`, :issue:`2556`, + :issue:`2565`, :issue:`2582`, :pr:`2583`). + +* **Changed** -- a project test fails on a shipped module with no + documentation entry, the placeholder sources under ``NotImplemented/`` + excepted, and the modules this release adds that had none -- + :mod:`pcapkit.const.linux`, the :mod:`pcapkit.vendor.tls` and + :mod:`pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have + one (:issue:`2559`, :pr:`2584`). * **Changed** -- the CPython unit-test legs run ``pytest -n logical``, a worker per logical CPU, where ``-n auto`` counted physical cores and ran 2 workers From 42280b49753a5512819f983a1c32e2a3a639c774 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 21:25:08 -0500 Subject: [PATCH 06/11] docs(changelog): fold DCCP, InARP's move and main's latest wave into 1.6.0 (#2449) - Added: DCCP, its checksum over the IP pseudo-header, register_dccp and register_dccp_option, and the pcapkit.const.dccp registries (#2593). - Folded into unreleased entries: LinuxSLL's stop on its ARPHRD path (#2587) and ARPHRD's __proto_dest__ (#2598), SSLv2 records chained in a segment (#2585), the checksum's move to pcapkit.corekit.checksum.internet (#2586), and IGMPv2's and IGMPv3's versioned name (#2590). - Breaking: InARP moves to pcapkit.protocols.link.inarp with no re-export, joining the DRARP entry (#2596). - Changed: the conventions' new rulings (#2599), the milestone workflow's next-open-milestone lookup (#2591), the PyPy protocols shards (#2604). Fixed: a handled next-layer refusal is quiet under beholder (#2597); the IPX crawlers restore sys.path (#2601). - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 22 +++++--- docs/source/changelog/1.6.0.rst | 96 +++++++++++++++++++++++++++------ 2 files changed, 93 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c840e26bbf..9cceab7760 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, verified and computed one's-complement checksums, stricter handling of a malformed IPv6 extension header, and HTTP dispatched to its HTTP/1 and HTTP/2 classes under the RFC names. -- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). +- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)), as does a protocol class declaring `code=ARPHRD.X`, since `ARPHRD` names `LinuxSLL` as its `__proto_dest__` ([#2533](https://github.com/JarryShaw/PyPCAPKit/issues/2533), [#2598](https://github.com/JarryShaw/PyPCAPKit/pull/2598)). A parse told to stop there -- at `LinuxSLL`, the version itself or the link layer -- keeps the payload `Raw` on either path ([#2532](https://github.com/JarryShaw/PyPCAPKit/issues/2532), [#2587](https://github.com/JarryShaw/PyPCAPKit/pull/2587)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). - **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). @@ -14,17 +14,19 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. + +- **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. - **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). -- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. +- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. - **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. -- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.protocols.misc.checksum.checksum` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. +- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. - **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). @@ -32,7 +34,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. -- **Changed** -- **a breaking change to** `DRARP`: it moves from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp`, with no re-export at the old path, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540)). `pcapkit.DRARP`, `pcapkit.protocols.DRARP` and a parsed RARP's `DRARP` alias are unchanged. +- **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. - **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`. @@ -44,17 +46,19 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). -- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). + +- **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). - **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). - **Fixed** -- `AppType.get` takes the `default` `EnumLookup.get` advertises, a registered member's value that it never mints, and a miss raises `EnumValueError`, still a `ValueError`, where it raised a builtin one ([#2545](https://github.com/JarryShaw/PyPCAPKit/issues/2545), [#2568](https://github.com/JarryShaw/PyPCAPKit/pull/2568)). -- **Fixed** -- under `beholder`, a next layer captured at full length whose parse raised, such as a malformed TCP header, reported its warnings before being replaced with `Raw`. Its trial now holds them however much was captured and drops them with the layer, so a kept layer's warnings follow those of the layers it carries ([#2516](https://github.com/JarryShaw/PyPCAPKit/issues/2516), [#2519](https://github.com/JarryShaw/PyPCAPKit/pull/2519)). +- **Fixed** -- under `beholder`, a next layer captured at full length whose parse raised, such as a malformed TCP header, reported its warnings before being replaced with `Raw`. Its trial now holds them however much was captured and drops them with the layer, so a kept layer's warnings follow those of the layers it carries ([#2516](https://github.com/JarryShaw/PyPCAPKit/issues/2516), [#2519](https://github.com/JarryShaw/PyPCAPKit/pull/2519)). A `ProtocolError` built while `beholder` runs a next layer is now quiet, so a refusal kept as `Raw` is recorded on that layer's `error` alone, where a loud one logged at `CRITICAL` and installed the exception hooks, and `beholder` logged it again at `WARNING`; building the refused layer directly stays loud ([#2588](https://github.com/JarryShaw/PyPCAPKit/issues/2588), [#2597](https://github.com/JarryShaw/PyPCAPKit/pull/2597)). - **Fixed** -- the PyPCAPFile engine's header checks reported the warnings of a trial parse they rejected. The extension-header trial now drops them as under `beholder`, and the IPv4 and TCP header check holds its trial's and discards them when it raises ([#2526](https://github.com/JarryShaw/PyPCAPKit/issues/2526), [#2577](https://github.com/JarryShaw/PyPCAPKit/pull/2577)). @@ -62,6 +66,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). +- **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). + --- Full changelog: diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index bd41bf3277..bcf369df32 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -22,7 +22,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. each of which has a dissector of its own, below (:issue:`2460`), reached through ``LinuxSLL``'s own registry keyed by ``arphrd_type``, which :func:`~pcapkit.foundation.registry.protocols.register_arphrd` extends - (:issue:`2510`, :pr:`2527`). The hardware types are + (:issue:`2510`, :pr:`2527`), as does a protocol class declaring + ``code=ARPHRD.X``, since ``ARPHRD`` names ``LinuxSLL`` as its + ``__proto_dest__`` (:issue:`2533`, :pr:`2598`). A parse told to stop there -- at ``LinuxSLL``, + the version itself or the link layer -- keeps the payload + :class:`~pcapkit.protocols.misc.raw.Raw` on either path (:issue:`2532`, + :pr:`2587`). The hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated by :class:`pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the ```` values that table lists, so any other, @@ -106,9 +111,29 @@ HTTP/1 and HTTP/2 classes under the RFC names. IGMPv1 takes the plain ``IGMP`` of :rfc:`1112` as its ``id()`` and alias, and IGMPv2's and IGMPv3's ``id()`` are their own names alone, ``('IGMPv2',)`` and ``('IGMPv3',)``; the base's is ``('IGMP', 'IGMPv2', 'IGMPv3')``, each - member's canonical name (:issue:`2512`, :pr:`2506`, :pr:`2518`). The - registries are :mod:`pcapkit.const.igmp`: the types crawled from IANA, the - group record types from a hand-maintained table. + member's canonical name (:issue:`2512`, :pr:`2506`, :pr:`2518`). IGMPv2's + and IGMPv3's ``name`` carries the version, + ``'Internet Group Management Protocol version 2'`` say, while IGMPv1's, as + in :rfc:`1112`, does not (:issue:`2558`, :pr:`2590`). The registries are + :mod:`pcapkit.const.igmp`: the types crawled from IANA, the group record + types from a hand-maintained table. + +* **Added** -- :class:`~pcapkit.protocols.transport.dccp.DCCP` (:rfc:`4340`) + for IP protocol 33, which was kept as + :class:`~pcapkit.protocols.misc.raw.Raw` (:issue:`2489`, :pr:`2593`). It + reads and builds the generic header with short and long sequence numbers, + the fields of every packet type, DCCP-Listen (:rfc:`5596`) and the options; + the Data Checksum option's CRC-32c is recorded rather than checked. Its + checksum is verified and computed as TCP's is, below, over the enclosing IP + layer's pseudo-header and the part of the packet its Checksum Coverage + gives (:rfc:`4340#section-9.2`): a coverage running past the packet is + ``INVALID`` and warns, and a ``make()`` that computes the checksum writes a + coverage reaching the end of the packet as 0, as Linux did. + :func:`~pcapkit.foundation.registry.protocols.register_dccp` registers a + port's dissector, and + :func:`~pcapkit.foundation.registry.protocols.register_dccp_option` an + option parser. The registries are :mod:`pcapkit.const.dccp`, crawled from + IANA. * **Added** -- :class:`~pcapkit.protocols.application.tls.TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.DTLS` @@ -141,8 +166,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays :class:`~pcapkit.protocols.misc.raw.Raw`; a kept one drops the rejected TLS trial's warnings (:issue:`2525`, - :pr:`2537`). Its constants are :mod:`pcapkit.const.sslv2`, from a - hand-maintained table. + :pr:`2537`). Each whole record after the first in a TCP segment, whatever + its message type, is a further SSLv2 layer, the previous record's payload, + as with TLS -- a ``CLIENT-HELLO`` then a ``CLIENT-MASTER-KEY`` read + ``SSLv2:SSLv2`` -- and one that fails to decode is + :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment + (:issue:`2567`, :pr:`2585`). Its constants are :mod:`pcapkit.const.sslv2`, + from a hand-maintained table. * **Added** -- :class:`pcapkit.foundation.reassembly.dtls.DTLS`, exported as ``DTLS_Reassembly``, which reassembles a DTLS handshake message fragmented @@ -170,10 +200,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Added** -- verification and computation of the one's-complement Internet checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, - in :mod:`pcapkit.protocols.misc.checksum.checksum` (:issue:`2470`, - :issue:`2472`, :issue:`2473`, :issue:`2474`, :issue:`2475`, :pr:`2543`). + in :mod:`pcapkit.corekit.checksum.internet` (:issue:`2470`, :issue:`2472`, + :issue:`2473`, :issue:`2474`, :issue:`2475`, :pr:`2543`), where + :mod:`pcapkit.corekit.checksum` holds a module per checksum family and + imports nothing from :mod:`pcapkit.protocols` (:issue:`2580`, :pr:`2586`). Parsing verifies only under ``PCAPKIT_CHECKSUM_VERIFY``, off by default, - recording a :class:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus` + recording a :class:`~pcapkit.corekit.checksum.internet.ChecksumStatus` as ``checksum_status`` and warning on a mismatch with :exc:`~pcapkit.utilities.warnings.ChecksumWarning`, a :exc:`~pcapkit.utilities.warnings.ProtocolWarning` a filter can silence @@ -224,12 +256,15 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``PCAPKIT_CHECKSUM_COMPUTE`` off restores zero. * **Changed** -- **a breaking change to** - :class:`~pcapkit.protocols.application.drarp.DRARP`: it moves from + :class:`~pcapkit.protocols.application.drarp.DRARP` and + :class:`~pcapkit.protocols.link.inarp.InARP`, which move from :mod:`pcapkit.protocols.application.rarp` to - :mod:`pcapkit.protocols.application.drarp`, with no re-export at the old - path, as a module holds one protocol class (:issue:`2529`, :pr:`2540`). - ``pcapkit.DRARP``, ``pcapkit.protocols.DRARP`` and a parsed RARP's - ``DRARP`` alias are unchanged. + :mod:`pcapkit.protocols.application.drarp` and from + :mod:`pcapkit.protocols.link.arp` to :mod:`pcapkit.protocols.link.inarp`, + with no re-export at the old paths, as a module holds one protocol class + (:issue:`2529`, :pr:`2540`, :pr:`2596`). ``pcapkit.DRARP``, + ``pcapkit.InARP``, their :mod:`pcapkit.protocols` exports and a parsed + RARP's ``DRARP`` or ARP's ``InARP`` alias are unchanged. * **Changed** -- **a breaking change to** HTTP: parsing octets through :class:`~pcapkit.protocols.application.http.HTTP`, building it from fields @@ -306,7 +341,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. constructor's fold, and ``Command.get`` and ``Method.get`` answering a name miss with an unregistered member -- and the folds that live outside any registry (:issue:`2551`, :issue:`2552`, :issue:`2553`, :issue:`2556`, - :issue:`2565`, :issue:`2582`, :pr:`2583`). + :issue:`2565`, :issue:`2582`, :pr:`2583`). They also record the SSL + family's grouping, and that a family base is bound, and dispatches, only + where the registry code cannot tell its members apart; that a pseudo-header + protocol belongs in :mod:`pcapkit.protocols.misc`; that + :mod:`pcapkit.utilities.compat` holds compatibility implementations and + re-exports only, behind version gates; and how a ruling given in a pull + request's review is cited (:issue:`2557`, :issue:`2563`, :pr:`2599`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -318,7 +359,17 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- the CPython unit-test legs run ``pytest -n logical``, a worker per logical CPU, where ``-n auto`` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's - leg too (:issue:`2571`, :pr:`2572`). + leg too (:issue:`2571`, :pr:`2572`). A PyPy leg runs + ``tests/protocols/internet``, ``tests/protocols/application`` and + ``tests/protocols/misc`` as shards of their own ahead of the rest of + ``tests/protocols``, which run whole outgrew the runner's memory and + stalled at its tail (:issue:`2600`, :pr:`2604`). + +* **Changed** -- once a release's milestone is closed while ``__version__`` + still names that release, the milestone workflow gives an item the lowest + open release milestone above it, compared as numbers so ``v1.10`` follows + ``v1.9``, and a backfill starts above the closed milestone's items + (:issue:`2548`, :pr:`2591`). * **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and @@ -338,7 +389,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. being replaced with :class:`~pcapkit.protocols.misc.raw.Raw`. Its trial now holds them however much was captured and drops them with the layer, so a kept layer's warnings follow those of the layers it carries (:issue:`2516`, - :pr:`2519`). + :pr:`2519`). A :exc:`~pcapkit.utilities.exceptions.ProtocolError` built + while ``beholder`` runs a next layer is now quiet, so a refusal kept as + ``Raw`` is recorded on that layer's ``error`` alone, where a loud one logged + at ``CRITICAL`` and installed the exception hooks, and ``beholder`` logged + it again at ``WARNING``; building the refused layer directly stays loud + (:issue:`2588`, :pr:`2597`). * **Fixed** -- the PyPCAPFile engine's header checks reported the warnings of a trial parse they rejected. The extension-header trial now drops them as @@ -358,3 +414,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. The layer now holds :class:`~pcapkit.protocols.misc.null.NoPayload`, the upper layer belonging to the enclosing IPv6 header (:issue:`2484`, :issue:`2487`, :pr:`2486`, :pr:`2488`). + +* **Fixed** -- importing :mod:`pcapkit.vendor.ipx`, or running its packet or + socket crawler, without ``requests`` installed lost an entry of + :data:`sys.path` on each failed attempt, so a :func:`pkgutil.walk_packages` + retrying it drained :data:`sys.path` until it raised :exc:`IndexError`. The + entry is now restored however the import ends (:issue:`2589`, :pr:`2601`). From 5ac9441beec98b39ebeed4064e6428b1f65d0eb2 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 22:10:06 -0500 Subject: [PATCH 07/11] docs(changelog): fold HTTP/2 frame chaining, SSLv3's module and main's latest wave into 1.6.0 (#2449) - Added: every whole HTTP/2 frame in a TCP segment is a further HTTPv2 layer, the previous frame's payload, as with TLS (#2609). - Folded into unreleased entries: SSLv3 lives in pcapkit.protocols.application.sslv3 (#2602), and a reassembled DTLS Datagram's packet holds the deferred handshake decode (#2605). tls.SSLv3 is in no release tag, so the entry records no move. - Changed: the remaining xdist jobs at -n logical (#2606) join the CI entry, and the member-order entry cites the last reorder (#2613). - No entry for the test-only #2611 and #2612, or for #2610's private registry sentinel, which changes no behaviour. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 10 ++++++---- docs/source/changelog/1.6.0.rst | 29 +++++++++++++++++++++++++---- 2 files changed, 31 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9cceab7760..0f8b4bf26d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,11 +18,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. -- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). +- **Added** -- dissection of every whole HTTP/2 frame in a TCP segment: each after the first, which was kept as `Raw`, is a further `HTTPv2` layer, the previous frame's payload, as with TLS -- a `SETTINGS` then a `WINDOW_UPDATE` frame read `TCP:HTTP/2:HTTP/2` ([#2536](https://github.com/JarryShaw/PyPCAPKit/issues/2536), [#2609](https://github.com/JarryShaw/PyPCAPKit/pull/2609)). One that fails to decode is `Raw` with the rest of the segment and the parser's `error`, and a frame the segment ends inside stays `Raw` with none. A parse told to stop at HTTP/2 keeps the rest of the segment raw, and `FRAME_LAYER_LIMIT` bounds how many frames are dissected. + +- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). A reassembled `Datagram` holds that decode as `packet`, run on its first read as an IP or TCP datagram's is, and `None` for an incomplete message ([#2531](https://github.com/JarryShaw/PyPCAPKit/issues/2531), [#2605](https://github.com/JarryShaw/PyPCAPKit/pull/2605)). - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. @@ -44,13 +46,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `Extractor` and `extract()`: their `dtls` parameter, above, follows `tcp`, so a positional argument from `buffer_size` on moves one place along, and `ReassemblyManager` takes a required `dtls` ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). -- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541)). +- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). - **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). -- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). - **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index bcf369df32..bd9261a003 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -150,7 +150,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. the previous record's payload, so each alias names its own version -- ``TCP:TLS 1.3:TLS 1.2`` -- and one that fails to decode is :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment - (:issue:`2517`, :pr:`2537`). :class:`~pcapkit.protocols.application.tls.SSLv3` + (:issue:`2517`, :pr:`2537`). + :class:`~pcapkit.protocols.application.sslv3.SSLv3`, in + :mod:`pcapkit.protocols.application.sslv3` (:issue:`2529`, :pr:`2602`), exists for explicit construction only. A record split across TCP segments is not reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also reads and builds DTLS 1.3's unified-header records @@ -174,6 +176,19 @@ HTTP/1 and HTTP/2 classes under the RFC names. (:issue:`2567`, :pr:`2585`). Its constants are :mod:`pcapkit.const.sslv2`, from a hand-maintained table. +* **Added** -- dissection of every whole HTTP/2 frame in a TCP segment: each + after the first, which was kept as :class:`~pcapkit.protocols.misc.raw.Raw`, + is a further :class:`~pcapkit.protocols.application.httpv2.HTTPv2` layer, the + previous frame's payload, as with TLS -- a ``SETTINGS`` then a + ``WINDOW_UPDATE`` frame read ``TCP:HTTP/2:HTTP/2`` (:issue:`2536`, + :pr:`2609`). One that fails to decode is + :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment and + the parser's ``error``, and a frame the segment ends inside stays + :class:`~pcapkit.protocols.misc.raw.Raw` with none. A parse told to stop at + HTTP/2 keeps the rest of the segment raw, and + :data:`~pcapkit.utilities.config.FRAME_LAYER_LIMIT` bounds how many frames + are dissected. + * **Added** -- :class:`pcapkit.foundation.reassembly.dtls.DTLS`, exported as ``DTLS_Reassembly``, which reassembles a DTLS handshake message fragmented across records (:rfc:`6347#section-4.2.3`), keyed by the UDP 4-tuple and the @@ -185,7 +200,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. :mod:`pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. :func:`~pcapkit.foundation.analysis.dtls.dtls_handshake` decodes a - reassembled message (:issue:`2496`, :pr:`2528`). + reassembled message (:issue:`2496`, :pr:`2528`). A reassembled + :class:`Datagram ` holds + that decode as ``packet``, run on its first read as an IP or TCP datagram's + is, and :data:`None` for an incomplete message (:issue:`2531`, :pr:`2605`). * **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, the IPv4 (:rfc:`793`) or IPv6 (:rfc:`8200#section-8.1`) pseudo-header an @@ -322,7 +340,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order (:issue:`2490`, :pr:`2522`, :pr:`2523`, :pr:`2538`, :pr:`2539`, - :pr:`2541`). + :pr:`2541`, :pr:`2613`). * **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module (:issue:`2529`, :pr:`2542`); that a @@ -359,7 +377,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- the CPython unit-test legs run ``pytest -n logical``, a worker per logical CPU, where ``-n auto`` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's - leg too (:issue:`2571`, :pr:`2572`). A PyPy leg runs + leg too (:issue:`2571`, :pr:`2572`). The integration, engine, parity, + project-test and gate jobs and the experimental workflow's Windows and + free-threaded 3.14t legs run ``-n logical`` too, and a project test fails + on a bare ``-n auto`` (:issue:`2573`, :pr:`2606`). A PyPy leg runs ``tests/protocols/internet``, ``tests/protocols/application`` and ``tests/protocols/misc`` as shards of their own ahead of the rest of ``tests/protocols``, which run whole outgrew the runner's memory and From ec47f7f552ebc58e3730046e813a366a489d7153 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 22:24:34 -0500 Subject: [PATCH 08/11] docs(changelog): fold TLS extensions and DTLS record chaining into 1.6.0 (#2449) - Folded into the TLS/DTLS entry: the hello and EncryptedExtensions extension lists, their registered body decoders, register_tls_extension and the pcapkit.const.tls extension types (#2595), and each whole DTLS record in a datagram as a further layer (#2603). - The conventions entry records #2611's per-test purge rule; the member-order entry cites #2491. - No entry for #2595's private ProtocolBase value helpers, as for the other private API. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 6 +++--- docs/source/changelog/1.6.0.rst | 32 ++++++++++++++++++++++++-------- 2 files changed, 27 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0f8b4bf26d..4e4fdb7e70 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,7 +18,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. @@ -46,9 +46,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `Extractor` and `extract()`: their `dtls` parameter, above, follows `tcp`, so a positional argument from `buffer_size` on moves one place along, and `ReassemblyManager` takes a required `dtls` ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). -- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). +- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index bd9261a003..1cd6025c62 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -144,9 +144,16 @@ HTTP/1 and HTTP/2 classes under the RFC names. operation, telling 1.3 apart by its ``supported_versions`` extension; ``id()`` stays ``('TLS',)`` or ``('DTLS',)``. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, - compression methods and extension types, any other message at type and - length -- as are an alert and ChangeCipherSpec, while encrypted records stay - opaque octets. Each whole TLS record in a TCP segment is a layer of its own, + compression methods and extensions, EncryptedExtensions down to its + extensions, any other message at type and length -- as are an alert and + ChangeCipherSpec, while encrypted records stay opaque octets. An extension's + body is read and built for ``server_name``, ``supported_groups``, + ``signature_algorithms``, ALPN, ``supported_versions`` and ``key_share``, a + HelloRetryRequest's included, and kept as octets for any other type or for a + body its parser rejects, which logs nothing; + :func:`~pcapkit.foundation.registry.protocols.register_tls_extension` + registers a parser and constructor for a further type (:issue:`2514`, + :pr:`2595`). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- ``TCP:TLS 1.3:TLS 1.2`` -- and one that fails to decode is :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment @@ -158,8 +165,14 @@ HTTP/1 and HTTP/2 classes under the RFC names. reads and builds DTLS 1.3's unified-header records (:rfc:`9147#section-4.1`), except one carrying a connection ID, which stays Raw as an :rfc:`9146` ``tls12_cid`` record does (:issue:`2467`, - :pr:`2494`). The content and handshake types are :mod:`pcapkit.const.tls`, - crawled from IANA (:issue:`2495`, :pr:`2503`). + :pr:`2494`). Each whole DTLS record in a UDP datagram is likewise a layer of + its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read + ``DTLS 1.2:DTLS 1.3`` -- a unified-header record with no length running to + the end of the datagram, and one that fails to decode is + :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the datagram + (:issue:`2535`, :pr:`2603`). The content, handshake and extension types are + :mod:`pcapkit.const.tls`, crawled from IANA (:issue:`2495`, :issue:`2514`, + :pr:`2503`, :pr:`2595`). * **Added** -- :class:`~pcapkit.protocols.application.sslv2.SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares @@ -339,8 +352,8 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the - order (:issue:`2490`, :pr:`2522`, :pr:`2523`, :pr:`2538`, :pr:`2539`, - :pr:`2541`, :pr:`2613`). + order (:issue:`2490`, :pr:`2491`, :pr:`2522`, :pr:`2523`, :pr:`2538`, + :pr:`2539`, :pr:`2541`, :pr:`2613`). * **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module (:issue:`2529`, :pr:`2542`); that a @@ -365,7 +378,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. protocol belongs in :mod:`pcapkit.protocols.misc`; that :mod:`pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull - request's review is cited (:issue:`2557`, :issue:`2563`, :pr:`2599`). + request's review is cited (:issue:`2557`, :issue:`2563`, :pr:`2599`). The + test conventions record that a test class whose tests each need a fresh + import purges in ``setUp``, with a comment saying why, rather than importing + once per class (:issue:`2562`, :pr:`2611`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` From 34f32f4f7f9d8ac06cc60d981274967dfd1e0e9d Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 23:05:33 -0500 Subject: [PATCH 09/11] docs(changelog): fold IEEE 802.11 and the IP/TCP analysis functions into 1.6.0 (#2449) - Added: IEEE802_11, the 802.11 MAC header behind radiotap, with the pcapkit.const.ieee802_11 type and subtype registries (#2607); the Radiotap entry drops its stale "no dissector yet" note and names its own registry and register_radiotap. - Added: ip_datagram and tcp_datagram beside dtls_handshake (#2619). - The conventions entry records #2607's ruling on a paywalled registry's pinned secondary source. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 8 ++++++-- docs/source/changelog/1.6.0.rst | 33 ++++++++++++++++++++++++++++++--- 2 files changed, 36 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4e4fdb7e70..358b12921a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)). The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. -- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded, and the 802.11 MAC frame behind them, which has no dissector in this package yet, as `Raw`. +- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). + +- **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` does not decode the Flags that mark them, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. @@ -26,6 +28,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). A reassembled `Datagram` holds that decode as `packet`, run on its first read as an IP or TCP datagram's is, and `None` for an incomplete message ([#2531](https://github.com/JarryShaw/PyPCAPKit/issues/2531), [#2605](https://github.com/JarryShaw/PyPCAPKit/pull/2605)). +- **Added** -- `ip_datagram()` and `tcp_datagram()`, siblings of `dtls_handshake`, above, which parse the payload of a datagram that IPv4, IPv6 or TCP reassembly reports, by its payload protocol or by its port pair, and return `None` for the tuple of received runs `strict=True` reports for an incomplete one ([#2581](https://github.com/JarryShaw/PyPCAPKit/issues/2581), [#2619](https://github.com/JarryShaw/PyPCAPKit/pull/2619)). + - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. - **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. @@ -48,7 +52,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 1cd6025c62..32b1237002 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -78,8 +78,25 @@ HTTP/1 and HTTP/2 classes under the RFC names. It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than - decoded, and the 802.11 MAC frame behind them, which has no dissector in this - package yet, as :class:`~pcapkit.protocols.misc.raw.Raw`. + decoded. The 802.11 MAC frame behind them is dissected as + :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, below, looked up in + ``Radiotap``'s own registry under + :attr:`LinkType.IEEE802_11 `, + and :func:`~pcapkit.foundation.registry.protocols.register_radiotap` + replaces that dissector (:issue:`2575`, :pr:`2607`). + +* **Added** -- :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, a + dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, + which :class:`~pcapkit.protocols.link.radiotap.Radiotap` dispatches to + (:issue:`2575`, :pr:`2607`). It reads Frame Control, the Control Frame + Extension and S1G Beacon layouts included, and the address, sequence, QoS + Control and HT Control fields each frame type carries; a Control frame of a + subtype it does not model, and an Extension frame, are read as the minimal + header. The frame body is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, + with the FCS and radiotap's pad octets where captured, as ``Radiotap`` does + not decode the Flags that mark them, and any other protocol version is + refused. The frame types and subtypes are :mod:`pcapkit.const.ieee802_11`, + from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. * **Added** -- :class:`~pcapkit.protocols.internet.icmpv4.ICMPv4` (:rfc:`792`) and :class:`~pcapkit.protocols.internet.icmpv6.ICMPv6` (:rfc:`4443`), on an @@ -218,6 +235,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. that decode as ``packet``, run on its first read as an IP or TCP datagram's is, and :data:`None` for an incomplete message (:issue:`2531`, :pr:`2605`). +* **Added** -- :func:`~pcapkit.foundation.analysis.ip.ip_datagram` and + :func:`~pcapkit.foundation.analysis.tcp.tcp_datagram`, siblings of + ``dtls_handshake``, above, which parse the payload of a datagram that IPv4, + IPv6 or TCP reassembly reports, by its payload protocol or by its port + pair, and return :data:`None` for the tuple of received runs ``strict=True`` + reports for an incomplete one (:issue:`2581`, :pr:`2619`). + * **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, the IPv4 (:rfc:`793`) or IPv6 (:rfc:`8200#section-8.1`) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit @@ -381,7 +405,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. request's review is cited (:issue:`2557`, :issue:`2563`, :pr:`2599`). The test conventions record that a test class whose tests each need a fresh import purges in ``setUp``, with a comment saying why, rather than importing - once per class (:issue:`2562`, :pr:`2611`). + once per class (:issue:`2562`, :pr:`2611`). Where a registry's primary source + is paywalled, the const-vendor pairing accepts a pinned secondary revision as + a crawler's ``DATA`` once its values are cross-checked against independent + implementations (:issue:`2575`, :pr:`2607`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` From 52c6ec3930e285734084296daf668a679cd595e1 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 23:40:14 -0500 Subject: [PATCH 10/11] docs(changelog): fold pad_length, the shared contextvars and the role kinds into 1.6.0 (#2449) - Added: pad_length and PaddingField's align/after form (#2624), and pcapkit.utilities.contextvars (#2623); the old private names get no entry, as for the other private API. - Changed: the weekly Issue and PR Kinds workflow, beside the milestone workflow's entry (#2626). - The conventions entry records where a context variable lives (#2623), and the role-kind citation rule with its test (#2626). - No entry for #2626's heading and Mermaid fixes, #2564 or fa9af520f. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 8 +++++++- docs/source/changelog/1.6.0.rst | 26 +++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 358b12921a..8f751e2b97 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,6 +36,10 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). +- **Added** -- `pcapkit.corekit.fields.strings.pad_length()`, the octets of padding that take a length to an `align`-octet boundary, and a `PaddingField` sized by it, given `align` and `after` -- the boundary, and the field or callable whose length the padding follows -- in place of `length` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2624](https://github.com/JarryShaw/PyPCAPKit/pull/2624)). + +- **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth`, `in_fragment` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). + - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. @@ -52,7 +56,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). @@ -60,6 +64,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). +- **Changed** -- a weekly Issue and PR Kinds workflow regenerates, with `util/issue_pr_kinds.py`, the snapshot of the tracker that the project test of each role's kind reads, and opens or updates a pull request from `bot/issue-pr-kinds` when it changes, never pushing `main` ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). + - **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). - **Fixed** -- `AppType.get` takes the `default` `EnumLookup.get` advertises, a registered member's value that it never mints, and a miss raises `EnumValueError`, still a `ValueError`, where it raised a builtin one ([#2545](https://github.com/JarryShaw/PyPCAPKit/issues/2545), [#2568](https://github.com/JarryShaw/PyPCAPKit/pull/2568)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 32b1237002..2b95fc5812 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -286,6 +286,18 @@ HTTP/1 and HTTP/2 classes under the RFC names. classmethods the IPX and Loopback wrappers call (:issue:`2511`, :pr:`2521`). +* **Added** -- :func:`pcapkit.corekit.fields.strings.pad_length`, the octets of + padding that take a length to an ``align``-octet boundary, and a + :class:`~pcapkit.corekit.fields.strings.PaddingField` sized by it, given + ``align`` and ``after`` -- the boundary, and the field or callable whose + length the padding follows -- in place of ``length`` (:issue:`2614`, + :pr:`2624`). + +* **Added** -- :mod:`pcapkit.utilities.contextvars`, holding the context + variables more than one module sets or reads -- ``beholding``, + ``frame_depth``, ``in_fragment`` and ``rebuilding`` -- and importing nothing + from :mod:`pcapkit` (:issue:`2616`, :pr:`2623`). + * **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as :class:`~pcapkit.protocols.misc.raw.Raw`, and the walk stops @@ -408,7 +420,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. once per class (:issue:`2562`, :pr:`2611`). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's ``DATA`` once its values are cross-checked against independent - implementations (:issue:`2575`, :pr:`2607`). + implementations (:issue:`2575`, :pr:`2607`). A context variable more than one + module uses lives in :mod:`pcapkit.utilities.contextvars`, and one that only + its own module uses stays private there (:issue:`2616`, :pr:`2623`). Each + ``:issue:`` and ``:pr:`` role in the documentation names a number of its own + kind, a ruling given in a pull request's review cited by the issue that pull + request closes, and a project test checks each role's kind against a + snapshot of the tracker (:issue:`2561`, :pr:`2626`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -435,6 +453,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``v1.9``, and a backfill starts above the closed milestone's items (:issue:`2548`, :pr:`2591`). +* **Changed** -- a weekly Issue and PR Kinds workflow regenerates, with + ``util/issue_pr_kinds.py``, the snapshot of the tracker that the project test + of each role's kind reads, and opens or updates a pull request from + ``bot/issue-pr-kinds`` when it changes, never pushing ``main`` + (:issue:`2561`, :pr:`2626`). + * **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and ``CONTRIBUTING.md`` and the pull request template say so (:issue:`2449`, From 7be441345a96327c80758a1deb5c4c84532be0ae Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 00:57:31 -0500 Subject: [PATCH 11/11] docs(changelog): fold Quick-Start's refusals, DTLS CIDs, 802.11 link types and the registry split into 1.6.0 (#2449) - Breaking: Quick-Start values their fields cannot carry raise, reversing 1.5.0's round-down of a TCP rate under 80 kbps (#2630). - Folded into unreleased entries: tls12_cid records and DTLSContext (#2631), LinkType 105 and 127 in Frame and PCAP-NG (#2632), and DTLS.analyze_handshake behind dtls_handshake and packet (#2634). - Changed: foundation.registry.protocols split into per-layer submodules (#2635); 1.6.0's register_* roles and 1.5.0's three register_protocol* roles point at their submodules, 1.5.0's text unchanged. - The conventions entry records the per-layer wrapper placement (#2635) and parse ownership (#2634). No entry for #2629, which changes no released behaviour. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 14 ++++-- docs/source/changelog/1.5.0.rst | 12 ++--- docs/source/changelog/1.6.0.rst | 86 +++++++++++++++++++++++++-------- 3 files changed, 82 insertions(+), 30 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8f751e2b97..8641183f4d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,9 +10,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)). The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. -- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). +- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11_RADIOTAP` (127) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). -- **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` does not decode the Flags that mark them, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. +- **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11` (105) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` does not decode the Flags that mark them, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. @@ -20,13 +20,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)), and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves a `tls12_cid` record's connection ID from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. - **Added** -- dissection of every whole HTTP/2 frame in a TCP segment: each after the first, which was kept as `Raw`, is a further `HTTPv2` layer, the previous frame's payload, as with TLS -- a `SETTINGS` then a `WINDOW_UPDATE` frame read `TCP:HTTP/2:HTTP/2` ([#2536](https://github.com/JarryShaw/PyPCAPKit/issues/2536), [#2609](https://github.com/JarryShaw/PyPCAPKit/pull/2609)). One that fails to decode is `Raw` with the rest of the segment and the parser's `error`, and a frame the segment ends inside stays `Raw` with none. A parse told to stop at HTTP/2 keeps the rest of the segment raw, and `FRAME_LAYER_LIMIT` bounds how many frames are dissected. -- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). A reassembled `Datagram` holds that decode as `packet`, run on its first read as an IP or TCP datagram's is, and `None` for an incomplete message ([#2531](https://github.com/JarryShaw/PyPCAPKit/issues/2531), [#2605](https://github.com/JarryShaw/PyPCAPKit/pull/2605)). +- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). A reassembled `Datagram` holds that decode as `packet`, run on its first read as an IP or TCP datagram's is, and `None` for an incomplete message ([#2531](https://github.com/JarryShaw/PyPCAPKit/issues/2531), [#2605](https://github.com/JarryShaw/PyPCAPKit/pull/2605)). Both run the classmethod `DTLS.analyze_handshake` -- `packet` through the reassembly class's own `__protocol_type__` -- which keeps a body that does not decode as raw octets, logging nothing ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). - **Added** -- `ip_datagram()` and `tcp_datagram()`, siblings of `dtls_handshake`, above, which parse the payload of a datagram that IPv4, IPv6 or TCP reassembly reports, by its payload protocol or by its port pair, and return `None` for the tuple of received runs `strict=True` reports for an incomplete one ([#2581](https://github.com/JarryShaw/PyPCAPKit/issues/2581), [#2619](https://github.com/JarryShaw/PyPCAPKit/pull/2619)). @@ -44,6 +44,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. +- **Changed** -- **a breaking change to** building a Quick-Start option in `IPv4`, `HOPOPT`, `IPv6_Opts` and `TCP`: a value its field cannot carry raises `ProtocolError`, where 1.5.x built it ([#2617](https://github.com/JarryShaw/PyPCAPKit/issues/2617), [#2630](https://github.com/JarryShaw/PyPCAPKit/pull/2630)). That is a negative or non-integer rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an IPv4 reserved value of 4 or more, and `True` as a TTL, nonce or reserved value. Refusing a TCP rate under 80 kbps reverses 1.5.0's rounding of one down to zero ([#1119](https://github.com/JarryShaw/PyPCAPKit/issues/1119)); a rate between two codes still rounds down to the lower. HOPOPT's and IPv6_Opts' Quick-Start maker builds an unassigned function code as the unassigned option their reader keeps it as, as IPv4's does. + - **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. - **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`. @@ -54,9 +56,11 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `Extractor` and `extract()`: their `dtls` parameter, above, follows `tcp`, so a positional argument from `buffer_size` on moves one place along, and `ReassemblyManager` takes a required `dtls` ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). +- **Changed** -- `pcapkit.foundation.registry.protocols` is a package with a submodule per layer -- `link`, `internet`, `transport`, `application` and `misc`, with the shared registrars in `protocol` -- and exports every public function they define, so each `register_*` imports from it and from `pcapkit.foundation.registry` as before ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A registrar's `__module__` names its submodule, and its log records carry that submodule's logger name, a child of the package's, so configuring the package's logger still applies. + - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index 9eeab1455e..c8b7d45ea0 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -1143,8 +1143,8 @@ Fixed ``register_reassembly`` and ``register_traceflow`` on :class:`.Extractor`, :meth:`.TraceFlow.register_dumper` (reached through ``register_traceflow_dumper``), and - :func:`pcapkit.foundation.registry.protocols.register_protocol`. Each guard gains an - explicit ``isinstance(x, type)`` test ahead of its ``issubclass``; in those that + :func:`pcapkit.foundation.registry.protocols.register_protocol `. + Each guard gains an explicit ``isinstance(x, type)`` test ahead of its ``issubclass``; in those that accept a :class:`.ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while ``register_protocol`` takes no descriptor. No guard's target class changes, and a wrong class still raises @@ -1239,8 +1239,8 @@ Fixed and ``register_transtype`` refuse a ``code`` that is not an enum member with :exc:`.RegistryError` before touching a registry, where they registered it and then raised ``AttributeError``, leaving the entry behind (:issue:`2186`, :pr:`2193`). -* :func:`pcapkit.foundation.registry.protocols.register_protocol_code`, and the - ``code=`` class keyword, check every destination's key before registering any of +* :func:`pcapkit.foundation.registry.protocols.register_protocol_code `, + and the ``code=`` class keyword, check every destination's key before registering any of them. A key belonging to another enum, such as ``{Frame: TransType.ICMP}``, is refused with :exc:`.RegistryError` where it replaced the shipped ``LinkType.ETHERNET`` entry, and a bad target late in the list no longer leaves @@ -1382,8 +1382,8 @@ Added Inference refuses rather than guesses: an enum member whose type names no known destination raises :exc:`.RegistryError`, and an unrecognised class keyword raises :exc:`.UnsupportedCall`, matching the other registrar families. Backed by - :func:`pcapkit.foundation.registry.protocols.register_protocol_code`, also callable - directly for a class that declined at definition time. It was written up as the + :func:`pcapkit.foundation.registry.protocols.register_protocol_code `, + also callable directly for a class that declined at definition time. It was written up as the mechanism :issue:`548` needs; :issue:`548` turned out to need a missing class instead (115 is :rfc:`3931` L2TPv3 over IP, not :rfc:`2661`; see the corresponding **Fixed** entry below), so its worked example names ``L2TPv3`` (:pr:`570`). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 2b95fc5812..aa4bffa393 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -21,7 +21,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``FRAD``, ``NETLINK``, ``IPGRE``/``IP6GRE`` and ``IEEE80211_RADIOTAP`` -- each of which has a dissector of its own, below (:issue:`2460`), reached through ``LinuxSLL``'s own registry keyed by ``arphrd_type``, which - :func:`~pcapkit.foundation.registry.protocols.register_arphrd` extends + :func:`~pcapkit.foundation.registry.protocols.link.register_arphrd` extends (:issue:`2510`, :pr:`2527`), as does a protocol class declaring ``code=ARPHRD.X``, since ``ARPHRD`` names ``LinuxSLL`` as its ``__proto_dest__`` (:issue:`2533`, :pr:`2598`). A parse told to stop there -- at ``LinuxSLL``, @@ -41,7 +41,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, and - :func:`~pcapkit.foundation.registry.protocols.register_frad` registers a + :func:`~pcapkit.foundation.registry.protocols.link.register_frad` registers a dissector for a further NLPID (:pr:`2527`). * **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for @@ -55,7 +55,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. Wireshark's netlink dissector does. Only the first message's header is dissected: what follows it, its per-family payload and any further message in the same capture, is kept as :class:`~pcapkit.protocols.misc.raw.Raw` - unless :func:`~pcapkit.foundation.registry.protocols.register_netlink` + unless :func:`~pcapkit.foundation.registry.protocols.link.register_netlink` registers a dissector for its ``nlmsg_type`` (:pr:`2527`). * **Added** -- :class:`~pcapkit.protocols.link.gre.GRE`, a dissector for the @@ -74,21 +74,23 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Added** -- :class:`~pcapkit.protocols.link.radiotap.Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` - dispatches to for ``ARPHRD_IEEE80211_RADIOTAP`` (:issue:`2502`, :pr:`2509`). - It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) + dispatches to for ``ARPHRD_IEEE80211_RADIOTAP`` (:issue:`2502`, :pr:`2509`), + as do PCAP and PCAP-NG frames of ``LINKTYPE_IEEE802_11_RADIOTAP`` (127) + (:issue:`2608`, :pr:`2632`). It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, below, looked up in ``Radiotap``'s own registry under :attr:`LinkType.IEEE802_11 `, - and :func:`~pcapkit.foundation.registry.protocols.register_radiotap` + and :func:`~pcapkit.foundation.registry.protocols.link.register_radiotap` replaces that dissector (:issue:`2575`, :pr:`2607`). * **Added** -- :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which :class:`~pcapkit.protocols.link.radiotap.Radiotap` dispatches to - (:issue:`2575`, :pr:`2607`). It reads Frame Control, the Control Frame + (:issue:`2575`, :pr:`2607`), as do PCAP and PCAP-NG frames of + ``LINKTYPE_IEEE802_11`` (105) (:issue:`2608`, :pr:`2632`). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal @@ -108,7 +110,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain ``ICMP`` of :rfc:`792` as its ``id()`` and alias. - :func:`~pcapkit.foundation.registry.protocols.register_icmp_type` registers a + :func:`~pcapkit.foundation.registry.protocols.internet.register_icmp_type` registers a message-body parser. The type and code registries are :mod:`pcapkit.const.icmp` and :mod:`pcapkit.const.icmpv6`, crawled from IANA. @@ -146,9 +148,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. gives (:rfc:`4340#section-9.2`): a coverage running past the packet is ``INVALID`` and warns, and a ``make()`` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. - :func:`~pcapkit.foundation.registry.protocols.register_dccp` registers a + :func:`~pcapkit.foundation.registry.protocols.transport.register_dccp` registers a port's dissector, and - :func:`~pcapkit.foundation.registry.protocols.register_dccp_option` an + :func:`~pcapkit.foundation.registry.protocols.transport.register_dccp_option` an option parser. The registries are :mod:`pcapkit.const.dccp`, crawled from IANA. @@ -168,7 +170,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``signature_algorithms``, ALPN, ``supported_versions`` and ``key_share``, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; - :func:`~pcapkit.foundation.registry.protocols.register_tls_extension` + :func:`~pcapkit.foundation.registry.protocols.application.register_tls_extension` registers a parser and constructor for a further type (:issue:`2514`, :pr:`2595`). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- @@ -181,8 +183,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. not reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also reads and builds DTLS 1.3's unified-header records (:rfc:`9147#section-4.1`), except one carrying a connection ID, which stays - Raw as an :rfc:`9146` ``tls12_cid`` record does (:issue:`2467`, - :pr:`2494`). Each whole DTLS record in a UDP datagram is likewise a layer of + Raw (:issue:`2467`, :pr:`2494`), and :rfc:`9146` ``tls12_cid`` records. A + :class:`~pcapkit.protocols.application.dtls.DTLSContext`, given to + ``extract()`` as its ``context``, resolves a ``tls12_cid`` record's + connection ID from those the capture's hellos negotiate in their + ``connection_id`` extension, which TLS and DTLS both decode, or from the + ``cid_length`` the caller gives it, and one it cannot resolve stays Raw + (:issue:`2515`, :pr:`2631`). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read ``DTLS 1.2:DTLS 1.3`` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is @@ -234,6 +241,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. :class:`Datagram ` holds that decode as ``packet``, run on its first read as an IP or TCP datagram's is, and :data:`None` for an incomplete message (:issue:`2531`, :pr:`2605`). + Both run the classmethod + :meth:`DTLS.analyze_handshake ` + -- ``packet`` through the reassembly class's own ``__protocol_type__`` -- + which keeps a body that does not decode as raw octets, logging nothing + (:issue:`2622`, :pr:`2634`). * **Added** -- :func:`~pcapkit.foundation.analysis.ip.ip_datagram` and :func:`~pcapkit.foundation.analysis.tcp.tcp_datagram`, siblings of @@ -273,11 +285,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is ``NO_PSEUDO_HEADER``. -* **Added** -- :func:`~pcapkit.foundation.registry.protocols.register_ipx`, - :func:`~pcapkit.foundation.registry.protocols.register_loopback`, - :func:`~pcapkit.foundation.registry.protocols.register_sctp_chunk`, - :func:`~pcapkit.foundation.registry.protocols.register_sctp_parameter` and - :func:`~pcapkit.foundation.registry.protocols.register_sctp_cause`, wrapping +* **Added** -- :func:`~pcapkit.foundation.registry.protocols.internet.register_ipx`, + :func:`~pcapkit.foundation.registry.protocols.link.register_loopback`, + :func:`~pcapkit.foundation.registry.protocols.transport.register_sctp_chunk`, + :func:`~pcapkit.foundation.registry.protocols.transport.register_sctp_parameter` and + :func:`~pcapkit.foundation.registry.protocols.transport.register_sctp_cause`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the @@ -322,6 +334,21 @@ HTTP/1 and HTTP/2 classes under the RFC names. A checksum given is written as given, and turning ``PCAPKIT_CHECKSUM_COMPUTE`` off restores zero. +* **Changed** -- **a breaking change to** building a Quick-Start option in + :class:`~pcapkit.protocols.internet.ipv4.IPv4`, + :class:`~pcapkit.protocols.internet.hopopt.HOPOPT`, + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` and + :class:`~pcapkit.protocols.transport.tcp.TCP`: a value its field cannot + carry raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, where 1.5.x + built it (:issue:`2617`, :pr:`2630`). That is a negative or non-integer + rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an + IPv4 reserved value of 4 or more, and ``True`` as a TTL, nonce or + reserved value. Refusing a TCP rate under 80 kbps reverses 1.5.0's + rounding of one down to zero (:issue:`1119`); a rate between two codes still + rounds down to the lower. HOPOPT's and IPv6_Opts' Quick-Start maker builds an + unassigned function code as the unassigned option their reader keeps it as, + as IPv4's does. + * **Changed** -- **a breaking change to** :class:`~pcapkit.protocols.application.drarp.DRARP` and :class:`~pcapkit.protocols.link.inarp.InARP`, which move from @@ -386,6 +413,19 @@ HTTP/1 and HTTP/2 classes under the RFC names. place along, and :class:`~pcapkit.foundation.reassembly.ReassemblyManager` takes a required ``dtls`` (:issue:`2496`, :pr:`2528`). +* **Changed** -- :mod:`pcapkit.foundation.registry.protocols` is a package with + a submodule per layer -- :mod:`~pcapkit.foundation.registry.protocols.link`, + :mod:`~pcapkit.foundation.registry.protocols.internet`, + :mod:`~pcapkit.foundation.registry.protocols.transport`, + :mod:`~pcapkit.foundation.registry.protocols.application` and + :mod:`~pcapkit.foundation.registry.protocols.misc`, with the shared + registrars in :mod:`~pcapkit.foundation.registry.protocols.protocol` -- and + exports every public function they define, so each ``register_*`` imports + from it and from :mod:`pcapkit.foundation.registry` as before + (:issue:`2578`, :pr:`2635`). A registrar's ``__module__`` names its + submodule, and its log records carry that submodule's logger name, a child + of the package's, so configuring the package's logger still applies. + * **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order (:issue:`2490`, :pr:`2491`, :pr:`2522`, :pr:`2523`, :pr:`2538`, @@ -426,7 +466,15 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``:issue:`` and ``:pr:`` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a - snapshot of the tracker (:issue:`2561`, :pr:`2626`). + snapshot of the tracker (:issue:`2561`, :pr:`2626`). A registry wrapper goes + in the submodule of :mod:`pcapkit.foundation.registry.protocols` for the + layer of the class owning its map (:issue:`2578`, :pr:`2635`). A protocol + class owns the parse of a reassembled outcome, which each + :mod:`pcapkit.foundation.analysis` function wraps without parsing of its + own, and a project test fails on a module under :mod:`pcapkit.protocols` + importing :mod:`pcapkit.foundation` outside ``TYPE_CHECKING``, + :mod:`pcapkit.foundation.registry.protocols` aside (:issue:`2622`, + :pr:`2634`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/``