refactor(corekit): move the Internet checksum into corekit.checksum.internet

docs/source/contributing/conventions/utilities-or-corekit.rst

@@ -25,3 +25,15 @@ question belongs here instead. :mod:`pcapkit.corekit.decorators` is the worked e
 protocol's file and falls back to protocol classes, so it moved out of ``utilities``
 on :issue:`1735`. ``corekit`` may import ``utilities``; the reverse is what the rule
 forbids.
+
+A second worked example is :mod:`pcapkit.corekit.checksum`, settled on :issue:`2580`.
+It fails the ``utilities`` question because
+:class:`~pcapkit.corekit.checksum.internet.ChecksumStatus` subclasses
+:class:`~pcapkit.corekit.enum.EnumLookup`. The protocols compute and verify their
+checksums through it, so it imports
+:mod:`pcapkit` only from ``corekit`` and ``utilities``, never from
+:mod:`pcapkit.protocols`; :file:`tests/corekit/test_checksum_import_boundary_2580_unit.py`
+checks that. What needs IPv4 and IPv6 stays beside
+:class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`. The package holds one
+module per checksum family, so a new family sits beside
+:mod:`~pcapkit.corekit.checksum.internet`.

docs/source/contributing/pep.rst

@@ -895,7 +895,7 @@ model for how to spell "expected, not an error".
 
 **The one's-complement half is done** (:issue:`2470`, :issue:`2472` to
 :issue:`2475`), on :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`
-(:issue:`2471`) and :mod:`pcapkit.protocols.misc.checksum.checksum`, settled as
+(:issue:`2471`) and :mod:`pcapkit.corekit.checksum.internet`, settled as
 below: the toggles by the owner on :issue:`2471`, the rest at implementation,
 as the offload question above asked; the owner accepted the toggles and the
 offload heuristic on :issue:`2470`.
@@ -912,17 +912,17 @@ offload heuristic on :issue:`2470`.
   and :class:`~pcapkit.protocols.internet.ipv6.IPv6` verify their upper layer's
   checksum once parsed, and compute it for an upper layer constructed as their
   payload; ``make()`` of TCP, UDP and ICMPv6 also takes the ``pseudo`` itself.
-* **Offload is** :attr:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus.NOT_COMPUTED`,
+* **Offload is** :attr:`~pcapkit.corekit.checksum.internet.ChecksumStatus.NOT_COMPUTED`,
   modelled on ``NO_SA``: a zero IPv4 or TCP checksum, or a TCP or UDP one holding
   the pseudo-header's sum alone, as Linux offload leaves it, never warns. So does
   a zero UDP checksum over IPv4 (:rfc:`768`), but not over IPv6, where it is
   mandatory (:rfc:`8200#section-8.1`).
 * **A fragment is**
-  :attr:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus.NO_PSEUDO_HEADER`,
+  :attr:`~pcapkit.corekit.checksum.internet.ChecksumStatus.NO_PSEUDO_HEADER`,
   atomic ones aside: TCP, UDP, ICMP and ICMPv6 checksums cover all of the
   datagram's data, which it carries part of. OSPF's Packet Length says what its
   checksum covers, so it is verified when the fragment holds that much, and
-  :attr:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus.TRUNCATED`
+  :attr:`~pcapkit.corekit.checksum.internet.ChecksumStatus.TRUNCATED`
   otherwise.
 
 The cryptographic half is sequenced for **wave 2 or 3**, and could be done sooner

docs/source/pcapkit/corekit/checksum/index.rst

@@ -0,0 +1,15 @@
+Checksums
+=========
+
+.. module:: pcapkit.corekit.checksum
+
+:mod:`pcapkit.corekit.checksum` holds one module per checksum family the
+protocols compute and verify, so that a new family sits beside the others
+(:issue:`2580`).
+
+.. versionadded:: 1.6.0
+
+.. toctree::
+   :maxdepth: 2
+
+   internet

docs/source/pcapkit/corekit/checksum/internet.rst

@@ -0,0 +1,31 @@
+Internet Checksum
+=================
+
+.. module:: pcapkit.corekit.checksum.internet
+
+:mod:`pcapkit.corekit.checksum.internet` contains the one's-complement
+Internet checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP, ICMP, ICMPv6
+and OSPF. Parsing verifies a checksum only if
+:data:`~pcapkit.utilities.config.CHECKSUM_VERIFY` is on, and records the
+outcome as ``checksum_status``; construction computes one left out unless
+:data:`~pcapkit.utilities.config.CHECKSUM_COMPUTE` is off (:issue:`2471`).
+A mismatch warns with a :exc:`~pcapkit.utilities.warnings.ChecksumWarning`,
+which a filter can silence alone (:issue:`2470`).
+
+It imports nothing from :mod:`pcapkit.protocols`, which builds on it
+(:issue:`2580`). A checksum over the IP pseudo-header is left to the enclosing
+IP layer, see :mod:`pcapkit.protocols.misc.checksum.ip_pseudo`. ICMP's
+checksum, over the message alone, is verified as it is parsed, unless the IP
+layer it is parsed from is a fragment, see :func:`fragment_payload`.
+
+.. versionadded:: 1.6.0
+
+.. autoclass:: pcapkit.corekit.checksum.internet.ChecksumStatus
+   :members:
+   :show-inheritance:
+
+.. autofunction:: pcapkit.corekit.checksum.internet.internet_checksum
+.. autofunction:: pcapkit.corekit.checksum.internet.compute_checksum
+.. autofunction:: pcapkit.corekit.checksum.internet.partial_checksum
+.. autofunction:: pcapkit.corekit.checksum.internet.verify_checksum
+.. autofunction:: pcapkit.corekit.checksum.internet.fragment_payload

docs/source/pcapkit/corekit/index.rst

@@ -5,7 +5,8 @@ Core Utilities
 
 :mod:`pcapkit.corekit` holds the core utilities :mod:`pcapkit` is built on:
 the :class:`~pcapkit.corekit.fields.field.Field` family for data parsing, the
-:class:`~pcapkit.corekit.context.ContextRegistry` channel for caller-supplied
+:mod:`~pcapkit.corekit.checksum` algorithms the protocols compute and verify,
+the :class:`~pcapkit.corekit.context.ContextRegistry` channel for caller-supplied
 information a protocol needs but the wire does not carry, the
 :mod:`~pcapkit.corekit.decorators` that protocols and schemas parse through,
 the :class:`~pcapkit.corekit.enum.EnumLookup`/
@@ -25,6 +26,7 @@ parsed on first read, the protocol collection
    :maxdepth: 2
 
    fields/index
+   checksum/index
    context
    decorators
    enum

docs/source/pcapkit/protocols/misc/checksum.rst

@@ -8,21 +8,22 @@ Checksums
 
 :mod:`pcapkit.protocols.misc.checksum` contains the pseudo-header that
 upper-layer checksums cover,
-:class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, and in
-:mod:`~pcapkit.protocols.misc.checksum.checksum` what the one's-complement
-checksums share, such as
-:class:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus`.
+:class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`. What the
+one's-complement checksums share, such as
+:class:`~pcapkit.corekit.checksum.internet.ChecksumStatus`, is in
+:mod:`pcapkit.corekit.checksum.internet`.
 
 IP Pseudo-Header
 ================
 
 .. module:: pcapkit.protocols.misc.checksum.ip_pseudo
 
 :mod:`pcapkit.protocols.misc.checksum.ip_pseudo` contains
-:class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo` only,
+:class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`,
 which implements the pseudo-header of the IP protocol family
-that the upper-layer checksums cover, whose structure is
-described as below:
+that the upper-layer checksums cover, and :func:`pseudo_header` and
+:func:`verify_upper`, through which the IP layers use it. The
+pseudo-header's structure is described as below:
 
 IPv4 (:rfc:`793#section-3.1`, :rfc:`768`):
 
@@ -48,6 +49,16 @@ Octets      Bits        Name                    Description
   39        312   ``ip_pseudo.protocol``      Next Header
 ======= ========= ========================= ===============================
 
+A checksum over the IP pseudo-header is verified and computed by the enclosing
+:class:`~pcapkit.protocols.internet.ipv4.IPv4` or
+:class:`~pcapkit.protocols.internet.ipv6.IPv6` layer, which builds the
+:class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, rather than by
+the upper layer reaching back to its parent. So an upper layer constructed
+alone, or behind another header such as an IPv6 extension header or an
+Authentication Header, has its checksum computed only if given the ``pseudo``
+to compute it over, and one parsed alone records no outcome. The sum itself is
+in :mod:`pcapkit.corekit.checksum.internet`.
+
 .. autoclass:: pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo
    :no-members:
    :show-inheritance:
@@ -71,6 +82,9 @@ Octets      Bits        Name                    Description
    .. automethod:: __post_init__
    .. automethod:: __index__
 
+.. autofunction:: pcapkit.protocols.misc.checksum.ip_pseudo.pseudo_header
+.. autofunction:: pcapkit.protocols.misc.checksum.ip_pseudo.verify_upper
+
 Header Schemas
 --------------
 
@@ -98,40 +112,3 @@ Data Models
 .. autoclass:: pcapkit.protocols.data.misc.checksum.ip_pseudo.IP_Pseudo
    :members:
    :show-inheritance:
-
-Internet Checksum
-=================
-
-.. module:: pcapkit.protocols.misc.checksum.checksum
-
-:mod:`pcapkit.protocols.misc.checksum.checksum` contains what the
-one's-complement Internet checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP,
-ICMP, ICMPv6 and OSPF shares. Parsing verifies a checksum only if
-:data:`~pcapkit.utilities.config.CHECKSUM_VERIFY` is on, and records the
-outcome as ``checksum_status``; construction computes one left out unless
-:data:`~pcapkit.utilities.config.CHECKSUM_COMPUTE` is off (:issue:`2471`).
-A mismatch warns with a :exc:`~pcapkit.utilities.warnings.ChecksumWarning`,
-which a filter can silence alone (:issue:`2470`).
-
-A checksum over the IP pseudo-header is verified and computed by the enclosing
-:class:`~pcapkit.protocols.internet.ipv4.IPv4` or
-:class:`~pcapkit.protocols.internet.ipv6.IPv6` layer, which builds the
-:class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, rather than by
-the upper layer reaching back to its parent. So an upper layer constructed
-alone, or behind another header such as an IPv6 extension header or an
-Authentication Header, has its checksum computed only if given the ``pseudo``
-to compute it over, and one parsed alone records no outcome. ICMP's checksum,
-over the message alone, is verified as it is parsed, unless the IP layer it is
-parsed from is a fragment.
-
-.. autoclass:: pcapkit.protocols.misc.checksum.checksum.ChecksumStatus
-   :members:
-   :show-inheritance:
-
-.. autofunction:: pcapkit.protocols.misc.checksum.checksum.internet_checksum
-.. autofunction:: pcapkit.protocols.misc.checksum.checksum.compute_checksum
-.. autofunction:: pcapkit.protocols.misc.checksum.checksum.partial_checksum
-.. autofunction:: pcapkit.protocols.misc.checksum.checksum.verify_checksum
-.. autofunction:: pcapkit.protocols.misc.checksum.checksum.pseudo_header
-.. autofunction:: pcapkit.protocols.misc.checksum.checksum.verify_upper
-.. autofunction:: pcapkit.protocols.misc.checksum.checksum.fragment_payload

pcapkit/corekit/checksum/__init__.py

@@ -0,0 +1,23 @@
+# -*- coding: utf-8 -*-
+# pylint: disable=unused-import
+"""Checksums
+=============
+
+.. module:: pcapkit.corekit.checksum
+
+:mod:`pcapkit.corekit.checksum` holds one module per checksum family the
+protocols compute and verify, so that a new family sits beside the others
+(:issue:`2580`). :mod:`~pcapkit.corekit.checksum.internet` is the
+one's-complement Internet checksum (:rfc:`1071`).
+
+.. versionadded:: 1.6.0
+
+"""
+from pcapkit.corekit.checksum.internet import (ChecksumStatus, compute_checksum, fragment_payload,
+                                               internet_checksum, partial_checksum, verify_checksum)
+
+__all__ = [
+    'ChecksumStatus',
+    'internet_checksum', 'compute_checksum', 'partial_checksum', 'verify_checksum',
+    'fragment_payload',
+]

pcapkit/protocols/misc/checksum/checksum.py → pcapkit/corekit/checksum/internet.py

@@ -2,36 +2,24 @@
 """Internet Checksum
 =====================
 
-.. module:: pcapkit.protocols.misc.checksum.checksum
+.. module:: pcapkit.corekit.checksum.internet
 
-:mod:`pcapkit.protocols.misc.checksum.checksum` contains what the
-one's-complement Internet checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP,
-ICMP, ICMPv6 and OSPF shares: :class:`ChecksumStatus`, the outcome a parse
-records, and the functions that compute and verify the checksum.
+:mod:`pcapkit.corekit.checksum.internet` contains the one's-complement
+Internet checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP, ICMP, ICMPv6
+and OSPF: :class:`ChecksumStatus`, the outcome a parse records, and the
+functions that compute and verify the checksum. It imports nothing from
+:mod:`pcapkit.protocols`, which builds on it (:issue:`2580`).
 
 Verification is opt-in and computation opt-out, by
 :data:`~pcapkit.utilities.config.CHECKSUM_VERIFY` and
 :data:`~pcapkit.utilities.config.CHECKSUM_COMPUTE` (:issue:`2471`). A
 mismatch warns with a :exc:`~pcapkit.utilities.warnings.ChecksumWarning`,
 which a filter can silence alone (:issue:`2470`).
 
-A checksum over the IP pseudo-header is left to the enclosing IP layer, which
-alone knows the pseudo-header, and builds it as an
-:class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, rather than the
-upper layer reaching back to its parent. The upper layer takes part through
-these members:
-
-* ``_verify_pseudo_checksum(pseudo)`` verifies the checksum against
-  ``pseudo``, or records :attr:`ChecksumStatus.NO_PSEUDO_HEADER` for
-  :data:`None`; the IP layer calls it once parsed, see :func:`verify_upper`;
-* ``make()`` takes the ``pseudo`` it computes the checksum over, and replaces
-  the length it carries with the upper layer's own;
-* ``_checksum_pending`` is :data:`True` once ``make()`` left out a checksum it
-  had no ``pseudo`` for. An IP layer constructed with that upper layer as its
-  payload then rebuilds it with its own pseudo-header.
-
-A checksum over the upper layer alone, as ICMP's, is verified as it is parsed,
-unless the IP layer it is parsed from is a fragment, see :func:`fragment_payload`.
+A checksum over the IP pseudo-header is left to the enclosing IP layer, see
+:mod:`pcapkit.protocols.misc.checksum.ip_pseudo`. A checksum over the upper
+layer alone, as ICMP's, is verified as it is parsed, unless the IP layer it is
+parsed from is a fragment, see :func:`fragment_payload`.
 
 .. versionadded:: 1.6.0
 
@@ -43,23 +31,26 @@
 from typing import TYPE_CHECKING
 
 from pcapkit.corekit.enum import EnumLookup
-from pcapkit.protocols.protocol import _rebuilding
-from pcapkit.utilities.exceptions import FieldValueError, ProtocolError, stacklevel
+from pcapkit.utilities.exceptions import stacklevel
 from pcapkit.utilities.warnings import ChecksumWarning, warn
 
 if TYPE_CHECKING:
     from typing import Collection, Iterator, Optional
 
-    from pcapkit.protocols.internet.ipv4 import IPv4
-    from pcapkit.protocols.internet.ipv6 import IPv6
-    from pcapkit.protocols.misc.checksum.ip_pseudo import IP_Pseudo
-
 __all__ = [
     'ChecksumStatus',
     'internet_checksum', 'compute_checksum', 'partial_checksum', 'verify_checksum',
-    'pseudo_header', 'verify_upper', 'fragment_payload',
+    'fragment_payload',
 ]
 
+#: contextvars.ContextVar[bool]: Whether a payload is being rebuilt by
+#: :meth:`Protocol._make_payload
+#: <pcapkit.protocols.protocol.Protocol._make_payload>`, so that the layer
+#: it is rebuilt for parses the same octets again; see
+#: :data:`~pcapkit.utilities.config.FRAME_LAYER_LIMIT`. It lives here so that
+#: :func:`verify_checksum` reads it without importing :mod:`pcapkit.protocols`.
+_rebuilding = contextvars.ContextVar('_rebuilding', default=False)
+
 #: contextvars.ContextVar[bool]: Whether the layer being parsed is the payload
 #: of an IP fragment, see :func:`fragment_payload`.
 _in_fragment = contextvars.ContextVar('_in_fragment', default=False)
@@ -208,59 +199,6 @@ def verify_checksum(alias: 'str', data: 'bytes', field: 'bytes', *, length: 'Opt
     return ChecksumStatus.INVALID
 
 
-def pseudo_header(pseudo: 'IP_Pseudo', length: 'int') -> 'bytes':
-    """Octets of ``pseudo`` with ``length`` as its upper-layer length.
-
-    Args:
-        pseudo: Pseudo-header of the enclosing IP layer.
-        length: Upper-layer length the checksum is computed for.
-
-    Returns:
-        :attr:`pseudo.data <pcapkit.protocols.protocol.Protocol.data>`, made
-        anew if it carries another length.
-
-    Raises:
-        FieldValueError: If ``length`` does not fit the length field.
-
-    """
-    if pseudo.info.len == length:
-        return pseudo.data
-    return type(pseudo)(src=pseudo.src, dst=pseudo.dst, protocol=pseudo.protocol,
-                        upper_length=length, zero=pseudo.info.zero).data
-
-
-def verify_upper(ip: 'IPv4 | IPv6') -> 'None':
-    """Verify the checksum of the upper layer ``ip`` carries over its pseudo-header.
-
-    The upper layer is the payload of ``ip``, past any Authentication Header
-    of IPv4. It is verified if it defines ``_verify_pseudo_checksum``, against
-    the :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo` of ``ip``,
-    or against :data:`None` if that cannot be derived or names another protocol.
-
-    Args:
-        ip: Parsed IPv4 or IPv6 layer.
-
-    """
-    from pcapkit.protocols.internet.ah import AH  # isort: skip # pylint: disable=import-outside-toplevel
-    from pcapkit.protocols.misc.checksum.ip_pseudo import IP_Pseudo  # isort: skip # pylint: disable=import-outside-toplevel
-
-    upper = ip.payload
-    # NOTE: IPv6 walks an AH as an extension header; IPv4 carries it as its payload.
-    while isinstance(upper, AH):
-        upper = upper.payload
-    verify = getattr(upper, '_verify_pseudo_checksum', None)
-    if verify is None:
-        return
-
-    try:
-        pseudo = IP_Pseudo(ip=ip)  # type: Optional[IP_Pseudo]
-    except (ProtocolError, FieldValueError):
-        pseudo = None
-    if pseudo is not None and pseudo.protocol != upper.__index__():
-        pseudo = None
-    verify(pseudo)
-
-
 @contextlib.contextmanager
 def fragment_payload(fragment: 'bool') -> 'Iterator[None]':
     """Mark the payload an IP layer parses as a fragment's, or not.

pcapkit/protocols/application/ospf.py

@@ -226,7 +226,7 @@ def read(self, length: 'Optional[int]' = None, **kwargs: 'Any') -> 'Data_OSPF':
         # NOTE: The checksum leaves out the 8-octet Authentication field at
         # octet 16, and is unused under cryptographic authentication (RFC 2328).
         if config.CHECKSUM_VERIFY and ospf.version == 2:
-            from pcapkit.protocols.misc.checksum.checksum import ChecksumStatus, verify_checksum  # isort: skip # pylint: disable=import-outside-toplevel
+            from pcapkit.corekit.checksum.internet import ChecksumStatus, verify_checksum  # isort: skip # pylint: disable=import-outside-toplevel
             if ospf.autype == Enum_Authentication.Cryptographic_authentication:
                 status = ChecksumStatus.NOT_COMPUTED
             else:
@@ -345,7 +345,7 @@ def make(self,
 
         if (checksum is None and config.CHECKSUM_COMPUTE and version == 2
                 and auth_type_ != Enum_Authentication.Cryptographic_authentication):
-            from pcapkit.protocols.misc.checksum.checksum import compute_checksum  # isort: skip # pylint: disable=import-outside-toplevel
+            from pcapkit.corekit.checksum.internet import compute_checksum  # isort: skip # pylint: disable=import-outside-toplevel
             # NOTE: Over the packet but the Authentication field, as read() checks it (#2475).
             octets = schema.pack()
             schema.checksum = compute_checksum(octets[:16] + octets[self.length:max(packet_length, self.length)])