Skip to content

Add native DynamoDB conversion and error mapping - #157

Draft
dkropachev wants to merge 7 commits into
mainfrom
feat/native-conversion-errors
Draft

dkropachev wants to merge 7 commits into
mainfrom
feat/native-conversion-errors

Conversation

@dkropachev

@dkropachev dkropachev commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Closes #134. Part of #131.

The staged native backend has construction and transport but needs a conversion boundary for its ten approved DynamoDB operations. Convert low-level Python dictionaries directly into owned Rust SDK inputs, including legacy parameters, nested attributes, exact numeric strings, binary values and list-based sets. Reject missing or unknown members, invalid types and malformed AttributeValue unions with parameter paths before native execution.

Project typed SDK responses into fresh Python dictionaries, retaining absent and empty fields. Use each operation's captured final response for defaulted scalar presence and metadata, including retries that had no HTTP response. The scope includes compact parsing of required field-presence paths while skipping large attribute values and preserving absent, null and empty distinctions, including nested batch fields. JSON scanning and a second decompression of compressed responses remain. Decode compressed bodies in Rust while preserving original wire bytes and headers.

Export the reserved validation/service exceptions and structured transport categories. A private Python adapter constructs modeled or unknown botocore-compatible errors using cached service models without creating an admin client. Native service errors retain unknown body members and message casing, with decoded binary Items and cancellation reasons.

The scope includes preserving unrelated nested fields in dictionary-valued Error and ResponseMetadata body members. Canonical exception, HTTP and retry field names remain reserved, even when their genuine values are absent. Non-dictionary values are replaced by generated containers; overwritten originals remain in raw .body.

Decisions

  • Scope: Conversion and errors only. Reason: operation execution is tracked in Implement Rust item, query, and scan methods #135 and Implement Rust batch and transaction methods #136, and facades in Add private sync and async facades with Python admin delegation #137. Source: the agreed implementation plan.
  • Rust dependency: Retain the current pinned revision. Reason: final dependency-pin qualification belongs to separate Add native binding package, configuration conversion, and runtime ownership #133 work. Source: the agreed implementation plan.
  • Interface: Retain version 1. Reason: these exception interfaces were already reserved by the staged contract. Source: the version-1 native interface contract.
  • Choice: Response presence: Include compact presence parsing in this PR. Parse only required field-presence paths and skip large attribute values. Preserve absent, null and empty distinctions, including nested batch fields. Accept the selective parser, generated paths and focused tests; JSON scanning and a second decompression of compressed responses remain. Reason: avoid temporary allocations that grow with item payloads. Source: the user selected option 1.
  • Choice: Error payload collisions: Preserve unrelated nested fields in dictionary-valued Error and ResponseMetadata payloads. Exception, HTTP and retry field names remain reserved, even when their genuine values are absent. Generated values take precedence; replace non-dictionary payload values at these two names. Overwritten originals remain in raw .body. These are explicit exceptions to decoded-payload retention. Reason: retain useful payload data while keeping exception identity and transport/retry details authoritative. Source: the user selected option 1 for collision handling.

Draft status

Native execute methods and factories remain staged; this PR does not claim end-to-end operation support. Exact-head native core qualification and the existing 20 interpreter/platform wheel cells are required before readiness. Final dependency-pin qualification remains separate #133 work.

@github-actions github-actions Bot added enhancement New feature or request docs test ci labels Oct 26, 2026
Comment thread native/tests/conversion_cases.py Fixed
Convert the ten approved operation dictionaries directly into owned Rust
SDK inputs before releasing the GIL. Generate typed converters from the
compatibility manifest and locked SDK source, including legacy members,
precise numeric strings and binary values.

Project SDK outputs with optional members intact and use the final captured
response for defaulted scalar presence and operation metadata. Decode
compressed service responses in Rust while retaining original wire context.

Map native validation, service failures and structured transport categories
through cached botocore exception models without creating a Python client
or invoking its serializers and transport.

Keep interface version 1, the current Rust pin and staged operation boundary.
Add embedded-Python differential checks and installed-wheel error coverage;
renew source-bound startup evidence for the manifest changes.

Refs #134
Part of #131.
Pop the Error member before checking it so result cleanup does not depend
on Python assertion evaluation.
Project only required final-response presence paths, borrowing raw JSON
branches so large attributes are not copied into a second value tree.
Generate routes from the pinned model and SDK, retaining nested batch
markers and typed output values.

Keep empty and non-JSON HTTP failures as service errors with wire context.
Preserve unknown integers, null error members and repeated headers while
retaining runtime failures for successful-status SDK decoding and malformed
HTTP. Leave modeled request maximums to the service, matching Python.

Renew the native manifest startup anchor for the existing serde_json
precision and raw-value features. Keep the Rust pin, Tokio features and
interface version unchanged.

Refs #134
Match the pinned SDK's single-member gzip and per-layer expansion limits.
Keep TLS handshake I/O retryable and wrap transport failures only once.

Preserve proxy service messages, explicit null messages and overflowing
unknown JSON floats while retaining wire context and exact integers.
Cover error-code precedence and these boundaries through the existing
converter and local-socket SDK tests.

Refs #134
Accept the pinned SDK's compression aliases and parameterized tokens.
Preserve expansion-limit diagnostics when considering raw-deflate
fallback, so oversized responses retain their actual failure reason.

Match botocore's generic server-error handling for empty and HTML bodies,
including HTML bytes outside UTF-8. Retain status/reason fields and wire
context while keeping non-generic decode failures as protocol errors.

Abort the TLS EOF test's peer task when its owner unwinds or times out,
so a failed assertion cannot leave its listener on the shared runtime.

Refs #134
@dkropachev
dkropachev force-pushed the feat/native-conversion-errors branch from d255012 to f82adcf Compare October 27, 2026 06:18
Retain unrelated fields in dictionary Error and ResponseMetadata payloads
while canonical exception, HTTP and retry fields remain authoritative.
Reserve optional metadata names even when genuine values are absent, and
keep overwritten originals in the raw body. Document these exceptions.

Normalize malformed modeled error shapes to protocol failures, preserve
supported null variants and sanitize error-code suffixes before namespaces.
Retain TLS protocol retries separately from their Python category and
preserve the original request URL in transport failures.

Keep test peers owned on unwind and isolate the metadata retry barrier
from the SDK upload observer's periodic timers. Preserve its timeout and
attempt assertions while qualifying the combined source.

Refs #134
@coderabbitai

coderabbitai Bot commented Oct 27, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Borrow path fragments through standard formatting instead of retaining
an allocated prefix at every recursive conversion frame. Render the
useful path only for a validation error, preserving the depth guard and
owned SDK values without adding input limits or path context state.

Report length bounds in plain language. Reject a second active member in
modeled error AttributeValues while preserving supported null members,
ignored protocol tags and single unknown variants.

Refs #134

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci docs enhancement New feature or request test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement native DynamoDB value conversion and Python error mapping

1 participant