fix(stack-encrypt)!: align and test context composition

conformance/stack-encrypt/README.md

@@ -5,6 +5,47 @@ This directory holds a shared test contract for Stack Encrypt bindings. The
 sets out the wider work; [coverage.md](coverage.md) maps existing evidence and
 gaps. These profiles are **experimental**, not certification of an entire SDK.
 
+## Files
+
+Each file in `v1/` is named `<profile>.<role>.json`. `<profile>` is the exact
+`profile` value in the file's header, so searching for a profile ID finds every
+file that belongs to it. `v1/` is the corpus format version, which is separate
+from each profile's own version. `<role>` is one of:
+
+- `cases`: the shared, language-neutral cases that every runner executes. When
+  an expectation is a category rather than bytes, such as an error class or a
+  retrieval count, it is written in the case itself.
+- `expected.<producer>`: frozen outputs keyed by the case IDs in `cases`,
+  produced by `<producer>`. `rust-native` is the native Rust engine. Only the
+  producer regenerates this file, deliberately; runners compare against it.
+- `inputs.<runner>`: extra inputs keyed by case ID that only `<runner>` needs,
+  such as Go host values that have no language-neutral form.
+
+Every file has a `version` and `profile` header. An `expected` file also names
+its `producer`, and an `inputs` file names its `runner`.
+`scripts/__tests__/stack-conformance-ci.test.mjs` fails on a file that breaks
+these rules.
+
+| File | Role | Rust reader | Go reader |
+|---|---|---|---|
+| `record-scalars-v1.cases.json` | cases | `tests/conformance.rs` | `conformance_test.go` |
+| `record-scalars-v1.expected.rust-native.json` | expected | `tests/conformance.rs` (writes it) | `conformance_test.go` |
+| `record-declarations-v1.cases.json` | cases | `tests/declarations.rs` | `declaration_conformance_test.go` |
+| `record-declarations-v1.inputs.go.json` | Go inputs | — | `declaration_conformance_test.go` |
+| `record-reads-v1.cases.json` | cases | `tests/read_conformance.rs` | `read_conformance_test.go` |
+| `record-runtime-v1.cases.json` | cases | `tests/runtime_conformance.rs` | `runtime_conformance_test.go` |
+| `record-contexts-v1.cases.json` | cases | `tests/context_conformance.rs` | `context_conformance_test.go` |
+| `record-contexts-v1.expected.rust-native.json` | expected | `tests/context_conformance.rs` (writes it) | `context_conformance_test.go` |
+| `record-context-boundaries-v1.cases.json` | cases | `tests/context_conformance.rs` | `context_conformance_test.go` |
+| `fixture-enrollment.json` | index | `tests/common/mod.rs` | `fixture_conformance_test.go` |
+
+Rust readers are under `packages/stack-encrypt/`; Go readers are under
+`languages/golang/encrypt/`. `fixture-enrollment.json` is not a profile: it maps
+requirement IDs onto existing fixtures and tests (see
+[the fixture contract](fixture-contract.md)), and
+`scripts/stack-conformance-selection.mjs` reads it to choose the Go tests that
+`mise run test:conformance` runs.
+
 ## Run
 
 From the repository root, with the pinned mise tools installed:
@@ -21,18 +62,27 @@ native Rust against the ciphertexts Go has just produced. It never regenerates
 expectations. CI also runs the Go cases on its other supported host platforms.
 The same task runs the shared declaration acceptance/rejection corpus in both
 languages, with native observations of normalization. It also runs the shared
-runtime and read cases, and enrolls the existing value-kind and query-operator
+runtime, read and context cases, and enrolls the existing value-kind, context-field and query-operator
 fixtures described below. Set no fixture-update variables when checking conformance.
 
 The Rust runner is `packages/stack-encrypt/tests/conformance.rs`; the Go runner
 is `languages/golang/encrypt/conformance_test.go`. Go calls the record interface
 used by generated code. Testing public SDK declarations is a separate layer.
 Python and TypeScript runners are pending; their absence is not a passing result.
 
+## Context composition and expectations
+
+The [context contract](context-contract.md) covers ordered extensions, canonical
+context encodings, typed/generated API calls, stored-context expectations,
+tampering and preflight boundaries. The required command runs the new
+`context_conformance` Rust binary and `TestContextConformance` /
+`TestContextBoundaries` Go runners, together with native `plan_grammar` and
+existing context-field fixture tests.
+
 ## Existing value-kind and query-operator fixtures
 
 The [fixture contract](fixture-contract.md) defines `record-value-kinds-v1` and
-`record-query-operators-v1`. Their [enrollment index](v1/fixture-profiles.json)
+`record-query-operators-v1`. Their [enrollment index](v1/fixture-enrollment.json)
 maps stable requirement IDs to the existing Rust and Go tests. The fixtures
 remain under `packages/stack-encrypt/tests/fixtures`; no vectors are copied.
 
@@ -48,7 +98,7 @@ operations remain separate work.
 
 ## `record-reads-v1` contract
 
-The [read contract](read-contract.md) defines cases in `v1/reads.json` for
+The [read contract](read-contract.md) defines cases in `v1/record-reads-v1.cases.json` for
 malformed stored records, authenticated read-type failures and context preflight.
 The native runner observes retrieval stages. Go exercises the same cases through
 an uninitialised guest and a configured cipher, plus the normal record adapter
@@ -58,7 +108,7 @@ crate and `encrypt/read_conformance_test.go` in the Go module.
 
 ## `record-runtime-v1` contract
 
-The [runtime contract](runtime-contract.md) defines cases in `v1/runtime.json`:
+The [runtime contract](runtime-contract.md) defines cases in `v1/record-runtime-v1.cases.json`:
 missing and explicit null values, SQL-null omission, sealed-null storage and
 equality, typed batch round trips, write validation and lossless query conversions.
 The runners are `packages/stack-encrypt/tests/runtime_conformance.rs` and
@@ -69,7 +119,7 @@ data-key call counts; Go host source validation preserves shared structured diag
 
 The [declaration grammar](declaration-grammar.md) defines the engine's current
 record-plan syntax, defaults, context rules, kind/index table and null policies.
-`v1/declarations.json` contains cases, including malformed frames and all
+`v1/record-declarations-v1.cases.json` contains cases, including malformed frames and all
 21 kinds against five scalar indexes. The resolver is explicitly `none`; EQL
 targets need their own profile.
 
@@ -80,11 +130,11 @@ frames document the different native-parser and WASI-boundary classifications.
 Native Rust also checks normalization observations. Go's checker does not expose
 the parsed plan, so it asserts acceptance and diagnostics against both release
 and deterministic guests. Representable cases also exercise `Checker.Check`
-through explicit host plan fixtures in `v1/go-declarations.json`.
+through explicit host plan fixtures in `v1/record-declarations-v1.inputs.go.json`.
 
 ## `record-scalars-v1` contract
 
-`v1/cases.json` has a corpus format `version`, a semantic `profile`, a public
+`v1/record-scalars-v1.cases.json` has a corpus format `version`, a semantic `profile`, a public
 deterministic-provider `seed`, a record `context`, a `field`, an exact list of
 `required_capabilities`, and uniquely identified `cases`. Unknown fields,
 versions, profiles, capabilities and input types fail. Every case must run.
@@ -128,7 +178,7 @@ profiles or cases. The full SQL behavior is not part of this initial profile.
 
 ## Artifacts and evolution
 
-`v1/native.json` is a frozen artifact set produced by the Rust owned scalar API.
+`v1/record-scalars-v1.expected.rust-native.json` is a frozen artifact set produced by the Rust owned scalar API.
 It records `version`, `profile`, `producer`, and a case-ID map containing hex
 `frame`, `ciphertext` and `equality` bytes. The map must contain exactly every
 positive case; missing or extra cases fail. Every key and plaintext here is

conformance/stack-encrypt/context-contract.md

@@ -0,0 +1,126 @@
+# Context composition
+
+`record-contexts-v1` (`v1/record-contexts-v1.cases.json`) specifies composition for the current
+record API. `record-context-boundaries-v1` (`v1/record-context-boundaries-v1.cases.json`) specifies
+stored-context expectations and refusal boundaries. Both are required in Rust
+and Go. Python and TypeScript must implement these profiles when their runners
+are added; their absence is not conformance evidence.
+
+## Current semantics
+
+A field's base is a plain label: the record's declared label followed by the
+field's identity. A context-field record obtains the record label from each
+row's stored field. A Rust plan built without a context can instead take that
+label from the call. Go generated types declare either a literal context or a
+context field; they do not expose Rust's contextless saved-plan builder.
+
+Caller input has two distinct purposes:
+
+- A **base label** supplies a missing base for a query or a contextless Rust plan.
+  For a record storing its own context, an expected label checks that stored
+  value. It does not replace or extend it. A caller label on a plan with a
+  literal base is refused.
+- An **extension** appends to the resolved field context for encryption,
+  decryption and search. Each call appends in order. Rust
+  `.extend(7u64).extend(nonempty!("eu"))` and Go
+  `.Extend(uint64(7)).Extend("eu")` produce `((label)/7u64)/eu`.
+  Rust `.extend(())` and Go `.Extend()` append nothing. A Rust
+  `DeclaredContext` containing several parts appends all of those parts.
+
+Grouping is significant. Rust `.extend(NonEmpty::from(7u64).with("eu"))` adds
+one composite part, producing `(label)/(7u64/eu)`. This has different AAD and
+terms from separate extensions. Nested `ContextPiece` values are supported by
+Rust's lower-level context APIs; the dynamic record grammar and Go `Extend`
+accept scalar extension parts only. Their refusal of a grouped extension is
+part of the supported API boundary, not permission to flatten it.
+
+Previously, repeated Rust chain extensions replaced earlier parts. Records
+written by those chains still require the
+**last extension alone** when opening them; callers must explicitly retain that
+old context when reading and querying old records. A query using repeated
+extensions now succeeds with different terms, silently missing old rows.
+Re-encrypt and re-index old rows to adopt the appended context. This change
+does not rewrite ciphertext or migrate stored terms.
+
+The native base and each appended scalar retain their type and structure.
+`7i32`, `7i64`, `7u64`, text `eu`, and bytes `6575` do not collapse to one AAD
+or PRF context. Descriptor rendering is deliberately coarser in some cases:
+text and bytes can render alike, as can positive signed and unsigned integers.
+A matching descriptor alone therefore does not prove matching AAD. Empty text
+is accepted by dynamic record/Go extension APIs when the base remains nonempty;
+Rust's `NonEmpty` constructor rejects an empty standalone extension. The
+composition runner records this API difference instead of bypassing it.
+
+## Composition evidence
+
+Each case has a unique `id`, `base` (`literal` or `field`), nonempty `label`, and
+ordered `parts`. Each part is `{ "type": ..., "text": ... }`: string, int32,
+int64, uint64, or bytes (hex). Integers are decimal strings, never JSON numbers.
+Every case encrypts and queries the string `hello` in the `text` field, using
+the public all-zero deterministic-provider test seed.
+
+`v1/record-contexts-v1.expected.rust-native.json` is keyed by exactly those case IDs. It pins the
+resolved context frame, the field context frame sent in a plan, descriptor,
+AAD, PRF bytes, one Rust ciphertext and equality term. Rust checks actual descriptors sent to the key provider, fresh typed and
+dynamic writes, queries, opens, and frozen ciphertexts. The typed chain calls
+`extend` repeatedly, including a final no-op, so replacement behavior cannot
+regress unnoticed.
+
+Go observes the actual plan buffer at the `se_encrypt_record` WASM boundary
+through a test-only function wrapper. It compares the field context in those
+bytes with the frozen native plan-context frame, then opens the Rust ciphertext.
+It compares fresh stored and query terms, both variadic and repeated extensions,
+and generated `Note` (field base) and `LiteralNote` (literal base) methods.
+Go does not expose a separate descriptor/AAD encoder: matching the actual plan
+and successful authenticated cross-opening provide indirect evidence for those
+observations. Native Rust checks the canonical descriptor/AAD/PRF bytes directly.
+Ciphertext bytes are randomized and are not compared for equality.
+
+Regenerate deliberately with `STACK_ENCRYPT_UPDATE_FIXTURES=1` and the native
+`context_conformance` binary. Required conformance rejects fixture updates.
+Changes to frozen context encodings require compatibility review.
+
+## Expectations and boundaries
+
+Boundary cases specify rows' `tenants`, the caller's `expected` label (the empty
+string explicitly means no expectation in this corpus), a mutation of the last
+stored row, an `outcome`, and `native_retrievals`. All properties are required;
+null is not an alternative spelling for omission or the empty string.
+
+Cases cover unrestricted reads, matching expectations, late mismatches, empty
+batches, changed stored labels with and without matching expectations, and
+missing/empty stored contexts late in a batch. A two-row ciphertext-tampering
+case reaches AEAD after retrieval in both languages and returns no partial
+plaintext. Typed Rust opening and generated Go opening exercise the boundary
+cases too, except that an absent member cannot be represented by a generated Go
+struct and is tested at the raw-tree boundary.
+
+The separate `queries` array specifies a caller `context`, an `outcome` and
+whether the query `matches` the stored term for `tenants/acme`. Missing and
+invalid contexts refuse; another tenant produces a valid nonmatching query.
+Rust typed query chains and generated Go query methods execute these cases.
+Key-independent refusals must make no retrieval. An empty valid batch also makes none. Changing the stored
+label does not confer permission to open the ciphertext even if the caller's
+expectation matches the replacement label.
+
+Native `CountingSource` checks exact data-key call counts. Its fake provider
+does not authenticate descriptors, so a tampered context fails AEAD after one
+retrieval. Go's deterministic provider authenticates the descriptor and refuses
+the same tampering with `ErrForbidden`. An uninitialised Go checker separately
+proves the key-free preflight boundary. This is not a claim that Go exposes
+provider call counters.
+
+The earlier read profile continues to cover context storage shape and exact
+structured diagnostics. The enrolled `record-context-fields-v1` fixture
+retains generated Go expectation, query, tampering and single-field update
+checks and Rust typed/dynamic cross-opening. The required native `plan_grammar`
+binary additionally covers missing caller bases, conflicting sources, invalid
+labels, caller-derived queries and empty-batch validation before keyset loading.
+
+## Future roles
+
+This profile describes the current shared context: it feeds the data-key
+descriptor, ciphertext AAD and index PRF context. It does not introduce separate
+ciphertext-binding and search-scope contexts. That API design remains separate work;
+any future split needs explicit compatibility rules and new profile coverage.
+It also does not assert plaintext erasure in Go, Python or TypeScript heaps.

conformance/stack-encrypt/coverage.md

@@ -8,7 +8,7 @@ Paths are relative to the repository root. New profile cases run from `v1/`.
 |---|---|---|
 | Scalar value widths and lossless transport | Initial scalar cases and [value-kind requirements](fixture-contract.md#value-kind-requirements), executed by the existing Rust and generated Go runners | Complete per-kind boundary matrices, bytes and broader floats; Python/TS mappings |
 | Owned opaque ciphertext and storage envelope | `packages/stack-encrypt/tests/scalar.rs`, `frozen_bytes.rs`; initial cross-opening cases | Blob profile; encoded targets; supported version matrix |
-| Record declaration and lowering | [Declaration grammar](declaration-grammar.md); `v1/declarations.json` shared by native Rust and Go; native normalization observations; existing record lowering suites | EQL-resolver profile; SDK-to-declaration mappings; runtime source/read shapes |
+| Record declaration and lowering | [Declaration grammar](declaration-grammar.md); `v1/record-declarations-v1.cases.json` shared by native Rust and Go; native normalization observations; existing record lowering suites | EQL-resolver profile; SDK-to-declaration mappings; runtime source/read shapes |
 | Context binding and composition | `packages/stack-encrypt/tests/target.rs`, `descriptor.rs`, `record_lowering.rs`; `languages/golang/encrypt/context_field_test.go`; initial wrong-context cases | Shared composition vectors; expected-context and required-input matrix |
 | Query terms and diagnostics | [Query-operator requirements](fixture-contract.md#query-operator-requirements), existing Rust/Go runners and initial equality/query-error cases | Common rejection corpus for every operator, capability and input shape; wider kind/index combinations |
 | Missing, SQL NULL and sealed null | `packages/stack-encrypt/src/dynamic/record.rs` null tests; `languages/golang/encrypt/null_test.go` | Explicit language mappings and common missing/null fixtures |
@@ -25,7 +25,7 @@ absence is a delivery gap, not a test skip that counts toward certification.
 
 ## Runtime checkpoint
 
-`v1/runtime.json` adds shared cases for typed writes and reads, null storage,
+`v1/record-runtime-v1.cases.json` adds shared cases for typed writes and reads, null storage,
 batches, equality queries and lossless numeric conversion. See the
 [runtime contract](runtime-contract.md) for exact observations and limits.
 Native data-key counters cover zero-call preflight failures and single-call
@@ -41,7 +41,7 @@ batches; Go provider-call instrumentation remains open.
 
 ## Read checkpoint
 
-`v1/reads.json` adds cases for stored-shape validation, authenticated types
+`v1/record-reads-v1.cases.json` adds cases for stored-shape validation, authenticated types
 and context preflight. The [read contract](read-contract.md) distinguishes native
 shape/pending stages and the guest's earlier `check_record` diagnostics. Native
 data-key counters and Go's uninitialised guest verify preflight boundaries.
@@ -60,7 +60,7 @@ raw-byte profile must cover that boundary explicitly.
 ## Consolidation policy
 
 The value-kind/query-operator enrollment adds named requirements for the
-existing fixtures through [the enrollment index](v1/fixture-profiles.json).
+existing fixtures through [the enrollment index](v1/fixture-enrollment.json).
 The conformance command selects their original runners; CI guards the mapping
 and selection. Typed Rust comparisons, generated Go methods, verification and
 fetch-limit checks stay in those runners. No fixture is copied and no existing
@@ -91,3 +91,18 @@ For each removal, link the old assertion to stable case IDs, compare coverage
 and failure behavior, and run both runners in CI. Preserve unrelated assertions
 from mixed tests. Enroll Python and TypeScript against the existing corpus so
 adding a language does not create another copy of the expected behavior.
+
+## Context checkpoint
+
+The [context contract](context-contract.md) adds 18 shared composition cases and
+eleven shared read boundary cases and four query cases, plus enrollment of the
+existing record-lowering
+context-field fixture. Rust typed write/query/open chains and Go generated
+methods execute the composition cases. Required native plan-grammar tests cover
+caller-provided bases and empty-batch source validation. Existing context tests
+remain: no mixed-purpose Rust or Go test is removed.
+
+Go proves canonical AAD indirectly through its actual guest plan frame and native
+ciphertext opening; provider-call instrumentation remains a separate gap.
+Nested composite extensions remain a native API capability rather than an
+unsupported feature silently advertised by the Go/dynamic record profile.