feat(data): default Plutus data to PLUTUS_DATA_OPTIONS

.changeset/data-default-plutus-layout.md

@@ -0,0 +1,24 @@
+---
+"@evolution-sdk/evolution": minor
+---
+
+Plutus data is now encoded by default with `CBOR.PLUTUS_DATA_OPTIONS`, the layout the node writes: non-empty lists and constructor fields indefinite, maps definite. Before, the default was `CBOR.CML_DATA_DEFAULT_OPTIONS`, which also wrote non-empty maps indefinite. `Data.DEFAULT_CBOR_OPTIONS` and `CBOR.TX_DEFAULT_OPTIONS.plutusData` are now `CBOR.PLUTUS_DATA_OPTIONS`, and `CBOR.toTxCodecOptions` maps `CBOR.CML_DEFAULT_OPTIONS` to it.
+
+Only data that holds a non-empty map changes, and only in its map headers. `Data.map([[1n, 2n]])` was `bf0102ff` and is now `a10102`. For such data these change:
+
+- datum hashes from `Data.toDatumHash`, which now match the hashes the node computes
+- inline datums, witness datums and redeemer data in new transactions
+- script data hashes of transactions that carry such datums or redeemers
+
+Applied script hashes do not change, since `UPLC.applyParamsToScript` already used `CBOR.PLUTUS_DATA_OPTIONS`. Decoding is unchanged, and a decoded transaction keeps the original bytes of its datums and redeemers.
+
+To keep the old bytes, pass the old options:
+
+```ts
+Data.toDatumHash(datum, CBOR.CML_DATA_DEFAULT_OPTIONS)
+Transaction.toCBORHex(tx, { ledger: CBOR.CML_DEFAULT_OPTIONS, plutusData: CBOR.CML_DATA_DEFAULT_OPTIONS })
+```
+
+The transaction options cover witness datums and redeemer data. An inline datum in a new output is written with the `Data` default under every option set, as before, so it takes the new layout.
+
+`CBOR.CML_DATA_DEFAULT_OPTIONS` is deprecated in favor of `CBOR.PLUTUS_DATA_OPTIONS`. It keeps its bytes, but matches no tool exactly, so use it only to reproduce bytes and datum hashes written before this release.

.changeset/plutus-data-tx-options.md

@@ -6,7 +6,7 @@ Witness datums and redeemer data are now written with their own Plutus data opti
 
 `CBOR.TxCodecOptions` holds two sets of options. `ledger` is for ledger structures: the body, the witness set, and the containers that hold datums and redeemers. `plutusData` is for Plutus data items: witness datums and redeemer data. Two presets come with it:
 
-- `CBOR.TX_DEFAULT_OPTIONS`: `ledger` is `CBOR.CML_DEFAULT_OPTIONS` and `plutusData` is `CBOR.CML_DATA_DEFAULT_OPTIONS`, the `Data` default.
+- `CBOR.TX_DEFAULT_OPTIONS`: `ledger` is `CBOR.CML_DEFAULT_OPTIONS` and `plutusData` is `CBOR.PLUTUS_DATA_OPTIONS`, the `Data` default.
 - `CBOR.TX_CANONICAL_OPTIONS`: `CBOR.CANONICAL_OPTIONS` for both.
 
 The encoders and schemas of `Transaction`, `TransactionWitnessSet`, `Redeemers` and `Redeemer`, and `Redeemers.toScriptDataHash`, take `CBOR.TxCodecOptions` and default to `CBOR.TX_DEFAULT_OPTIONS`:
@@ -17,6 +17,6 @@ Transaction.toCBORHex(tx, CBOR.TX_CANONICAL_OPTIONS)
 Transaction.toCBORHex(tx, { ledger: CBOR.CML_DEFAULT_OPTIONS, plutusData: CBOR.PLUTUS_DATA_OPTIONS })
 ```
 
-Plain `CBOR.CodecOptions` are still accepted, read as `CBOR.toTxCodecOptions` reads them: the options serve as both, except that `CBOR.CML_DEFAULT_OPTIONS` writes Plutus data with `CBOR.CML_DATA_DEFAULT_OPTIONS`, as passing no options does. Decoded datums and redeemers keep their bytes, and a datum or redeemer added to a decoded transaction is written with the default Plutus data options.
+Plain `CBOR.CodecOptions` are still accepted, read as `CBOR.toTxCodecOptions` reads them: the options serve as both, except that `CBOR.CML_DEFAULT_OPTIONS` writes Plutus data with `CBOR.PLUTUS_DATA_OPTIONS`, as passing no options does. Decoded datums and redeemers keep their bytes, and a datum or redeemer added to a decoded transaction is written with the default Plutus data options.
 
-Script transactions whose redeemer data or witness datums hold a non-empty list, map or constructor fields change bytes. Simple data such as `Data.constr(0n, [])` or an integer keeps its bytes. The builder wrote redeemer data definite, for example `d87982a101028103`, and now writes it in the data default with indefinite lists, maps and constructor fields: `d8799fbf0102ff9f03ffff`. To keep the old redeemer layout, pass `{ ledger: CBOR.CML_DEFAULT_OPTIONS, plutusData: CBOR.CML_DATA_DEFINITE_OPTIONS }` to the encoders and to `Redeemers.toScriptDataHash`.
+Script transactions whose redeemer data or witness datums hold a non-empty list, map or constructor fields change bytes. Simple data such as `Data.constr(0n, [])` or an integer keeps its bytes. The builder wrote redeemer data definite, for example `d87982a101028103`, and now writes it in the data default with indefinite lists and constructor fields and a definite map: `d8799fa101029f03ffff`. To keep the old redeemer layout, pass `{ ledger: CBOR.CML_DEFAULT_OPTIONS, plutusData: CBOR.CML_DATA_DEFINITE_OPTIONS }` to the encoders and to `Redeemers.toScriptDataHash`.

docs/content/docs/encoding/cbor.mdx

@@ -123,8 +123,8 @@ The plain `toCBORHex`/`toCBORBytes` APIs accept a `CodecOptions` argument that c
 |--------|----------|
 | `CML_DEFAULT_OPTIONS` *(default)* | General Cardano use — definite lengths, minimal integer encoding |
 | `CANONICAL_OPTIONS` | RFC 8949 canonical: sorted keys, minimal encoding |
-| `PLUTUS_DATA_OPTIONS` | Plutus data in the node layout: indefinite lists and constructor fields, definite maps. The default for `UPLC.applyParamsToScript` and `UPLC.dataConstant`; matches `cardano-cli`, Aiken `cbor.serialise()`, and `aiken blueprint apply` except for constructor indices above 127 and the integer -2^64 |
-| `CML_DATA_DEFAULT_OPTIONS` | Matches CML `PlutusData.to_cardano_node_format()`: indefinite lists and maps. CML also sorts map keys; this preset keeps insertion order. The default for `Data` encoding |
+| `PLUTUS_DATA_OPTIONS` | Plutus data in the node layout: indefinite lists and constructor fields, definite maps. The default for `Data` encoding, `UPLC.applyParamsToScript` and `UPLC.dataConstant`; matches `cardano-cli`, Aiken `cbor.serialise()`, and `aiken blueprint apply` |
+| `CML_DATA_DEFAULT_OPTIONS` | Deprecated. Indefinite lists, constructor fields and maps, keys in insertion order. The `Data` default before `PLUTUS_DATA_OPTIONS`; use it only to reproduce datum bytes and hashes the SDK wrote before. It matches no tool exactly: CML `PlutusData.to_cardano_node_format()` also sorts map keys |
 | `CML_DATA_DEFINITE_OPTIONS` | Matches CML `PlutusData.to_cbor_hex()` on new data: definite lists and maps |
 | `AIKEN_DEFAULT_OPTIONS` | Deprecated alias of `PLUTUS_DATA_OPTIONS` |
 | `CARDANO_NODE_DATA_OPTIONS` | Deprecated alias of `CML_DATA_DEFINITE_OPTIONS` |

docs/content/docs/encoding/data.mdx

@@ -199,7 +199,7 @@ const datum = Data.constr(0n, [
 
 // Encode to hex string
 const cborHex = Data.toCBORHex(datum)
-// "d8799fa2646265..."
+// "d8799fa24b62656e..."
 
 // Encode to bytes  
 const cborBytes = Data.toCBORBytes(datum)
@@ -210,6 +210,8 @@ const decoded = Data.fromCBORHex(cborHex)
 // Returns original PlutusData structure
 ```
 
+By default the encoder writes the layout the node writes, `CBOR.PLUTUS_DATA_OPTIONS`: non-empty lists and constructor fields indefinite, maps definite. The map above starts with `a2`, a definite map of two entries. Earlier releases wrote maps indefinite. To reproduce those bytes or datum hashes, pass the deprecated `CBOR.CML_DATA_DEFAULT_OPTIONS` to `Data.toCBORHex`, `Data.toCBORBytes` or `Data.toDatumHash`.
+
 ## Equality Comparison
 
 Check if two PlutusData structures are equal:

docs/content/docs/introduction/important-defaults.mdx

@@ -78,16 +78,25 @@ const tx2 = await template.build() // No contamination from tx1
 
 ### CBOR: Aiken-Compatible by Default
 
-`UPLC.applyParamsToScript()` encodes parameters with `CBOR.PLUTUS_DATA_OPTIONS` by default: indefinite-length lists and constructor fields, definite-length maps. This is the layout the node, `cardano-cli`, and `aiken blueprint apply` use. If you're using a different Plutus toolchain, you may need different options.
+Plutus data is encoded with `CBOR.PLUTUS_DATA_OPTIONS` by default: indefinite-length lists and constructor fields, definite-length maps. This is the layout the node, `cardano-cli`, and `aiken blueprint apply` use. The default covers `Data.toCBORHex()`, `Data.toDatumHash()`, inline and witness datums, redeemer data, and the parameters of `UPLC.applyParamsToScript()`. If you're using a different Plutus toolchain, you may need different options.
 
 ```typescript
 // Default: node layout, same as aiken blueprint apply
 UPLC.applyParamsToScript(script, params)
 
-// Indefinite lists and maps, like CML to_cardano_node_format() (map keys keep insertion order):
-UPLC.applyParamsToScript(script, params, CBOR.CML_DATA_DEFAULT_OPTIONS)
+// Definite lists and maps, like CML to_cbor_hex() on new data:
+UPLC.applyParamsToScript(script, params, CBOR.CML_DATA_DEFINITE_OPTIONS)
 ```
 
+Before this default, `Data` wrote non-empty maps indefinite, so a datum that holds a map had different bytes and a different datum hash. To reproduce those, pass the deprecated `CBOR.CML_DATA_DEFAULT_OPTIONS`:
+
+```typescript
+Data.toDatumHash(datum, CBOR.CML_DATA_DEFAULT_OPTIONS)
+Transaction.toCBORHex(tx, { ledger: CBOR.CML_DEFAULT_OPTIONS, plutusData: CBOR.CML_DATA_DEFAULT_OPTIONS })
+```
+
+The transaction options cover witness datums and redeemer data. An inline datum in a new output always uses the `Data` default.
+
 ### Scripts Are Double-CBOR Encoded
 
 Compiled Plutus scripts from Aiken/Plutarch are typically double-CBOR encoded (flat bytes → CBOR bytes → CBOR bytes). The SDK handles this automatically, but if you're working with raw script bytes, be aware of the encoding level.

docs/content/docs/smart-contracts/apply-params.mdx

@@ -134,11 +134,11 @@ const applied = UPLC.applyParamsToScript(compiledScript, [
   Data.int(42n),
 ])
 
-// With indefinite lists and maps, like CML to_cardano_node_format() (map keys keep insertion order)
+// With definite lists and maps, like CML to_cbor_hex() on new data
 const appliedCml = UPLC.applyParamsToScript(
   compiledScript,
   [Data.int(42n)],
-  CBOR.CML_DATA_DEFAULT_OPTIONS,
+  CBOR.CML_DATA_DEFINITE_OPTIONS,
 )
 ```
 

packages/evolution-devnet/test/Transaction.InlineDatumBytes.test.ts

@@ -84,10 +84,11 @@ describe("Inline datum bytes (Devnet Submit)", () => {
     const client = createTestClient()
     const myAddress = await client.address()
 
-    // The builder writes both maps indefinite (bf...ff). The second one is then
-    // rewritten as a definite map (a1...), one byte shorter, so the fee still covers it.
-    const indefiniteDatum = "bf0102ff"
-    const definiteDatum = "a10304"
+    // The builder writes both maps definite (a1...). The second one, {3: 1000}, is
+    // then rewritten as the indefinite map {3: 4} (bf...ff), one byte shorter, so
+    // the fee still covers it.
+    const definiteDatum = "a10102"
+    const indefiniteDatum = "bf0304ff"
     const signBuilder = await client
       .newTx()
       .payToAddress({
@@ -98,14 +99,14 @@ describe("Inline datum bytes (Devnet Submit)", () => {
       .payToAddress({
         address: myAddress,
         assets: Cardano.Assets.fromLovelace(5_000_000n),
-        datum: new InlineDatum.InlineDatum({ data: PlutusData.map([[PlutusData.int(3n), PlutusData.int(4n)]]) })
+        datum: new InlineDatum.InlineDatum({ data: PlutusData.map([[PlutusData.int(3n), PlutusData.int(1000n)]]) })
       })
       .build({ availableUtxos: [...genesisUtxos] })
 
     const builtHex = Transaction.toCBORHex(await signBuilder.toTransaction())
-    expect(builtHex.split("d81844" + indefiniteDatum).length).toBe(2)
-    expect(builtHex.split("d81844bf0304ff").length).toBe(2)
-    const txHex = builtHex.replace("d81844bf0304ff", "d81843" + definiteDatum)
+    expect(builtHex.split("d81843" + definiteDatum).length).toBe(2)
+    expect(builtHex.split("d81845a1031903e8").length).toBe(2)
+    const txHex = builtHex.replace("d81845a1031903e8", "d81844" + indefiniteDatum)
     const txBytes = Bytes.fromHex(txHex)
     const bodyBytes = Transaction.extractBodyBytes(txBytes)
     const expectedTxId = TransactionHash.toHex(TransactionBody.toHashFromBytes(bodyBytes))
@@ -131,6 +132,6 @@ describe("Inline datum bytes (Devnet Submit)", () => {
     const datums = (utxos.result as Array<{ index: number; datum?: string }>)
       .sort((a, b) => a.index - b.index)
       .map((utxo) => utxo.datum)
-    expect(datums).toEqual([indefiniteDatum, definiteDatum])
+    expect(datums).toEqual([definiteDatum, indefiniteDatum])
   })
 })

packages/evolution-devnet/test/Transaction.RedeemerMapBytesKey.test.ts

@@ -129,11 +129,11 @@ describe("Redeemer map with byte-string key (Devnet Submit)", () => {
     const scriptUtxos = await client.getUtxos(scriptAddress)
     expect(scriptUtxos.length).toBe(1)
 
-    // The builder writes {h'01': [100, 2]} in the data default, an indefinite map and
-    // list (bf 41 01 9f 1864 02 ff ff). It is rewritten as {h'01': [1, 2]} with a
-    // definite map and an indefinite list (a1 41 01 9f 01 02 ff), which is shorter,
-    // so the fee still covers it. The script ignores the redeemer.
-    const builtRedeemerData = "bf41019f186402ffff"
+    // The builder writes {h'01': [100, 2]} in the data default, a definite map and
+    // an indefinite list (a1 41 01 9f 1864 02 ff). It is rewritten as {h'01': [1, 2]}
+    // in the same layout (a1 41 01 9f 01 02 ff), which is shorter, so the fee still
+    // covers it. The script ignores the redeemer.
+    const builtRedeemerData = "a141019f186402ff"
     const redeemerData = "a141019f0102ff"
     const signBuilder = await client
       .newTx()

packages/evolution-devnet/test/TxBuilder.WitnessDatum.test.ts

@@ -11,7 +11,7 @@
  * signs with Transaction.addVKeyWitnessesBytes, and submits. Each case uses
  * one set of options for the script data hash and the encoding, and their
  * plutusData options for the datum hash: no options, CBOR.CML_DEFAULT_OPTIONS,
- * CBOR.TX_CANONICAL_OPTIONS, or plutusData set to PLUTUS_DATA_OPTIONS. The
+ * CBOR.TX_CANONICAL_OPTIONS, or plutusData set to CML_DATA_DEFAULT_OPTIONS. The
  * node accepts the spend only when the written datum hashes to the locked
  * datum hash and the script data hash covers the datum and redeemer as
  * written.
@@ -57,8 +57,8 @@ describe("Witness datum layout (Devnet Submit)", () => {
   const script = new PlutusV3.PlutusV3({ bytes: Bytes.fromHex(alwaysSucceed.compiledCode) })
   const scriptAddress = Address.Address.make({ networkId: 0, paymentCredential: ScriptHash.fromScript(script) })
 
-  // Constr 0 [{1: 2}, [3]]: the data default writes its map and lists
-  // indefinite, PLUTUS_DATA_OPTIONS its map definite
+  // Constr 0 [{1: 2}, [3]]: the data default writes its lists indefinite and
+  // its map definite, CML_DATA_DEFAULT_OPTIONS its map indefinite
   const mapAndList = Data.constr(0n, [Data.map([[Data.int(1n), Data.int(2n)]]), Data.list([Data.int(3n)])])
 
   // Each datum, the options it is written with, and the bytes they give for
@@ -75,15 +75,15 @@ describe("Witness datum layout (Devnet Submit)", () => {
       name: "Constr 0 [{1: 2}, [3]] with no options",
       datum: mapAndList,
       options: undefined,
-      datumHex: "d8799fbf0102ff9f03ffff",
-      redeemerHex: "d8799fbf0102ff9f03ffff"
+      datumHex: "d8799fa101029f03ffff",
+      redeemerHex: "d8799fa101029f03ffff"
     },
     {
       name: "Constr 0 [1] with CBOR.CML_DEFAULT_OPTIONS",
       datum: Data.constr(0n, [Data.int(1n)]),
       options: CBOR.CML_DEFAULT_OPTIONS,
       datumHex: "d8799f01ff",
-      redeemerHex: "d8799fbf0102ff9f03ffff"
+      redeemerHex: "d8799fa101029f03ffff"
     },
     {
       name: "Constr 0 [{1: 2}, [3]] with CBOR.TX_CANONICAL_OPTIONS",
@@ -93,11 +93,11 @@ describe("Witness datum layout (Devnet Submit)", () => {
       redeemerHex: "d87982a101028103"
     },
     {
-      name: "Constr 0 [{1: 2}, [3]] with plutusData PLUTUS_DATA_OPTIONS",
+      name: "Constr 0 [{1: 2}, [3]] with plutusData CML_DATA_DEFAULT_OPTIONS",
       datum: mapAndList,
-      options: { ledger: CBOR.CML_DEFAULT_OPTIONS, plutusData: CBOR.PLUTUS_DATA_OPTIONS },
-      datumHex: "d8799fa101029f03ffff",
-      redeemerHex: "d8799fa101029f03ffff"
+      options: { ledger: CBOR.CML_DEFAULT_OPTIONS, plutusData: CBOR.CML_DATA_DEFAULT_OPTIONS },
+      datumHex: "d8799fbf0102ff9f03ffff",
+      redeemerHex: "d8799fbf0102ff9f03ffff"
     }
   ]
 
@@ -283,7 +283,7 @@ describe("Witness datum layout (Devnet Submit)", () => {
       .build()
     const builtHex = Transaction.toCBORHex(await signBuilder.toTransaction())
     expect(builtHex).toContain(`05a182000082${Data.toCBORHex(mapAndList)}`)
-    expect(Data.toCBORHex(mapAndList)).toBe("d8799fbf0102ff9f03ffff")
+    expect(Data.toCBORHex(mapAndList)).toBe("d8799fa101029f03ffff")
 
     const txHash = await (await signBuilder.sign()).submit()
     expect(await client.awaitTx(txHash, 1000)).toBe(true)

packages/evolution/src/CBOR.ts

@@ -228,7 +228,8 @@ export const CML_DEFAULT_OPTIONS: CodecOptions = {
  * - Maps: definite-length
  * - Empty list: `80`; empty map: `a0`
  *
- * `Data.toCBORBytes` with these options writes the bytes the node writes.
+ * The `Data` default. `Data.toCBORBytes` with these options writes the bytes
+ * the node writes.
  * The rules that do not depend on options are applied by the Plutus data
  * encoder under every preset: byte strings over 64 bytes in 64-byte chunks,
  * bignums only outside -2^64 to 2^64 - 1 with their bytes chunked the same
@@ -248,17 +249,19 @@ export const PLUTUS_DATA_OPTIONS: CodecOptions = {
 } as const
 
 /**
- * CBOR encoding options for PlutusData matching CML
- * `PlutusData.to_cardano_node_format()`, except that CML also sorts map keys
- * and this preset keeps insertion order. This is the current default for
- * `Data` encoding.
+ * CBOR encoding options for PlutusData with non-empty lists, constructor
+ * fields and maps indefinite, map keys in the order given. This was the
+ * `Data` default before {@link PLUTUS_DATA_OPTIONS}.
  *
- * Uses indefinite-length lists, constructor fields, and maps. It differs from
- * the node layout ({@link PLUTUS_DATA_OPTIONS}) by writing non-empty maps
- * indefinite. The `bounded_bytes` constraint (Conway CDDL: byte strings of at
- * most 64 bytes) is applied by the Plutus data encoder, independent of these
- * codec options.
+ * It matches no tool exactly. The node, cardano-cli and Aiken write maps
+ * definite. CML 6.2.0 `PlutusData.to_cardano_node_format()` writes maps
+ * indefinite but also sorts map keys. For a constructor index above 127 the
+ * Plutus data encoder writes the tag 102 `[index, fields]` pair definite
+ * under every preset, including this one. Use it only to reproduce the bytes
+ * and datum hashes the SDK wrote before {@link PLUTUS_DATA_OPTIONS} became
+ * the default.
  *
+ * @deprecated Use {@link PLUTUS_DATA_OPTIONS}.
  * @since 1.0.0
  * @category constants
  */
@@ -357,8 +360,7 @@ export type TxCodecOptions = {
  *
  * - `ledger` is {@link CML_DEFAULT_OPTIONS}, CML's transaction encoding: map
  *   keys in insertion order, definite lengths.
- * - `plutusData` is {@link CML_DATA_DEFAULT_OPTIONS}, the current `Data`
- *   default.
+ * - `plutusData` is {@link PLUTUS_DATA_OPTIONS}, the `Data` default.
  *
  * It does not match cardano-cli, which sorts map keys and writes indefinite
  * lengths above 23 entries.
@@ -368,7 +370,7 @@ export type TxCodecOptions = {
  */
 export const TX_DEFAULT_OPTIONS: TxCodecOptions = {
   ledger: CML_DEFAULT_OPTIONS,
-  plutusData: CML_DATA_DEFAULT_OPTIONS
+  plutusData: PLUTUS_DATA_OPTIONS
 }
 
 /**
@@ -386,7 +388,7 @@ export const TX_CANONICAL_OPTIONS: TxCodecOptions = {
 /**
  * Turns plain options into transaction options: `ledger` is the options and
  * `plutusData` is the options too, except that {@link CML_DEFAULT_OPTIONS}
- * gives {@link CML_DATA_DEFAULT_OPTIONS}, as {@link TX_DEFAULT_OPTIONS} does.
+ * gives {@link PLUTUS_DATA_OPTIONS}, as {@link TX_DEFAULT_OPTIONS} does.
  * Transaction options are returned as they are.
  *
  * @since 2.0.0
@@ -395,7 +397,7 @@ export const TX_CANONICAL_OPTIONS: TxCodecOptions = {
 export const toTxCodecOptions = (options: TxCodecOptions | CodecOptions): TxCodecOptions =>
   "ledger" in options
     ? options
-    : { ledger: options, plutusData: options === CML_DEFAULT_OPTIONS ? CML_DATA_DEFAULT_OPTIONS : options }
+    : { ledger: options, plutusData: options === CML_DEFAULT_OPTIONS ? PLUTUS_DATA_OPTIONS : options }
 
 const DEFAULT_OPTIONS: CodecOptions = {
   mode: "custom",