diff --git a/antd-kotlin/README.md b/antd-kotlin/README.md index 907b26f9..4c92e2fc 100644 --- a/antd-kotlin/README.md +++ b/antd-kotlin/README.md @@ -71,6 +71,10 @@ All methods are `suspend` functions for use with Kotlin coroutines. | **Files** | `filePut`, `fileGet`, `filePutPublic`, `fileGetPublic`, `fileCost`. Private variants return a caller-held DataMap (NOT stored on-network); public variants store the DataMap on-network at the returned address. All puts and `fileCost` accept a `PaymentMode` parameter. | | **Pointers** | `pointerKeys`, `pointerCreate`, `pointerUpdate`, `pointerTransfer`, `pointerGet`, `pointerResolve`, `pointerFinality`, `pointerController`, `pointerPrepare`, `pointerFinalize`. See [Pointers](#pointers). | +### Upload results: `sha256` + +`DataPutResult`, `DataPutPublicResult`, `FilePutResult`, `FilePutPublicResult`, `FinalizeUploadResult` and `FinalizeMerkleUploadResult` carry `sha256`: the plaintext's SHA-256 (hex); requires antd 0.17.0 or later, none on older daemons (`null`). It is the last constructor parameter, defaulting to `null`, so existing code that builds these results keeps compiling. + ## Pointers A pointer is a small, owner-signed, mutable reference. Its address is derived from the owner's key and never changes; what it points at (a chunk, such as a public file's data map, or another pointer) can be changed by the owner. The daemon holds the owner keys, loaded at startup with `antd --pointer-keys ` (files in the format `ant pointer keygen` writes), so every write names the pointer by its address (`owner`), never by a key. An owner the daemon holds no key for is a `BadRequestException`. Addresses are 64 hex characters; the daemon accepts a `0x` prefix but always returns bare lowercase hex, so normalise before comparing. The pointer methods need antd 0.16.0 or later, the first release with pointers; both `AntdRestClient` and `AntdGrpcClient` implement them. @@ -133,11 +137,11 @@ decoded.entries.forEach { println("${it.effectiveName}: ${it.exactSize} bytes") | Method | Description | |---|---| -| `buildManifest(entries, options = BuildManifestOptions())` | Build a manifest. Each `ManifestEntryInput` sets exactly one of `dataMap` (hex, embedded) or `address` (hex, public), plus an optional `path` and `size` (the size to record, as given). Both or neither throws `BadRequestException` (status 400) before anything is sent, with the daemon's own message listing every bad entry (`2 invalid entries: entries[0]: give data_map or address, not both; ...`). `BuildManifestOptions` has `name`, `torrentInfoHashes` and `embedRoots`, which defaults to `true`: the daemon resolves each shrunk DataMap and embeds its root when that is at most 64 KiB, so recipients skip the fetch. Returns `BuiltManifest`: `manifest` (the raw `.ant` bytes), `link` (null when it would not fit 1,500 bytes) and `entries` | +| `buildManifest(entries, options = BuildManifestOptions())` | Build a manifest. Each `ManifestEntryInput` sets exactly one of `dataMap` (hex, embedded) or `address` (hex, public), plus an optional `path`, `size` (the size to record, as given) and `sha256` (the file's plaintext SHA-256 as hex, passed through as given and checked by the daemon, which answers a malformed one with a 400 `BadRequestException`; requires antd 0.17.0 or later). Both or neither throws `BadRequestException` (status 400) before anything is sent, with the daemon's own message listing every bad entry (`2 invalid entries: entries[0]: give data_map or address, not both; ...`). `BuildManifestOptions` has `name`, `torrentInfoHashes` and `embedRoots`, which defaults to `true`: the daemon resolves each shrunk DataMap and embeds its root when that is at most 64 KiB, so recipients skip the fetch. Returns `BuiltManifest`: `manifest` (the raw `.ant` bytes), `link` (null when it would not fit 1,500 bytes) and `entries` | | `decodeManifest(manifest, resolve = false)` | Decode `.ant` bytes. Returns `DecodedManifest`: `source` (`"manifest"`), `name`, `torrentInfoHashV1`, `torrentInfoHashV2` and `entries`. With `resolve`, the daemon fetches each entry's DataMap (not its content) to report `exactSize` and confirm it is on the network | | `decodeManifestLink(link, resolve = false)` | Decode a manifest link, a file link or a bare 64-hex address. The last two come back as `source == "file_link"` with one public entry and no path | -Each entry is a `ManifestEntryInfo`: `effectiveName` (the path, or the content address when there is none), `path`, `size` (the recorded size), `kind` (`"embedded"` or `"public"`), `contentAddress` (hex; the public DataMap address, the same for private and public uploads of one file), `dataMap` (an embedded entry's DataMap in the hex form `dataGet` / `fileGet` accept; decodes only), `exactSize` and `error`. +Each entry is a `ManifestEntryInfo`: `effectiveName` (the path, or the content address when there is none), `path`, `size` (the recorded size), `kind` (`"embedded"` or `"public"`), `contentAddress` (hex; the public DataMap address, the same for private and public uploads of one file), `dataMap` (an embedded entry's DataMap in the hex form `dataGet` / `fileGet` accept; decodes only), `exactSize`, `error` and `sha256` (the plaintext's SHA-256 the manifest records for the entry, hex; null when it records none; requires antd 0.17.0 or later). - **Bytes:** the methods take and return the raw `.ant` bytes on both transports. REST carries them as base64 inside the JSON, and the client converts. - **Sizes** are `ULong?`. Unset is `null`, never `0`, on both transports: REST omits the field, and gRPC's proto3 `optional` is read by presence. A recorded `size` is only the creator's claim. `exactSize` comes from an embedded root DataMap or a resolving decode. diff --git a/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/AntdGrpcClient.kt b/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/AntdGrpcClient.kt index 0b6134a9..822c45f7 100644 --- a/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/AntdGrpcClient.kt +++ b/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/AntdGrpcClient.kt @@ -152,7 +152,7 @@ class AntdGrpcClient internal constructor( this.data = ByteString.copyFrom(data) this.paymentMode = paymentMode.wire }) - DataPutPublicResult(address = resp.address, chunksStored = resp.chunksStored.toULong(), paymentModeUsed = resp.paymentModeUsed) + DataPutPublicResult(address = resp.address, chunksStored = resp.chunksStored.toULong(), paymentModeUsed = resp.paymentModeUsed, sha256 = resp.sha256.ifEmpty { null }) } override suspend fun dataGetPublic(address: String): ByteArray = grpc { @@ -165,7 +165,7 @@ class AntdGrpcClient internal constructor( this.data = ByteString.copyFrom(data) this.paymentMode = paymentMode.wire }) - DataPutResult(dataMap = resp.dataMap, chunksStored = resp.chunksStored.toULong(), paymentModeUsed = resp.paymentModeUsed) + DataPutResult(dataMap = resp.dataMap, chunksStored = resp.chunksStored.toULong(), paymentModeUsed = resp.paymentModeUsed, sha256 = resp.sha256.ifEmpty { null }) } override suspend fun dataGet(dataMap: String): ByteArray = grpc { @@ -347,6 +347,7 @@ class AntdGrpcClient internal constructor( gasCostWei = resp.gasCostWei, chunksStored = resp.chunksStored.toULong(), paymentModeUsed = resp.paymentModeUsed, + sha256 = resp.sha256.ifEmpty { null }, ) } @@ -366,6 +367,7 @@ class AntdGrpcClient internal constructor( gasCostWei = resp.gasCostWei, chunksStored = resp.chunksStored.toULong(), paymentModeUsed = resp.paymentModeUsed, + sha256 = resp.sha256.ifEmpty { null }, ) } @@ -536,6 +538,7 @@ class AntdGrpcClient internal constructor( chunksStored = resp.chunksStored, dataMap = resp.dataMap, dataMapAddress = resp.dataMapAddress, + sha256 = resp.sha256.ifEmpty { null }, ) } @@ -558,6 +561,7 @@ class AntdGrpcClient internal constructor( chunksStored = resp.chunksStored, dataMap = resp.dataMap, dataMapAddress = resp.dataMapAddress, + sha256 = resp.sha256.ifEmpty { null }, ) } diff --git a/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/AntdRestClient.kt b/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/AntdRestClient.kt index 6965ffaa..09350bed 100644 --- a/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/AntdRestClient.kt +++ b/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/AntdRestClient.kt @@ -216,7 +216,7 @@ class AntdRestClient( put("payment_mode", paymentMode.wire) }.toString() val resp = postJson("/v1/data/public", body) - return DataPutPublicResult(resp.address, resp.chunksStored, resp.paymentModeUsed) + return DataPutPublicResult(resp.address, resp.chunksStored, resp.paymentModeUsed, resp.sha256?.ifEmpty { null }) } override suspend fun dataGetPublic(address: String): ByteArray { @@ -230,7 +230,7 @@ class AntdRestClient( put("payment_mode", paymentMode.wire) }.toString() val resp = postJson("/v1/data", body) - return DataPutResult(resp.dataMap, resp.chunksStored, resp.paymentModeUsed) + return DataPutResult(resp.dataMap, resp.chunksStored, resp.paymentModeUsed, resp.sha256?.ifEmpty { null }) } override suspend fun dataGet(dataMap: String): ByteArray { @@ -384,6 +384,7 @@ class AntdRestClient( gasCostWei = resp.gasCostWei, chunksStored = resp.chunksStored, paymentModeUsed = resp.paymentModeUsed, + sha256 = resp.sha256?.ifEmpty { null }, ) } @@ -407,6 +408,7 @@ class AntdRestClient( gasCostWei = resp.gasCostWei, chunksStored = resp.chunksStored, paymentModeUsed = resp.paymentModeUsed, + sha256 = resp.sha256?.ifEmpty { null }, ) } @@ -556,6 +558,7 @@ class AntdRestClient( chunksStored = resp.chunksStored, dataMap = resp.dataMap, dataMapAddress = resp.dataMapAddress, + sha256 = resp.sha256?.ifEmpty { null }, ) } @@ -579,6 +582,7 @@ class AntdRestClient( chunksStored = resp.chunksStored, dataMap = resp.dataMap, dataMapAddress = resp.dataMapAddress, + sha256 = resp.sha256?.ifEmpty { null }, ) } diff --git a/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/Manifests.kt b/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/Manifests.kt index ab07c7e2..3aa43e0d 100644 --- a/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/Manifests.kt +++ b/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/Manifests.kt @@ -48,6 +48,12 @@ data class ManifestEntryInput( val dataMap: String? = null, /** A public file's DataMap address (hex, 32 bytes), as `dataPutPublic` / `filePutPublic` return it. */ val address: String? = null, + /** + * The plaintext's SHA-256 to record (hex), passed through as given: the + * daemon validates it (a bad value is a 400). Null or empty: none is + * sent. Requires antd 0.17.0 or later. + */ + val sha256: String? = null, ) /** The optional parts of an [IAntdClient.buildManifest] call. */ @@ -93,6 +99,11 @@ data class ManifestEntryInfo( val exactSize: ULong? = null, /** Why a resolving decode failed for this entry; null when it did not. */ val error: String? = null, + /** + * The plaintext's SHA-256 (lowercase hex) the manifest records for this + * entry; null when it records none. Requires antd 0.17.0 or later. + */ + val sha256: String? = null, ) /** The result of [IAntdClient.buildManifest]. */ @@ -195,6 +206,7 @@ internal fun manifestBuildBody(entries: List, options: Build // A ULong is not a Number; JsonPrimitive(ULong) writes the exact u64. e.size?.let { put("size", JsonPrimitive(it)) } if (!e.dataMap.isNullOrEmpty()) put("data_map", e.dataMap) else put("address", e.address) + e.sha256?.takeIf { it.isNotEmpty() }?.let { put("sha256", it) } } } } @@ -228,6 +240,7 @@ internal data class ManifestEntryInfoDto( @SerialName("data_map") val dataMap: String? = null, @SerialName("exact_size") val exactSize: ULong? = null, val error: String? = null, + val sha256: String? = null, ) @Serializable @@ -256,6 +269,7 @@ internal fun ManifestEntryInfoDto.toManifestEntryInfo() = ManifestEntryInfo( dataMap = dataMap?.ifEmpty { null }, exactSize = exactSize, error = error?.ifEmpty { null }, + sha256 = sha256?.ifEmpty { null }, ) /** @throws InternalException if the daemon's `manifest` is not valid base64. */ @@ -294,6 +308,7 @@ internal fun buildManifestRequestOf(inputs: List, options: B e.path?.takeIf { it.isNotEmpty() }?.let { path = it } e.size?.let { size = it.toLong() } if (!e.dataMap.isNullOrEmpty()) dataMap = e.dataMap else address = e.address.orEmpty() + e.sha256?.takeIf { it.isNotEmpty() }?.let { sha256 = it } } } } @@ -307,6 +322,7 @@ internal fun Manifests.ManifestEntryInfo.toManifestEntryInfo() = ManifestEntryIn dataMap = dataMap.ifEmpty { null }, exactSize = if (hasExactSize()) exactSize.toULong() else null, error = error.ifEmpty { null }, + sha256 = sha256.ifEmpty { null }, ) internal fun Manifests.BuildManifestResponse.toBuiltManifest() = BuiltManifest( diff --git a/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/Models.kt b/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/Models.kt index a4122149..79450289 100644 --- a/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/Models.kt +++ b/antd-kotlin/lib/src/main/kotlin/com/autonomi/sdk/Models.kt @@ -51,6 +51,11 @@ data class DataPutResult( val dataMap: String, val chunksStored: ULong = 0u, val paymentModeUsed: String = "", + /** + * The plaintext's SHA-256 (lowercase hex, 64 characters). Requires antd + * 0.17.0 or later; null on older daemons. + */ + val sha256: String? = null, ) /** @@ -63,6 +68,11 @@ data class DataPutPublicResult( val address: String, val chunksStored: ULong = 0u, val paymentModeUsed: String = "", + /** + * The plaintext's SHA-256 (lowercase hex, 64 characters). Requires antd + * 0.17.0 or later; null on older daemons. + */ + val sha256: String? = null, ) /** @@ -75,6 +85,11 @@ data class FilePutResult( val gasCostWei: String, val chunksStored: ULong, val paymentModeUsed: String, + /** + * The plaintext's SHA-256 (lowercase hex, 64 characters). Requires antd + * 0.17.0 or later; null on older daemons. + */ + val sha256: String? = null, ) /** @@ -87,6 +102,11 @@ data class FilePutPublicResult( val gasCostWei: String, val chunksStored: ULong, val paymentModeUsed: String, + /** + * The plaintext's SHA-256 (lowercase hex, 64 characters). Requires antd + * 0.17.0 or later; null on older daemons. + */ + val sha256: String? = null, ) /** Wallet address response. */ @@ -140,6 +160,11 @@ data class FinalizeUploadResult( @SerialName("chunks_stored") val chunksStored: Long, @SerialName("data_map") val dataMap: String = "", @SerialName("data_map_address") val dataMapAddress: String = "", + /** + * The plaintext's SHA-256 (lowercase hex, 64 characters). Requires antd + * 0.17.0 or later; null on older daemons. + */ + @SerialName("sha256") val sha256: String? = null, ) /** @@ -152,6 +177,11 @@ data class FinalizeMerkleUploadResult( @SerialName("chunks_stored") val chunksStored: Long, @SerialName("data_map") val dataMap: String = "", @SerialName("data_map_address") val dataMapAddress: String = "", + /** + * The plaintext's SHA-256 (lowercase hex, 64 characters). Requires antd + * 0.17.0 or later; null on older daemons. + */ + @SerialName("sha256") val sha256: String? = null, ) /** @@ -572,6 +602,7 @@ internal data class DataPutPublicDto( val address: String, @SerialName("chunks_stored") val chunksStored: ULong = 0u, @SerialName("payment_mode_used") val paymentModeUsed: String = "", + val sha256: String? = null, ) @Serializable @@ -579,6 +610,7 @@ internal data class DataPutDto( @SerialName("data_map") val dataMap: String, @SerialName("chunks_stored") val chunksStored: ULong = 0u, @SerialName("payment_mode_used") val paymentModeUsed: String = "", + val sha256: String? = null, ) @Serializable @@ -588,6 +620,7 @@ internal data class FilePutPublicDto( @SerialName("gas_cost_wei") val gasCostWei: String, @SerialName("chunks_stored") val chunksStored: ULong, @SerialName("payment_mode_used") val paymentModeUsed: String, + val sha256: String? = null, ) @Serializable @@ -597,6 +630,7 @@ internal data class FilePutDto( @SerialName("gas_cost_wei") val gasCostWei: String, @SerialName("chunks_stored") val chunksStored: ULong, @SerialName("payment_mode_used") val paymentModeUsed: String, + val sha256: String? = null, ) @Serializable @@ -676,6 +710,7 @@ internal data class FinalizeUploadDto( @SerialName("chunks_stored") val chunksStored: Long = 0, @SerialName("data_map") val dataMap: String = "", @SerialName("data_map_address") val dataMapAddress: String = "", + val sha256: String? = null, ) @Serializable diff --git a/antd-kotlin/lib/src/test/kotlin/com/autonomi/sdk/LiveSha256Test.kt b/antd-kotlin/lib/src/test/kotlin/com/autonomi/sdk/LiveSha256Test.kt new file mode 100644 index 00000000..c3091b97 --- /dev/null +++ b/antd-kotlin/lib/src/test/kotlin/com/autonomi/sdk/LiveSha256Test.kt @@ -0,0 +1,96 @@ +package com.autonomi.sdk + +import kotlinx.coroutines.runBlocking +import org.junit.jupiter.api.Assumptions.assumeTrue +import java.security.MessageDigest +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertNull +import kotlin.test.assertTrue + +/** + * Live `sha256` checks (antd 0.17.0 or later) against a running antd with a + * network behind it. Everything is free except one small public data upload + * per transport. + * + * The whole test is skipped unless all of these are set: + * + * - `ANTD_TEST_URL`: the daemon's REST base URL, e.g. `http://127.0.0.1:8082` + * - `ANTD_TEST_SHA_LINK`: a manifest link with entries `public.txt` and + * `private.txt` (each with a sha256) and `nohash.txt` (none) + * - `ANTD_TEST_SHA_PUBLIC_ADDRESS` / `ANTD_TEST_SHA_PUBLIC_SHA256`: a public + * upload and its plaintext's SHA-256 + * - `ANTD_TEST_SHA_DATA_MAP_SHA256`: the private entry's SHA-256 + * + * Used when set: + * + * - `ANTD_TEST_GRPC`: the daemon's gRPC target, e.g. `127.0.0.1:50051`, to + * repeat the checks over gRPC + * + * ```text + * ANTD_TEST_URL=http://127.0.0.1:8082 ANTD_TEST_SHA_LINK=ant://manifest/... \ + * ./gradlew :lib:test --tests '*LiveSha256Test*' + * ``` + */ +class LiveSha256Test { + + private fun env(name: String): String? = System.getenv(name)?.trim()?.takeIf { it.isNotEmpty() } + + private fun sha256Hex(bytes: ByteArray): String = + MessageDigest.getInstance("SHA-256").digest(bytes).joinToString("") { "%02x".format(it) } + + private suspend fun check( + label: String, + c: IAntdClient, + link: String, + publicAddress: String, + publicSha: String, + dataMapSha: String, + ) { + + // 1. The fixture link's entries carry (or lack) their hashes. + val decoded = c.decodeManifestLink(link, resolve = false) + val byName = decoded.entries.associateBy { it.effectiveName } + assertEquals(publicSha, byName.getValue("public.txt").sha256, "$label public.txt") + assertEquals(dataMapSha, byName.getValue("private.txt").sha256, "$label private.txt") + assertNull(byName.getValue("nohash.txt").sha256, "$label nohash.txt has no hash") + + // 2. A build carries a given hash back on the entry; 3. a bad one is a bad request. + val built = c.buildManifest(listOf(ManifestEntryInput(path = "a.txt", address = publicAddress, sha256 = publicSha))) + assertEquals(publicSha, built.entries.single().sha256, "$label build") + assertEquals(publicSha, c.decodeManifest(built.manifest).entries.single().sha256, "$label decode of the build") + val none = c.buildManifest(listOf(ManifestEntryInput(path = "a.txt", address = publicAddress))) + assertNull(none.entries.single().sha256, "$label an unset hash stays unset") + val bad = assertFailsWith("$label bad sha256") { + c.buildManifest(listOf(ManifestEntryInput(path = "a.txt", address = publicAddress, sha256 = "zz"))) + } + assertEquals(400, bad.statusCode) + assertTrue(bad.message!!.contains("sha256"), bad.message) + + // 4. The one paid call: a small public upload reports its plaintext's hash. + val payload = "sha-fanout kotlin ${System.currentTimeMillis() / 1000} $label".toByteArray(Charsets.UTF_8) + val expected = sha256Hex(payload) + val put = c.dataPutPublic(payload) + println("LiveSha256Test $label: uploaded ${payload.size} bytes, local sha256=$expected, returned sha256=${put.sha256}") + assertEquals(expected, put.sha256, "$label dataPutPublic sha256") + } + + @Test + fun `live sha256 on manifests and uploads`() = runBlocking { + val restUrl = env("ANTD_TEST_URL") + val link = env("ANTD_TEST_SHA_LINK") + val publicAddress = env("ANTD_TEST_SHA_PUBLIC_ADDRESS") + val publicSha = env("ANTD_TEST_SHA_PUBLIC_SHA256") + val dataMapSha = env("ANTD_TEST_SHA_DATA_MAP_SHA256") + assumeTrue( + restUrl != null && link != null && publicAddress != null && publicSha != null && dataMapSha != null, + "set ANTD_TEST_URL, ANTD_TEST_SHA_LINK, ANTD_TEST_SHA_PUBLIC_ADDRESS, ANTD_TEST_SHA_PUBLIC_SHA256 " + + "and ANTD_TEST_SHA_DATA_MAP_SHA256 to run", + ) + val grpcTarget = env("ANTD_TEST_GRPC")?.removePrefix("http://") + + AntdRestClient(restUrl!!).use { check("rest", it, link!!, publicAddress!!, publicSha!!, dataMapSha!!) } + if (grpcTarget != null) AntdGrpcClient(grpcTarget).use { check("grpc", it, link!!, publicAddress!!, publicSha!!, dataMapSha!!) } + } +} diff --git a/antd-kotlin/lib/src/test/kotlin/com/autonomi/sdk/Sha256GrpcTest.kt b/antd-kotlin/lib/src/test/kotlin/com/autonomi/sdk/Sha256GrpcTest.kt new file mode 100644 index 00000000..08cac8ce --- /dev/null +++ b/antd-kotlin/lib/src/test/kotlin/com/autonomi/sdk/Sha256GrpcTest.kt @@ -0,0 +1,179 @@ +package com.autonomi.sdk + +import antd.v1.* +import io.grpc.ManagedChannel +import io.grpc.Server +import io.grpc.Status +import io.grpc.StatusException +import io.grpc.inprocess.InProcessChannelBuilder +import io.grpc.inprocess.InProcessServerBuilder +import kotlinx.coroutines.test.runTest +import kotlin.test.AfterTest +import kotlin.test.BeforeTest +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertFalse +import kotlin.test.assertTrue + +/** + * The `sha256` fields antd 0.17.0 added, over gRPC against in-process + * services: every upload result reads it, with "" (proto3's zero value) as + * null; a manifest entry input sets it only when given and non-empty + * (proto3 `optional` presence); a manifest entry info reads it, "" as null; + * INVALID_ARGUMENT is a [BadRequestException]. + */ +class Sha256GrpcTest { + + private lateinit var server: Server + private lateinit var channel: ManagedChannel + private lateinit var client: AntdGrpcClient + + /** The sha256 every upload service answers with; "" is the proto3 zero value. */ + private var sha: String = SHA + private var buildResponse: Manifests.BuildManifestResponse = buildManifestResponse {} + private var decodeResponse: Manifests.DecodeManifestResponse = decodeManifestResponse {} + private var lastBuild: Manifests.BuildManifestRequest? = null + private var buildFailure: StatusException? = null + + companion object { + val SHA = "ab".repeat(32) + val ADDR = "11".repeat(32) + val CA = "cc".repeat(32) + } + + private inner class MockData : DataServiceGrpcKt.DataServiceCoroutineImplBase() { + override suspend fun put(request: Data.PutDataRequest) = putDataResponse { + dataMap = "abcd" + chunksStored = 3 + sha256 = sha + } + + override suspend fun putPublic(request: Data.PutPublicDataRequest) = putPublicDataResponse { + address = ADDR + chunksStored = 3 + sha256 = sha + } + } + + private inner class MockFiles : FileServiceGrpcKt.FileServiceCoroutineImplBase() { + override suspend fun put(request: Files.PutFileRequest) = putFileResponse { + dataMap = "abcd" + chunksStored = 3 + sha256 = sha + } + + override suspend fun putPublic(request: Files.PutFileRequest) = putFilePublicResponse { + address = ADDR + chunksStored = 3 + sha256 = sha + } + } + + private inner class MockUpload : UploadServiceGrpcKt.UploadServiceCoroutineImplBase() { + override suspend fun finalizeUpload(request: Upload.FinalizeUploadRequest) = finalizeUploadResponse { + chunksStored = 4 + dataMap = "abcd" + sha256 = sha + } + } + + private inner class MockManifest : ManifestServiceGrpcKt.ManifestServiceCoroutineImplBase() { + override suspend fun build(request: Manifests.BuildManifestRequest): Manifests.BuildManifestResponse { + lastBuild = request + buildFailure?.let { throw it } + return buildResponse + } + + override suspend fun decode(request: Manifests.DecodeManifestRequest): Manifests.DecodeManifestResponse = decodeResponse + } + + @BeforeTest + fun setUp() { + val name = InProcessServerBuilder.generateName() + server = InProcessServerBuilder.forName(name).directExecutor() + .addService(MockData()).addService(MockFiles()).addService(MockUpload()).addService(MockManifest()) + .build().start() + channel = InProcessChannelBuilder.forName(name).directExecutor().build() + client = AntdGrpcClient(channel) + } + + @AfterTest + fun tearDown() { + client.close() + channel.shutdownNow() + server.shutdownNow() + } + + private suspend fun results(): List = listOf( + client.dataPut("x".toByteArray()).sha256, + client.dataPutPublic("x".toByteArray()).sha256, + client.filePut("/tmp/x").sha256, + client.filePutPublic("/tmp/x").sha256, + client.finalizeUpload("up-1", mapOf("q" to "t")).sha256, + client.finalizeMerkleUpload("up-1", "0xwinner").sha256, + ) + + @Test + fun `every upload result reads sha256`() = runTest { + assertEquals(List(6) { SHA }, results()) + } + + @Test + fun `every upload result reads an empty sha256 as null`() = runTest { + sha = "" + assertEquals(List(6) { null }, results()) + } + + @Test + fun `buildManifest sets sha256 only on an entry that gives a non-empty one`() = runTest { + client.buildManifest( + listOf( + ManifestEntryInput(path = "a.txt", address = ADDR, sha256 = SHA), + ManifestEntryInput(path = "b.txt", address = ADDR), + ManifestEntryInput(path = "c.txt", address = ADDR, sha256 = ""), + ), + ) + val (a, b, c) = lastBuild!!.entriesList + assertTrue(a.hasSha256()) + assertEquals(SHA, a.sha256) + assertFalse(b.hasSha256(), "unset sha256 must be absent") + assertFalse(c.hasSha256(), "an empty sha256 must be absent, not set to an empty string") + } + + @Test + fun `buildManifest passes sha256 through exactly as given`() = runTest { + client.buildManifest(listOf(ManifestEntryInput(address = ADDR, sha256 = "ZZ"))) + assertEquals("ZZ", lastBuild!!.entriesList.single().sha256) + } + + @Test + fun `build and decode read sha256 on entries, with an empty one as null`() = runTest { + val infos = listOf(SHA, "").map { s -> + manifestEntryInfo { + effectiveName = "x" + kind = "public" + contentAddress = CA + sha256 = s + } + } + buildResponse = buildManifestResponse { entries += infos } + decodeResponse = decodeManifestResponse { + source = "manifest" + entries += infos + } + assertEquals(listOf(SHA, null), client.buildManifest(listOf(ManifestEntryInput(address = ADDR))).entries.map { it.sha256 }) + assertEquals(listOf(SHA, null), client.decodeManifest(byteArrayOf(1)).entries.map { it.sha256 }) + assertEquals(listOf(SHA, null), client.decodeManifestLink("ant://manifest/wUFOVAE").entries.map { it.sha256 }) + } + + @Test + fun `INVALID_ARGUMENT for a bad sha256 is a BadRequestException keeping the daemon's message`() = runTest { + buildFailure = Status.INVALID_ARGUMENT.withDescription("entries[0]: sha256 must be 32 bytes, got 1").asException() + val ex = assertFailsWith { + client.buildManifest(listOf(ManifestEntryInput(address = ADDR, sha256 = "ab"))) + } + assertEquals(400, ex.statusCode) + assertTrue(ex.message!!.contains("entries[0]: sha256 must be 32 bytes, got 1"), ex.message) + } +} diff --git a/antd-kotlin/lib/src/test/kotlin/com/autonomi/sdk/Sha256RestTest.kt b/antd-kotlin/lib/src/test/kotlin/com/autonomi/sdk/Sha256RestTest.kt new file mode 100644 index 00000000..2bba3c7b --- /dev/null +++ b/antd-kotlin/lib/src/test/kotlin/com/autonomi/sdk/Sha256RestTest.kt @@ -0,0 +1,193 @@ +package com.autonomi.sdk + +import kotlinx.coroutines.test.runTest +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.jsonArray +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import okhttp3.mockwebserver.Dispatcher +import okhttp3.mockwebserver.MockResponse +import okhttp3.mockwebserver.MockWebServer +import okhttp3.mockwebserver.RecordedRequest +import kotlin.test.AfterTest +import kotlin.test.BeforeTest +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFailsWith +import kotlin.test.assertFalse +import kotlin.test.assertNull +import kotlin.test.assertTrue + +/** + * The `sha256` fields antd 0.17.0 added, over REST against a mock daemon: + * every upload result reads it when present and as null when the key is + * absent (an older daemon) or empty; a manifest entry input sends it only + * when set and non-empty; a manifest entry info reads it from build and + * decode; a bad hash is the daemon's 400 as a [BadRequestException]. + */ +class Sha256RestTest { + + private lateinit var server: MockWebServer + private lateinit var client: AntdRestClient + private lateinit var daemon: Daemon + + companion object { + val SHA = "ab".repeat(32) + val ADDR = "11".repeat(32) + val CA = "cc".repeat(32) + const val HEADER_B64 = "wUFOVAE=" + + fun json(body: String, code: Int = 200): MockResponse = + MockResponse().setResponseCode(code).setHeader("Content-Type", "application/json").setBody(body) + } + + class Daemon : Dispatcher() { + val responses = mutableMapOf() + val requests = mutableListOf>() + + override fun dispatch(request: RecordedRequest): MockResponse { + val key = "${request.method} ${request.path}" + requests += key to request.body.readUtf8() + return responses[key] ?: json("""{"error":"no route $key","code":"NOT_FOUND"}""", 404) + } + + fun on(key: String, body: String, code: Int = 200) { + responses[key] = json(body, code) + } + + fun body(key: String): JsonObject = Json.parseToJsonElement(requests.last { it.first == key }.second).jsonObject + } + + @BeforeTest + fun setUp() { + server = MockWebServer() + daemon = Daemon() + server.dispatcher = daemon + server.start() + client = AntdRestClient(baseUrl = server.url("/").toString()) + } + + @AfterTest + fun tearDown() { + client.close() + server.shutdown() + } + + // ── upload results ── + + @Test + fun `dataPutPublic and dataPut read sha256`() = runTest { + daemon.on("POST /v1/data/public", """{"address":"$ADDR","chunks_stored":3,"payment_mode_used":"single","sha256":"$SHA"}""") + daemon.on("POST /v1/data", """{"data_map":"abcd","chunks_stored":3,"payment_mode_used":"single","sha256":"$SHA"}""") + assertEquals(SHA, client.dataPutPublic("x".toByteArray()).sha256) + assertEquals(SHA, client.dataPut("x".toByteArray()).sha256) + } + + @Test + fun `filePutPublic and filePut read sha256`() = runTest { + val tail = """"storage_cost_atto":"1","gas_cost_wei":"2","chunks_stored":3,"payment_mode_used":"single","sha256":"$SHA"""" + daemon.on("POST /v1/files/public", """{"address":"$ADDR",$tail}""") + daemon.on("POST /v1/files", """{"data_map":"abcd",$tail}""") + assertEquals(SHA, client.filePutPublic("/tmp/x").sha256) + assertEquals(SHA, client.filePut("/tmp/x").sha256) + } + + @Test + fun `finalizeUpload and finalizeMerkleUpload read sha256`() = runTest { + daemon.on("POST /v1/upload/finalize", """{"address":"","chunks_stored":4,"data_map":"abcd","data_map_address":"$ADDR","sha256":"$SHA"}""") + assertEquals(SHA, client.finalizeUpload("up-1", mapOf("q" to "t")).sha256) + assertEquals(SHA, client.finalizeMerkleUpload("up-1", "0xwinner").sha256) + } + + @Test + fun `upload results read sha256 as null when the key is absent or empty`() = runTest { + val tail = """"storage_cost_atto":"1","gas_cost_wei":"2","chunks_stored":3,"payment_mode_used":"single"""" + daemon.on("POST /v1/data/public", """{"address":"$ADDR","chunks_stored":3,"payment_mode_used":"single"}""") + daemon.on("POST /v1/data", """{"data_map":"abcd","sha256":""}""") + daemon.on("POST /v1/files/public", """{"address":"$ADDR",$tail}""") + daemon.on("POST /v1/files", """{"data_map":"abcd",$tail,"sha256":""}""") + daemon.on("POST /v1/upload/finalize", """{"address":"","chunks_stored":4}""") + assertNull(client.dataPutPublic("x".toByteArray()).sha256) + assertNull(client.dataPut("x".toByteArray()).sha256) + assertNull(client.filePutPublic("/tmp/x").sha256) + assertNull(client.filePut("/tmp/x").sha256) + assertNull(client.finalizeUpload("up-1", mapOf("q" to "t")).sha256) + assertNull(client.finalizeMerkleUpload("up-1", "0xwinner").sha256) + } + + @Test + fun `result constructors still compile without sha256, which defaults to null`() { + assertNull(DataPutResult("dm").sha256) + assertNull(DataPutResult("dm", 1u, "single").sha256) + assertNull(DataPutPublicResult(ADDR, 1u, "single").sha256) + assertNull(FilePutResult("dm", "1", "2", 3u, "single").sha256) + assertNull(FilePutPublicResult(ADDR, "1", "2", 3u, "single").sha256) + assertNull(FinalizeUploadResult("", 4, "dm", ADDR).sha256) + assertNull(FinalizeMerkleUploadResult("", 4, "dm", ADDR).sha256) + assertNull(ManifestEntryInput("a", 1u, "aa", null).sha256) + assertNull(ManifestEntryInfo("a", "public", CA, "a", 1u, "dm", 2u, "err").sha256) + } + + // ── manifest entry input ── + + @Test + fun `buildManifest sends sha256 only on an entry that sets it, non-empty`() = runTest { + daemon.on("POST /v1/manifests/build", """{"manifest":"$HEADER_B64","entries":[]}""") + client.buildManifest( + listOf( + ManifestEntryInput(path = "a.txt", address = ADDR, sha256 = SHA), + ManifestEntryInput(path = "b.txt", address = ADDR), + ManifestEntryInput(path = "c.txt", address = ADDR, sha256 = ""), + ), + ) + val entries = daemon.body("POST /v1/manifests/build")["entries"]!!.jsonArray.map { it.jsonObject } + assertEquals(SHA, entries[0]["sha256"]!!.jsonPrimitive.content) + assertFalse("sha256" in entries[1], "unset sha256 must be omitted: ${entries[1]}") + assertFalse("sha256" in entries[2], "an empty sha256 must be omitted, never sent as an empty string: ${entries[2]}") + } + + @Test + fun `buildManifest passes sha256 through exactly as given`() = runTest { + daemon.on("POST /v1/manifests/build", """{"manifest":"$HEADER_B64","entries":[]}""") + client.buildManifest(listOf(ManifestEntryInput(address = ADDR, sha256 = "ZZ"))) + val entry = daemon.body("POST /v1/manifests/build")["entries"]!!.jsonArray.single().jsonObject + assertEquals("ZZ", entry["sha256"]!!.jsonPrimitive.content) + } + + // ── manifest entry info ── + + @Test + fun `build and decode read sha256 on entries, and null when absent or empty`() = runTest { + val entries = """[ + {"effective_name":"a.txt","path":"a.txt","kind":"public","content_address":"$CA","sha256":"$SHA"}, + {"effective_name":"b.txt","path":"b.txt","kind":"public","content_address":"$CA"}, + {"effective_name":"c.txt","path":"c.txt","kind":"public","content_address":"$CA","sha256":""} + ]""" + daemon.on("POST /v1/manifests/build", """{"manifest":"$HEADER_B64","entries":$entries}""") + daemon.on("POST /v1/manifests/decode", """{"source":"manifest","entries":$entries}""") + + val built = client.buildManifest(listOf(ManifestEntryInput(address = ADDR))) + assertEquals(listOf(SHA, null, null), built.entries.map { it.sha256 }) + val decoded = client.decodeManifest(byteArrayOf(1)) + assertEquals(listOf(SHA, null, null), decoded.entries.map { it.sha256 }) + val linked = client.decodeManifestLink("ant://manifest/wUFOVAE") + assertEquals(listOf(SHA, null, null), linked.entries.map { it.sha256 }) + } + + // ── errors ── + + @Test + fun `a 400 for a bad sha256 is a BadRequestException keeping the daemon's message`() = runTest { + daemon.on( + "POST /v1/manifests/build", + """{"error":"Bad request: entries[0]: invalid hex sha256: Invalid character 'z' at position 0","code":"BAD_REQUEST"}""", + 400, + ) + val ex = assertFailsWith { + client.buildManifest(listOf(ManifestEntryInput(address = ADDR, sha256 = "zz"))) + } + assertEquals(400, ex.statusCode) + assertTrue(ex.message!!.contains("entries[0]: invalid hex sha256"), ex.message) + } +}