Repository navigation
No way to fail closed when the idempotency store is unavailable #53
Description
Version: idempotency-kit 0.4.1.
When the store cannot be reached, the coordinator always runs the action without protection. That is documented ("Storage and decode trouble never raise: the coordinator degrades to running the action"), but there is no way to choose otherwise. With Redis unreachable, two calls with the same key run the action twice:
import asyncio
import logging
from redis.asyncio import Redis
from idempotency_kit import AsyncIdempotencyCoordinator, IdempotencyDomainService, JsonResultAdapter
from idempotency_kit.infra.storage.redis.aio import RedisAsyncIdempotencyRepository
async def main():
unreachable = Redis.from_url("redis://127.0.0.1:1/0", socket_connect_timeout=0.2)
coordinator = AsyncIdempotencyCoordinator(
RedisAsyncIdempotencyRepository(unreachable), IdempotencyDomainService(), in_flight="raise",
)
effects = []
async def charge():
effects.append(1)
return {"charge_id": f"ch_{len(effects)}"}
for _ in range(2):
await coordinator.coordinate("payment.charge", "same-key", 3600, JsonResultAdapter(), charge)
print("effects", len(effects)) # effects 2
await unreachable.aclose()
logging.basicConfig(level=logging.CRITICAL)
asyncio.run(main())Fail-open is the right default for many operations. It is the wrong one for an effect that is expensive or impossible to repeat when nothing downstream deduplicates it, such as a charge at a provider without its own idempotency keys, an email or a ledger write in another system. For those, the service would rather answer 503 and let the client retry later, which today requires wrapping the store in a separate health check that cannot close the race between the check and the write.
Proposal
An option on the coordinator, for example on_storage_error: Literal["run", "raise"] = "run":
"run"keeps today's behaviour."raise"raises a dedicatedIdempotencyStorageUnavailableErrorwhen the reservation or the initial read fails, before the action runs.
A failure to save the result after the action has run is a different case (the effect already happened) and can keep today's logging in both modes.
Activity
AlexeyShalaev commentedon Oct 16, 2026
Reproduced on master:
effects 2with Redis unreachable.Plan: your option, as
on_storage_error: Literal["run", "raise"] = "run"on the coordinator, onBaseIdempotencySettingsand through the Dishka provider, the same wayin_flightgoes."run"is today's behaviour. With"raise", a storage failure on the way in, the reservation or the read, raises before the action runs. A failure after the action (saving the result, releasing the reservation) is logged as today in both modes: the effect has happened, and raising would make the client retry a completed operation.The exception is the existing
IdempotencyStorageErrorrather than a newIdempotencyStorageUnavailableError. It is already exported and already means "the store is unavailable or fails"; the Redis repository raises exactly that, and the coordinator passes it on with itsoperationandoriginal_error. Out ofcoordinate()it can only mean the action did not run, so it maps straight to 503. A custom repository that lets a raw driver exception through (the protocol asks forIdempotencyStorageError) gets it wrapped in one, since the point of the mode is not to run when in doubt.What does not count as unavailable: a record that cannot be read (corrupted JSON, an adapter that cannot decode it). The store answered, and today's rule holds: the action runs and its result replaces the record, so the key heals instead of failing every retry until the TTL. Default stays
"run"; fail-open is right for most operations and changing it would turn every Redis blip into errors for everyone who did not ask.AlexeyShalaev commentedon Oct 16, 2026
#55 is up. With Redis unreachable, your script on that branch prints
effects 2by default and, withon_storage_error="raise":The option is on the coordinator, on
BaseIdempotencySettingsand in the Dishka provider. With"raise", a failure on the reservation or the read raises the repository's ownIdempotencyStorageErrorbefore the action runs (a raw driver error from a custom repository gets wrapped in one), so out ofcoordinate()it always means nothing ran: answer 503. A failure to save the result after the action stays a log line, as you suggested, and so does a record the store returned but cannot be read, since that one is not an outage and the fresh result heals the key.It is per coordinator, so a service with both kinds of operation builds two. Nothing changes for anyone who does not set it.
feat: let the coordinator fail closed when the idempotency store is u…