Repository navigation
feat(idempotency): production-grade idempotency across http, mediator, messaging and ef core - #155
Merged
Merged
Conversation
Generated with Python's standard library so the expected keys and UUIDv5 values come from an implementation independent of the library under test. Adds idempotency to the allowed commit scopes.
…tract kit Add CSharpEssentials.Idempotency (net11.0, net10.0, net9.0, netstandard2.1): IIdempotencyRecordStore with reservation tokens and opaque payloads, InMemoryIdempotencyRecordStore (atomic in one process) and DistributedCacheIdempotencyRecordStore (best effort), IdempotencyRecordKey with a length-prefixed v3 storage key and an explicit global scope, IdempotencyScopes resolvers (User, Tenant, UserAndTenant, Client, Custom, Global) and IdempotencyKeys (For, FromMessage, RFC 9562 UUIDv5, IsValid) pinned by tests/vectors/idempotency-key-vectors.json. Add CSharpEssentials.Idempotency.Testing, an xUnit contract test base class for custom stores. Wire both into the solution, central package versions, the publish script, README, AGENTS.md and the API reference.
…y core The Idempotency-Key middleware now stores its responses through IIdempotencyRecordStore from CSharpEssentials.Idempotency. Records use the new Namespace option (default "http") and a scope from the new ScopeResolver option (default IdempotencyScopes.User). AllowUnscopedKeys keeps its meaning: when true, an unresolved scope uses a global key. Stored responses use serializer version 3; an unreadable completed payload returns 500 without running the handler again. BREAKING CHANGE: IIdempotencyStore, IdempotencyReservation, IdempotencyReservationStatus (CSharpEssentials.AspNetCore), InMemoryIdempotencyStore, DistributedCacheIdempotencyStore, IdempotencyOptions.KeyScope and IdempotencyOptions.DefaultKeyScope are removed without an obsolete bridge. Use IIdempotencyRecordStore, IdempotencyRecordReservation, IdempotencyReservationStatus, InMemoryIdempotencyRecordStore, DistributedCacheIdempotencyRecordStore and IdempotencyOptions.ScopeResolver from CSharpEssentials.Idempotency. Storage keys change to the idem:v3: format and entries written by earlier versions (v2) are no longer read. See plan section 10 R9 for the mapping.
…erministic keys Run the contract kit against the in-memory and distributed cache stores, add store-specific cases (sweep, versioned entries, storage key prefix), and check scope resolvers, record key encoding and IdempotencyKeys against the key vectors.
Map the removed AspNetCore store types and KeyScope to the CSharpEssentials.Idempotency API, show how a custom store moves to IIdempotencyRecordStore and the contract kit, and describe the v3 record format and rolling deploy risk (plan section 10 R9).
Version 2 pins the scope strings (user, tenant, client, custom, global and userAndTenant) and the length-prefixed storage key under idem:v3: so every implementation builds identical records.
…em codes
- accept an Idempotency-Key sent as a structured-field string ("abc") and
store it under the same record as the bare value
- validate the unquoted key with IdempotencyKeys.IsValid; malformed quoting,
invalid characters, an empty value or several headers return 400
Idempotency.KeyInvalid
- every idempotency problem (KeyMissing, KeyInvalid, InFlight, KeyReused) is
application/problem+json built from an Error, with a top-level code member
and errorCodes; no idempotency type URI is published
- the 409 Idempotency.InFlight always sends Retry-After
BREAKING CHANGE: IdempotencyOptions.RetryAfter is a non-nullable TimeSpan
(default 1 second, must be positive); the header can no longer be omitted.
Keys are limited to printable ASCII without space, '"' and '\', so keys
with other characters that were accepted before now return 400.
… them as this repo's own
…rted Idempotent endpoints now get an execution filter that marks the handler as started: .WithIdempotency() adds an endpoint filter on Minimal API endpoints and groups, and [Idempotent] is the action filter on MVC actions. The middleware uses that mark to decide what happens to the key: - the handler did not start (rate limiter, load shedding, auth, binding or model validation failure): the key is released; - the status is in NeverStoreStatusCodes (503, 504 by default), or the request failed or was aborted after the handler started: the key stays in flight until its lease expires, so a retry gets 409 and runs once more afterwards; - otherwise ShouldStore decides. IdempotencyStorePolicies.Default stores 2xx, and 3xx and 5xx from a started handler; 5xx is kept for ServerErrorRetention (10 minutes). SuccessOnly stores 2xx only and releases 3xx and 5xx. A response that cannot be stored (handler-set Content-Encoding, text/event-stream, larger than MaxResponseBodySize) releases the key only on 4xx and stays in flight otherwise. Unscoped keys follow UnscopedKeyBehavior: Skip (default, logs Idempotency.UnscopedKeySkipped), Reject (400 Idempotency.ScopeRequired) or Global. Retry-After is replayed by default. An endpoint without the filter logs a warning once, or throws under RequireExecutionFilter, and a request timeout longer than InFlightTimeout logs a warning once per endpoint. Startup validation checks InFlightTimeout against RetentionPeriod and MaxInFlightTimeout (15 minutes), ServerErrorRetention against RetentionPeriod, and NeverStoreStatusCodes. BREAKING CHANGE: AllowUnscopedKeys is replaced by UnscopedKeyBehavior, and an unscoped request now logs a warning. ShouldStore takes an IdempotencyOutcome instead of a status code. 3xx and handler 5xx responses are stored, 503 and 504 keep the key in flight, an exception or abort after the handler started keeps the key in flight instead of releasing it, and a response that cannot be stored no longer releases the key unless it is 4xx. Retry-After is added to ReplayedHeaders. Idempotent endpoints need .WithIdempotency() or [Idempotent] for the handler-started mark; without it every outcome releases the key. New startup validation rejects InFlightTimeout above RetentionPeriod or MaxInFlightTimeout, ServerErrorRetention outside (0, RetentionPeriod] and an empty or out-of-range NeverStoreStatusCodes.
…options Describe the outcome table, ServerErrorRetention, NeverStoreStatusCodes, IdempotencyStorePolicies, UnscopedKeyBehavior, the execution filter, Retry-After replay and the new startup validation, and add the behavior changes and option mappings to the 6.x to 7.0 migration guide.
…ut the idempotency filter An idempotent endpoint without the execution filter cannot report whether its handler started. Its 2xx responses are now stored as before, and a 2xx that cannot be stored keeps the key in flight; 3xx, 4xx, 5xx (503 and 504 included) and exceptions before the response started release the key. ShouldStore receives the outcome with HandlerStarted false. The warning and RequireExecutionFilter are unchanged.
…torage keys
Scope strings now carry their kind (user:, tenant:, client:, {kind}: for custom scopes), UserAndTenant length-prefixes only the tenant part, and a global key writes its scope part as "global" instead of "-". The scope and storage key vectors are consumed by the tests.
BREAKING CHANGE: IdempotencyScopes.Custom takes a scope kind; scope strings carry user:/tenant:/client: prefixes and global keys store the scope as "global".
…hare global records A global key writes its scope part as 'global', so an owner scope with that raw value would have reached the global records. IdempotencyScope.Of and IdempotencyRecordKey now reject it.
IdempotencyScope is now created only by User, Tenant, Client,
UserAndTenant, Custom(kind, value) and Global. Every owner scope carries
its kind, so Tenant("user:42") is tenant:user:42 and never matches
User("42"), and no owner scope can be the reserved global scope part.
The storage key format and the key vectors are unchanged.
BREAKING CHANGE: IdempotencyScope.Of is internal. IdempotencyRecordKey
takes an IdempotencyScope? (null or Global gives a global key); the
constructor taking a raw scope string is no longer public.
IdempotencyScope.Custom and IdempotencyScopes.Custom reject a kind that
contains ':' or consists only of digits. The protected Key helper of
IdempotencyRecordStoreContractTests takes an IdempotencyScope?.
The scope factories (User, Tenant, Client, UserAndTenant, Custom kind and value) and the IdempotencyRecordKey constructor (namespace, scope, key) now throw ArgumentException for an unpaired surrogate, so an invalid value fails at construction instead of in ToStorageKey. HTTP keys already reject it as Idempotency.KeyInvalid; a test covers bare and quoted header values. Custom kinds (IdempotencyScope.Custom and IdempotencyScopes.Custom) also reject '/' and the reserved built-in kinds user, tenant, client and global in any casing. Document that stores must compare Namespace, Scope and Key ordinally and case-sensitively, and add a contract scenario for case isolation.
IIdempotencyExecutor reserves a key, replays a stored result, reports an in-flight or reused key, and runs the operation at most once. A key is released when the operation never started, stays in flight when it threw after starting, and is completed with the codec's classification (Store, StoreAsFailure, Release) otherwise. Modes Off, Observe and Enforce; settings resolve operation > profile > defaults with a built-in "messaging" profile; AddIdempotencyExecutor validates them at startup.
IdempotencyInbox runs a message handler at most once per consumer: the
namespace is messaging:{consumer}, the key is IdempotencyKeys.FromMessage
and the fingerprint is constant because the message id is authoritative.
It uses the messaging profile (7-day retention) unless the consumer names
another profile, and AddIdempotencyExecutor registers it.
IdempotencyExecutorOptions.ResolveMode returns the mode an operation runs in (operation, else profile, else default), so a caller can skip key and fingerprint work in Off and decide whether a preparation problem blocks it in Enforce or is only logged in Observe.
IdempotencyBehavior runs an IIdempotentRequest at most once per key and caller through IIdempotencyExecutor and replays the stored response to a retry. Scopes come from the request or from IdempotencyScope factories via a resolver fed by IIdempotencyScopeAccessor. A failed Result whose first error is Unexpected is kept for the failure retention; any other failure is a business failure and is kept as long as a success. A failed ITransactionalRequest releases its key. AddMediatorIdempotencyBehavior places the behavior outside the transaction (before the lock) or right inside it, regardless of registration order.
…pe in-flight results
…g spaces distinct
… to the EF Core store
…and the money movement guide
…d money movement sections
Client retry with an Idempotency-Key, a WithIdempotency endpoint, Mediator commands on the EF Core store under the payments profile, a message inbox and a signed provider webhook, tested against PostgreSQL through Testcontainers.
…e's key normalization The middleware unquotes a structured-field Idempotency-Key before validating it. A command that read the raw header saw the quotes and rejected the key. The middleware now parses through the same public method, so both layers see one key.
…to a command Document IdempotencyKeyHeader.TryParse in the API reference and the agent skill, and parse the header in the transactional money movement sample so a quoted key reaches the command as the middleware saw it.
…and reload the payments mode
- Parse Idempotency-Key with IdempotencyKeyHeader.TryParse so a quoted key is one key in both layers
- Send the payment provider IdempotencyKeys.For("charge", merchant, key); the providers dedupe on it
- Register a ConfigurationChangeTokenSource so a reload switches the payments mode without a restart
- Answer a failed webhook with 500 and scope settlements to Custom("account", ...)
- Compare webhook signatures as decoded HMAC bytes
- Redeliver in-flight and failed messages after a delay instead of in a hot loop
- Dispose the host's clients, release the held charge in finally, check Build() in the client
- Move PostgresCollection to its own file
- Cover the quoted key, a lost provider response, KeyReused from the database layer and the mode reload
…er the raw body and answer failures with a 5xx
…ader.TryParse in the migration guide
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
…brary as a test project
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Production-grade idempotency across HTTP, Mediator, messaging and EF Core, per
plans/2026-10-19-idempotency-PLAN.md.CSharpEssentials.Idempotency)IdempotencyScope(User,Tenant,Client,UserAndTenant,Custom,Global).idem:v3:storage keys.IIdempotencyExecutorwith profiles.IdempotencyKeys.FromMessage).IOptionsMonitor.idempotency.decisions) and EventIds 7001–7017. Logs carry hashed keys only.KeyReused422,InFlight409 +Retry-After, 400 for the rest.IdempotencyKeyHeader.TryParse, which the middleware also uses.IdempotencyBehaviorwith retention based on the failure kind (FailureRetention).SetIdempotencyKey,IdempotencyKeyHandler, andIdempotentRetrythat honoursRetry-After(capped byMaxRetryAfter).EfCoreIdempotencyRecordStore.KeyHash binary(32)with an ordinal re-check, safe under case-insensitive and PAD SPACE collations.IdempotencyRecordStoreContractTestscontract kit for custom stores.paymentsprofile → inbox → webhook. Both run in CI.docs/migration/v6-to-v7-idempotency.md,API_REFERENCE.md, agent skills.Breaking changes (v7)
IdempotencyOptionsis resolved as transientCurrentValue.AllowUnscopedKeys→UnscopedKeyBehavior.idem:v3:; 6.x entries are not read. Drain before a rolling deploy.See the migration guide.
New dependencies
Microsoft.Extensions.Options[9.0.4,); EF projects pin it withVersionOverride[10.0.1,).StackExchange.Redis2.8.31, examples only.Known gaps
Test plan
dotnet build --no-incremental: 0 errors, the 7 known warningsCSharpEssentials.Tests: 8121/8121Examples.Idempotency.EndToEnd: 14/14 (Postgres via Testcontainers)Examples.Idempotency.Redis: 28/28--job Drypasses on net9.0, net10.0 and net11.0BEGIN_COMMIT_OVERRIDE
feat(idempotency)!: production-grade idempotency across http, mediator, messaging and ef core
BREAKING CHANGE: idempotency moves to the new CSharpEssentials.Idempotency core; see the migration guide.
IIdempotencyStore,IdempotencyReservation,IdempotencyReservationStatus,InMemoryIdempotencyStoreandDistributedCacheIdempotencyStoreare removed fromCSharpEssentials.AspNetCore; useIIdempotencyRecordStore,InMemoryIdempotencyRecordStoreandDistributedCacheIdempotencyRecordStore.IdempotencyOptions.KeyScopeandDefaultKeyScopeare removed; useScopeResolverwithIdempotencyScopes. Scopes are built only through theIdempotencyScopefactories, andAllowUnscopedKeysis replaced byUnscopedKeyBehavior.IdempotencyOptions.ShouldStoreis aFunc<IdempotencyOutcome, bool>instead of aFunc<int, bool>;IdempotencyStorePolicies.SuccessOnlyrestores the 6.x rule.IdempotencyOptions.RetryAfteris a non-nullableTimeSpan; the 409 always carriesRetry-After.ServerErrorRetention; 503, 504, exceptions and aborts after the handler started keep the key in flight untilInFlightTimeoutinstead of releasing it."and\; other keys get 400Idempotency.KeyInvalid.IdempotencyOptionsis resolved as a transientCurrentValue, and records use theidem:v3:storage key format, so 6.x entries are not read.END_COMMIT_OVERRIDE