Idempotency Keys in Practice

"Just add an idempotency key" is the kind of advice that sounds complete and isn't. I've reviewed enough payment integrations to know the gap is never the header — every client remembers to send Idempotency-Key: <uuid>. The gap is what the server does with it once two requests carrying the same key arrive close enough together that "check if we've seen this before" and "record that we're processing it" aren't actually one atomic step. This one's for the backend engineer who's about to build that step and wants to see where it actually breaks. 

The retry that isn't safe by default 

A client calls POST /payments, the request reaches your service, the charge succeeds, and the response times out on the way back. The client — correctly, per its own retry policy — sends the same request again. Nothing about HTTP tells your service these are "the same" request. As far as your payment endpoint is concerned, it received two POST calls, and unless something explicitly ties them together, it will process two charges. 

An idempotency key is that tie. The client generates a key — a UUIDv4 is the standard choice — and sends it once, reusing the identical value on every retry of that same logical operation. The server's job is to guarantee that no matter how many times a given key arrives, the underlying charge happens at most once, and every caller gets the same response. That guarantee sounds simple. Implementing it correctly requires answering three questions your design either handles explicitly or gets wrong silently: 

Same key, same payload, arriving again → return the original result, don't reprocess. 

Same key, different payload → reject. A key isn't a request cache; reusing it for a different amount or recipient is a client bug, and the server needs to catch it rather than silently process whichever payload arrived first. 

Same key, arriving concurrently — two requests racing each other before either has finished — → exactly one of them executes the charge; the other waits or is told to retry. 

Case 3 is the one most naive implementations miss, because it's the one that doesn't show up in a single-threaded test. It's also the one that actually causes double-charges in production, because it's exactly what a client's retry-on-timeout logic produces: the original request is still in flight, slow, and the client — having given up waiting — fires the retry while the first call is still running. 

The fast path: claiming a key atomically in Redis 

Redis is a good fit for the hot path because the operation you need — "check if this key exists, and if not, claim it, in one atomic step" — is exactly what SET key value NX gives you. NX means "only set if the key doesn't already exist," and Redis executes it as a single atomic command, which is what closes the race in case 3. 

 1 @Service 
 2 public class IdempotencyService { 
 3  
 4     private final StringRedisTemplate redis; 
 5     private static final Duration LOCK_TTL = Duration.ofSeconds(30); 
 6     private static final Duration RESULT_TTL = Duration.ofDays(7); 
 7  
 8     public IdempotencyResult claim(String idempotencyKey, String payloadHash) { 
 9         String lockKey = "idem:lock:" + idempotencyKey; 
10         String resultKey = "idem:result:" + idempotencyKey; 
11  
12         // Already completed? Return the stored response — this is the 
13         // "same key, same payload, return original result" path. 
14         String existing = redis.opsForValue().get(resultKey); 
15         if (existing != null) { 
16             IdempotencyRecord record = deserialize(existing); 
17             if (!record.payloadHash().equals(payloadHash)) { 
18                 throw new IdempotencyConflictException(idempotencyKey); 
19             } 
20             return IdempotencyResult.replay(record.response()); 
21         } 
22  
23         // Not completed yet — try to claim the in-flight lock atomically. 
24         Boolean acquired = redis.opsForValue() 
25             .setIfAbsent(lockKey, payloadHash, LOCK_TTL); 
26  
27         if (Boolean.TRUE.equals(acquired)) { 
28             return IdempotencyResult.proceed(); // caller does the real work 
29         } 
30  
31         // Someone else is already processing this key right now. 
32         String inFlightPayload = redis.opsForValue().get(lockKey); 
33         if (inFlightPayload != null && !inFlightPayload.equals(payloadHash)) { 
34             throw new IdempotencyConflictException(idempotencyKey); 
35         } 
36         throw new IdempotencyInProgressException(idempotencyKey); // 409, client retries later 
37     } 
38  
39     public void recordResult(String idempotencyKey, String payloadHash, String response) { 
40         String resultKey = "idem:result:" + idempotencyKey; 
41         redis.opsForValue().set(resultKey, serialize(payloadHash, response), RESULT_TTL); 
42         redis.delete("idem:lock:" + idempotencyKey); 
43     } 
44 } 

 The payloadHash — a hash of the normalized request body, not the raw JSON — is what implements case 2. Two requests with the same key but different amounts produce different hashes, and the mismatch is caught explicitly instead of silently resolved in favor of whichever request happened to win the race. 

The controller wraps the actual charge logic around this: 

 1 @PostMapping("/payments") 
 2 public ResponseEntity createPayment( 
 3         @RequestHeader("Idempotency-Key") String idempotencyKey, 
 4         @RequestBody PaymentRequest request) { 
 5  
 6     String payloadHash = hashService.hash(request); 
 7     IdempotencyResult claim = idempotencyService.claim(idempotencyKey, payloadHash); 
 8  
 9     if (claim.isReplay()) { 
10         return ResponseEntity.ok(claim.response()); 
11     } 
12  
13     PaymentResponse response = paymentService.charge(request); // the real work 
14     idempotencyService.recordResult(idempotencyKey, payloadHash, serialize(response)); 
15     return ResponseEntity.ok(response); 
16 } 

  

Why Redis alone isn't the whole answer 

Redis gives you a fast, correctly-atomic claim. It doesn't give you durability guarantees a payment system should actually depend on. An eviction under memory pressure, a Redis failover that loses a few seconds of writes, or a TTL that expires while a downstream system is still retrying, and the in-memory record of "we already processed this" is gone — which reopens exactly the race the whole mechanism exists to close. 

The fix is the same one we've written about for reconciliation: don't trust a single system of record for something this consequential. The database — not Redis — should be the actual source of truth, enforced with a unique constraint: 

 1 @Entity 
 2 @Table(name = "payment_intents", 
 3        uniqueConstraints = @UniqueConstraint(columnNames = "idempotency_key")) 
 4 public class PaymentIntent { 
 5  
 6     @Id @GeneratedValue 
 7     private UUID id; 
 8  
 9     @Column(name = "idempotency_key", nullable = false) 
10     private String idempotencyKey; 
11  
12     @Column(name = "payload_hash", nullable = false) 
13     private String payloadHash; 
14  
15     private BigDecimal amount; 
16     private String status; 
17     // getters/setters omitted 
18 } 
19  @Transactional 
20 public PaymentResponse charge(PaymentRequest request, String idempotencyKey, String payloadHash) { 
21     try { 
22         PaymentIntent intent = new PaymentIntent(idempotencyKey, payloadHash, request.amount()); 
23         paymentIntentRepository.saveAndFlush(intent); // fails fast on duplicate key 
24         return processGatewayCharge(intent); 
25     } catch (DataIntegrityViolationException e) { 
26         // Another process won the DB-level race — Redis missed it, the 
27         // constraint didn't. Load and return the existing result instead 
28         // of surfacing a 500. 
29         PaymentIntent existing = paymentIntentRepository 
30             .findByIdempotencyKey(idempotencyKey) 
31             .orElseThrow(); 
32         return toResponse(existing); 
33     } 
34 } 

 Redis handles the common case fast, without a database round-trip on every retry. The unique constraint is what makes the guarantee actually hold when Redis doesn't — a DataIntegrityViolationException here isn't a bug, it's the safety net catching a race Redis's own atomicity was supposed to prevent but, under failover or eviction, didn't. Treat both layers as required, not one as an optimization of the other. 

TTL is a business decision, not a config default 

How long an idempotency record needs to live is worth deciding deliberately rather than leaving at whatever a tutorial defaults to. Adyen's own API guidance — which we've cited before in the context of retry design against payment gateways — recommends holding idempotency keys valid for at least a week, which matches the reality that a client's own retry logic, queued webhooks, or a delayed reconciliation pass can all resurface a request well after the original call. Set the Redis TTL and any DB-level cleanup job to match whatever retry window your actual clients operate on, not a number that felt reasonable in a demo. 

What this actually protects against 

None of this is defending against a hypothetical. It's defending against the exact sequence that happens every time a payment gateway goes slow instead of down — the failure mode we've written about at length elsewhere: a client times out, retries in good faith, and two requests for the same logical charge land close enough together that only atomicity, not application logic running "fast enough," decides whether the customer gets charged once or twice. Get the claim step atomic, back it with a database constraint that doesn't trust Redis to be perfect, and set the TTL to match how long a retry can realistically still be coming. That's the whole mechanism — the code is short. What's not short is remembering that "add an idempotency key" was never really about the header.