[ SRE_ARENA ]
DISTRIBUTED TRANSACTIONS

Monoliths Hide the Pain. Microservices Expose It.

Visualization: 1. The Saga Pattern
DOWNLOADING CONCEPT: 1. THE SAGA PATTERN...
  • ↳You are building an e-commerce app.
  • ↳A user clicks 'Buy'.
  • ↳You must: 1.
  • ↳Deduct $50 (Payment Service) 2.
  • ↳Reduce inventory by 1 (Inventory Service) 3.
  • ↳Create shipping label (Shipping Service)
  • ↳In a monolith, you wrap all three in an SQL BEGIN...COMMIT.
  • ↳If Shipping fails, the DB safely rolls back the money and inventory automatically.
  • ↳Magic.
  • ↳In microservices, these are 3 different databases.
  • ↳You cannot lock them all (2-Phase Commit is too slow).
  • ↳Enter the Saga Pattern.
  • ↳The Orchestrator calls Payment.
  • ↳Success.
  • ↳It calls Inventory.
  • ↳Success.
  • ↳It calls Shipping.
  • ↳Shipping is DOWN.
  • ↳The Orchestrator cannot simply 'rollback' the first two.
  • ↳It must issue Compensating Transactions.
  • ↳It explicitly calls Inventory to add 1 back.
  • ↳It explicitly calls Payment to refund $50.
  • ↳This is incredibly complex to build correctly, which is why you should avoid microservices until you absolutely need them.
▲ LIVE INCIDENTCustomer support is flooded with complaints of double-billing after a brief network outage.
sre@arena:~/distributed-transactions
sre@arena:~$ grep 'POST /charge' /var/log/payment-gateway.log | grep 'user_881'