concurrency in practice

every concurrency bug i have chased came down to the same question. what invariant needs protecting, and what is the cheapest mechanism that protects it?

take a payments table. the invariant is that debits equal credits and no account goes below its floor, and the dangerous case is two transfers touching the same accounts at once. the classic protection is pessimistic: select for update on both account rows, always locked in a fixed order. concurrent transfers either queue cleanly or run independently. the fixed order is what makes deadlock impossible, since no two transactions can ever hold pieces of each other’s lock set. the alternative is optimistic concurrency, a version column checked at commit. optimistic wins when conflicts are rare. money breaks that assumption. payment workloads have hot rows, one busy account hit constantly, and under real contention optimistic retries snowball into a storm of replayed transactions. waiting a few milliseconds on a lock is cheaper than replaying work.

a price feed protects a different invariant, which is that each symbol’s ticks apply in order. you can protect it with no locks at all. partition the stream by symbol, the way kafka does, and two symbols never contend while each partition serializes its own updates. that is concurrency by isolation rather than by locking, and it is the cheapest mechanism on this list when your invariant is per-key ordering. the catch is delivery. streams give you at-least-once, so the same message will eventually arrive twice, and the write on the far end has to be idempotent, landing on the same state no matter how many times it applies.

overload is the third shape. a batch job that fans out unbounded work will hang the moment traffic outgrows it, and the fix is rarely clever. chunk the work, bound what is in flight, and keep headroom instead of running the pipe at its ceiling. backpressure is a concurrency mechanism too, just one that protects the invariant “this system stays up”.

locks, partitions, idempotent writes, and bounded queues are not competing philosophies. they are tools, and the invariant picks the tool. most of the concurrency bugs i have seen came from someone adding mechanism without naming the invariant first.