r/erlang • u/ancatrusca0 • 11m ago
The BEAM Book came from a production outage caused by a function that used only 20 reductions - what does that tell us about reduction accounting?
New BEAM There, Done That with Erik Stenman (Happi) - Erlang researcher since 1994, BEAM Book author, former architect of Klarna's payment core.
The scar is worth dissecting for this community: on Christmas Eve, Klarna's BEAM scheduler throttled to a single active scheduler due to low overnight traffic. A large batch job arrived. An XML-to-binary serialization function used only 20 reductions per call regardless of input size. It ran on the single scheduler, never yielded, and Heart (the external watchdog process) couldn't get a response from the VM. Heart killed the system. The failover node hit the same condition. So did the third. Half-hour outage on Christmas Eve.
The fix required going deep into runtime behavior - understanding the relationship between reductions, yielding, and scheduler availability under load. That diagnostic process is directly what motivated the BEAM Book.
The current work (Eldrim - sovereign AI compute routing) applies the same principle at a different layer: you need to know where resources actually run, not where you've been told they run.