A Cluster Stall Looks Healthy on Every Host. The Cause Is in the Pattern Across Hosts.

Eight ranks on two hosts. Every per-host metric reads healthy. Rank 5 enters the barrier 290ms late. The cause lives in a cross-rank query, not in any single host’s trace. Watch the demo (90 seconds): Echo starts on :4317, two echo-stress invocations push 2,000 events combined (node-a100 + node-h100), then DuckDB verifies they all landed and the causal-chain markers survived the wire. TL;DR Eight ranks on two hosts run an all-reduce. Token throughput drops 4x. Every per-host nvidia-smi reads 95-