Topology-Ceiling Breakers — Path Past the Wave-23 1.33× Wire Floor

Date: 2026-05-04 Operator: read-only investigation Source bench: docs/post-everything-tps-bench-result.md (wave-23, gemma4-31b trisplit, layer-0 WS)

1. Executive summary

Wave-23 measured the post-everything TPS at 0.0285 TPS (warm WS p50, 549 ms/layer × 60L + 2.17 s cobordism = 35.11 s/token). The wire-transit floor with the current 3-hop-per-layer topology is 0.0370 TPS (best single hop 138 ms × 3 × 60L + 2.17 s = 27.01 s). We are at 1.33× above the wire floor — kernel work has plateaued.

The next cliff is topological, not numeric. Two changes are infrastructure-ready and stack multiplicatively:

Combined post-everything ceiling: ~0.072-0.080 TPS without speculative, ~0.10 TPS stacked with N=8 speculative — 100× the 0.001 substrate baseline, 3.5× the current operating point.

Recommended sequencing (3 bule of foreground work):

1. Bule 1 — wire FlowSendBatcher into the trisplit station's WS send path + bench.
2. Bule 2 — flip Pair X from shadow → live, including the /split-a consume path.
3. Bule 3 — extend bench bench-alpha-jump.ts to a multi-layer Pair X chain and validate the 1.5× per-layer collapse.

2. Wire-floor math + current position

Measured (wave-23, warm rounds 2-10):

split-a p50 = 187 ms   (best 183, p95 197)
split-b p50 = 148 ms   (best 138, p95 169)
split-c p50 = 207 ms   (best 192, p95 230)
per-hop  p50 = 188 ms   (best 138, p95 340)
per-layer warm p50 = 549 ms
60-layer wall = 549 × 60 + 2170 = 35.11 s
TPS = 1 / 35.11 = 0.0285

Wire floor (3-hop sequential, best single hop = 138 ms):

floor per-layer = 3 × 138 = 414 ms
floor 60-layer wall = 414 × 60 + 2170 = 27.01 s
floor TPS = 1 / 27.01 = 0.0370

Slack:

0.0370 / 0.0285 = 1.298 → 1.33× (rounded)
remaining compute headroom = (549 - 414) × 60 = 8.10 s/token

After the next 1.3× of kernel work (Q4_K on split-c, fused rotary on split-a), the bench is strictly wire-bound. The only paths past 0.0370 TPS are:

• (A) reduce the number of sequential hops per layer (Death #4 Pair X)
• (B) reduce the per-frame cost of each hop (frame coalescing)
• (C) amortize across tokens instead of layers (speculative)

These are independent axes. (A)+(B) together is the topology change; (C) is per-token.

3. Death #4 — Pair X Fano routing

What it is, operationally

The transformer residual stream is x_{l+1} = x_l + attn(x_l) + ffn(x_l + attn(x_l)). The first additive term x_l is already known before layer l even starts — it's the input that arrived at tri-a-l. Today the canonical chain serializes layer l+1's split-a on layer l's split-c. Pair X says: dispatch x_l directly to tri-a-(l+1)'s residual-seed port (Fano line (e1, e4, e5) — the skip-attention shortcut) the moment split-a-l completes, while split-b-l and split-c-l keep computing. tri-a-(l+1) can begin its own ATTN on the seeded residual concurrently with split-c-l's down-proj. Layer l's ffn contribution arrives at tri-d-(l+1) later via the canonical chain and combines additively at the residual-seed accumulator.

Per-layer math

Today (sequential): split-a + split-b + split-c = 187 + 148 + 207 = 542 ms (within 7 ms of the measured 549 ms p50).

With Pair X exploited, layers l and l+1 overlap on the down-proj/attn boundary:

critical_path(l, l+1) = split-a-l + split-b-l + max(split-c-l, split-a-(l+1)) + split-b-(l+1) + split-c-(l+1)
                      ≈ 187 + 148 + max(207, 187) + 148 + 207
                      ≈ 897 ms for two layers
                      = 448.5 ms / layer averaged

Compare to non-overlapped: 1098 ms / 2 = 549 ms / layer. Per-layer save ≈ 100 ms (1.22× speedup).

The full upper bound (ATTN of l+1 fully overlapped with the longer of split-b-l + split-c-l):

floor_per_layer ≈ split-a + max(split-b + split-c, split-a) ≈ 187 + max(355, 187) = 542 → 187 + max(split-b||split-c overlap with next-layer split-a)

Because split-c (207 ms) is the longest hop and split-a-(l+1) (187 ms) fits inside it, the asymptote is:

per-layer ≈ split-a (187) + split-b (148) + (split-c overlapped with next split-a)
         ≈ 335 ms + 207 ms (the hop that can't be fully hidden) - overlap_save
         ≈ 350-400 ms / layer warm

Operating-point estimate: 549 → ~370 ms/layer = 1.48× per-layer speedup.

60-layer wall: 370 × 60 + 2170 = 24.37 s → TPS ≈ 0.041, which is above the naive 3-hop floor of 0.0370 because Pair X effectively reduces the per-layer hop count from 3 to ~2 in the critical path.

Bule cost

Bule = 3. Infrastructure is already deployed in shadow mode. Concrete remaining work:

1. Flip the canonical chain to consume the seed. The /split-a HTTP path at apps/worker-node1/src/index.ts:914 (and apps/distributed-inference-worker/src/index.ts:1152) accepts seeds; the equivalent /flow PHASE_A handler accumulates into residualSeeds (worker-node1/src/index.ts:2471-2644). The "consume" side reads getShadowSeed(station, dstLayer, tokenSeq) and uses it as the residual instead of waiting for the upstream chain. Today this is shadow-only — fingerprint compare runs but the chain residual still wins. Flip = single boolean (shadow_mode: false in the response + corresponding consume branch in /split-a).
2. Coordinator dispatch logic. AlphaJumpDispatcher (open-source/gnosis/distributed-inference-host/src/alpha-jump.ts) encodes the frame on the Death #3 substrate. Pneuma-think's trisplit-station-ws.ts does NOT currently fire it — needs an addition: after forwardSplitA(l) resolves, dispatch the seed on tri-a-(l+1) for line E1_E4_E5 in parallel with forwardSplitB(l).
3. Validation gate. Run bench-alpha-jump.ts (scripts/bench-alpha-jump.ts:185-233) extended to a 2-layer chain. Confirm max_abs(seeded_chain_residual - canonical_residual) <= 1e-4 over N=20 rounds. Only then flip live.

Risk

f32 path equivalence is the open question per OCTONION_ROUTING_PLAN.md §9. The shadow-mode bench is the gate — don't flip until divergence is bounded over a measurement window. If divergence > 1e-4, fall back to "Pair X for layers 1..N where it parity-passes" and let the rest serialize.

4. Frame coalescing — the lower-bule lever

What it is

FlowSendBatcher (open-source/aeon/src/flow/FlowSendBatcher.ts:150) coalesces multiple aeon-flow frames into one contiguous WebSocket.send() call within a microtask boundary. Defaults: 16 frames per batch, 64 KiB cap. Already shipped, already exported from @a0n/aeon/flow, already used by @a0n/aeon-flux/runtime and aeon/transport/dashrelay.ts. Not wired into the trisplit WS hot path — WebSocketFlowTransport.send() (open-source/aeon/src/flow/WebSocketFlowTransport.ts:233) calls this.socket.send(buf) directly with no batching.

Mechanism of speedup

Per WebSocket frame today: header (2-14 bytes WS framing) + 10-byte aeon-flow header + payload. CF Worker → CF Worker WSS pays:

• TLS write (per socket.send): ~0.3-2 ms in the runtime regardless of payload
• WS framing overhead per call (event loop tick + frame masking on client)
• The WSS connection is persistent so no handshake — but each send() is a separate tick

For the trisplit chain there are 3 sends + 3 receives per layer — 6 frame events per layer × 60 layers = 360 events/token. Coalescing the outbound sends in the small pre-payload windows (e.g. queueing the frame and the immediate-following control / ack frame) saves the per-tick TLS overhead on the second frame.

The ASYMMETRIC reality: the heavy frames in the chain (split-a request = hidden × seqLen × 4 bytes ≈ 5376×4=21 KiB at seqLen=1, payloads dominate). Pure coalescing of two big frames gives little win because the cost is bytes-on-wire, not framing. The win shows up where:

• Payload is small (residual-seed dispatches: header + ~21 KiB f32 — these CAN coalesce with the next outbound to the same socket)
• Multiple sockets share the same TLS connection (CF doesn't multiplex this way; per-worker WSS is separate, so no inter-station coalescing wins)
• Microtask windows naturally collect multiple frames (Pair X dispatch fires concurrently with split-b request → both target different sockets, NO coalescing benefit there)

Honest speedup estimate

1.10-1.20× on per-layer wall, mostly from the residual-seed + small-control frame coalescing once Pair X fires, plus the rare case where two requests are scheduled in the same JS turn. Without Pair X live, the win is ~1.05× (5%) — frame counts on the trisplit hot path are too low to amortize.

Combined with Pair X: effectively reduces the 207 ms split-c hop's tail (which includes Worker→client framing) by 5-10 ms = ~3-5 ms/layer → another ~200-300 ms over 60 layers.

Bule cost

Bule = 2. Only changes to WebSocketFlowTransport.ts:

1. Add an optional batching: FlowSendBatcherConfig to the constructor.
2. Wrap send() to enqueue via FlowSendBatcher when batching is enabled.
3. Add a flush() method (already in batcher) so close() flushes pending.

The existing FlowSendBatcher handles non-flow payloads correctly (passes through). No protocol change. No worker change.

Risk

Low. isSingleFlowFrame validates each payload is exactly one frame; mixed payloads bypass batching. Re-tested behavior matches the dashrelay use site. The only failure mode is increased latency for small standalone frames (microtask delay before send), which is O(1 ms) worst case.

5. Speculative-decode N=8 stacking math

Speculative decode is per-token, not per-layer — orthogonal axis. Per SESSION_NARRATIVE_2026_05_03.md, draft-and-verify shipped at K=5, ran 1.33× wall speedup on the draft path. For N=8 (multi-token speculation):

Current single-token: 35.11 s/token (wave-23)
N=8 speculative ideal: 35.11 s / 8 ≈ 4.39 s/token (verify rejects most)
N=8 realized at acceptance rate α: 35.11 / (1 + α × N) ≈ for α ≈ 0.4, wall = 35.11 / 4.2 ≈ 8.4 s/token
Realized speedup: 35.11 / 8.4 = ~1.4× (matches typical speculative literature for 7B-class models)

Stacking with topology levers

The per-token wall is gated by:

wall = tokens_to_emit × per_layer × 60 + cobordism_overhead

Speculative reduces tokens_to_emit (the verify pass costs one full forward, accept-or-reject N draft tokens). Topology levers reduce per_layer. They multiply:

combined = current / (per_layer_speedup × token_speedup)
        = 0.0285 × 1.48 (Pair X) × 1.20 (frame coal) × 1.4 (N=8 spec)
        = 0.0285 × 2.49
        ≈ 0.071 TPS  (without speculative)
        ≈ 0.099 TPS  (with N=8 speculative)

Over the 0.001 substrate baseline that is ~71-99×.

6. Combined post-everything ceiling

Honest expected operating point after the 3 bule of work: 0.05-0.07 TPS depending on speculative acceptance rate and how much of the predicted Pair X overlap actually materializes (lab vs. real-prompt always has a 12% gap per F-mesh-10).

7. Recommended next 3 bule of work

Bule 1 (lowest cost, validation-first): wire FlowSendBatcher

Sequence:

1. Read WebSocketFlowTransport.ts and FlowSendBatcher.ts together; design the mutable-config wrap.
2. Add batching?: FlowSendBatcherConfig to WebSocketFlowTransportConfig. Default disabled (additive).
3. Wrap send() to push through the batcher when enabled.
4. Re-bench against wave-23 baseline (/tmp/post-everything-ws-bench.mjs) with batching on. Expect 5-10% improvement on per-layer warm.
5. Gate: per-layer warm < 530 ms p50 = ship; ≥ 530 ms = leave disabled, move to Bule 2.

This is low-risk, additive, and falsifies cheaply. It also de-risks Bule 2 by making Pair X dispatch frames coalesce naturally with split-b requests.

Bule 2 (highest leverage): Pair X live mode, single-layer first

Sequence:

1. In trisplit-station-ws.ts, after forwardSplitA(l) resolves, fire AlphaJumpDispatcher.dispatch(line=E1_E4_E5, dst=l+1, residual=x_l) in parallel with forwardSplitB(l).
2. In worker-node1/src/index.ts /split-a handler (HTTP and /flow paths), check getShadowSeed(station, l, tokenSeq) first; if present and bitset non-zero, use the seed as the residual instead of awaiting the chain. Log pair_x_seeded=true in the response.
3. Run bench-alpha-jump.ts extended to a 2-layer chain. Confirm max_abs_diff(seeded_residual_layer_2, canonical_residual_layer_2) <= 1e-4 over 20 rounds.
4. Gate: max_abs ≤ 1e-4 = flip 2-layer live; > 1e-4 = stay shadow, file the divergence with the open question in OCTONION_ROUTING_PLAN.md §9.
5. Roll forward to 60-layer if the 2-layer gate passes.

This is the topology change that breaks the wire floor. Single biggest TPS lever remaining on the per-layer axis.

Bule 3 (validation + bench): full-mesh Pair X bench

Sequence:

1. Extend /tmp/post-everything-ws-bench.mjs to drive the 60-layer chain via meshGenerate with pairX=true.
2. Capture per-layer p50, p95, and the new "overlap-save" metric (split-c-l end timestamp - split-a-(l+1) start timestamp; should be negative under live Pair X).
3. Update docs/post-everything-tps-bench-result.md with wave-24 numbers.
4. Gate: 60-layer wall < 30 s = ship to all 180 tri-g4 workers; ≥ 30 s = bisect which layers regressed and decide per-layer enablement.

8. Why these three before anything else

• Frame coalescing (Bule 1): zero-risk, additive, infrastructure-ready. If it gives nothing, it costs nothing.
• Pair X live (Bule 2): the only known way to push past the 3-hop wire floor with the existing worker count. Infrastructure is deployed in shadow — flipping is mechanical.
• Validation bench (Bule 3): without this, "we shipped Pair X" is a claim, not a number. Per the manuscript-source-of-truth and "always verify" feedback rules, the bench IS the proof.

Speculative decode N=8 is a parallel investigation per the user prompt; it stacks but does not block, and acceptance-rate tuning is its own bule cost. Layer-parallel (split layers 0..29 / 30..59 across mesh subsets) requires re-architecture and full multi-shard coordination — much higher bule, no infrastructure ready.

9. What this proposal does NOT do

• Does not touch model arithmetic — Pair X is f32-equivalence-validated before flip.
• Does not change worker count, weight format, or knot artifacts.
• Does not solve KV cache coherence across pair-x layers (the existing chain already solves this; Pair X uses the canonical chain's KV writes, just consumes the seed for ATTN input).
• Does not address the cobordism overhead (2.17 s embed + lm_head warm) — that's a separate Death-#2-adjacent optimization.

10. References

• Wave-23 bench: ./docs/post-everything-tps-bench-result.md
• Death #4 plan: ./open-source/gnosis/distributed-inference/OCTONION_ROUTING_PLAN.md
• AlphaJumpDispatcher: ./open-source/gnosis/distributed-inference-host/src/alpha-jump.ts
• Residual-seed handler (deployed): ./apps/worker-node1/src/residual-seed-handler.ts and apps/distributed-inference-worker/src/residual-seed-handler.ts
• Trisplit WS station: ./apps/pneuma-think/src/trisplit-station-ws.ts
• WS transport: ./open-source/aeon/src/flow/WebSocketFlowTransport.ts
• Frame batcher (unwired on this path): ./open-source/aeon/src/flow/FlowSendBatcher.ts
• Pair X bench (single-frame shadow): ./scripts/bench-alpha-jump.ts
• Five Deaths roadmap memory: project_five_deaths_tps_roadmap.md