distributed-inference-host scripts

Parent: distributed-inference-host

Utility runners for distributed-inference experiments and Zodiac attack experiments, including optional Colab runtime.

Getting Started

• What: scripts and automation helpers for scripts.
• Why: it keeps repeatable generation, validation, or packaging work out of hand-run one-offs.
• How: prefer the package script that wraps the file, then inspect the script only when changing its contract.
• Next: check the parent README for the safest command surface.

Contents

• bench-mesh-tps.mjs: streaming tok/s benchmark for the OpenAI-compatible mesh (/v1/chat/completions). Warms each model, then reports DECODE tok/s + prefill TTFT. Built to A/B Q8 vs Q4_K knot variants (e.g. qwen3-8b vs qwen3-8b-q4k). Run: node bench-mesh-tps.mjs <BASE_URL> <model...>.
• zodiac-overnight.ts: long-running TypeScript orchestration entrypoint.
• amplituhedron-partial-replay-bench.ts: local microbenchmark for Grassmannian app-cache skips, sparse topology diffs, cold prefill, full Amplituhedron replay, and packed-XOR partial replay suffix recompute.
• amplituhedron-maha-sharding-bench.ts: local microbenchmark for the Maha 21-virtual-shard routing surface across small and replicated physical meshes.
• zodiac_gpu_solver.py: GPU swarm solver implementation.
• zodiac-colab-run.py: optional Colab/local launcher that can run GPU solver or Rust solver.
• colab-bootstrap.sh: Colab environment bootstrap for Python + optional Rust toolchain.
• Zodiac.ipynb: canonical interactive notebook for local and Colab runs.

Amplituhedron partial replay bench

Run:

GNOSIS_AMPH_BENCH_HIDDEN=512 \
GNOSIS_AMPH_BENCH_PROMPT=256 \
GNOSIS_AMPH_BENCH_ITERATIONS=50 \
GNOSIS_AMPH_BENCH_WARMUP=5 \
pnpm run gnode -- run open-source/gnosis/distributed-inference-host/scripts/amplituhedron-partial-replay-bench.ts

Latest local snapshot, measured on May 27, 2026:

The Grassmannian rows use precompiled seed/current topology plans, matching the runtime shape where app identity is compiled at deploy/start rather than per request. grassmannian-skip is the exact fingerprint path and does not allocate a lane diff; grassmannian-diff mutates one station bracket and measures sparse lane invalidation. The partial row simulates a three-station mesh where station 0's prefix boundary remains hot and stations 1-2 miss. The packed-XOR replay certificate selects the suffix recompute path, skipping embed plus station 0 prefill work.

For capture-only calls (maxNewTokens=0), the host now skips final lm_head while preserving tail residual capture. The synthetic bench's mock lm_head is tiny, so the main proof is the coordinator counter test; real meshes avoid the exit-station output projection on prefix-warming requests.

Direct pipelinedPrefill calls now omit full per-station residual capture by default. The Amplituhedron coordinator opts back in for volume capture, while ordinary prefill and prefill-window suffix calls avoid copying and stitching every station's full prefix stream.

prefix-hash-u64 measures the normal ArrayBuffer-backed fast path; prefix-hash-copy forces the old copy-before-digest behavior for comparison. The normal path avoids the extra byte copy, but the microbench is dominated by WebCrypto timing variance at this prompt size.

Trace-off dispatch no longer formats trace samples or records timestamps unless PIPELINE_TRACE=1 or a SpectralNoiseMonitor is configured. This keeps the normal prefill loop focused on station dispatch rather than disabled telemetry.

After capture-only prefill or partial suffix recompute, generate(maxNewTokens=0) now returns the prompt directly instead of re-entering generateFromPrefill just to rebuild the same token array.

Decode output assembly uses a preallocated Uint32Array instead of Array.from(prompt) -> push -> Uint32Array. The isolated bench shows assemble-output-prealloc at 1.4us versus assemble-output-array at 22.0us for a 256-token prompt plus 8 generated tokens; the full generate-8-tokens row remains dominated by station forward variance.

Amplituhedron Maha sharding bench

Run:

pnpm run gnode -- run open-source/gnosis/distributed-inference-host/scripts/amplituhedron-maha-sharding-bench.ts

Latest local snapshot, measured on May 27, 2026:

The coordinator always maps the physical mesh onto 21 virtual Maha shards. A single station owns all 21 virtual shards; two stations split them as 11/10; three stations split them as 7/7/7; 21 stations map one-to-one; replicated meshes keep the same 21-shard coordinate system with multiple physical stations per virtual shard.

The synchronous local-touch rows in the bench are intentionally cheap and mostly measure route overhead. On the latest 21-station snapshot, hoisting the Maha BigInt modulus anchor kept route tax at 0.109us for bigint hashes and 0.135us for precomputed decimal-string hashes. A hand-written decimal modulo path was tested and rejected because V8's BigInt parser was already faster at this u64 hash size. The async rows are the useful proxy for station fanout: once the mesh has enough stations, Maha routing avoids broad Promise.all probes and captures while preserving stable prefix ownership.

Host integration only trims the Amplituhedron replay/capture fanout when the stations are replica-equivalent over the same layer range. Sequential layer meshes still probe and capture every layer station, so partial replay keeps global node-index semantics while replica cache meshes get the virtual-21 route.