Qwen2.5-0.5B Cheap Knot — CI-Fast Quality Gate

Date: 2026-05-22

Why

The quality gate currently runs against the coder-7B dense knot (~/.edgework/models/qwen-coder-7b.knot, ~4.7 GB). That model is large and slow, which makes the gate expensive to run in CI. Qwen2.5-0.5B-Instruct is ~400 MB and runs in a fraction of the time. This document records producing a dense KNOT from the 0.5B GGUF and verifying it answers the PARIS baseline, so the gate can use it instead.

Note: this is about acquiring open model WEIGHTS and converting them into our own KNOT format for our own fat-station / paris-probe runtime. It is not a third-party inference engine.

Artifacts produced

Both magic-checked: GGUF starts with 47 47 55 46 ("GGUF"), KNOT starts with 4b 4e 4f 54 ("KNOT").

Model config read from GGUF: arch=qwen2, hidden_dim=896, layers=24, heads=14, kv_heads=2, head_dim=64, ffn=4864, vocab=151936, rope_theta=1000000, rms_norm_eps≈1e-6. 290 tensors, all Q4_K/Q6_K/F32 (0 skipped).

Steps (reproducible)

1. Download (public, no auth):

   curl -L -o ~/.edgework/models/Qwen2.5-0.5B-Instruct-Q4_K_M.gguf \
     https://huggingface.co/bartowski/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-Q4_K_M.gguf

2. Convert to a dense KNOT (verbatim quantized-byte repack, no re-quantize):

   cd open-source/gnosis/distributed-inference
   python3 gguf-to-knot.py \
     ~/.edgework/models/Qwen2.5-0.5B-Instruct-Q4_K_M.gguf \
     ~/.edgework/models/qwen2.5-0.5b-instruct.knot \
     --name qwen2.5-0.5b-instruct

   The converter's own reverse round-trip check (first/middle/last tensor) all reported match=True and the tool printed OK.

3. Probe the PARIS baseline:

   ./target/release/paris-probe-rknot \
     --dense ~/.edgework/models/qwen2.5-0.5b-instruct.knot \
     --dense-only \
     --tokens 785,6722,315,9625,374 \
     --expected 12095 \
     --top-k 8

Probe result — PARIS baseline

Prompt tokens [785, 6722, 315, 9625, 374] decode (via the GGUF vocab, loadQwenTokenizerFromGguf(...).decode([...])) to exactly:

"The capital of France is"

The 0.5B argmax is token 12095, which decodes to " Paris" — the same answer the coder-7B gives. The probe exits 0 (PASS).

Top-8 logits, with decoded text:

Logit health: all 151936 logits finite, 0 NaN/Inf/zero. Max finite logit +14.34 at token 12095, min -15.29, mean |finite| +3.78. No residual NaN/Inf at any of the 24 layers.

Timings (wall clock, time -p real)

Notes on timing:

• The KNOT loads in 0.00 s reported by the probe (mmap), so probe wall time is forward pass + lm_head, not load. The lm_head over the 151936-vocab takes ~0.20 s; per-layer forward is ~0.03–0.19 s.
• Warm timings on this machine were noisy (3.5 s up to 11.8 s) due to host CPU contention during measurement, not because of cold/warm cache behavior — the cold run was actually the fastest observed (1.81 s). The honest read is: a single Paris probe on the 0.5B knot is a low-single-digits-to-low-tens-of- seconds operation, dominated by the dense forward pass, with no large fixed cost. This is dramatically cheaper than running the same gate against the 4.7 GB coder-7B.

Recommendation — make the gate CI-cheap

The 0.5B passes PARIS and gives the identical " Paris" argmax as coder-7B, so it is a valid drop-in for the cheap gate. Configure:

MONSTER_SWARM_QWEN_KNOT=~/.edgework/models/qwen2.5-0.5b-instruct.knot

with expected token id 12095 (" Paris") on prompt tokens 785,6722,315,9625,374 ("The capital of France is").

Exact gate invocation (exit 0 == pass):

paris-probe-rknot \
  --dense $MONSTER_SWARM_QWEN_KNOT \
  --dense-only \
  --tokens 785,6722,315,9625,374 \
  --expected 12095 \
  --top-k 8

To reproduce the knot from scratch in CI, run the download + convert steps above; both together complete in well under a minute and need ~800 MB of disk (GGUF + KNOT) under ~/.edgework/models/.