Glossolalia V2 Integration Plan — Rust Distributed Inference Mesh

Status: Phase 1 Complete (MOA + COLTRANE modules), Phase 2 In Progress
Start: 2026-05-02
Target Completion: 2026-05-03

Phase 1: ✅ Core Algorithm Implementation

Completed:

glossolalia_moa.rs (700 LOC): Fork/race/fold with deficit-weighted merge, INTERFERE feedback
coltrane_constraints.rs (300 LOC): Topological FSM (fold, pitch, eddy) with Lean-backed tables
Unit tests: 8/8 passing (4 MOA, 4 COLTRANE)
All modules: Compile cleanly, integrate into lib.rs

Key Functions:

pub fn create_moa_agents(config: &MOAConfig) -> Vec<MOAAgent>
pub fn filter_finite_agents(logits: &[Vec<f32>], indices: Option<&[usize]>) -> FilteredAgentLogits
pub fn deficit_weighted_merge(logits: &[Vec<f32>], vocab_size: usize, strategy: &MergeStrategy, use_kl: bool) -> MOAMergedResult
pub fn interfere(state: &mut InterfereState, deficit: f32, entropy: f32, token: i32, sensitivity: f32)

pub struct ConstraintFSM {
    pub fn compute_valid_token_mask(&self, logits: &[f32]) -> Vec<bool>
    pub fn advance(&mut self, token: u32)
    pub fn is_valid(&self) -> bool
}

Phase 2: 🚀 NativeLlamaPipeline Integration

Goal:

Wrap NativeLlamaPipeline's forward pass with MOA scheduling so that every token generation is:

FORK: k parallel agents at diverse temperatures
RACE: Filter non-finite logits
FOLD: Merge with deficit weighting
COLTRANE: Apply topological masking (if enabled)
SAMPLE: Select token from merged distribution
INTERFERE: Adjust next temperature

Architecture:

┌─────────────────────────────────────────────────────────────────┐
│                     MOAInferenceWrapper                         │
├─────────────────────────────────────────────────────────────────┤
│  
│  For each token position in decode loop:
│    1. FORK: spawn k temperature-diverse agents
│    2. For each agent → call NativeLlamaPipeline.forward_single()
│    3. RACE: filter agents[all finite logits]
│    4. FOLD: deficit_weighted_merge(agents) → merged_logits
│    5. COLTRANE: fsm.compute_valid_token_mask(merged_logits)
│    6. SAMPLE: sample_token(merged_logits[valid_mask])
│    7. INTERFERE: update temperature spread based on deficit
│    8. Write merged_logits + fsmState + metrics to session state
│  
│  Returns: final token, metadata (deficit, entropy, violations)
└─────────────────────────────────────────────────────────────────┘
        ↓ (calls)
    ┌───────────────────────────────┐
    │   NativeLlamaPipeline         │
    │  .forward_range_attn_chunk()  │
    │  .forward_range_ffn_*_chunk() │
    │  (single forward pass)        │
    └───────────────────────────────┘

Implementation Tasks:

Task 2.1: Expose decode_single_token() interface

File: model.rs
Current: forward_range_*_chunk() processes entire layers for a sequence
Need: forward_single_token(position: usize) → Vec (logits for next token)
Approach: Wrapper around existing forward, processes one position at a time

Task 2.2: Create MOAInferenceWrapper

File: glossolalia_moa_pipeline.rs (NEW)
Implements:
- struct MOAInferencePipeline { pipeline: NativeLlamaPipeline, config: MOAConfig, ... }
- fn decode_with_moa(&mut self, tokens: Vec<u32>, length: usize) → DecodingResult
- Manages INTERFERE state across decode loop
- Maintains COLTRANE FSM state for constraint tracking

Task 2.3: Parallel agent execution (rayon)

File: glossolalia_moa_pipeline.rs
Use: rayon::scope() for fork phase (k agents in parallel)
Each lane: calls pipeline.forward_single_token() with modified temperature
Collect: all k logit vectors → RACE phase

Task 2.4: COLTRANE masking integration

File: glossolalia_moa_pipeline.rs
Logic: After FOLD, apply fsm.compute_valid_token_mask(merged_logits)
Tier-1: Stateless (fold_cost, pitch_class) — applied directly
Tier-2: Stateful (pitch transitions, eddy detection) — requires FSM advance
Tier-3: Oracle fallback (if <5% valid tokens) — fallback to unmasked

Task 2.5: Integration test

File: glossolalia_moa_pipeline.test.rs (NEW)
Test: Compare Rust MOA output vs TypeScript reference
- Same config (3 agents, base=0.7, spread=0.4, deficit-weighted merge)
- 10-token decode sequence
- Verify: merged_logits match within f32 precision, deficit converges to φ
- Verify: COLTRANE mask reduces violations by expected %

Task 2.6: Benchmark & validation

File: src/bin/moa-decode-benchmark.rs (NEW)
Measure: latency impact of MOA vs single-agent
- Expected: 12-15% overhead (k=3 agents, parallel execution with rayon)
- Compare to TypeScript baseline (edge-workers distributed-inference)

Technical Decisions

Temperature Sampling Strategy

Current: Explicit agent temperatures from MOAConfig
Sampling: temperature → logits * (1/T) before softmax
Rayon scope: Each agent calls forward with modified temperature parameter

INTERFERE Convergence

Golden ratio: deficit_ratio → φ over decode sequence
Metric: track "distance from φ" to detect convergence
Fallback: if distance > threshold, widen spread (safety valve)

COLTRANE Masking Cost

Tier-1: O(vocab_size) table lookup per token (negligible)
Tier-2: O(5) recent-tokens + FSM state advance (minimal)
Tier-3: Oracle fallback <0.5% of tokens (rare)
Expected overhead: <5ms per token (compared to MOA overhead 12-15%)

Memory Footprint

MOA state: k agents × vocab_size f32 ≈ 3 × 152K × 4B = 1.8 MB
COLTRANE FSM: fold_cost_table (152K) + pitch_class_table (152K) ≈ 300 KB
INTERFERE state: O(1) registers + entropy history
Total: ~2.1 MB per MOA session (well under 128MB Worker cap)

Testing Strategy

Unit Tests (Phase 1) ✓

Fork/race/fold correctness
INTERFERE convergence
COLTRANE FSM state transitions

Integration Tests (Phase 2)

MOA + NativeLlamaPipeline end-to-end
COLTRANE masking compliance
INTERFERE eigenvalue convergence

Benchmarks (Phase 2)

Latency: MOA vs single-agent
Throughput: tokens/sec with k agents
Memory: resident FSM state

Validation (Phase 2)

Compare Rust MOA output vs TypeScript (open-source/aether/glossolalia-moa.ts)
Verify merged_logits statistical properties (entropy, deficit)

Deployment Path (After Phase 2)

Staging (Day 1, 6 hours)

Deploy to staging with feature flag ENABLE_MOA_SCHEDULING=false (single-agent baseline)
Smoke test: 1K tokens on JSON/poetry/code tasks
Enable MOA (ENABLE_MOA_SCHEDULING=true)
10K tokens with constraints enabled, monitor violation rate (expect 12-15%)

Validation (Day 2, 6 hours)

10M token validation with MOA + COLTRANE
Quality checks: JSON parsing, poetry harmonic sense, code syntax
Constraint compliance: fold-depth <5%, harmonic closure >95%, eddy avoidance >90%

Production Ramp (Day 3, 12 hours)

Deploy to prod with MOA disabled (backward compatible)
Traffic ramp: 10% → 50% → 100% (each hour monitored)
Auto-rollback if violation rate >20% or latency >25% increase

References

TypeScript canonical: open-source/aether/src/glossolalia-moa.ts
Constraint spec: open-source/gnosis-math/Gnosis/*.lean (59 theorems)
Integration doc: open-source/gnosis/distributed-inference/ARCHITECTURE.md
Deployment: docs/COLTRANE_DEPLOYMENT_PLAYBOOK.md

Next Step: Task 2.1 — expose forward_single_token() interface in model.rs

forkjoin-ai/gnosis

Glossolalia V2 Integration Plan — Rust Distributed Inference Mesh

Glossolalia V2 Integration Plan — Rust Distributed Inference Mesh

Phase 1: ✅ Core Algorithm Implementation

Completed:

Key Functions:

Phase 2: 🚀 NativeLlamaPipeline Integration

Goal:

Architecture:

Implementation Tasks:

Task 2.1: Expose decode_single_token() interface

Task 2.2: Create MOAInferenceWrapper

Task 2.3: Parallel agent execution (rayon)

Task 2.4: COLTRANE masking integration

Task 2.5: Integration test

Task 2.6: Benchmark & validation

Technical Decisions

Temperature Sampling Strategy

INTERFERE Convergence

COLTRANE Masking Cost

Memory Footprint

Testing Strategy

Unit Tests (Phase 1) ✓

Integration Tests (Phase 2)

Benchmarks (Phase 2)

Validation (Phase 2)

Deployment Path (After Phase 2)

Staging (Day 1, 6 hours)

Validation (Day 2, 6 hours)

Production Ramp (Day 3, 12 hours)

References