Saturation Masks Test Fixtures

Pre-computed saturation masks for model benchmarking and integration testing.

Getting Started

What: the saturation masks subtree inside Gnosis.
Why: it keeps this local concern documented where readers will look before editing or using it.
How: read this entry first, then follow the local commands, child links, or file list below.
Next: go back to the parent README when you need the wider package context.

Structure

saturation_masks/
├── phi3-mini.masks.pkl         # Phi-3-mini (32 layers, 3072 hidden)
├── qwen2.5-7b.masks.pkl        # Qwen2.5-7B (28 layers, 4096 hidden)
├── gemma-9b.masks.pkl          # Gemma-9b (42 layers, 3584 hidden)
├── llama-70b.masks.pkl         # Llama-70B (80 layers, 8192 hidden)
│
├── phi3-mini.saturation.json    # Metadata for Phi-3-mini
├── qwen2.5-7b.saturation.json   # Metadata for Qwen2.5-7B
├── gemma-9b.saturation.json     # Metadata for Gemma-9b
├── llama-70b.saturation.json    # Metadata for Llama-70B
│
└── MASK_STATS.md               # Summary statistics

Fixture Generation

Generate pre-computed masks for a model:

python saturation_mask_encoder.py \
  --model Qwen/Qwen2.5-7B \
  --method variance_gradient \
  --output masks.safetensors \
  --num-samples 512 \
  --save-pickle fixtures/saturation_masks/qwen2.5-7b.masks.pkl

Generation Parameters

Phi-3-mini:

python saturation_mask_encoder.py \
  --model microsoft/phi-3-mini-4k-instruct \
  --method variance_gradient \
  --variance-quantile 0.2 \
  --num-samples 512 \
  --save-pickle fixtures/saturation_masks/phi3-mini.masks.pkl

Qwen2.5-7B:

python saturation_mask_encoder.py \
  --model Qwen/Qwen2.5-7B \
  --method variance_gradient \
  --variance-quantile 0.2 \
  --num-samples 512 \
  --save-pickle fixtures/saturation_masks/qwen2.5-7b.masks.pkl

Gemma-9b:

python saturation_mask_encoder.py \
  --model google/gemma-2-9b \
  --method statistical \
  --variance-quantile 0.25 \
  --num-samples 512 \
  --save-pickle fixtures/saturation_masks/gemma-9b.masks.pkl

Llama-70B:

python saturation_mask_encoder.py \
  --model meta-llama/Llama-2-70b-hf \
  --method variance_gradient \
  --variance-quantile 0.2 \
  --num-samples 512 \
  --save-pickle fixtures/saturation_masks/llama-70b.masks.pkl

Using Fixtures in Tests

Python

import pickle
from pathlib import Path

# Load fixture
fixture_dir = Path(__file__).parent / "fixtures" / "saturation_masks"
with open(fixture_dir / "qwen2.5-7b.masks.pkl", "rb") as f:
    masks = pickle.load(f)

# Use in tests
assert len(masks) == 28  # 28 layers
for layer_id, mask in masks.items():
    assert mask.num_frozen() > 0

Rust

// Load masks in rust tests
#[test]
fn test_saturation_mask_loading() {
    let fixture_path = "fixtures/saturation_masks/qwen2.5-7b.masks.pkl";
    let masks = load_saturation_masks_from_pickle(fixture_path)
        .expect("Failed to load fixture");
    
    assert_eq!(masks.len(), 28);
}

Expected Statistics

Phi-3-mini

Layers: 32
Avg frozen %: 12.5%
Method: variance_gradient
Estimated speedup: 1.125x
Blob size: 48 KB

Qwen2.5-7B

Layers: 28
Avg frozen %: 10.2%
Method: variance_gradient
Estimated speedup: 1.102x
Blob size: 35 KB

Gemma-9b

Layers: 42
Avg frozen %: 8.1%
Method: statistical
Estimated speedup: 1.081x
Blob size: 42 KB

Llama-70B

Layers: 80
Avg frozen %: 6.3%
Method: variance_gradient
Estimated speedup: 1.063x
Blob size: 60 KB

Fixture Maintenance

Regenerating Fixtures

When updating saturation detection algorithms or calibration corpus:

# Regenerate all fixtures
for model in \
    "microsoft/phi-3-mini-4k-instruct:phi3-mini" \
    "Qwen/Qwen2.5-7B:qwen2.5-7b" \
    "google/gemma-2-9b:gemma-9b" \
    "meta-llama/Llama-2-70b-hf:llama-70b"
do
    IFS=: read model_id fixture_name <<< "$model"
    python saturation_mask_encoder.py \
        --model "$model_id" \
        --method variance_gradient \
        --num-samples 512 \
        --save-pickle "fixtures/saturation_masks/${fixture_name}.masks.pkl"
done

Verifying Fixture Integrity

python saturation_mask_decoder.py \
    --model fixtures/saturation_masks/phi3-mini.masks.pkl \
    --layer-summary \
    --show-metadata

Known Issues

Memory Usage

Large models (Llama-70B) require significant VRAM to compute masks:

Minimum: 40 GB VRAM (with activation offloading)
Recommended: 80 GB VRAM

For constrained environments, use smaller --num-samples (256 instead of 512).

Reproducibility

Masks are deterministic for a given model, tokenizer, and random seed, but may vary slightly:

Across torch versions (due to numerical precision)
With different attention/normalization implementations
If calibration corpus is shuffled differently

Difference tolerance: < 1% variation in frozen neuron counts across runs.

Citation

If using these saturation masks in research, cite:

@inproceedings{saturation_masks_2024,
  title={Saturation Masks for Efficient Transformer Inference},
  author={Forkjoin Contributors},
  booktitle={Distributed Inference Systems},
  year={2024}
}

License

Saturation mask fixtures are available under the same license as the forkjoin.ai project (UNLICENSED / proprietary).

Model weights remain under their original licenses (Phi-3: Microsoft, Qwen: Alibaba, Gemma: Google, Llama: Meta).

forkjoin-ai/gnosis