Variant-specific crosscoder features are seed-stable but not detectably task-causal in a GRPO-LoRA math setting

Authors: Nozomu Fujisawa, Masaaki Kondo (Graduate School of Science and Technology, Keio University)

Paper: ICML 2026 Mechanistic Interpretability Workshop (Spotlight)

Supplementary material — code, results, and pre-registration — for the paper above.

Quickstart: How to reproduce Figures 1–4

If you only want to verify the figures from cached results (no GPU, no model download):

pip install -r requirements.txt
jupyter nbconvert --to notebook --execute notebooks/04_diagnostics_jaccard_and_effectiveness.ipynb
jupyter nbconvert --to notebook --execute notebooks/05_manual_answer_audit.ipynb

These two notebooks read from results/*.{csv,json} and reproduce Figure 4 and the symbolic-audit table in <2 minutes on CPU. The remaining figures (Figures 1–3) require activation extraction and crosscoder training; see the Reproducing the figures table below for runtimes and hardware. Full follow-up protocols (E.1–E.6 in the paper) are in docs/SUPPLEMENTARY_EXPERIMENTS.md.

TL;DR

We test whether the variant-specific features detected by joint-norm pairwise crosscoders trained on (base, RL-tuned) activation pairs reflect genuine RL-induced computation. Three lines of evidence support a measurement artifact interpretation in this setting:

1. The number of features with ν > 0.7 appears only after a capacity threshold (expansion ≥ 16) and shows a weak descriptive log-log trend, but the pattern is non-monotonic and drops at the largest capacity. At fixed capacity the count is identically zero across five token budgets.
2. Cross-seed Jaccard similarity of detected features' top-activating tokens is statistically indistinguishable from a size-matched random baseline: 4/6 seed-pairs fall inside the random 95% interval, 0 are significantly above, and 2 are significantly below. However, non-lexical metrics give the opposite verdict: under decoder cosine and held-out activation correlation, every detected variant-specific feature pairs with a counterpart in another seed at percentile 1.0 versus a 1000-feature random pool (Notebook 08).
3. Ablating the detected features in the residual stream of the RL-tuned model perturbs generated outputs (mean Levenshtein ratio 0.60 vs. clean) but does not yield a detectable MATH-500 accuracy drop in either a small-n ablation (clean 0.500, ablate-vs 0.550, n=20) or an n=100 paired evaluation against a magnitude-matched random control (audited clean 0.57, ablate-vs 0.56, ablate-magmatched 0.56; paired Δ between conditions = 0, 95% CI [-0.03, +0.03], Notebooks 06–07).
4. Two complementary base-vs-base controls diagnose what the high-ν gate is responding to:
   • Paired-identity (A_t = B_t, Notebook 11): zero high-ν features in all 12 trained crosscoders; ν std collapses to 0.004. The gate does not gratuitously emit asymmetry.
   • Disjoint halves (Notebook 09/10): 50–200× as many features as base/GRPO with similar non-lexical seed-stability (520/577 = 90% above the best-of-m_vs random 97.5-percentile under decoder cosine). This is unpaired-pair reconstruction asymmetry, not crosscoder architecture per se.

The high-ν gate detects something real about between-side distributional difference, but most of the volume in our setting is unpaired-pair noise; the small remaining signal in the base/GRPO setting (1–5 features per crosscoder) reflects real model-pair distributional drift but is not detectably the task-causal RL computation.

Repository layout

.
├── notebooks/
│   ├── 01_scaling_token_axis.ipynb              — Figure 1
│   ├── 02a_scaling_capacity_sweep.ipynb         — Figure 2 (low-capacity points)
│   ├── 02b_capacity_extension_and_inspection.ipynb — Figure 2 (high-capacity points)
│   ├── 03_cross_seed_inspection_and_ablation.ipynb — Figure 3
│   ├── 04_diagnostics_jaccard_and_effectiveness.ipynb — Figure 4 (CPU-only)
│   ├── 05_manual_answer_audit.ipynb             — Symbolic audit (CPU, executed)
│   ├── 06_magmatched_control.ipynb               — Magnitude-matched control
│   ├── 07_n100_ablation.ipynb                   — n=100 paired ablation (protocol)
│   ├── 08_cross_seed_geometric_stability.ipynb  — Non-lexical seed stability (protocol)
│   ├── 09_base_vs_base_control.ipynb            — Negative control crosscoder (protocol)
│   ├── 10_b2b_geometric_stability.ipynb         — Geometric stability on b2b (executed)
│   └── 11_b2b_paired_identity_control.ipynb     — Paired-identity b2b control (executed)
├── results/                                     — All CSV/JSON used by figures + supplementary
├── figures/                                     — Final paper figures (PNG)
├── prereg/                                      — Pre-registration + amendments
└── docs/
    ├── REPRODUCING_FIGURES.md
    ├── ARTIFACTS.md
    ├── KNOWN_ISSUES.md
    └── SUPPLEMENTARY_EXPERIMENTS.md             — All five supplementary protocols

Reproducing the figures

Figure	Notebook	Runtime	Hardware
1	01_scaling_token_axis.ipynb	~2.5 h	A6000+
2 (low cap)	02a_scaling_capacity_sweep.ipynb	~1.5 h	A6000+
2 (high cap)	02b_capacity_extension_and_inspection.ipynb	~1 h	A6000+
3	03_cross_seed_inspection_and_ablation.ipynb	~2.5 h	A6000+
4	04_diagnostics_jaccard_and_effectiveness.ipynb	~5 min	CPU only
audit	05_manual_answer_audit.ipynb	~1 min	CPU only
(exec.)	06_magmatched_control.ipynb	~2.5 h	A6000+
(exec.)	07_n100_ablation.ipynb	~2 h	A6000+
(exec.)	08_cross_seed_geometric_stability.ipynb	~10 min	A6000+
(exec.)	09_base_vs_base_control.ipynb	~12 h	A6000+
(exec.)	10_b2b_geometric_stability.ipynb	~5 min	A6000+
(exec.)	11_b2b_paired_identity_control.ipynb	~12 h	A6000+

Notebooks 04 and 05 reproduce Figure 4 and the audit table directly from the CSV/JSON files in results/ without any model loading or GPU work. See docs/REPRODUCING_FIGURES.md for details and docs/SUPPLEMENTARY_EXPERIMENTS.md for the audit and the four follow-up protocol notebooks.

Setup

pip install -r requirements.txt
export MPCD_ROOT=/path/to/your/workspace

For figures 1–3 you also need to download the pre-computed activation tensors and trained crosscoders (available on request; see docs/ARTIFACTS.md). Figure 4 needs only this repository.

Pre-registration

The original written analysis protocol and the chronological log of amendments are in prereg/. We do not claim the protocol was externally timestamp-certified before any analysis; we provide it as a record of the analytical choices that produced the verdicts in the paper. The amendments file documents every deviation from the protocol, including the methodological amendments motivated by the diagnostics that produced the negative result.

Limitations

The negative result rests on four notable caveats — see docs/KNOWN_ISSUES.md for full discussion:

• The random control set in the ablation experiment is sampled from features in the magnitude-balanced band (ν ∈ [0.40, 0.60]); these features have systematically smaller decoder norms than the variant-specific features, so the random control under-perturbs the model and is byte-identical to clean on all 20 problems.
• The ablation evaluation uses only 20 problems, so confidence intervals on accuracy are wide.
• The MATH parser normalizes whitespace and a few delimiters but does not perform symbolic equivalence, so the reported accuracy is a coarse signal rather than a fine-grained measurement.
• All experiments use a single base model (Qwen3-4B) at a single layer (18) on a single domain (math). Generalization to other settings is left to future work.

License

MIT (see LICENSE).