Catch abliterated models at registration with AUROC 0.95 audit

Abliterated models are open-weight checkpoints with their safety guardrails surgically removed. For any protocol that relies on LLM outputs to make on-chain decisions, they’re a growing threat. Until now, catching them meant expensive runtime evaluation or white-box access. A new paper changes that: it proposes a two-signal checkpoint audit that detects abliteration at registration time using only cheap internal signals. On a 273-checkpoint registry spanning Qwen, DeepSeek-distilled Qwen, Llama, and Gemma, it hits AUROC 0.95 [^claim_1412].

The audit combines two signals into a threshold-free check: a reference-anchored activation refusal-gap and a weight-recovery energy of the base-to-candidate weight difference [^claim_1410]. The signals are negatively correlated and label-complementary — the gap supplies refusal-specificity, the weight energy supplies recall [^claim_1411]. Together, their z-sum separates 57 public abliterations from 37 benign fine-tunes, merges, and instruction-tunes at AUROC 0.95, well above either signal alone (0.84 and 0.90) [^claim_1417]. A Youden-calibrated threshold transfers to held-out families at balanced accuracy 0.89 (FPR 0.11), missing only 4 of the 57 abliterations [^claim_1413].

For crypto inference markets — Bittensor subnet validators, oracle networks using LLM outputs, or DAO-governed model registries — this is a pre-deployment triage tool that shifts safety costs from runtime to registration. Runtime guards can’t detect abliteration because they score generations, not the artifact itself [^claim_1416]. By catching abliterated models before they serve a single generation, the audit aligns incentives: stake slashing can be tied to passing the audit at registration rather than after harmful outputs are served.

But the audit has two failure modes that bound its crypto applicability. A spoofed reference evades both axes with no training — ΔW=0, ρ=1 by construction [^claim_1414]. That maps directly to the need for a tamper-evident reference registry anchored on-chain: the reference model hash must be committed to a smart contract at a known timestamp. Without an on-chain attestation of the reference, the whole audit is spoofable. The second failure mode is more fundamental: a white-box owner can train a checkpoint past the threshold while it stays guard-unsafe and coherent [^claim_1414]. That sets a hard limit on what permissionless verification can guarantee. The audit is effective triage, not tamper-proofing — it presumes an attested reference, and its claims are bounded by the registry it was evaluated on [^claim_1415].

For any crypto protocol that relies on model safety — say, an autonomous AI agent managing a DeFi vault that must refuse harmful transactions — the audit is a first line of defense, not a silver bullet. Protocols need additional runtime monitoring layered on top of the pre-deployment check: TEE-enforced generation guards, on-chain slashing conditions for detected unsafe outputs. The paper’s key insight is that artifact-level detection is possible and cheap, but its crypto utility stops where the integrity of the reference chain ends.