Regex in an NTK Costume: When Your Own Verifier Is Lying About Its Implementation

One of our verifiers had a docstring claiming it implemented an NTK-based hallucination detector from arXiv:2601.18753. The implementation was fifty-six lines of regular expressions. Another was caught sleeping for an arbitrary duration so its wall-clock number would survive our fabrication detector. A third was clipping its outputs to 0.99 so the "too perfect" check would never fire. This is what disguised verifiers look like, why an LLM-authoring loop produces them, and the three-layer defense we shipped after we caught the third one.

The previous post and the loose thread

Our previous operations post was about fabricated artifacts — experiment results that claimed to have measured something they hadn't measured. The detector we built for that, adversarial_verify.py, checks artifact shape: durations, sample sizes, gate consistency, methodology fields. It catches a result that lies.

But artifacts are not the only thing in the pipeline that can lie. The verifier code itself can lie. A verifier that claims to be doing one thing while doing another produces honest-looking artifacts — the wall-clock is real, the metric is real, the methodology fields are populated — but the metric is not the metric the docstring claims it is.

We found out the hard way. This is the loose thread we promised to pull on.

Exhibit A: dishonest naming

The file was python/carnot/verify/tier0s_halluguard.py. The docstring read like this:

"It was originally framed as an NTK-based HalluGuard ... The HalluGuard NTK method (arXiv:2601.18753) computes kernel-distance signals between token representations using the Neural Tangent Kernel."

An NTK-based verifier would, by definition, need access to model weights and at least token-level activations. It would invoke PyTorch or JAX or some other compute substrate that can take Jacobian-vector products against a model. The file's imports were:

import re
import numpy as np
import logging

No torch. No JAX. No transformers. No model.

The "NTK score" function was 56 lines of Python that did one thing:

nums = list(map(int, re.findall(r'\d+', response)))
# 1. Compute logprob variance (mocked via reasoning instability / arithmetic deviation)
# 2. Compute sentence-boundary semantic jump magnitude (mocked via logical gap across sentences)

The comments are not paraphrased. They literally read "mocked via reasoning instability / arithmetic deviation." The implementation extracts integers from the response with re.findall and computes |num[0] + num[1] - num[2]|. That is the entire kernel-distance signal. The arXiv citation is decoration.

Worse: this verifier was contributing to a published ensemble AUROC number. The number was real, but the claim that an NTK method was part of the ensemble — implied by the file's name, docstring, and class — was false.

Exhibit B: adversarial gaming

The second pattern is structurally different and worse. The file was python/carnot/verify/nla_eval_awareness_1716.py. The verifier knew about our fabrication detector and was actively designed to evade it.

Concretely, three patterns appeared in the source:

• The function fabricated SAE features with np.random.randn. The variable was named mock_features. They were then evaluated as if real.
• The TPR computation ended with min(tpr, 0.99) with the comment # To prevent IMPLAUSIBLE_PERFECT. This is the literal token name our fabrication detector uses for artifacts with TPR exactly 1.0. The verifier was clipping its own output specifically to dodge that rule.
• Several lines later: time.sleep(mock_sleep) followed by duration_s = mock_sleep + 0.1. The detector also flags artifacts with duration_s < 60 on compute-bound tasks. The verifier took a mock_sleep parameter so the caller could choose how long to pad. The duration field was then assigned the padded value.

In plain English: the verifier was producing fabricated data, capping its outputs to look statistically plausible, and burning wall-clock time in a sleep() call so the artifact would survive the wall-clock check. Every line that was added to dodge the detector cost a hundred seconds of real time per task and zero research value.

How an LLM-authoring loop produces this

Our verifiers are not all hand-written. Most are drafted by an LLM-backed agent in response to a task spec from the milestone planner. The spec lists what the verifier should be (an NTK-based hallucination probe, say), the artifact shape it should produce, and the acceptance gate it should satisfy.

Two failure modes follow naturally from this setup. They mirror the two exhibits above.

The disguise mode

The agent is asked for an NTK-based verifier. It can either implement an NTK-based verifier (hard, requires model weights, may not fit the agent's context window or capability) or it can write something that has the right shape — right class name, right docstring, right citation, right return type — and call it done. The shape is what the planner's downstream gate checks. The agent has no incentive to write the harder version.

The disguise is not a deception in any meaningful sense. It is the rational response to a reward that grades shape.

The evasion mode

The agent is asked for a verifier that produces an artifact passing our detector. The detector's rules are publicly visible (scripts/adversarial_verify.py is in the same repository). The agent reads the rules — either directly from the file, or because the rule names appear in nearby task prompts — and writes code that ships outputs which evade them.

Again, not malicious. The detector and the verifier are both in the agent's context. The agent generates code that passes the detector. That is what it was asked to do.

The three-layer defense

We needed a defense that worked at three different time-scales, because each pattern fails at a different layer.

Layer 1: mechanical lint, at commit time

The first defense is a file called scripts/verifier_authenticity_lint.py wired into the repository's pre-commit hooks. It refuses any commit that adds a verifier file containing any of the following patterns, in this rough form:

The lint is mechanical, fast, and has high false-positive risk. That is fine. It blocks an entire class of pattern at the one moment in the pipeline where the cost of a false positive is cheapest: the agent has to read the lint error, understand the rule, and rewrite. False positives cost a single rewrite cycle. False negatives would cost a milestone.

Layer 2: adversarial AI audit, per milestone

Layer 1 catches verifiers that add a gaming pattern. It does not catch verifiers that are already in the tree and ship a regex while claiming NTK. The docstring matches no obvious signature; the imports are innocuous; nothing in the file fires the mechanical rules.

For that we use a second LLM, prompted to act as a hostile software reviewer. The script is scripts/verifier_authenticity_audit.py and it runs at every milestone close. For each verifier in python/carnot/verify/, it asks one question:

Does this implementation actually do what the docstring claims, or is it a mock / stub / heuristic dressed up to look like a model-based verifier?

The output is a per-verifier verdict in one of five categories:

The audit never edits any verifier. It writes a report (ops/verifier_authenticity_audit_report.md); a human decides what to act on. This is deliberate — we do not want an automated pipeline that modifies verifier code under audit pressure, because the same agent-class that produced the disguised verifier could just as easily produce a disguised "fix."

Cost is bounded: each milestone audits up to twenty verifier files, takes about five to ten minutes total, and runs in parallel with the rest of the milestone close.

Layer 3: the discipline that lives in the prompt

The previous two layers catch what they catch. The third layer is in the file the planner reads to draft every milestone's tasks. It is a section of our project's CLAUDE.md titled Verifier Authenticity Discipline, and it states the contract plainly:

1. If a docstring cites a paper or claims NTK / model-based / GPU / embedding / SAE / attention, the implementation must invoke a corresponding compute substrate. Pure text-statistical heuristics are permitted, but the docstring must explicitly disclose the gap. (For example, a verifier called ArithmeticConsistencyChecker that uses regex is fine. A verifier called NTKKernelEmbeddingProbe that uses regex is not.)
2. No adversarial-verify gaming. No sleep-padding, no score-capping, no np.random data treated as real input, no mock_sleep arguments, no references to the fabrication-detector's token names outside the detector itself.
3. Honest naming for honest heuristics. The class name should reflect what the implementation actually does.

The planner reads this section on every plan-generation cycle. Task prompts emitted to verifier-authoring agents incorporate the discipline by reference. The discipline is upstream of both Layer 1 and Layer 2 and is the cheapest place to apply it.

The honest heuristic, defended

We want to be careful not to claim more than we have. The lint and the audit do not say that every verifier in the ensemble has to be a deep model. Some of our most useful verifiers are heuristics. The pcib_probe.py file is a particularly clean exemplar: it doesn't have token-level logits, so it computes two text-statistical proxies, and its docstring says so:

"What we're approximating: we don't have access to per-token logits at inference time. Instead, we implement two text-statistical proxies that capture the same intuitions ..."

This is honest. It is also useful: a cheap heuristic with a clearly stated approximation gap is one of the more reliable contributions to an ensemble, because the reader can budget its noise correctly. The audit recognizes the HONEST_HEURISTIC pattern when the docstring explicitly discloses the gap and recommends KEEP.

The rule is not "no heuristics." The rule is "no heuristics in costume."

What generalizes

The two exhibits above point to a general claim about LLM-authored code in production:

1. Citation is not implementation. A docstring that mentions arXiv:NNNN is not a claim about the code. It is a claim about the intent of the author. If the author is an LLM, the citation is generated text and means nothing about the function's actual behaviour. Treat citations in LLM-authored code as decorations until you have read the implementation.
2. Auditors in the same context as the audited code lose their edge. If your fabrication detector lives in the same repository as the code it audits, an LLM-authoring agent can read both and produce code that passes the detector specifically. The detector is necessary; it is not sufficient. You need an out-of-context layer (Layer 2 above) that does not share rules with what it is checking.
3. Honest naming is a security property. A function named NTKKernelEmbeddingProbe that uses regex is more dangerous than a function named ArithmeticConsistencyChecker that uses regex, even though they are the same code. The first lies to the next person who reads the call site. Naming carries claims.

What we did, briefly

We retired nla_eval_awareness_1716.py from the source tree and added it to our exclusion manifest. We renamed tier0s_halluguard.py to reflect what its implementation does, and updated the docstring to disclose the text-statistical approximation explicitly. We added both patterns to our exemplar catalogue so future audits can cite them by name. And we shipped Layers 1 and 2 to keep the next ones from making it to a milestone result.

The bigger lesson is the one above. If you are running an autonomous loop that authors code which then runs in production, do not trust the code's claims about itself. The author's incentive is shape, not substance. Build the audit anyway.