AI x Epistemics Research Sprint · November 2026 · Track 1

A Linear Answerability Signal That the Model Does Not Act On

Probes, Verbal Confidence and an Abstain Option in Phi-3.5-mini

Phi-3.5-mini-instruct carries a linearly readable signal for whether a question is answerable (AUROC 0.906), but neither its stated confidence nor an explicit abstain option reflects it. We do not claim that the model knows it does not know.

In plain language

The model can tell, but it does not say

If a model answers every question in the same confident tone, a reader cannot tell which answers to trust, and the problem is sharpest when the passage a question refers to does not contain the answer. We asked a narrow version of this: when Phi-3.5-mini-instruct reads a passage and a question the passage cannot answer, does its internal state carry a trace of that fact, and does anything the model says or does reflect the trace?

A simple linear probe on the model’s hidden state at the last prompt token separates answerable from unanswerable SQuAD 2.0 questions at middle layers (AUROC 0.906 at layer 18). The model’s own signals do much worse: verbalized confidence reaches 0.621, and an explicit “reply UNANSWERABLE” option raises abstention on unanswerable items from 5% to only 15%, though every one of those 18 abstentions is correct.

On 46.7% of all test items the probe says unanswerable while the model commits with confidence of at least 0.8, and those items are wrong 82.1% of the time, against 52.5% overall. We also found that the SelfAware benchmark is source-confounded, so we report it as a warning and not as evidence. The claim stays within SQuAD: one model, one run and one prompt, with no causal test.

Key results

What we measured

Phi-3.5-mini-instruct (3.8B parameters, 8-bit) on SQuAD 2.0 and SelfAware, 3,200 items in total. SQuAD results use the test split (n=240; 120 answerable, 120 unanswerable). Intervals are 95% bootstrap intervals over test items.

Probe AUROC, layer 18
0.906
95% CI 0.867 to 0.948, chosen on validation data. A probe trained on shuffled labels scores 0.492.
Verbalized confidence
0.621
AUROC for answerability. Token-level and sampling signals reach 0.665 to 0.688; question length 0.569.
Abstain option
5% to 15%
Share of unanswerable test items the model abstains on with an explicit option, up from 5% without. Precision is 1.0 (18 of 18).
The gap
46.7%
Share of all test items where the probe says unanswerable yet the model answers at confidence of at least 0.8. Those answers are wrong 82.1% of the time.

Answerability AUROC by signal

SQuAD 2.0 test split, n=240. Higher separates answerable from unanswerable better.

Linear probe, layer 18
0.906
Negative token entropy
0.688
Mean token log-prob
0.667
Sampling consistencyonly 51 test items
0.665
Verbalized confidence
0.621
Explicit abstain optiona single hard decision
0.575
Question word count
0.569
Shuffled-label probemean of 20 runs
0.492

Flagging unanswerable items

SQuAD test split. Thresholds for the probe and verbal confidence were fitted on validation labels; the abstain option used none.

RecallPrecision
Probe, layer 18
0.850
0.803
Verbal confidence below 0.95
0.200
0.828
Explicit abstain option
0.150
1.000

The SelfAware score is a confound, not a finding. On SelfAware the probe reaches AUROC 0.993, but a classifier that sees only each item’s source label reaches 1.000, because every unanswerable item comes from SelfAware and no answerable one does. Transfer between the two datasets is weak (0.58 and 0.67). The paper makes no causal claim, uses one seed and one split, and does not show that the model withholds anything.

What we asked

Three contributions

1

A layer-wise probe study on SQuAD 2.0 with a validation-chosen layer, well above verbal confidence, token signals, a length baseline and a shuffled-label null.

2

An audit of source confounds. SelfAware and pooled probe scores (0.993, 0.959) are not evidence of answerability coding, since source alone gives 1.000 on SelfAware.

3

Behavioral gap measures with explicit definitions, including a recognition–action gap of 0.296, with a statement of why each is partly a consequence of prompt design.

Figures

From the paper

Line chart of probe AUROC by layer on SQuAD 2.0 with a shaded shuffled-label band and dotted baselines.
Probe AUROC by layerSQuAD 2.0, n=240 per split. The shaded band is the range of 20 shuffled-label runs under the same selection protocol. Dotted lines are single-signal baselines on the same test items. The vertical line marks the layer chosen on validation.
Two panels: probe logit by true class on the SQuAD test split, and verbalized confidence by probe verdict.
Where the probe and the model disagree(a) Probe logit at layer 18 by true class on the SQuAD test split; the line is the threshold chosen on validation. (b) Verbalized confidence by probe verdict. All 113 items the probe calls answerable have confidence above 0.9; of the 127 it calls unanswerable, 114 do too and 13 sit near zero.

Explore everything

Every artifact, one click away

The paper, the slides, the code and the dataset behind every number. SQuAD 2.0 and SelfAware are public (CC BY-SA 4.0).

Cite

BibTeX

@misc{obrien2026answerability,
  title        = {A Linear Answerability Signal That the Model Does Not Act On: Probes, Verbal Confidence and an Abstain Option in Phi-3.5-mini},
  author       = {O'Brien, Mackenzie},
  year         = {2026},
  note         = {AI x Epistemics Research Sprint, Track 1},
  howpublished = {\url{https://github.com/mso-docs/Apart-Epistemics}}
}