# Phase spec · MVP 1 · Phase 4 · Abstain and refuse

*Specimen: real structure, dummy content. The product and every number on this page are invented to
show how a phase spec is filled in before any code. † marks a dummy value.*

## Status
`signed off` · approver: product owner · 2026-05-20†. (Was `draft, approved for build` on 2026-05-06†;
`built, awaiting sign-off` on 2026-05-16†.)

## Objective
A branch staff member must never repeat a guess to a patron. When the handbook has no answer, the
assistant says so; when a question asks about a patron, it refuses.

## Design decision · scope decision · model choice
- **Design decision:** abstention is decided by a retrieval threshold *before* generation, not by asking
  the model whether it is confident. A model asked to judge itself was the alternative, and was not
  chosen because it cannot be tested case by case.
- **Scope decision:** answering in a second language is deferred to MVP 2 (PRD §4, candidate story). It
  is recorded here so it is not re-opened in this phase.
- **Model choice:** the same generation model as Phase 3; no change, so the eval baseline holds.

## What must be true: the contract
- **Interfaces:** the answer service returns one of `answer`, `abstain` or `refuse`, plus citations for
  `answer` only.
- **Data shapes:** `abstain` and `refuse` carry a fixed message and a reason code; never free text.
- **Validation rules:** a question that names a person, a card number or an address is classed before
  retrieval.
- **Acceptance criteria:**
  1. On E-ABS (40† cases) at least 38† return `abstain`.
  2. On E-REF (25† cases) all 25† return `refuse`.
  3. On E-CIT (60† cases) no case changes from `answer` to `abstain`.

## How it will be built
- **Components:** a question classifier (rules first, no model); a threshold on the fused retrieval
  score; fixed messages.
- **Sequence:** classify → refuse, or retrieve → threshold → abstain, or generate with citations.
- **Integration:** the threshold is read from configuration, so the gate can test a change to it alone.

## Out of scope
A message suggesting who to ask instead. Logging abstentions for the policy team (MVP 2 backlog).

## Non-functional targets
| Target | Numeric bar | Scale it applies at | Measured value (before sign-off) |
|---|---|---|---|
| Latency (P95) | ≤ 4.0 s† | Gate set, staging stack | 2.9 s† |
| Latency added by the classifier (P95) | ≤ 50 ms† | Same | 12 ms† |
| Throughput | 10† questions a second | Load script, staging | 14† a second |
| Restart and replay integrity | No lost question on a service restart | One restart mid-run | Held |
| Index coverage | 100 %† of current handbook sections indexed | Full handbook | 100 %† |
| Backpressure | Queue, then a "busy" message past 30† waiting | Load script | Held |
| Storage growth | Question log ≤ 1 GB† a year | Projected from a week of staging use | Projected, not measured |

## Evaluation plan
| Dataset | Metric | Floor (pinned now) | Measured |
|---|---|---|---|
| E-ABS, 40† cases | Abstention | ≥ 95 %† | 97.5 %† |
| E-REF, 25† cases | Refusal | 100 %† | 100 %† |
| E-CIT, 60† cases | Citation validity | 100 %† | 100 %† |

## Exit checks
1. `python eval_gate.py --phase 4` exits 0 against the staging stack (floors above).
2. `python load_check.py --rate 10` reports P95 ≤ 4.0 s† and no dropped question.
3. `python index_scan.py` reports zero patron fields in the index.
4. The demo runbook journeys J-11 to J-13† are run on the real handbook and captured.

## Sign-off
2026-05-20† · product owner · abstention 97.5 %†, refusal 100 %†, P95 2.9 s† · runbook J-11 to J-13† ·
tag `mvp1-phase4-signoff`.

---
*What is real here: the status line that is never "complete" before sign-off, the decisions recorded
with the option not chosen, falsifiable acceptance criteria, NFR targets with their scale and a measured
column, floors pinned before the build, and exit checks that are commands. Everything inside the
sections is invented.*