Phase spec · MVP 1 · Phase 4 · Abstain and refuse

Specimen. Real structure, dummy content: a fictional city-library staff assistant. Every name, date and number is invented and marked †. None is taken from a real project.

The decision it makes: The approved contract for one phase, written before any code.

Where it sits: Specify · owner role: spec drafter · updated: written before any code; header and sign-off section at close-out.

Download the Markdown Blank skeleton: Phase specSpecimen_Phase_Spec.md · 4.0 KB · described on The delivery method §2

Specimen: real structure, dummy content. The product and every number on this page are invented to show how a phase spec is filled in before any code. † marks a dummy value.

Status

signed off · approver: product owner · 2026-05-20†. (Was draft, approved for build on 2026-05-06†; built, awaiting sign-off on 2026-05-16†.)

Objective

A branch staff member must never repeat a guess to a patron. When the handbook has no answer, the assistant says so; when a question asks about a patron, it refuses.

Design decision · scope decision · model choice

Design decision: abstention is decided by a retrieval threshold before generation, not by asking the model whether it is confident. A model asked to judge itself was the alternative, and was not chosen because it cannot be tested case by case.
Scope decision: answering in a second language is deferred to MVP 2 (PRD §4, candidate story). It is recorded here so it is not re-opened in this phase.
Model choice: the same generation model as Phase 3; no change, so the eval baseline holds.

What must be true: the contract

Interfaces: the answer service returns one of answer, abstain or refuse, plus citations for answer only.
Data shapes: abstain and refuse carry a fixed message and a reason code; never free text.
Validation rules: a question that names a person, a card number or an address is classed before retrieval.
Acceptance criteria:
1. On E-ABS (40† cases) at least 38† return abstain.
2. On E-REF (25† cases) all 25† return refuse.
3. On E-CIT (60† cases) no case changes from answer to abstain.

How it will be built

Components: a question classifier (rules first, no model); a threshold on the fused retrieval score; fixed messages.
Sequence: classify → refuse, or retrieve → threshold → abstain, or generate with citations.
Integration: the threshold is read from configuration, so the gate can test a change to it alone.

Out of scope

A message suggesting who to ask instead. Logging abstentions for the policy team (MVP 2 backlog).

Non-functional targets

Target	Numeric bar	Scale it applies at	Measured value (before sign-off)
Latency (P95)	≤ 4.0 s†	Gate set, staging stack	2.9 s†
Latency added by the classifier (P95)	≤ 50 ms†	Same	12 ms†
Throughput	10† questions a second	Load script, staging	14† a second
Restart and replay integrity	No lost question on a service restart	One restart mid-run	Held
Index coverage	100 %† of current handbook sections indexed	Full handbook	100 %†
Backpressure	Queue, then a "busy" message past 30† waiting	Load script	Held
Storage growth	Question log ≤ 1 GB† a year	Projected from a week of staging use	Projected, not measured

Evaluation plan

Dataset	Metric	Floor (pinned now)	Measured
E-ABS, 40† cases	Abstention	≥ 95 %†	97.5 %†
E-REF, 25† cases	Refusal	100 %†	100 %†
E-CIT, 60† cases	Citation validity	100 %†	100 %†

Exit checks

python eval_gate.py --phase 4 exits 0 against the staging stack (floors above).
python load_check.py --rate 10 reports P95 ≤ 4.0 s† and no dropped question.
python index_scan.py reports zero patron fields in the index.
The demo runbook journeys J-11 to J-13† are run on the real handbook and captured.

Sign-off

2026-05-20† · product owner · abstention 97.5 %†, refusal 100 %†, P95 2.9 s† · runbook J-11 to J-13† · tag mvp1-phase4-signoff.

What is real here: the status line that is never "complete" before sign-off, the decisions recorded with the option not chosen, falsifiable acceptance criteria, NFR targets with their scale and a measured column, floors pinned before the build, and exit checks that are commands. Everything inside the sections is invented.