RCA-01 · Superseded handbook sections served after a re-index
Specimen. Real structure, dummy content: a fictional city-library staff assistant. Every name, date and number is invented and marked †. None is taken from a real project.
The decision it makes: Why a process or state went wrong, and what prevents it.
Where it sits: Learn · owner role: session · updated: when something goes materially wrong.
Download the MarkdownSpecimen_RCA.md · 3.2 KB · described on The operating model §4
Specimen: real structure, dummy content. The product, the incident and every number on this page are invented to show how an RCA and its retrospective entry are written. † marks a dummy value.
Header block: product · Library staff assistant · RCA-01 · 2026-05-30† · written by: the session · reviewed by: product owner · related: sign-off log item IS-02, risk register R-01
1 · What happened
On 2026-05-28† the policy team published handbook version 2026-06†, which moved the fines policy from §6 to §7. The re-index ran and reported success. The next day's end-to-end test found that 3† questions about fines were answered from the old §6, with a citation that resolved, because the old section was still in the index. No patron was given a wrong answer: the assistant was on staging only.
2 · Root causes
- The re-index added new sections and never removed sections that no longer existed.
- "Re-index succeeded" meant "every new section was written", not "the index matches the handbook".
- The citation check confirmed that a cited section exists in the index, not that it exists in the current version of the handbook.
3 · Why it was not caught earlier
- Every handbook update before this one only added sections; none moved or removed one.
- The eval set E-CIT was written against one handbook version and never re-run after a re-index.
4 · Preventions
| Prevention | Where it lives | Owner (role) | Done |
|---|---|---|---|
| Re-index replaces the whole index for a version; never appends | Re-index script | Tech lead | 2026-05-31† |
| A check compares the section list in the index with the handbook's contents page; a difference fails the run | Exit check of any re-index | Tech lead | 2026-05-31† |
| Citation validity is checked against the current version, not the index | Gate script | Tech lead | 2026-06-02† |
| E-CIT is re-run after every re-index | Phase spec template | Product manager | 2026-06-02† |
5 · What was not found
- No sign that any other policy area was affected: the section comparison found only the three† fines sections.
- No cause in the model or the retrieval ranking; both behaved as specified on the index they were given.
Retrospective · entry for RCA-01
| Field | Entry |
|---|---|
| Scenario | A handbook update moved a section; the re-index kept the old one |
| Risk | Staff repeat a superseded policy with a citation that looks valid (R-01) |
| Root cause | The re-index appended and never removed; "success" did not mean "matches the handbook" |
| Fix | Replace the index per version; compare sections with the contents page |
| Prevention | Citation check against the current version; E-CIT re-run after every re-index |
| Source | RCA-01 (this page). The retrospective summarises and points; it does not repeat the RCA |
What is real here: what happened stated before any cause; numbered root causes; a separate answer to why it was not caught; preventions with a place, an owner and a date; what was looked for and not found; and a retrospective entry that points to the RCA instead of copying it. Everything inside the sections is invented.