Assure
How do we know each output is good enough, at runtime, before it ships?
What we will measure
Definitions are published before results. No measurement for this programme has been published yet.
- Cost per correct answer
- Spend required for an answer that meets the quality contract
- Abstention-adjusted accuracy
- Accuracy after accounting for allowed abstention
- Calibration error
- Gap between estimated quality and observed quality
- Escalation rate
- Share of requests sent to a stronger model or reviewer
- Time to detect drift
- Delay between a quality shift and a raised alert
Connected projects
| Project | Type | Programme | HOPn role | Status |
|---|---|---|---|---|
| ServAlign Constraints and policy as machine-checkable objects. | Research project | Assure | Developer | Planned |
| Invoice automation Shared testbed for the Route, Ground and Assure programmes. | Research project | Route, Ground, Assure | Developer | In progress |
Planned work
Planned work for the next twelve months, counted from the start of the programme.
| months | Programme | Summary | Status |
|---|---|---|---|
| 0–3 | Assure | Quality orchestration charter and shared invoice benchmark | Planned |
| 3–6 | Assure | First quality-routing results | Planned |