An AI that is not allowed to decide

The problem

The vendor feed carries no prescription flag, and no category can supply one. "Pain & Fever" contains both paracetamol and controlled analgesics. The prescription answer is orthogonal to the browse taxonomy by design.

The tempting move is to let the model answer it. We refused, and the reasoning generalises well past pharmacy.

The asymmetry

These are not comparable risks, so they must not share a decision threshold.

How that became architecture

The Rx classifier returns three states, not two:

• true — believes a prescription is required
• false — believes one is not
• null — declines to guess

null exists specifically so an unsure classifier cannot answer false. Abstaining leaves the row in the rx: unreviewed filter where a human will see it. Answering false would quietly park it as OTC, which is the failure mode we cannot accept.

The model never writes the truth

Downstream, the model opinion and the operational truth are separate columns. rxSuggested holds the AI view; requiresPrescription is only ever written by a human action. The bulk-apply is on-only — it sets the flag where the suggestion is true and never clears it.

The target is recall, not symmetric accuracy. A classifier that is 99% accurate but misses 1% of prescription drugs is worse than useless in a regulated context.

The shortcut we rejected

We investigated automating an authoritative registry lookup against MOHAP and EDE data, and parsed the sources rather than assuming.

• The EDE controlled and semi-controlled list parses cleanly into 268 generic-name entries with control levels
• The 2025 National Essential Medicines List parses into 83 therapeutic sections

But neither carries a per-product dispensing mode across the whole catalogue, and the complete registered-products list is a paid request service, not an API. Automating it gives certainty on a subset, not a complete answer — so we documented that instead of shipping a partial solution dressed as a full one.