Project 04 / Healthcare / Unwarranted variation
Prescribing Variation and Medicines Optimisation
Separating warranted from unwarranted variation across 6,300 English GP practices. 8.8 million prescribing rows, six BNF sections, three years, filtered at source through the NHSBSA Open Data API.
The finding in one paragraph
A medicines optimisation team ranks practices on prescribing rate and writes to the top of the list. That list correlates +0.73 with the share of patients aged over 65: it is substantially a ranking of how old the registered population is. Roughly half the practices on it are prescribing entirely appropriately for the patients they have, and each of those letters costs a working relationship.
Indirect age-sex standardisation removes the association with age completely (r falls to +0.00). Funnel plots then separate the practices whose remaining variation is larger than chance and ordinary between-practice difference would produce. At the same workload the funnel list is 66% precise against the league table's 46% — 27 more real problems found and 14 fewer wasted conversations.
The overdispersion adjustment, and why the textbook reason is not the reason here
Every account of funnel plots says the problem with a league table is that small units sit at both extremes, because small denominators carry more sampling variation.
On this data that is barely true, and it is worth knowing why. Practices issue between 29,000 and 190,000 items over 36 months. The Poisson coefficient of variation on 29,000 is 0.6%. Genuine differences in prescribing behaviour are worth tens of percent. Sampling noise is not in the same league, so it does not push small practices to the extremes — measured on the high-volume rate, the flagged practices are no smaller than average. It is true on low-volume measures: liothyronine, where a practice may issue a handful of items in three years.
So the adjustment here is needed for the opposite reason to the textbook one. Pure Poisson limits are about half a percent wide at these counts, so they flag 97% of England. Estimating the genuine between-practice variance (DerSimonian-Laird, tau-squared = 0.022) and adding it to the sampling variance brings that to 477 at 95% and 108 at 99.8%.
An earlier attempt used the variance-stabilised form with Winsorising at the 10th and 90th percentiles and returned a tau-squared six times too small, flagging 45% of practices. The self-check that caught it — comparing the estimate against the observed variance of the ratios minus the average sampling variance — is printed in the analysis output.
95 practices were given a genuine prescribing-style effect by the data generator. Neither method was told which. This check is only possible because the data is synthetic, and it is the most useful thing in the project.
| method | practices_flagged | true_outliers_found | precision_pct | recall_pct |
|---|---|---|---|---|
| League table, top and bottom 95 | 95 | 44 | 46.3 | 46.3 |
| League table, top and bottom 200 | 200 | 50 | 25.0 | 52.6 |
| Funnel plot, outside 95% limits | 477 | 92 | 19.3 | 96.8 |
| Funnel plot, outside 99.8% limits | 108 | 71 | 65.7 | 74.7 |
Reading the table honestly
The 95% funnel list has the highest recall (97%) and the lowest precision (20%), because it flags 471 practices. The 99.8% list flags 111 — a workload comparable to the league table's 95 — and is 65% precise with 76% recall.
Which you choose depends on what the list is for. A 99.8% list is the right basis for a letter, because the cost of being wrong is a damaged relationship. A 95% list is the right basis for an internal watchlist, where the cost of being wrong is only your own time.
That choice is a judgement about consequences, not a statistical one, and it should be made explicitly rather than by whichever threshold the software defaults to.
Two things stated rather than buried
Nothing here assumes 100% switching. The generic figure applies an 80% achievable rate and the low-priority figure 70%, because some patients genuinely cannot switch — narrow therapeutic index drugs, formulation intolerance, specialist-initiated therapy the practice cannot unilaterally change. NHS England's own guidance says explicitly that low-priority items may still be right for individual patients. A pharmacist reading a 100% figure knows immediately that nobody asked one.
The low-priority figure is a demonstration, not a business case. Low-priority items are 13% of spend in this extract against 1-2% nationally. The individual products are costed realistically; the six sections around them are under-weighted in the generator, which inflates the share. The funnel-plot and generic-switching findings do not depend on it. This is in the reconciliation note too, because a figure that flatters the analysis is exactly the one to flag.
The deliverable that actually changes anything
A dashboard does not change prescribing. A conversation does, and the conversation needs one page that a pharmacist can put on a desk. Below is the template, generated for the most extreme practice in the dataset.
The three design decisions in it are all about the same thing — making the conversation possible rather than defensive:
- it states what has been adjusted for, before any number
- it has a section headed what we are not saying, because the first thing a GP will assume is that they are being accused of prescribing badly
- it offers a route OFF the list, permanently, if there is an explanation the data cannot see. A list that practices cannot get off is a list they stop answering.
outputs/practice_report_example.md
| # Prescribing review: Practice 03836 (A03836) |
| **North East and Yorkshire · ICB03 · list size 8,431 · IMD decile 8 · 17.8% aged 65+** |
| ## Why this practice is on the list |
| After adjusting for the age and sex of your registered population, this practice issued **297 items for every 100 expected** across the six BNF sections in scope. The median practice in England sits at 97. |
| That is **outside the 99.8% control limit** on a funnel plot of all 6,300 English practices — meaning the gap is larger than random variation and ordinary between-practice differences together would produce at your list size. |
| **This is not a crude ranking.** Age and sex are already accounted for. Practices with older populations are expected to prescribe more, and that expectation is built into the figure above. |
| ## The numbers |
| | Measure | This practice | England median | | |
| |---|---:|---:|---| |
| | Standardised prescribing ratio | **297** | 97 | +208% | |
| | Items per 1,000 patients per month | 990.6 | 317.9 | +212% | |
| | Low-priority spend, 36 months | GBP 30,659 | | 7.4% of your total | |
| | Broad-spectrum antibacterials | 15.1% | 8.9% | target below 10% | |
| ## What we are NOT saying |
| - We are not saying this prescribing is clinically wrong. We cannot see your patients. |
| - We are not saying it is unjustified. Age and sex are adjusted for; a specialist clinic, a nursing home on your list, or a local service you provide on behalf of others are not, and any of those would explain the gap entirely. |
| - We are not asking you to reduce prescribing to a target. |
| ## What we are asking |
| One conversation, to establish whether there is a reason we cannot see in the data. If there is, we record it and this practice comes off the list — permanently, not until the next report. |
| --- |
| *Method: indirect age-sex standardisation against national rates, with funnel plot control limits including an overdispersion adjustment. Measure definitions follow OpenPrescribing and the NHS England low-priority list. Full method and code in the repository.* |