By Ritvars Mētra, Founder of ReadinessCompassThe readiness heat map has a cell showing 42 — the lowest score on the page, in Legal, in Spain. It gets fifteen minutes of the steering committee’s attention and an action to investigate.
Legal in Spain is four people, three of whom responded. The 42 is the average of three answers, one of which came from someone having a bad week.
Two separate problems, often conflated
Reliability. A mean from three responses carries an enormous margin of error. It is not a weak signal; it is close to no signal, and it will move dramatically between waves for reasons that have nothing to do with readiness. A programme that reacts to it is chasing noise, and one that watches it over time will see a trend that does not exist.
Identifiability. With three respondents in a named function and country, the results are attributable. Colleagues can work out who said what, and so can a manager. This is the problem that changes behaviour on the next survey, because people are perfectly capable of anticipating it — which is why reporting rules have to be published before the survey opens.
The two problems have the same remedy, which is fortunate, but they justify it for different reasons. Suppression protects the respondent; it also protects the programme from acting on a number that was never real.
There is an established standard for this
Change programmes tend to invent their own rules here. They do not need to: national statistical offices have solved this problem carefully, and the guidance is public.
The UK Government Statistical Service’s disclosure control guidance for tables produced from surveys sets out three ideas worth importing wholesale.
- Minimum cell size. Tables using confidential variables should avoid small cell counts, and where that is not possible the cells should be suppressed. Pick a floor and apply it mechanically.
- Complementary suppression. Suppressing the sensitive cell is not enough. If the row and column totals are published, the suppressed value can be recovered by arithmetic — so additional cells must also be suppressed to prevent recovery.
- Disclosure by differencing. Suppression does not protect against comparing two overlapping tables. Publish a country breakdown one month and a country-by-function breakdown the next, and the difference reveals what was hidden. Fixed categories, held constant, are the defence.
The second point is the one nearly every readiness report gets wrong. Suppressing Legal while publishing the Spain total and every other Spanish function means Legal’s score can be calculated in about thirty seconds by anyone motivated to do it.
A workable set of rules
| Responses in the cell | What to publish |
|---|---|
| Fewer than 5 | Suppress. Show “n/a — below reporting threshold”, not a blank |
| 5 to 9 | Publish with an explicit low-confidence marker; never report movement between waves |
| 10 to 29 | Publish. Treat only large movements as signal |
| 30 or more | Publish and trend normally |
Two details matter as much as the thresholds themselves.
Show suppressed cells rather than dropping the row. Marking a cell “below threshold” preserves the fact that the group exists and was surveyed. Deleting the row makes a population invisible, and invisible populations are the ones nobody prepares. A heat map should show the gap honestly.
Aggregate up rather than abandoning the question. If Legal in Spain is too small, report Legal across all countries, or all functions in Spain. You lose specificity and keep the signal, which is the right trade.
Set the threshold before you see the data
A suppression rule chosen after the results arrive is not a rule. It is an editorial decision, and it will be made — consciously or not — with an eye on which cells it hides.
Write the threshold into the survey communication, apply it mechanically, and resist the request to make an exception for the one small team everybody is worried about. If that team genuinely matters, the answer is not to publish an unreliable number about them — it is to go and find out properly.
When a group is too small to report and too important to ignore
This is common and it has a good answer. Small groups are frequently the highest-risk ones — a four-person team running a critical process is exactly where a capability gap hurts most.
Do not solve it with statistics. Solve it by observation: watch those four people attempt the task and record who completed it unaided. With a group that size, a structured observation is both more reliable than a survey and less exposing, because it produces a finding about the process rather than a score attached to individuals.
That is the general principle behind all of this. Suppression is not an obstacle to understanding a small team. It is a prompt to use a method that suits the size of the group — and for four people, half a day of watching them work beats any number a survey could have produced.
More on readiness measurement and reporting in the Change Readiness Hub, or see how to turn survey data into a signal leaders can act on.