By Ritvars Mētra, Founder of ReadinessCompassUAT finishes on a Friday. The defect log is down to a handful of low-severity items, the business signs the acceptance document, and the programme moves to cutover planning with a sense of earned relief.
Eight weeks later, the same organisation is dealing with data errors, escalating support tickets, and finance users quietly rebuilding a reconciliation in Excel.
Nothing went wrong between those two moments. The problem is that UAT answered a question nobody had a problem with — does the system do what we specified? — while leaving a different question almost entirely unmeasured: can these people do this work, in this system, under real conditions, without help?
Almost everything written about UAT is written from a testing perspective: scripts, coverage, defect severity, entry and exit criteria. That work matters. But UAT is also the single richest source of people-readiness evidence a programme will ever get, and most organisations throw that evidence away because nobody was assigned to collect it.
This article is about how to read UAT as an adoption instrument.
What UAT is formally for — and the gap that leaves
The ISTQB glossary defines user acceptance testing as “acceptance testing carried out by future users in a (simulated) operational environment focusing on user requirements and needs.”
Read that definition slowly, because it is doing more work than most programmes ask of it. Future users. Simulated operational environment. ISTQB is also explicit that the goal of acceptance testing is to establish confidence in the system rather than to find defects.
So the standard already frames UAT as a confidence exercise. The gap is that programmes interpret “confidence” as an institutional verdict — a signature on a document — rather than as something measurable in the people who will actually operate the process on Monday morning.
That distinction is not academic. Kwahk and Lee, in Information & Management, found that readiness for change shaped intention to use an ERP system indirectly, through perceived usefulness and perceived ease of use, and that readiness itself was driven by organisational commitment and perceived personal competence. Perceived personal competence is precisely the variable UAT is in a position to observe — and precisely the one a signed acceptance form tells you nothing about.
UAT is the only point before go-live when real business users do real work in the real system. It is free instrumentation. Most programmes leave it uninstrumented.
Nine adoption signals hiding inside UAT
None of these require extra tooling. They require someone whose job during UAT is to watch people rather than defects.
1. Who actually participates
The most common substitution in ERP programmes: UAT is staffed by super-users, project team members and the same enthusiastic volunteers who attended every design workshop.
These people are unrepresentative by construction. They have seen the system for months, they know who to ask, and they have absorbed the design rationale. If they pass UAT comfortably, you have learned that the system works for the people who least needed convincing.
What to record: for every participant, their actual role, their tenure, and their prior exposure to the programme. Then calculate what proportion of UAT effort came from people who will be ordinary users on day one. If that number is below half, your UAT result is a measure of your project team, not your organisation. It is also an early warning of expert overload — the same handful of people carrying the change on their backs.
2. Whether the right roles are represented
Participation counts are not coverage. What matters is whether every materially affected role is present, weighted by how much their work changes.
The roles most often missing are the ones nobody thinks of as system users: upstream data providers, occasional approvers, month-end-only participants, night-shift operators, and third parties. Their absence is invisible during UAT and extremely visible afterwards, because they are usually where data quality breaks.
What to do: take the role list from your change impact and mitigation register and check it against the UAT participant list. Every high-impact role with no UAT representation is an untested adoption risk, and should be recorded as one. Where the gaps are about influence and engagement rather than headcount, stakeholder analysis will tell you who is being left out and why.
3. Whether users can handle exceptions
Most UAT scripts walk the happy path. Real operational life is largely exceptions: the blocked invoice, the partial delivery, the credit note against a closed period, the customer who is also a supplier.
Adoption almost never fails on the standard transaction. It fails at the first exception, because that is the moment a user has to reason about the system rather than follow a script — and the moment they discover whether they understand the process or merely memorised the clicks.
What to do: deliberately include unscripted exceptions, and score them separately from the main pass rate. A 95% pass rate on scripted happy paths and a 40% pass rate on exceptions is not a 90% result. It is a warning.
4. Whether the scenarios are realistic
UAT usually runs on clean data, in a quiet room, with no competing deadline, no phone ringing, and a tester who has been told to take their time.
Month-end is none of those things. A process that works under laboratory conditions and collapses under load has not been tested for adoption; it has been tested for existence.
What to do: run at least one session under realistic pressure — production-like data volumes, messy records, concurrent activity, a time-boxed deadline. If realism is impractical in your environment, that is itself a finding, and one a change complexity assessment would generally have anticipated: complex changes need rehearsal conditions closer to reality, not further from it.
5. Whether users are leaning on experts to finish
This is the highest-yield signal in the whole exercise, and it is almost never recorded.
Watch how tasks are completed rather than whether they were completed. A consultant leaning over a shoulder. A super-user taking the keyboard “just to get past this bit.” A test marked pass because someone at the next desk knew the workaround.
Every one of those is a pass in the test log and a fail in adoption terms, because on go-live day that helper is supporting two hundred people instead of one.
What to record: unaided completion rate — the proportion of scenarios completed with no intervention. Track it as a first-class metric. It is a far better predictor of go-live day than the headline pass rate, and it exposes the difference between exposure and capability that makes training completion such a weak proxy for adoption.
6. Which questions keep recurring
During UAT, users ask questions constantly. Those questions are usually answered helpfully and then evaporate.
They should be logged, because a question asked by eleven different people is not eleven support interactions. It is one design, training or documentation defect, and it will reappear on go-live day multiplied by your entire user population.
What to do: keep a running question log and cluster it weekly. Recurring questions tend to fall into three buckets: the process is genuinely ambiguous, the screen is misleading, or the training covered the wrong thing. Those need different fixes, so trace the top few to their cause with a 5 Why root cause analysis rather than answering the surface question again.
7. Which workarounds emerge
If a workaround appears during UAT, it will be standard practice within a month of go-live. It has already been invented, socialised and proven to work.
Steven Alter’s Theory of Workarounds is useful here: workarounds are goal-driven adaptations people create when they hit an obstacle in the system as they experience it. A workaround invented during UAT is therefore a precise, free description of an obstacle — delivered weeks before it can hurt you, by someone motivated enough to solve it themselves.
What to do: log every workaround with the obstacle that produced it, and treat it as a design finding rather than a testing anecdote. The nine underlying causes are the same ones that later drive users back to Excel after go-live — process misfit, data distrust, missing functionality, approval friction and the rest. Catching them in UAT is dramatically cheaper. Anything that cannot be resolved before cutover belongs in the transformation RAID log with an owner and a date.
8. Whether managers are seeing the problems
In most programmes, UAT findings travel from tester to test lead to programme report. The line manager of the affected team often never sees the raw picture.
That matters because the manager is the person who will decide, after go-live, whether the new way of working is enforced or quietly tolerated. A manager who has watched their team struggle with an exception in UAT arrives at go-live with a mental model of where the risks are. A manager who only received a green status report arrives expecting everything to be fine, and interprets the first difficulty as a failure of their people.
What to do: require line managers to observe at least one UAT session for their own team. Not to participate, not to help — to watch. It is one of the cheapest interventions available and it directly strengthens the reinforcement that makes middle management the deciding factor at go-live.
9. Whether UAT confidence matches readiness-survey confidence
This is the comparison that turns UAT from a test event into a readiness instrument, and almost nobody runs it.
You have two independent measurements available at the same moment: what people say about their readiness in a survey, and what they demonstrably did in UAT. Comparing them is more informative than either alone, because self-assessment is not a reliable proxy for capability. Kruger and Dunning’s work on inflated self-assessment established that the least skilled tend to overestimate their ability most, partly because judging competence in a domain requires the same skills as having it. The practical implication for ERP is uncomfortable: the group most likely to report high confidence may be the group least equipped to notice what they cannot yet do.
Plotting the two together gives four groups, and they need completely different responses.
| High observed capability in UAT | Low observed capability in UAT | |
|---|---|---|
| High self-reported confidence | Ready. Use them as local support at go-live. | Highest risk. Unaware of the gap, so they will not ask for help. Address first. |
| Low self-reported confidence | Capable but anxious. Needs reassurance and reps, not more content. | Honest gap. Easiest group to help, because they will tell you what they need. |
The top-right cell is the one that damages go-lives. Those users sail through the readiness survey, are not flagged by any dashboard, and discover the gap in production — usually during month-end, usually in front of their manager. If your readiness assessment is survey-only, it cannot distinguish that group from the genuinely ready one. UAT can.
The signals at a glance
| Signal | What to measure | What it predicts |
|---|---|---|
| Participation | Share of UAT effort from ordinary day-one users | Whether the result generalises beyond the project team |
| Role coverage | High-impact roles with no UAT representation | Where adoption is completely untested |
| Exception handling | Pass rate on exceptions, scored separately | Behaviour at the first non-standard case |
| Scenario realism | Sessions run under production-like load | Whether competence survives pressure |
| Expert reliance | Unaided completion rate | Day-one support demand and expert overload |
| Recurring questions | Clustered question log | Support ticket volume and its causes |
| Workarounds | Workarounds logged with their obstacle | Post-go-live spreadsheet reversion |
| Manager exposure | Managers who observed a session | Whether reinforcement will happen |
| Confidence gap | Survey confidence vs observed capability | Which groups will fail silently |
Instrumenting UAT without breaking it
A fair objection: UAT is already stressful, time-boxed and politically loaded. Bolting a change-management exercise onto it will not be welcomed.
So do it lightly.
- Observe, do not interrogate. One person in the room whose job is to watch and note. No extra forms for testers.
- Add three fields to the existing test record: completed unaided (yes/no), help needed (what kind), confidence after the task (1–5). That is perhaps fifteen seconds per scenario and it produces most of the value described above.
- Keep a second scoresheet. The test log answers “does the system work?” The readiness log answers “can these people run it?” Report both to the steering committee.
- Separate the verdicts. Do not let a people-readiness finding be closed as a non-defect. It is not a defect. It is a different class of risk, and it needs its own owner.
Where several signals fire at once and it is not obvious which cause dominates, map them before choosing actions. A fishbone analysis across people, process, data, training and reinforcement is faster than arguing about it in a go/no-go meeting.
Using the evidence at go/no-go
Go/no-go decisions are usually made on technical criteria plus a general sense of organisational mood. UAT readiness evidence lets you replace the mood with something defensible.
It rarely changes the date — and it usually should not. What it changes is what you do with the weeks you have left, and what you plan for hypercare. Knowing that unaided completion is 55% in one function and 90% in another lets you put support where it is needed instead of spreading it evenly. Knowing which roles never appeared in UAT tells you where to expect the first surprises.
This is what genuine adoption evidence before go-live looks like: observed behaviour rather than milestone completion. And it is the same evidence chain that determines whether benefits leak after go-live, because the gaps visible in UAT are the gaps that quietly consume the business case afterwards.
The question UAT should be answering
A signed acceptance document records that the system met its specification. That is worth having, and it is not the same as knowing the organisation can operate it.
The useful reframe is small. Alongside “did the test pass?”, ask: who passed it, unaided, under realistic conditions, including the awkward cases — and does that match what they told us about their own readiness?
Nothing in that question requires new tooling, a new workstream, or a delayed date. It requires deciding, before UAT starts, that you are going to measure people as well as software.
Every programme runs UAT. Very few read it. The evidence is already being generated in the room — the only question is whether anyone is writing it down.
The tools referenced above — complexity, impact, stakeholders, RAID, 5 Why and fishbone — are collected in the AI-assisted diagnostic toolkit, and the ERP readiness resources set out how UAT evidence fits alongside readiness assessment and adoption measurement across the programme.