Should the AI programme expand to twenty sites?
Use cohort evidence and operating capacity to set a rollout wave.
The question
A fictional local-government programme tests AI-assisted document extraction. Staff review each output before use. The rollout rule requires at least 40 checked records in each document cohort and at least 95% correct outputs in each cohort. A site also needs two trained reviewers before it joins a wave.
| Checked records | Correct outputs | |
|---|---|---|
| Printed documents | 80 | 78 |
| Scanned documents | 20 | 19 |
These are invented practice counts, not results from the government source. Correctness is checked against the original record.
The sponsor wants 20 sites in the next wave. Passing the sample rule does not remove the reviewer requirement.
“Decide whether the pilot supports expansion and how large the next wave can be.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Check each cohort
What would you do here? Open the move you’d make.
Collect more scanned-document evidence before expansion.
The strong move. A passing rate does not replace a minimum sample.
Use the pooled 97% score to pass every cohort.
Tempting, but it misses something. The scanned cohort has only 20 checks; the pooled result cannot satisfy its 40-check minimum.
See how a strong candidate says it
“Printed documents score 78/80 = 97.5% and have enough checks. Scanned documents score 19/20 = 95%, but their 20 checks miss the 40-record minimum.”
Why it works: A passing rate does not replace a minimum sample.
Step 2: Keep human review in scope
What would you do here? Open the move you’d make.
Review quality and service capacity together.
The strong move. Programme readiness includes the operating work around the model.
Treat the model score as the whole rollout gate.
Tempting, but it misses something. A model score does not show that trained reviewers and exception handling are ready.
See how a strong candidate says it
“I would retain source checks, name the service and quality owners, and confirm reviewer training and escalation before inviting sites. An AI extraction score alone does not prove that sites can run the service.”
Why it works: Programme readiness includes the operating work around the model.
Step 3: Bound the next wave
The scanned cohort reaches 44 checked records, with 43 correct. Printed-document results stay unchanged. There are 32 trained reviewers available, each assigned to one site, and each site still needs two.
What would you do here? Open the move you’d make.
Cap the wave at 16 fully staffed sites.
The strong move. The tighter constraint is now reviewer capacity.
Approve all 20 sites because quality passed.
Tempting, but it misses something. Thirty-two reviewers can staff only 16 sites at two reviewers per site.
See how a strong candidate says it
“Scanned documents now score 43/44, about 97.7%, and meet the minimum. Thirty-two trained reviewers support at most 16 sites at two each. I would offer a 16-site wave after the remaining operating gates pass.”
Why it works: The tighter constraint is now reviewer capacity.
Step 4: Define the next decision
What would you do here? Open the move you’d make.
Name the evidence and capacity needed for four more sites.
The strong move. Scaling creates another decision with explicit conditions.
Assume the pilot rate stays fixed as volume rises.
Tempting, but it misses something. A pilot score does not establish quality or review capacity at a larger volume.
See how a strong candidate says it
“I would track quality by cohort, review backlog and exceptions during the wave. The next expansion needs both continued quality and eight more trained reviewers for the remaining four sites.”
Why it works: Scaling creates another decision with explicit conditions.
The answer, in short
- Decision
- Offer a gated wave of at most 16 sites.
- Because
- The updated cohort passes, but reviewer staffing does not cover 20 sites.
- What would change it
- Eight additional trained reviewers plus sustained quality could support the remaining four.
Common traps
- Pooling away a weak cohort.
- Dropping human work from the rollout plan.
Check your answer
- I checked each sample size.
- I kept human review in the plan.
- I calculated the 16-site limit.
- I named the next expansion conditions.
Last reviewed Nov 10, 2026