Picture a respiratory specialist in Japan, a few weeks into a launch. She has just read the new edition of the national COPD guideline. She has also seen the headline of a fresh trial read-out for a biologic. The rep in front of her needs to connect the two in one sentence: this is where your guideline says escalation belongs, and this is what the new evidence shows for that patient.
That was the brief behind this program, which ran in early 2025. A global pharmaceutical company was launching a biologic into COPD in Japan, and its field team had to master two new bodies of knowledge at the same moment. In 32 days, across two waves, 257 people closed 80% of the 4,503 knowledge gaps we measured at first attempt, and readiness rose from 75% to 95%. The more useful finding is what the remaining 20% looked like, and why those gaps went to managers instead of into another round of questions. One boundary up front: these are verified corrections inside the assessment, not proof of what anyone said in a specialist's office.
Two sources of truth landed in the same quarter
Most launches ask the field to learn one new story. This one asked for two.
The first was the sixth edition of the Japanese COPD guideline: spirometric diagnosis, staging, the symptom and quality of life scores (mMRC, CAT and SGRQ), treatment goals and the treatment algorithm. The second was the product evidence from a new trial read-out: exacerbation rates, FEV1, FeNO subgroups, safety and a Japanese subgroup. A specialist will not accept one without the other. The guideline tells her where a patient sits and what she is trying to achieve. The trial data tells her whether this biologic belongs at that point in the pathway.
The audience was mixed on purpose. Sales representatives and medical science liaisons were in the same program, in Japanese, measured against the same approved definitions of the disease and the evidence. Their roles stayed separate: reps hold promotional conversations within the label, while MSLs handle scientific exchange under medical governance, and what each may say, and to whom, is set by those rules, not by a quiz. What the program checked was the shared knowledge base underneath both. If the two roles hold different versions of the guideline, the physician notices.
The stakes are familiar to anyone who has run a launch. McKinsey's analysis of 210 launches between 2003 and 2009, published in 2014, found that about two thirds of new drugs missed pre-launch consensus sales expectations in their first year. Field readiness is one factor among many in that, but it is the one this program could measure.
We split disease and product into separate waves, deliberately
The obvious design would have been one large question bank covering everything, opened on day one. We argued against it, for three reasons.
- Guideline knowledge and trial evidence are learned differently. One is a framework (definitions, stages, goals, an algorithm). The other is a set of numbers and subgroups that hang off that framework.
- If both run at once, you cannot tell which kind of gap you are looking at when readiness stalls.
- Evidence makes more sense once the framework is solid. A rep who is shaky on when escalation to triple therapy is recommended will struggle to place an exacerbation reduction figure correctly.
So the program ran as two editions. The disease wave opened first and covered the guideline, diagnosis, staging, scoring, burden and the treatment algorithm. The product wave followed and covered the trial read-out. Together they ran 32 days, with 69 questions in play and 514 player slots across the two waves.
The mechanics were standard for us. Players met real colleagues in one-minute challenges, five questions in 60 seconds, inside synchronised battle windows, with notifications pulling people back in. Every question was built from approved material only and went through the client's review before it went live. When a player got something wrong, Gap Fixing sent that person the fix for their own mistake, not a generic recap, and the question came back later to check it had stuck.
The program recorded around 80,000 answers; the most active participants reached a maximum of 310 answers per person in a single wave. Repeated attempts by the same person on the same question are what show whether a gap closed.
Four in five gaps closed, and the trial evidence had a higher closure rate
We count a gap as one person wrong on one question at their first attempt. It only counts as closed when that same person answers the same question correctly later. That definition is strict, and it is why the numbers mean something.
At first attempt the program identified 4,503 gaps. By the close, 3,602 of them were verified closed, which is 80%. The remaining 901 were flagged for follow-up coaching. Readiness moved from 75% at first attempt to 95% at last attempt.
By content block, closure was high everywhere:
- Trial evidence (exacerbation rates, FEV1 and FeNO, SGRQ, safety, the Japanese subgroup): 1,948 gaps, 83% closed.
- Guideline and management (goals, algorithm, exacerbation care, biologics): 1,842 gaps, 78% closed.
- GOLD classification and the ABE assessment tool: 329 gaps, 75% closed.
- Disease definition and burden in Japan: 323 gaps, 77% closed.
- Asthma and COPD overlap and eosinophilic phenotypes: 61 gaps, 84% closed.
Two things stand out. The two big blocks account for more than 80% of all gaps, partly because they held most of the questions. And the trial evidence had a higher closure rate than the guideline. Numbers are sticky once they are rehearsed. Frameworks are harder, because a wrong framework feels right to the person holding it.
Numbers are sticky once they are rehearsed. Frameworks are harder, because a wrong framework feels right to the person holding it.
The gaps that stayed open were the plausible ones
Averages hide the interesting part, so we tracked individual questions from first attempt to last. Three of them tell the story well, and they happen to fall into the three patterns we see across our library: closed, half closed and still open.
Epidemiology: a common myth almost disappeared
One question dealt with the trend in COPD deaths in Japan and which age groups they concentrate in. At first attempt, 41 people chose an option claiming that deaths had levelled off at a fixed annual figure in recent years. By the last attempt, one person still believed it. Of 64 people wrong at first attempt, 59 were verified as corrected on a later attempt, about 92%.
This is the kind of gap a game fixes on its own: a half-remembered statistic, replaced once the right one was seen a few times.
Treatment algorithm: most corrected, some started from the wrong rung
The question reps meet most often in the room is when triple therapy is recommended in stable COPD without asthma features. The right answer combines two conditions: frequent exacerbations despite dual bronchodilation, plus blood eosinophilia. Of 82 people wrong at first attempt, 47 were verified as corrected, 57%. The other 35 were not, and at the last attempt 24 people were still choosing the option that starts the escalation from the wrong combination of inhalers.
That is not a memory problem. It is a slightly wrong model of the algorithm, and a specialist will catch it in one exchange.
Guideline goals: the answer everyone agrees with, and the wrong one
The most stubborn gap sat in the guideline's treatment goals. Asked which outcomes the guideline groups under its goal of reducing future risk, 56 people chose wrongly at first attempt. The popular wrong answer framed it as maintaining healthy life expectancy and survival prognosis, which sounds like exactly the kind of thing a guideline would say. The correct grouping centres on preventing exacerbations and slowing progression. Only 30 of the 56 were verified as corrected, leaving 26 who were not. Of those 26, 24 were still picking the prognosis option at their last attempt; the other two either chose a different wrong answer or did not return to the question, so their status is simply unverified.
Think about what that means in practice. A rep and a physician agree on a treatment goal. The rep then links the biologic's exacerbation data to that goal. If the rep has quietly swapped the goal for a different one, the agreement is hollow, and the evidence lands in the wrong place.
Five lessons we took from this launch
One strong result proves little on its own. These lessons match what we see elsewhere in our library.
- Sequence the framework before the evidence. Running the disease wave first gave the product wave something to attach to. We cannot prove the counterfactual in this program, but the trial block closing at 83% after the guideline wave is consistent with it.
- Separate waves make the diagnosis readable. Because the waves were split, we could see that guideline concepts and trial numbers closed at different rates. That told the client where coaching time would pay, instead of spreading it evenly.
- Mixed audiences are worth the extra care. Putting reps and MSLs through the same approved content meant both roles were measured against the same definitions.
- The last 20% is a different problem, not a smaller version of the first 80%. The gaps that stayed open were plausible misreadings of the guideline, not forgotten facts. Across our library, in one in five of the example questions we tracked, fewer than 20% of the people who were wrong corrected it; some of those were never retested, so not all are proven misconceptions. The ones that are need a conversation, not another question.
- A business moment focuses minds. In our library, the 16 programs tied to a business moment closed 55% of their gaps against 41% for the other 84. That is an observed association, not proof that a deadline causes closure, and this program fits the pattern rather than proving it.
A fair objection: is 80% just an easy question bank?
High closure could mean the questions were too simple, or that participants memorised answer positions.
We would answer it in three ways. First, closure is measured per person and per question, and the same question returns later. One later right answer does not rule out a lucky guess, which is why we read the per-topic pattern rather than any single correction. Second, 4,503 first-attempt gaps across 257 people is not the profile of an easy bank. Third, and most telling, the program still left 901 gaps open, clustered in exactly the places you would expect a thoughtful field to struggle, such as treatment goals and escalation logic. An easy bank would not produce such a specific residue.
The honest caveat is that 80% is the top of our observed range. We would not promise it to another team, and we would be suspicious of anyone who did. Across the 100-program library, 43% of identified gaps were verified closed, and the spread from 18% to 80% depends on design, timing and how close the content sits to a real business moment.
What we would do differently, and what comes next
The 901 open gaps went to managers as a coaching list, organised by topic and by person. That was the right call, but we would make it earlier next time. The guideline goals and triple therapy patterns were visible within the first week of the disease wave. A manager with that list in hand before the product wave could have used a ride-along or a cycle meeting slot to address the model, not just the answer.
We would also bring the MSL and rep views together more explicitly. The data lets you compare how each role performed on the same guideline questions. Sharing that comparison with medical and commercial leads would help both sides agree on one version of the story before the next data release.
Finally, we would plan the reinforcement. Murre and Dros's replication of the forgetting curve, a relearning experiment with nonsense syllables, shows that unrehearsed material fades fast and that spacing helps. It says nothing direct about clinical knowledge, but a 95% readiness score at day 32 is still only a snapshot. Short, spaced challenges timed to the next guideline discussion or congress would help check and reinforce retention.
Where Atrivity fits
This program ran on Atrivity: battles, one-minute challenges, Gap Fixing and readiness tracked by topic and by person. The platform did the measuring. The design choices, disease before product and coaching for the stubborn gaps, came from the client team and ours working through the data together. If you have a launch where the field has to hold two new sources of truth at once, the question to bring to your next launch meeting is simple: do you know which half your team will get wrong?