A field force can report 98% completion on launch training and still deliver three different versions of the efficacy story in the same city. We have seen it more than once. The dashboard was green. The launch was not.
The problem is usually not the people and not the training. It is the dashboard. Many learning reports were built to monitor programs, not knowledge, so they answer "did it happen?" well and say little about "does the field know it?" This piece sets out the six measures we think every learning leader in a Life Sciences commercial organisation should have in front of them, what each one tells you, and how to get it without building a data warehouse.
Completion describes the program. It says little about what people know.
Look at a typical quarterly learning report. Completion rate. Hours delivered. Satisfaction score. Average assessment score at the end of the module. Maybe a count of certifications issued.
Almost every one of those numbers is about delivery, not about what people still know weeks later. They tell you the course ran, who was reached and how it was received. That is useful operational information, and completion in particular stays essential for compliance and audit. It runs out of road when someone senior asks a different question: is the field ready for the launch in six weeks? Is the off-label risk on the new indication contained? Can the team defend the product against the competitor's congress data?
A few situations we recognise from the programs we run:
- A pharma field force is certified before launch, while a specific misconception about the mechanism of action sits concentrated in two regions.
- A compliance module reaches near-perfect completion, and six months later the same procedural error keeps appearing in call notes.
- A MedTech team passes product training with high scores, yet cannot explain which patient anatomy makes the new device the better choice.
In none of these cases did people fail to complete the training. Completion was asked to answer a question it was never meant to answer.
There is a commercial reason to care. McKinsey's analysis of 210 launches between 2003 and 2009, published in 2014, found that about two thirds missed pre-launch consensus expectations in year one, and most of those that started behind stayed behind. Field knowledge is only one of many things behind that pattern, and we would not claim a knowledge dashboard fixes it. But when the first months matter this much, finding out in month six that the field was never ready is finding out late.
Knowledge moves, so the dashboard has to move with it
Knowledge does not stand still in a commercial organisation. Every label update, every new publication, every competitor launch and every access change creates something new the field has to know. At the same time, what people learned last quarter fades. Murre and Dros's 2015 replication of Ebbinghaus, a single participant relearning nonsense syllables, found most of the loss within the first day and savings of about 4% at 31 days. It illustrates forgetting in that specific experimental setting; it does not quantify field-team knowledge loss.
So learning is not a sequence of events. It is a loop: find out what people know, reinforce what is weak, check that it held. A useful dashboard follows that loop. Instead of activity, it shows how capability changes. Instead of attendance, it shows where knowledge is thinning. Instead of assuming everyone needs the same refresher, it points at the people, teams and topics that carry the most risk.
That is the difference between learning reporting and what we call knowledge intelligence. It sounds grand. In practice it is six numbers.
The six measures worth a place on the screen
1. Where the gaps are, and which ones matter
Every wrong answer is a signal, but not every wrong answer has the same consequence. A rep who misremembers the year a molecule was approved has a trivia problem. A rep who gets the dosing adjustment in renal impairment wrong has a patient safety and compliance problem.
So the first measure is the gap, counted precisely: one person, wrong on one question, at first attempt. Then the gaps are weighted. In most programs a small set of critical questions carries most of the commercial and compliance risk: the primary endpoint, the key safety message, the patient who sits inside the indication, the objection the competitor has trained its field to raise. A dashboard that averages everything together hides exactly the gaps you most need to see. Denominator: every person-question pair answered at first attempt; report gaps as a count and as a share of first attempts, separately for critical questions.
2. Whether knowledge is still there weeks later
The end-of-module score measures short-term memory under ideal conditions. The real question is whether the critical facts are available three or six weeks later, in a car park, between calls, with a doctor who has two minutes.
Retention is measured by asking again, later, without warning. Denominator: the people who answered a critical question correctly earlier and were asked it again after an interval you state (for example four weeks); the measure is the share still correct. A team that scored well at the kickoff and drifts down across the following month is telling you something important about how the event was designed, and about when to schedule the next reinforcement.
3. How knowledge has changed since the baseline
One measurement is a snapshot. Two is a trend. Comparing a baseline with later cycles separates a temporary bump after a campaign from knowledge that has actually settled.
This matters for credibility. If you can show that accuracy on the ten launch-critical facts went from one level before the program to a clearly higher level after it, and held at the next cycle meeting, you are reporting a change in knowledge, not just activity. Compare only the same questions, or questions of similar difficulty, answered by the same population.
4. Whether the gaps you found actually closed
Finding gaps only helps if they close. The fourth measure tracks closure: of all the gaps identified, how many did the same person later answer correctly, and how many are still open?
The definition matters. A gap should only count as closed when the same person gets that question right on a later attempt, and for critical questions it is worth insisting on more than one correct answer, because one later right answer does not rule out a lucky guess. We count a critical gap as closed only at 5 of 5 later check questions on the same concept. The historical case below used one later correct answer by the same person on the same question. Denominator: all gaps identified at first attempt, including those never retested, which should be reported as their own line rather than silently dropped.
Here is what this looks like in a real program. A pharma field force in Latin America ran a three-day battle on the principles of virtual selling in 2021, with 725 reps. The first attempts produced 14,868 gaps. By the end, 10,789 of them, 73%, were verified closed, and readiness moved from 72% on first attempts to 92% on last attempts. Just as useful: the 4,079 gaps that remained open came with names attached, so managers knew exactly who needed coaching and on what. Those are verified corrections within the game; how the reps actually ran virtual calls is a question for observation.
A gap should count as closed only when the same person answers it right later. Anything else is a hopeful average.
5. Who is ready, by team, country and topic
Executives rarely ask whether people completed training. They ask whether their teams are ready. A readiness index answers part of that: the share of critical questions each person, team or country answers correctly at their latest attempt, broken down by topic. State the denominator (critical questions answered, not questions assigned) and show how many people are behind each cell, because a region of eight people moves further on one bad week than a region of eighty.
The value is in the breakdown, and in agreeing thresholds before the data arrives. For example: a cell below an agreed readiness level triggers a targeted reinforcement round; a critical question where fewer than one in five of the people who were wrong have corrected it triggers a manager conversation and a check of the material; anything near the label boundary goes to Medical. Set your own levels, write them down, and keep them stable so a sales director can spot in thirty seconds which first-line manager needs a conversation this week.
6. Where the open gaps touch the business
This is the measure that changes the conversation, and the easiest one to overstate.
Knowledge gaps can have consequences. A rep who cannot explain the differentiator against the main competitor is unlikely to win that argument in the room. Incorrect product information can erode a specialist's trust. A compliance error creates regulatory exposure. A misunderstanding of access criteria can send a pharmacist the wrong way.
We would not put a euro figure on any gap, and we would not claim that closing gaps moves sales by a given amount; many things move sales at once, and the arithmetic is always more confident than the evidence behind it. You do not need a number to make the point. Map each critical gap to the part of the business it touches: a launch message, a safety statement, a formulary argument. Then show the executive team where the open gaps sit. "Our biggest remaining gap is on the dosing message in the two regions with the highest target list" is a sentence a commercial director will act on. "Average score 81%" is not.
The questions the dashboard should let you answer
The shift here is conceptual more than technical. For years, dashboards described what happened during a program. The next generation should point to where performance problems are about to appear. Instead of "how many people completed the course?", a learning leader should be able to answer:
- Which critical gaps are still open, and in which teams?
- Which topics need reinforcement before the next cycle meeting?
- How has knowledge changed since the last launch wave?
- Which managers should be coaching, and on what?
- What could the unresolved gaps mean for the launch plan?
Answer those and learning becomes part of the commercial conversation, not only a report on hours.
A fair objection: we already have too many dashboards
This is a reasonable complaint. Commercial teams are drowning in dashboards from the CRM, the marketing automation tool, the sales data provider and the LMS. Why add another?
Two answers. First, this is not an extra dashboard for its own sake. It fills a hole that none of the others cover. The CRM tells you how many calls were made. The sales data tells you what happened to prescriptions. Nothing in between tells you whether the field knows what it needs for those calls. Knowledge data arrives weeks before sales data; it is an early signal worth acting on, not a predictor of sales.
Second, the most useful version is small. One page. Six measures. If it cannot be read in two minutes by a sales director between meetings, it is too big, and it will be ignored like the others.
Keep the LMS, and keep completion. Add what it cannot show.
None of this means abandoning the LMS. It is good at what it was built for: delivering and administering content at scale, and keeping the audit trail compliance needs. Completion, attendance and satisfaction keep their place as indicators of delivery.
What changes is the claim you make with them. They show that training happened and who it reached. The six measures above show what people know afterwards, and where they do not. In our experience, the teams that get the most from their programs know what their people know, spot new gaps as they form, and act on them early.
At Atrivity we build this kind of view from short, competitive question challenges that the field plays in a few minutes, with gaps tracked person by person, showing verified corrections and those still open. Whatever you use, try one exercise before your next launch meeting: take your current learning dashboard and ask which of these six measures it shows. If the answer is none, you know what to fix first.