Compliance training has a reputation problem, and some of it is deserved. In many organisations it is signed, filed and rarely revisited. Everyone completes it, auditors are satisfied, and the completion record alone cannot tell you whether a rep setting up a virtual meeting next month will get the privacy rules right. This case is from a program that ran in 2021; we are publishing it in 2024 because the pattern has not changed.
In November 2021 the compliance function of a pharma affiliate in Spain decided to find out. Over four days, 249 people from the field force, Medical and Marketing played 180 questions on their own code and procedures, in a league format that ran alongside, not instead of, the mandatory compliance training. Readiness rose from 82% at first attempt to 88% at last attempt. But the more useful finding was underneath: 35% of the individual knowledge gaps closed, and the ones that did not close were mostly about applying a rule, not knowing it.
The situation: a compliance function that wanted evidence, not signatures
The compliance team had done everything the standard playbook asks. Policies were written and approved. Training was delivered and completed. Monitoring was carried out, and the previous year's monitoring cycle had produced a set of findings that were shared with the organisation.
What the team did not have was evidence that people could apply the rules to the situations they actually face. Can a rep tell whether a given activity is promotional or non-promotional? Does a medical colleague know what is required before supporting an HCP to attend an event? Does a brand manager know the rules for an advisory board or an interaction with a patient organisation? Does anyone remember what the monitoring findings actually said?
One scope note before going further. Every question was written from this company's own code of practice, internal procedures and monitoring findings. The "right" answers below reflect those internal rules, which sit on top of industry codes and law and are sometimes stricter; they are not statements of universal regulation, and another company's policy could legitimately differ.
The stakes are different from a commercial programme. In a product programme, a residual gap is a missed opportunity. In compliance, a residual gap is an exposure. One person who misunderstands the rules for a sponsored event can create a problem that no amount of good selling fixes.
What we designed, and why a league instead of another course
The brief was simple: turn the code and the internal procedures into something people would actually engage with, and measure who could answer questions about applying them. A new e-learning would have produced another completion record. We needed answers.
So the programme was built as a short, competitive league:
- One game, 180 questions, all in Spanish, written from the company's own approved policies, procedures and monitoring findings.
- A four-day battle window, with one-minute challenges against real colleagues and notifications to bring people back.
- Six content areas: support for HCPs attending events and the rules for virtual events; promotional versus non-promotional activity; advisory boards, patient organisations and difficult conversations; data protection and the internal privacy roles; the findings of the previous monitoring cycle; and a set of remaining modules.
- Scenario questions wherever possible, so people had to apply a rule to a situation rather than recite it.
- A strict measure: a gap is one person wrong on one question at first attempt, and it closes only when that same person answers correctly later.
Why four days and not a month? Compliance is not a topic people choose to spend evenings on, and a long, open-ended campaign would have faded after the first week. A short, synchronised window turns it into an event: everyone plays at the same time, the leaderboard moves every hour, and colleagues talk about the questions at lunch. That conversation is part of the learning. When a rep and a medical colleague argue over whether a given activity counts as promotional, both of them leave with a clearer view than a policy document would have given them.
We also agreed with the compliance team, before launch, what each result would trigger, and who owned it. A verified correction within the assessment was recorded; follow-up on critical topics and retention remained subject to the compliance team's criteria. A gap still open at the end would go into the next wave. And a wrong answer shared by many people would go back to the policy owner as a question about the material, not only about the people.
The audience mix mattered. Compliance risk does not sit only in the field. Medical and Marketing design many of the activities that create exposure, so they played alongside the reps, on the same questions.
What happened: high baseline, stubborn gaps
Engagement was strong, especially for a compliance topic. Players averaged 222 answers each over four days, about 55,000 answers in total.
The headline figure was reassuring. Readiness went from 82% at first attempt to 88% at last attempt. This was a knowledgeable population, and the league made it more so.
But the gap data told a more demanding story. We identified 8,239 individual knowledge gaps at first attempt. Of these, 2,844 were verified closed on a later attempt, 35%, and 5,395 were flagged for follow-up. Closure by area:
- Data protection and privacy roles: 561 gaps, 50% closed.
- Monitoring findings from the previous cycle: 932 gaps, 40% closed.
- Events and support for HCP attendance: 2,032 gaps, 38% closed.
- Interactions, advisory boards and patient organisations: 1,538 gaps, 32% closed.
- Promotional versus non-promotional activity: 670 gaps, 29% closed.
- Other modules: 2,506 gaps, 29% closed.
Why would readiness be high and closure modest at the same time? A high baseline and a modest closure rate can coexist. Retesting coverage, question difficulty and repeated errors need to be examined separately.
In compliance, over-applying a rule is still a gap. It means people learned the words and not the reason.
Three questions that show the pattern
Internal privacy roles: fixed by repetition. One question asked which internal role backs up the data protection officer. The answer was a specific internal network of privacy champions, a fact about the company rather than about the law. At first attempt, many chose the legal department or "all of the above". Of 163 people who were wrong at first, 92 corrected themselves, about 56%, and correct answers roughly doubled from first to last attempt. This is the easy kind of gap: a fact people had not been told clearly, fixed once they saw it.
A process step: half closed. Another question asked when, in the life of an activity, the satisfaction survey belongs. The procedure places it in the approval stage. At first attempt, 168 people placed it after the event had finished, which is the intuitive answer. Of 170 people wrong at first, 74 corrected themselves, about 44%, and 71 people still placed it after the event at their last attempt. Process details that run against intuition need more than one correction.
Privacy notices for event registration: still open. The hardest case asked whether event registration pages always need a privacy notice, and if not, when. Under the company's procedure, the correct answer is: whenever personal data is collected. At first attempt, 113 people said it is needed in every case, without exception. Of the 116 people wrong at first, only 24 corrected themselves, about 21%, and 92 remained wrong. At the end, 90 of 249 people still believed a notice is always required.
That last result is interesting precisely because the wrong answer sounds safe. Over-applying a rule looks like caution. But it means people have memorised a slogan rather than understood the principle, and a person who does not understand why a rule exists will struggle the first time a situation does not fit the slogan.
Four lessons from the league
1. A good average can hide the real risk
Readiness of 88% at the end is a number most compliance teams would be happy to report. It is also the number that would have ended the conversation. The gap data showed where the exposure really sat: in the promotional boundaries, the interactions with advisory boards and patient organisations, and a few process details that many people shared the same wrong idea about. Report the average, but manage the gaps.
2. Closure varied by topic
The areas that closed best were those built on facts about the organisation: who does what, what the monitoring found. The areas that closed worst required judgement: is this activity promotional, what applies to this interaction. That is not surprising, but it changes what you do next. Facts can be fixed with a clear answer. Judgement needs scenarios, discussion and, often, a conversation with someone from compliance.
3. Over-application is a gap too
It is tempting to treat "always require it" as a harmless error. It is not. It shows the rule has not been understood, and it creates friction in the business, because people refuse or delay legitimate activities out of misplaced caution. A fair objection is that it is better to err on the safe side. Sometimes. But a compliance function that wants to be seen as a partner, not a brake, needs people who understand the reasons.
4. Compliance can be engaging, but knowing is not doing
About 222 answers per person in four days, on a compliance topic, is not the behaviour of people ticking a box. The competitive format made people want to get the answers right, which means a function can run these checks more often, on more topics. Two limits apply. The league complemented mandatory training and the company's other compliance obligations (approvals, monitoring, audits); it did not replace any of them. And a correct answer shows knowledge of the procedure, not compliant behaviour. Whether people follow the rules when they set up an event is something only monitoring can show.
What we would do next
The design already assumed a second wave. In compliance, the remaining gaps are exposures, so they should go straight into the next round rather than wait for next year's training.
We would do three things in that wave. First, send each person targeted fixes for the specific questions they got wrong, with a critical gap counted as closed only at 5 of 5 later check questions on the same concept. Second, rewrite the weakest areas as richer scenarios, especially the promotional boundaries and interactions, so that people practise the judgement and not just the rule. Third, take the stubborn misconceptions, like the privacy notice, back to the policy owners. When 90 people share the same wrong idea, the explanation in the policy may need rewriting.
We would also run a short check a few months later, ideally before the next monitoring cycle, to see whether the gaps that closed stayed closed.
Why this matters beyond compliance
The same pattern shows up in commercial programmes: a respectable average, a long tail of individual gaps, and a small set of confident misconceptions that repetition does not fix. Compliance just makes the stakes more obvious. If your organisation can measure whether people know how to apply the code, it can measure the same for the label, the positioning and the objection handling too.
At Atrivity we run this kind of programme for compliance, medical and commercial teams: short competitions on your own approved content, Gap Fixing for each person's mistakes, and readiness views that show where the real exposure is. If your compliance training is completed every year and nobody knows what stuck, a four-day league is a good place to start.