The demo went beautifully. The platform had an AI assistant, a slick mobile app, forty integrations and a dashboard in the client's brand colours. Eighteen months later, the commercial director asked what it had done for the launch, and nobody could answer.
We see versions of this story often, and the vendor is rarely the main culprit. The mistake is usually made before the first demo, when the buying process starts from a software category instead of a business problem. This guide is for Commercial Excellence, learning and development (L&D) and sales leaders in pharma and MedTech who are about to renew a learning management system (LMS) contract, evaluate a new learning tool or answer a CFO who wants to know what the last one delivered. It sets out five questions to ask before you look at any vendor, the traps specific to Life Sciences, and a short checklist to take into procurement.
Buying used to be simple. The questions have changed.
Ten years ago, buying learning technology meant buying an LMS. It delivered courses, tracked completion, managed certifications and kept an audit trail for compliance. The decision was mostly about administration, and the criteria were clear.
That need has not gone away. Compliance training still has to be tracked. Auditors still ask who completed which policy module. Content still has to be stored, versioned and retired when the label changes.
What has changed is what the business expects from the money. Finance directors ask what a program changed, not how many people finished it. Commercial leaders want to know whether the field is ready for the launch, not whether the training ran. And since generative AI arrived, every vendor's slides look roughly the same: personalised, intelligent, AI-powered.
So the hard part is no longer finding a platform. It is working out which capabilities will make your people effective faster, and how you will prove it.
Start with the business problem you need to fix, not the software category you think you need to buy.
1. What business problem are we actually trying to solve?
Many buying processes begin with a platform search. They should begin with a sentence that describes a business problem in commercial terms. Some examples that come up in Life Sciences:
- New reps take too long before they can run a credible call without a manager.
- Launch messages are delivered differently from one region to the next.
- The field consistently fails to handle one competitor objection.
- Knowledge from the kickoff has faded by the first cycle meeting.
- Pharmacy or channel staff give inconsistent advice on a consumer brand.
Each of these calls for different capabilities. A compliance-driven organisation may put auditability and governance first. A commercial team with several launches and indication extensions a year needs reinforcement, readiness measurement and speed of adoption. A MedTech team selling complex devices needs people who can explain which patient anatomy makes the product the right choice, and that is a different kind of knowledge again.
In our sector, a knowledge gap rarely stays internal. It surfaces in the next conversation with a specialist, a hospital pharmacist or a formulary committee. That is why the problem statement matters: it tells you what "working" will look like, and therefore what to test in the pilot.
A useful discipline: write the problem statement, then write the metric that would show it is solved, before you invite a single vendor. If you cannot agree the metric internally, no platform will solve the problem for you.
2. How will we know people actually learned something?
This is the most overlooked question in the process, and the one vendors are least keen to be pressed on.
Most platforms report completion. Completion does not show understanding, and it certainly does not show retention. The forgetting curve has been replicated since Ebbinghaus; Murre and Dros's 2015 replication, a single-participant experiment with nonsense syllables, shows that unrehearsed material fades fast and that spacing helps. It says nothing precise about product knowledge, but the direction is enough: any tool that measures learning only at the moment a course ends is measuring the best case.
So ask every vendor how their platform answers these:
- Can it identify knowledge gaps for each person, on each question, not just an average score?
- Can it show whether a gap has closed, meaning the same person later answers the same question correctly?
- Can it track retention weeks after the event, without a new course?
- Can a manager see which people in their team carry which gaps, today?
Then ask for evidence from a real program, not a demo dataset. For example, in a three-day battle we ran in 2021 for a pharma field team in Latin America, 725 reps generated 14,868 gaps at first attempt and 73% were verified closed by the end, with the rest handed to managers as a named list. You want vendors to show you that kind of number, with the definition behind it, from their own clients.
3. Will it fit into a rep's actual day?
A field rep's day does not have a free hour for e-learning. It has ten minutes in a car park, a gap between two hospital appointments, a delayed train. Medical science liaisons and key account managers are no different, just with longer drives.
Any tool that depends on people setting aside time at a desk will struggle in the field. Ask what the experience looks like on a phone, how long a typical session lasts, and whether it works on a weak connection. Ask how the platform brings people back without relying only on reminder emails that everybody ignores.
In our experience, the formats that fit field life are short and competitive: a few questions in a minute, played against a colleague, with the correction for your own mistake arriving immediately. They turn reinforcement into part of the working week instead of another isolated training event. But whatever format the vendor offers, test it with real reps in the pilot, not with the project team.
Design the pilot like a small experiment. Pick two or three teams with a real business moment coming up, such as a cycle meeting or an indication extension, agree the critical facts with Medical and Marketing, and measure what people know before and after. Four to six weeks is usually enough. Write the success criteria down before the pilot starts, with thresholds: for example, a minimum share of invited reps active each week without manual chasing, a minimum share of first-attempt gaps verified closed, and a maximum number of hours per week your team spends on content and review. Then name who judges them: the business owner of the problem (usually a sales or Commercial Excellence lead) decides, with L&D, Medical and IT each confirming their own criterion. A pilot that only measures satisfaction tells you whether people liked it, which is a different question.
4. What will the AI do, and what evidence is there?
Almost every vendor now has an AI story. Some of it is genuine and useful. Some of it is a chatbot bolted onto an old product.
Instead of asking "does the platform use AI?", ask:
- What exactly does the AI do today, in production, for current clients?
- Which business problem does that solve, in terms of the metric from question one?
- What content does it draw on, and can it be restricted to our approved documents?
- Who reviews what it produces before it reaches the field?
- What happens when it is wrong?
In Life Sciences, the fourth and fifth questions are not optional. Anything that reaches the field has been through medical-legal review for a reason. A generated question that misstates a safety warning, or a knowledge assistant that answers a rep's question with off-label information, is a compliance problem. The useful applications of AI in 2024 are the ones that save your team time on work it already does, such as drafting questions and translations from approved materials, while keeping an expert sign-off in the loop.
A fair objection is that asking all this slows the purchase down while competitors move ahead. It does, by a few weeks. A failed platform costs a lot more than that, in licence fees, in project time and above all in the field's goodwill, which is very hard to win back after a tool they were told to use turned out to be a waste of their time.
5. What happens after the training ends?
This may be the most important question of all, because it is where most learning investment stops.
Budgets tend to go into the event: the kickoff, the launch meeting, the certification. Far less goes into the weeks afterwards. Yet commercial performance is decided in those weeks, when reps have to recall the data in front of a doctor who did not ask for the visit. McKinsey's analysis of 210 launches between 2003 and 2009, published in 2014, found that about two thirds of the launches in that historical sample missed pre-launch sales expectations in year one, and that underperformers mostly stayed behind. The field's first months are not a rehearsal.
So ask each vendor what their platform does in week two, week six and month three after a launch event. Can it keep reinforcing the critical facts? Can it tell you which ones are fading? Can it route the open gaps to managers so coaching happens at the next ride-along? If the answer is a new course, the platform is solving the delivery problem again.
The traps specific to Life Sciences
A few issues come up again and again in our sector, and they are worth raising early with IT, Compliance and Medical.
Review workload. If every piece of content needs medical, legal and regulatory (MLR) approval, the platform's workflow has to respect that, and someone has to do the reviewing. Ask how questions and corrections are approved, versioned and withdrawn when the label changes, and estimate the reviewer hours per program. A tool that generates content fast can simply move the bottleneck to Medical.
Total cost, not licence cost. Add implementation, content creation, translation, review time, integration work and internal administration. The licence is often the smallest line.
Data location and privacy. Where is the data hosted? Is each client's data isolated? What does the vendor do with your content, and is any of it used to train models for others? European affiliates will ask about hosting in the EU, and they are right to. Ask for the data processing agreement early, not at contract stage.
Data export and exit. Can you export every answer, gap and result per person in a usable format? What happens to your data and content if you leave? A platform you cannot leave cleanly is a risk, however good it is.
Integrations that actually exist. Ask which integrations are live with current clients (single sign-on, HR user feeds, LMS or CRM exports) and which are on a roadmap. Then ask your own IT team what they will support.
Support. Who answers when a rep in a small affiliate cannot log in the night before a battle? Ask for support hours, languages and response times in writing.
Language and local adaptation. A global program that only works in English will be adopted in headquarters and ignored elsewhere. Ask how many languages are live in current programs, not how many are supported in theory.
Ownership. Who in the affiliate runs the platform day to day after go-live? Many tools fail not because they are bad but because nobody owns them once the project team moves on.
Keep what works. Add what is missing.
Most organisations already have plenty of content and an LMS that does its job. Replacing it is rarely the answer. The gap is often a layer many LMS set-ups do not provide: knowing what each person actually remembers, where the critical gaps are, and whether they close.
That is the layer Atrivity was built for, with short competitive challenges for the field, gap-by-gap tracking and readiness views by team, country and topic, working alongside the systems you already have. But the scorecard above applies to any vendor, including us. If it helps, bring your problem statement to a working session with your team and ours and fill it in together before any demo. Take it into your next evaluation meeting, and ask the first question before anyone opens a laptop.