Author: Carlos Eid, MD (views are strictly personal)
One of the major challenges that medical affairs professionals face is how to demonstrate their value and impact to the organization. Traditionally, the metrics used to assess the performance of field medical associates (MSLs) were based on quantitative indicators, such as the number of visits, the duration of interactions, or the coverage of key opinion leaders. Additionally, new metrics started surfacing like number of actionable insights, advocacy based on agreement with scientific statements, etc... However, these metrics do not capture the impact on behavioral change or change in patient outcomes objectively. Therefore, there is a need for a more comprehensive and robust approach to measure the impact of field medical activities, one that considers both the process and the impact of the engagement. There has been enough talk about metrics and impact measures, some have made it too complex and others too simple, and personally I feel we are missing a practical, pragmatic approach (which may not be perfect) but a good starting point if we really want to drive change!
Therefore, in this article, I will share with you a personally devised, practical example of how you could implement an impact measurement framework for your field medical teams (based on my own experience and views strictly). I will share how to define your desired outcomes, identify your key actions, collect and analyze your data, and communicate your findings. I will also illustrate how to use this framework to monitor and improve your performance, as well as to showcase your value and achievements to management and cross-functional partners.
First, let me quickly remind you of the impact framework I proposed months ago where our thinking needs to shift from transactional interactions and a focus on the number of activities to a bigger focus on the patient outcome that we hope to impact.
Based on this figure, focus on key 5 steps:
If you are asking yourself what “Impactful Contributions (ICs)” are, take a look at the figure below and note that we use the term contributions as some of these activities might not be primarily led by MSLs or have a shared accountability.
Let’s jump to the "practical example"
To help with this I will start with creating something that we will call the Medical Impact Plan (MIP). For this I will make an assumption that any CRM can incorporate this template and its functionality.
STEP 1: Start with the patient outcome/output.
PS: Please note that this example is purely fictional, given for the purpose of explaining the model and not related to any current strategy or tactics in my organization.
The disease I will tackle is COPD. As a patient outcome I will choose “National reduction in hospitalization rate for COPD exacerbations by XX% over X years”. This is longer term, so I will couple it with a shorter-term patient output (that will ultimately impact the outcome over time) “XX% increase in GDMT (guideline directed medical therapy) implementation in COPD patients discharged after a COPD exacerbation”.
(PS: you may decide to have more than one patient output for a disease area, but we will choose only one for demonstration purposes)
STEP 2: Decide what impactful contributions can help achieve this.
I will assume that we will need to (and these could look too basic, few or simplified versus the ones you will eventually choose):
Reminder: Implementation Science focuses on how to effectively integrate evidence-based interventions into real-world settings. It studies the methods and strategies to ensure that these interventions are adopted, implemented, and sustained in practice. Real-World Evidence involves collecting and analyzing data from real-world settings (outside of controlled clinical trials) to understand the effectiveness and safety of interventions
STEP 3: Based on the strategy and needs the team will have to then:
Example of an IC with milestones linked to it:
A. Co-Develop or support medical educational programs on overall COPD management and guideline implementation targeting XXX HCPs across XX different regions.
If we assume that the above Medical Impact Plan can be incorporated in their CRM, MSLs can now engage and link their engagement to either of the three impactful contributions, and most importantly document the progress in the “notes” and “progress” tabs. Once extracted, the table could look like the below:
STEP 4: Once all milestones are achieved (if milestones are needed of course) and an impactful contribution is marked as fully accomplished the MSL will be prompted to add
The example below shows a case where the MSL decided to have 5 educational activities for IC A (ie each is treated as an individual Impactful Contribution not a milestone, where the milestones are the gradual steps to complete an IC).
In summary if the MSL decided 5 individual activities for IC A, the development of 3 protocols for IC B, and 1 evidence generation activity for IC C, then his total is 9 ICs to be completed.
What can an impact dashboard look like?
A dashboard that brings all this to life can look like the below (more of a leadership view and not individual MSL view), but certainly you can also consider:
How is that helpful?
I see you had 15 engagements this quarter for IC B, and based on the notes and progress, there are still no discharge protocols embracing the updated guidelines in these centers. Why do you think this is the case? Are we targeting the right decision makers? Is there anything I can help you with?
Time to protocol creation in hospital A took less time and less engagements than hospital B, what are the key differences, best practices or challenges?
We did 10 educational activities, however the feedback from customers is neutral and there was no difference in pre and post surveys, What can we do differently? Are we choosing the right speakers, content and audiences?
So what??? Have we impacted the patient outputs or outcomes??!!
The exercise doesn’t end here, as (per the above) we just measured activity metrics and the completion of Impactful contributions, which we “assume” or “hope” to impact patient outputs/outcomes. Now, we need to take this to the next level and try to map the achievement of these ICs with the progress of the patient output, which in this specific case is: XX% increase in GDMT (guideline directed medical therapy) implementation in COPD patients discharged after a COPD exacerbation and ultimately the rate of COPD hospitalizations post discharge.
These can be measured using claims data, registries, databases, market research, real world evidence and other methods (including AI) that are possible and already being implemented by several Pharma companies. Only caveat is that we expect to see a faster impact on the patient output as the patient outcome will require more time. Therefore, you focus on the output on a yearly basis and outcome long term. (Long-term vs short-term impact of medical affairs...rings a bell?)
An example chart could look like the below where you map the yearly # of ICs the team is doing vs the cumulative % change in patient output.
What's next... The MOST IMPORTANT step: Course correction
What happens if the patient output is not being impacted? This is when you should consider reflection and course correction.
I propose 3 levels of course correction (with examples of questions that you can reflect on and ask)
Scenario 1: Impactful contributions are not progressing or being completed
Scenario 2: Patient output is not being impacted
Scenario 3: Patient outcome is not being impacted
Final Thoughts and Conclusion
Measuring the impact of medical affairs has long been a challenging endeavor. While we have made significant strides in quantifying various metrics such as the number of activities, percentage coverage of Key Opinion Leaders (KOLs), number of visits, subjective advocacy assessments, and customer satisfaction, these measures alone do not provide a complete picture.
What I am sharing with you is not intended to be the ultimate solution; rather, it is a different perspective (which some might already be thinking of or adopting). My aim is to link strategy to desired patient outcomes and focus on activities that serve this outcome (or “north star”) with course correction (a step we often miss). Additionally, I propose adding a few measurable (and possible to visualize) layers between what is easy to measure (activities) and what is difficult to measure (patient outcomes). These layers are:
Finally, I would like to thank all the colleagues who reviewed my article as a draft and shared their valuable insights. Let's keep striving to highlight the impact of medical affairs and aim for better patient outcomes.
The ideas and concepts presented in this article are the intellectual property of the author. Readers are kindly requested to respect the author's work and not claim these ideas as their own. Proper citation and credit should be given when referencing or using any of the information presented in this article.
