Author: Lisa Power (EATG)

Acknowledgements: Giorgio Barbareschi (EATG), Bojan Cigan (EATG), Léa Darvey (VU Amsterdam), Daniel De Schryver (Janssen)

How a monitoring and evaluation framework was used to measure patient engagement through the European AIDS Treatment Group’s European Community Advisory Board (ECAB)

Community Advisory Boards (CABs) are a growing feature of patient engagement strategies to inform and improve drug development within a range of medical conditions. The creation of these ongoing expert patient consultative panels has made substantial contributions to improved drug development but more systematic evaluation of the evidence is needed. While many CABs are national, they can also work across Europe and the longest-running of these is the European AIDS Treatment Group (EATG)’s CAB. 

Founded out of an ad-hoc meetings system in the very earliest days of HIV treatment exploration during the early 1990s, the ECAB has grown and developed since 1997 to become a much-cited example of productive patient engagement. But Giorgio Barbareschi of EATG explains “we were good at putting out one fire after another but we never stopped to ask, how can we fireproof the house?”

Becoming involved in PARADIGM allowed EATG room to explore the role that CABs could play more widely but also, at the same time, to bring some rigour and a more robust approach to monitoring and evaluating the work within EATG’s own long-standing ECAB system. 

Key issues addressed included:

  • reviewing historic core activities which had been taken for granted as useful
  • structuring meetings which they already knew were important to show more clearly what the impact of them had been and the wider lessons for community engagement 
  • needing to know where and when the ECAB made a difference over time and capturing those more subtle impacts on a long term basis rather than ad hoc
  • improving planning and prioritisation of issues and celebrating successes

“It was really informative to compare systems with other patient groups,” says Barbareschi. “We tend to work in silos so it was good to work with a wider partnership and to see where and how our needs can align with industry and others. The process of this exercise allowed time to reflect on this and to see that there were possible improvements we could make to how we work.” It wasn’t all plain sailing: “there are lots of challenges to working with multiple stakeholders with diverse priorities but it’s really helped us establish better ways of collaborating in future”.

EATG were previously aware that the industry people they interacted with must do some monitoring but had been unclear what it showed. Through the PARADIGM work, they were not only able to understand better how the ECAB is of benefit to both sides but also to evaluate for the first time the quality of the interaction alongside its impact on drug development. This systematisation of monitoring and evaluation of their work was captured in the Framework now available as one of the tools on the PARADIGM website.

The results of this development work and the resulting Framework were presented at a recent online meeting of PARADIGM. But, while this was the end of the process for the PARADIGM project itself, it was only the beginning for EATG and their ECAB. They will now use the Framework guidance to realign the ongoing work of their drug development interactions with industry. “We will adapt the framework for M&E to ECAB’s everyday realities. All of our previous tools, such as questionnaires, will be unified and refined. And now Covid has brought a need for new and simplified procedures which this will help with,” says Giorgio. This has also made them more aware of the need to incorporate this learning into other areas of EATG work, he adds.

The development of such tools in work package three of PARADIGM was also important to industry partners. Daniel de Schryver, Patient Engagement and Advocacy Lead for Janssen in Europe and one of the participants, says he was “a big fan of the concept. It was hard to know quite where to start with PARADIGM and patient engagement – the issue is vast and we were aware of some overlaps with existing projects such as EUPATI, but we were trying to be comprehensive.” But as the work progressed, he says, it moved faster and he looks forward to these concrete tools being used to, amongst other things, create and evaluate CABs in other health conditions and new disease areas. 

De Schryver expressed surprise that some industry people could still be sensitive about high levels of patient involvement and how differently they may need to phrase or frame things for them to be understood by all, but he accepts that these concepts are still not familiar to everyone. “Tools needed to be carefully written to get the idea across.” But he feels that the PARADIGM approach of having partners from all sectors has been vital in showing that there is no bias in its work and that what they are doing is co-creation rather than collusion. He is clear that the early engagement of patients in drug development processes leads to better healthcare outcomes”. The tools it has created are based on real-life needs and, over time, those such as the ECAB M&E framework will be shown to have real-life, transferable learning and benefits for all.