Prefer a quick summary over a long read? Use this link to hear PPI member Araya Gautam break down this blog in under three minutes, or alternatively, listen to the audio version of this blog.
Patient and public involvement in research (PPI) refers to an activity done ‘with’ or ‘by’ service users and carers rather than ‘about’, ‘on’ or ‘for’ them (INVOLVE, 2009).
Evaluation of PPI is important to ensure that such involvement activities are conducted with a good degree of quality and in a meaningful way.
Although the importance of community engagement in research is recognised, there are few evidence-based approaches for measuring the level of community engagement in research projects. The 4Pi Questionnaire has been created in response to this challenge.
It is based on the 4Pi National Involvement Standards of principles, purpose, presence, process and impact. The principles were developed by the National Survivor User Network with input from survivors and service users.
“The questionnaire’s intended purpose is to be used by service providers and other stakeholders that carry out patient and public involvement activities, who would like to assess the experiences of service users and carers involved in those activities,” explains Manuela Russo, a research fellow at King’s Improvement Science (KIS). “It was developed by KIS in collaboration with South London and Maudsley (SLaM) NHS Foundation Trust.”
We created this guide to provide a robust and standardised method to evaluate patient and public involvement. Patients, service users, carers and the public were involved throughout the design process
It is so important to listen to patients when doing health research, and my own experiences as a patient are what make me passionate about being involved in it. Collecting data about the things we don’t know enough about can create great change. Hearing about the involvement experience itself is a very important part of that data collection process
The questionnaire is a helpful tool for thinking back on your involvement experience and pinpointing areas that need improvement. The statements we ask you to consider are specific, so that staff can see exactly what worked and what didn't. We use detailed statements such as: "I had enough practical support (Examples: printing and posting information, technology support, scheduling meetings at a convenient time, support with travel arrangements, training.)," rather than generic statements like "I was involved."
The questionnaire should be used by staff members in charge of managing patient and public engagement as part of an organised strategy to track and enhance involvement procedures. It provides insights into how individuals felt about their role and influence within a project, going beyond basic indicators (like attendance).
Staff can identify a breakdown in continuity and influence if, for instance, a pattern appears in multiple questionnaires. If multiple respondents report that their feedback during the research planning phase was acknowledged, but by the implementation stage, recommendations had not been taken forward, and involvement members were no longer invited to the follow-up meetings, this could lead to an internal review of decision-making processes and how co-production is implemented.
Additionally, when used across multiple projects, the questionnaire data can be aggregated and analysed, to track whether improvements have occurred over time. This helps demonstrate accountability and supports learning. By acting on the feedback and feeding back to involvement members on how their input led to change, staff help close the loop - making involvement feel authentic and impactful.
We're very excited that the 4Pi Questionnaire is available to use. Information gathered through the 4Pi Questionnaire will be immensely helpful in our continued efforts to meaningfully involve service users in quality improvement. We know from our experience and our evaluation that when service users are involved from the start of quality improvement that the results are better for all
Do people enjoy filling out surveys? In both my empirical experience as a lay person partaking in quite a few surveys, and through being involved with design working groups, I find that in general, service users do not particularly enjoy this activity!
The hashtag ‘#NotAnotherSurvey’ remains popular on social media, and the term ‘survey fatigue’ was coined to describe this feeling of discomfort. This can refer to a situation when respondents are no longer interested in completing researchers’ surveys due to the volume of surveys or the effort it takes to complete them.
This type of fatigue is harmful in many ways. It can lead to lower survey response rates, when respondents who are over-surveyed experience fatigue that manifests before they take a survey, causing them to skip it entirely. It can also lead to inaccurate data or abandonment of the survey altogether.
Being aware of the risks associated with surveys as a research tool, our working group reflected hard to include the most important and relevant questions in the survey with the goal of helping respondents manage survey fatigue.
Our thinking went through the following lines:
As involvement members, we spoke about what we could do to make the questionnaire more accessible for people who struggle with longer documents. One idea we included in the access section of the guide is that instead of filling out a paper or online form, respondents could complete the questionnaire over the phone, by video call or one-to-one in person. We want as many people as possible to be able to contribute
Explore the profiles of this blog’s co-writers, Araya Gautum, Chris Pavlakis and Charlie Costa, and find out more about their research interests.