Outcome List Generation
Identifying outcomes
One of the first stages of COS development is to identify an initial long list of outcomes for use in further consensus work (we will refer to this stage as the outcomes scoping stage). Generating this list might involve one or more methods. For example:
- Systematic reviews of clinical trials and reviews of qualitative literature
- Qualitative interviews and focus groups
Systematic reviews are used to identify which outcomes researchers have used in previous studies of the health condition/topic. Qualitative reviews discover what outcomes have been identified as important in previous qualitative studies in this health condition/topic. Qualitative interviews and focus groups are used to explore what relevant stakeholders feel are important outcomes in the health condition/topic.
When undertaking the systematic review of outcomes or a qualitative literature review (also known as qualitative evidence syntheses), Public Research Partners (PRPs) can collaborate with the research team to consider:
- The eligibility criteria and scope for the review
- Should the review exclude or include specific types of studies?
- Is the scope of the review too broad or narrow?
When planning and analysing qualitative interviews/focus groups with patients about outcomes of importance to them, PRPs can collaborate with the research team to consider:
- What questions do you think should be used to find out what outcomes are important to patients / the public and why they are important?
- Are the intended questions in the topic guide understandable – could anything be misinterpreted?
- What information needs to go into the participant information sheets to help patients understand the purpose of the study, what will happen in the study and any ethical considerations?
- Do the participant information sheets cover everything that you think should be covered, without being overly detailed?
- How readable and understandable are the participant information sheets and consent forms for your COS interview / focus group study? Does anything confuse you? Could anything be misinterpreted?
- How can we ensure the sampling strategy is appropriate and inclusive for the COS interview / focus group study?
- What should we pay particular attention to in terms of the sampling participants for the study?
- What are the best ways of accessing relevant patients / potential public participants for participation in the COS interview / focus group study?
- What should the recruitment strategy be?
- How should the study should be promoted to patients / the public?
- What ethical issues might there may be with patient participation in the COS interview / focus group study (for example, any risks from participation)?
- Is there anything about the interview / focus group design that you feel could cause distress to potential patient / public participants? If so, what could be done to address this?
- Is there anything about the interview /focus group study design that you feel could impact the confidentiality or data protection of the study participants? If so, what could be done to address this?
- Consider whether the interviews/focus groups would best be held in-person, online, by telephone or hybrid?
- If in-person, how can we ensure the venue is accessible for patients / the public?
- What adaptations might be needed to make sure patients / the public can participate in the in-person interview / focus groups (e.g. room layout, breaks, venue accessibility, baby changing / breast feeding facilities, prayer room, timing and day of the interviews / focus group, interpreters etc)?
- What adaptations might be needed to make sure patients / the public can participate in the online interview / focus group to ensure (e.g. instructions for joining, breaks, timing and day of the interviews / focus group, interpreters etc)?
- Do you feel the intended coding and analysis of the data is appropriate from a patient perspective or are important outcomes potentially going to be missed?
Reporting and merging outcomes from different sources
When reporting and merging the outcomes from different sources (e.g. from a systematic review and an interview study), PRPs can collaborate with the research team to consider:
- Which outcomes could be merged from the various sources?
- Why those outcomes should / shouldn’t be merged
- Is the list of outcomes still too long – is further merging needed?
- Are the merged outcomes appropriate from a patient perspective or have important distinct outcomes been lost in the merging?
- Are patient important outcomes included in the final list? (For example, less frequently reported outcomes could be really important outcomes from a patient perspective)
Reporting and merging outcomes from different sources
When reporting and merging the outcomes from different sources (e.g. from a systematic review and an interview study), PRPs can collaborate with the research team to consider:
- Does the reporting of outcomes adequately highlight any differences found between the different sources?
- Which outcomes could be merged from the various sources?
- Why those outcomes should / shouldn’t be merged
- Is the list of outcomes still too long – is further merging needed?
- Are the merged outcomes appropriate from a patient perspective or have important distinct outcomes been lost in the merging?
- Are patient important outcomes included in the final list? (For example, less frequently reported outcomes could be really important outcomes from a patient perspective)
Defining the outcomes
When defining the outcomes, PRPs can collaborate with the research team to consider:
- Is the list missing any outcomes?
- Why do you think these outcomes should be included?
- What plain language descriptions of each outcome should be used?
- Are the plain language descriptions understandable and do they adequately cover what the outcome is about?
- Are the grouped headings appropriate / do you feel they will make sense to patients/the public?
- Would any of the groupings confuse you?
- Is the order in which the outcomes presented important? (Typically, the groups of outcomes are randomised but there might be reasons not to do this).
Access a PDF of the questions on this page.
Relevant PPI resources
The following resources may be useful when involving patients / the public in outcome list generation
Resources for PRPs
Populations and sampling– This resource has been developed by Understanding Health research to explain populations and sampling in plain language.
Resources for COS developers involving patients / the public in their work
- The following article provides an example of PPI in outcome list generation: https://researchinvolvement.biomedcentral.com/track/pdf/10.1186/s40900-018-0091-5.pdf)
- NIHR – A brief guide to patient and public involvement and qualitative methods within health and social care research.
- Piloting a qualitative topic guide with PPI partners (Ryan Combs)
- PPI reflection log
- COMET PoPPIE guidance on Ethics and COS studies https://www.comet-initiative.org/Patients/Ethics