Quality and generalisability

Assessment Tips

When should quality and risk of bias be assessed?

We suggest assessing in two stages:

• Before receiving data: assessments should be based on the original publications
• After receiving data: assessments can be revised/updated based on the participant-level data. In some situations, assessments may differ for different participants from the same primary study.

Additional considerations

• IPD may allow the researchers to address risk of bias concerns
  • For example, they can apply additional inclusion/exclusion criteria and exclude or reinstate individual participants based on the revised criteria
• IPD may also allow the researchers to address generalizability concerns
  • For example, the researchers can conduct subgroup analyses based on study or participant characteristics

Existing Tools

Below you will find existing tools for assessing risk of bias and applicability/generalizability in conventional meta-analyses of various study designs.

Extensions to the IPD context are currently in progress by members of our team.

• Randomized controlled trials:
  • RoB 2 tool: A revised Cochrane risk of bias tool for randomized trials

• Prediction model studies:
  • PROBAST: Prediction model Risk Of Bias ASsessment Tool

• Prognostic factor studies
  • QUIPS: QUality In Prognosis Studies tool

• Diagnostic accuracy studies:
  • QUADAS-2: A revised tool for QUality Assessment in Diagnostic Accuracy Studies