Appraising evidence without getting lost
You do not need to treat every published study as equally persuasive. You also do not need to reject every study that is not a randomized trial. Appraisal begins by asking what the study can reasonably tell you and how well it supports that claim.
JBI provides different appraisal tools for different study designs. [1] This reflects an important distinction: a study’s design and the quality of its conduct are related but not identical. A design label does not replace a careful reading of the methods.
Read for three different judgments
The level or type of evidence concerns the design and question. Quality concerns how well the study was conducted and whether important biases threaten its findings. Applicability concerns whether the intervention and evidence fit your setting.
| Judgment | A practical question |
|---|---|
| Design | Was the design capable of answering the stated question? |
| Quality | Were selection, measurement, comparison, and missing data handled credibly? |
| Applicability | Are the people, resources, workflow, and outcome relevant to your project? |
Do not average these judgments into a vague “good study” label. A carefully conducted interview study might be valuable for understanding barriers, while it would not establish the size of an intervention’s causal effect.
An encouraging result may not transfer directly
Hypothetical example. A published education intervention includes a workshop, supervised practice, feedback, and follow-up coaching. Your site can offer a brief presentation but no practice or coaching.
The evidence does not automatically support treating the presentation alone as the same intervention. Record what was actually delivered, how often it was delivered, who delivered it, and which parts appear important. A locally feasible adaptation may be reasonable, but describe the difference rather than implying exact replication.
Likewise, a study that reports improved performance immediately after training does not establish sustained performance months later. Match the outcome and timing in the evidence to the claim you plan to make.
Build an evidence table that helps you decide
A useful working table includes the study question, design, participants and setting, intervention components, comparison, measures and timing, main findings, limitations, and local fit. Its purpose is to support a choice, not to collect as many articles as possible.
Use the appraisal tool that fits the design, and record short reasons for your judgments. Keep the evidence review proportionate to your project requirements. A focused review supporting implementation is not automatically a systematic review.
Your next decision
For the intervention you are considering, write what the evidence supports, what remains uncertain, and what you would need to change locally. Review the proposed adaptation with your site partner. See Did the intervention actually happen? Measuring fidelity for documenting delivery and Can your outcome change in time? for choosing a realistic observation window.