Choosing a statistical approach for your DNP project
Choosing an analysis is not a search for the most sophisticated test. It is a process of matching the analysis to the question and the way the data were generated.
Before naming a test, complete this sentence: “My data contain one observation per ___, and I want to describe or compare ___.” A patient, an encounter, a staff member, and a week are different units. The same spreadsheet can contain more than one level, which is a reason to pause rather than guess.
Narrow the decision in a sensible order
First decide whether the goal is to describe a program, compare groups or times, or understand a process over time. Then identify whether observations are linked. Paired data have a meaningful correspondence, such as the same person measured twice; independent comparisons involve different observational units. [1]
Finally, identify the outcome. A score, a yes or no response, an ordered rating, a count, and a rate do not carry the same information.
| Data situation | A useful starting approach | What needs checking |
|---|---|---|
| One program measured after implementation | Descriptive summaries and implementation findings | What can be concluded without a baseline or comparison? |
| Linked numerical scores before and after | Describe individual changes; consider paired t or signed rank methods | Distribution of differences, scoring properties, and complete pairs. |
| Different people in two groups | Compare group summaries; consider an independent group method | Group comparability, distribution, and independence. |
| Linked yes or no responses | Describe how responses changed; consider McNemar’s test | The paired transitions, not only the two overall percentages. |
| Weekly process results | Plot the series; consider a run chart | Comparable periods, stable definitions, and changing denominators. |
| Repeated observations within people or sites | Seek a method that accounts for dependence | Number of people or clusters as well as number of records. |
This table is a starting map, not a complete decision algorithm. The named tests are examples to discuss with your statistical advisor, not automatic recommendations.
The same outcome can require different analyses
Hypothetical example. You have 20 baseline scores and 20 follow-up scores from a staff workshop. If the scores belong to the same 20 people and are linked correctly, you can examine each person’s change. If they belong to two different cohorts, you have a group comparison. If participants may overlap but cannot be identified, neither complete pairing nor independence can simply be assumed.
Similarly, twelve weekly percentages are not twelve patient outcomes. They summarize a process over time. IHI recommends plotting improvement measures chronologically because timing can reveal information that a single before and after average hides. [2]
Make the plan before seeing the result
Write down your primary outcome, unit, main comparison, and planned descriptive summaries. Note any clustering, repeated measures, or missing follow-up. Choosing among many tests after seeing which produces a favorable p value weakens the interpretation.
Your next decision
Bring a small fictional data layout to the planning meeting. Ask whether its structure supports your intended analysis. See Pre-post surveys: Same instrument, linked responses for linkage, Paired t-test or Wilcoxon? Analyzing pre-post data for paired scores, Analyzing Likert data: Items vs scale scores for Likert responses, and Building and reading a run chart for run charts.
Sources
[2] Institute for Healthcare Improvement. (n.d.). Model for Improvement: Establishing measures.