EMS World

JUL 2018

EMS World Magazine is the most authoritative source in the world for clinical and educational material designed to improve the delivery of prehospital emergency medical care.

Issue link: https://emsworld.epubxp.com/i/996071

Contents of this Issue


Page 41 of 51

EMSWORLD.com | JULY 2018 41 those two subgroups within the sample. By calculating a confidence inter val around the obser ved proportions, researchers can estimate the effect in the target popula- tion. If the 95% CI for the inter vention group is 43%–57%, then investigators are 95% sure that between 43% and 57% of con- gestive heart failure patients in the target population who receive nitroglycerin will require ICU admission. If the 95% CI in the control group is 52% –68%, those t wo confidence inter vals overlap (see Figure 1). That is, ICU admission rates in the target population could be the same in both sub- groups. On the other hand, if the 95% CIs for the two subgroups do not overlap, we would be 95% certain the ICU admission rates for two subgroups differ in the target population. As with descriptive uses, this comparative use of confidence inter vals works for almost ever y type of measure. Using Statistical Tests Another way researchers compare sub- groups within a sample is through sta- tistical testing. There are many different tests, and the chosen test depends on the type and characteristics of the data being analyzed. Whatever test is used, the most commonly reported measure produced by statistical tests is the p-value. The p-value represents the probability of finding some statistical test result sim- ply by random chance. More practically, the p-value is the probabilit y of finding an obser ved difference (or association, or whatever is being measured) in two sub- groups of a sample if those subgroups truly represent the same target population. Using the congestive heart failure exam- ple above, the question is whether patients who receive nitroglycerin and patients who do not receive nitroglycerin are all just part of the same big target population (i.e., the differences are just random variation), or are they really two separate target popu- lations (i.e., there are true, consistent dif- ferences; see Figure 2)? Researchers usually use a threshold (called an alpha value) of 5% to estab- lish statistical significance. If the statisti- cal test generates a p-value less than the threshold alpha value, that means there is less than a 5% chance that the data come from two subgroups in a sample rep- resenting one big target population with some natural variation. Instead the data probably represent two samples from two truly different target populations. Importantly, p-values—like confidence inter vals—are strongly influenced by the number of subjects included in an analysis. If a study repor ts a p-value greater than or equal to 0.05, we are lef t to wonder whether the data for the two subgroups truly represent the s ame larger target population or whether the study was just too small to detect that the two samples actually represent t wo dif ferent target populations. The power of a study is the probability that it will detect a difference if one exists. Researchers typically design studies to have at least 80% power, but this is not always possible. Studies with ver y large numbers of sub- jects (especially retrospective analyses of databases with thousands of subjects) have ex treme power and can produce p-values less than 0.05 even when dif- feren ce s b et ween t wo subgroups are quite small. For these kinds of analyses, researchers sometimes use a more con- ser vative alpha value threshold of 0.01 to establish statistical significance. Clinical Signifi cance Statistical tests and p-values are mea- sures of probability, not the size or strength of a difference or association. Whether a dif ference bet ween t wo subgroups in a sample is practically meaningful is a ques- tion of clinical significance. If a finding isn't clinically significant, it doesn't really mat- ter whether it's statistically significant. Clinical significance requires profes- sional judgement informed by experience and practicalities. A study of an inter ven- tion that reduces mor talit y from 18% to 15% with a p-value of 0.003 likely has less practical impact than one of an inter ven- tion that reduces mor talit y from 18% to 5% with a p-value of 0.038—even though the first study produces a much smaller p-value. Similarly, a study of an inter ven- tion that reduces admission rates from 23.7% to 23.4% would have little practi- cal significance even if the p-value were 0.001. Thus researchers think of statistical significance as a binar y yes (p < 0.05) or no (p > 0.05) concept and avoid describing findings as "slightly significant" (e.g., p = 0.048) or "ver y significant" (e.g., p = 0.001). Putting It All Together By combining a basic understanding of sampling, confidence inter vals, p-values, and statistical and clinical significance, readers can better judge studies they read and understand their clinical importance. They can extrapolate data in a study to the target population of their own patients; they can determine the probabilit y that a study's ef fect is simply a function of random variation within the target popu- lation; and they can determine whether the findings of the study are clinically or practically meaningful. This is the crux of analyzing research findings. ABOUT THE AUTHOR Lawrence H. Brown, PhD, a former paramedic, is an associate professor and direc tor of research education for the Emergenc y Medicine program at the Universit y of Texas' Dell Medical School. MORE ONLINE! For a guide to searching the scientifi c literature visit www.emsworld.com/article/220373. Figure 2: One vs. two target populations This article was produced in partnership with the Prehospital Care Research Forum at UCLA. Visit: https://www.cpc.mednet.ucla.edu/pcrf

Articles in this issue

Links on this page

Archives of this issue

view archives of EMS World - JUL 2018