Effect size

In statistics, an effect size is a value measuring the strength of the relationship between two variables in a population, or a sample-based estimate of that quantity. It can refer to the value of a statistic calculated from a sample of data, the value of a parameter for a hypothetical population, or to the equation that operationalizes how statistics or parameters lead to the effect size value.[1] Examples of effect sizes include the correlation between two variables,[2] the regression coefficient in a regression, the mean difference, or the risk of a particular event (such as a heart attack) happening. Effect sizes are a complement tool for statistical hypothesis testing, and play an important role in power analyses to assess the sample size required for new experiments.[3] Effect size are fundamental in meta-analyses which aim to provide the combined effect size based on data from multiple studies. The cluster of data-analysis methods concerning effect sizes is referred to as estimation statistics.

Effect size is an essential component when evaluating the strength of a statistical claim, and it is the first item (magnitude) in the MAGIC criteria. The standard deviation of the effect size is of critical importance, since it indicates how much uncertainty is included in the measurement. A standard deviation that is too large will make the measurement nearly meaningless. In meta-analysis, where the purpose is to combine multiple effect sizes, the uncertainty in the effect size is used to weigh effect sizes, so that large studies are considered more important than small studies. The uncertainty in the effect size is calculated differently for each type of effect size, but generally only requires knowing the study's sample size (N), or the number of observations (n) in each group.

Reporting effect sizes or estimates thereof (effect estimate [EE], estimate of effect) is considered good practice when presenting empirical research findings in many fields.[4][5] The reporting of effect sizes facilitates the interpretation of the importance of a research result, in contrast to its statistical significance.[6] Effect sizes are particularly prominent in social science and in medical research (where size of treatment effect is important).

Effect sizes may be measured in relative or absolute terms. In relative effect sizes, two groups are directly compared with each other, as in odds ratios and relative risks. For absolute effect sizes, a larger absolute value always indicates a stronger effect. Many types of measurements can be expressed as either absolute or relative, and these can be used together because they convey different information. A prominent task force in the psychology research community made the following recommendation:

Always present effect sizes for primary outcomes...If the units of measurement are meaningful on a practical level (e.g., number of cigarettes smoked per day), then we usually prefer an unstandardized measure (regression coefficient or mean difference) to a standardized measure (r or d).[4]

  1. ^ Kelley, Ken; Preacher, Kristopher J. (2012). "On Effect Size". Psychological Methods. 17 (2): 137–152. doi:10.1037/a0028086. PMID 22545595. S2CID 34152884.
  2. ^ Rosenthal, Robert, H. Cooper, and L. Hedges. "Parametric measures of effect size." The handbook of research synthesis 621 (1994): 231–244. ISBN 978-0871541635
  3. ^ Cohen, J. (2016). "A power primer". In A. E. Kazdin (ed.). Methodological issues and strategies in clinical research (4th ed.). American Psychological Association. pp. 279–284. doi:10.1037/14805-018. ISBN 978-1-4338-2091-5.
  4. ^ a b Wilkinson, Leland (1999). "Statistical methods in psychology journals: Guidelines and explanations". American Psychologist. 54 (8): 594–604. doi:10.1037/0003-066X.54.8.594. S2CID 428023.
  5. ^ Nakagawa, Shinichi; Cuthill, Innes C (2007). "Effect size, confidence interval and statistical significance: a practical guide for biologists". Biological Reviews of the Cambridge Philosophical Society. 82 (4): 591–605. doi:10.1111/j.1469-185X.2007.00027.x. PMID 17944619. S2CID 615371.
  6. ^ Ellis, Paul D. (2010). The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results. Cambridge University Press. ISBN 978-0-521-14246-5.[page needed]