p-Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results

Journals tend to publish only statistically significant evidence, creating a scientific record that markedly overstates the size of effects. We provide a new tool that corrects for this bias without requiring access to nonsignificant results. It capitalizes on the fact that the distribution of signi...

Full description

Saved in:
Bibliographic Details
Published in:Perspectives on psychological science 2014-11, Vol.9 (6), p.666-681
Main Authors: Simonsohn, Uri, Nelson, Leif D., Simmons, Joseph P.
Format: Article
Language:English
Subjects:
Arithmetic mean
Computer Simulation
Estimation bias
Meta analysis
P values
Publication Bias
Sample size
Simulations
Special Section on Moving Toward a Cumulative Science: Maximizing What Our Research Can Tell Us
Standard deviation
Statistical significance
Statistics as Topic
Sunk costs
T tests
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Journals tend to publish only statistically significant evidence, creating a scientific record that markedly overstates the size of effects. We provide a new tool that corrects for this bias without requiring access to nonsignificant results. It capitalizes on the fact that the distribution of significant p values, p-curve, is a function of the true underlying effect. Researchers armed only with sample sizes and test results of the published findings can correct for publication bias. We validate the technique with simulations and by reanalyzing data from the Many-Labs Replication project. We demonstrate that p-curve can arrive at conclusions opposite that of existing tools by reanalyzing the meta-analysis of the "choice overload" literature.
ISSN:1745-6916
1745-6924
DOI:10.1177/1745691614553988