Computational Reproducibility in Finance: Evidence from 1,000 Tests

Abstract We analyze the computational reproducibility of more than 1,000 empirical answers to 6 research questions in finance provided by 168 research teams. Running the researchers’ code on the same raw data regenerates exactly the same results only 52% of the time. Reproducibility is higher for re...

Full description

Saved in:
Bibliographic Details
Published in:The Review of financial studies 2024-06
Main Authors: Pérignon, Christophe, Akmansoy, Olivier, Hurlin, Christophe, Dreber, Anna, Holzmeister, Felix, Huber, Jürgen, Johannesson, Magnus, Kirchler, Michael, Menkveld, Albert J, Razen, Michael, Weitzel, Utz
Format: Article
Language:English
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Abstract We analyze the computational reproducibility of more than 1,000 empirical answers to 6 research questions in finance provided by 168 research teams. Running the researchers’ code on the same raw data regenerates exactly the same results only 52% of the time. Reproducibility is higher for researchers with better coding skills and those exerting more effort. It is lower for more technical research questions, more complex code, and results lying in the tails of the distribution. Researchers exhibit overconfidence when assessing the reproducibility of their own research. We provide guidelines for finance researchers and discuss implementable reproducibility policies for academic journals.
ISSN:0893-9454
1465-7368
1465-7368
DOI:10.1093/rfs/hhae029