BOX INFORMATIONSUPPLEMENTARY FILES FOR
Uri SImonsohn, ' Hyping Fisher: The Most Cited 2019 QJE Paper Relied on an Outdated STATA Default to Conclude Regression p-values Are Inadequate', Data Coladahttps://datacolada.org/99LICENSE FOR USE
All content posted to ResearchBox is under a
CC By 4.0 License
(all use is allowed as long as authorship of the content is attributed). When using content from ResearchBox please
cite the original work, and provide a link to the URL for this box (https://researchbox.org/431).
BOX PUBLIC SINCE
October 12, 2021 (files may not be changed, deleted, or added
Uri Simonsohn (firstname.lastname@example.org) ABSTRACT
The paper titled "Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results" (.htm) is currently the most cited 2019 article in the Quarterly Journal of Economics (372 Google cites). It delivers bad news to economists running experiments: their p-values are wrong. To get correct p-values, the article explains, they need to run, instead of regressions, something called "randomization tests" (I briefly describe randomization tests in the next subsection). For example, the QJE article reads: "[the results] show the clear advantages of randomization inference . . . [it] is superior to all other methods" (p.585). In this post I show that this conclusion only holds when relying on an unfortunate default setting in STATA. In contrast, when regression results are computed using the default setting in R, the supposed superiority of the randomization test goes away.