Read on Twitter
Some interesting (to me) details about the retracted Clark et al. article. One thread per detail.
This is thread #1: The preregistration.
(There may only be one thread, depending on my motivation levels; it& #39;s a beautiful day here and the park beckons.) /1
This is thread #1: The preregistration.
(There may only be one thread, depending on my motivation levels; it& #39;s a beautiful day here and the park beckons.) /1
Clark et al. received Psychological Science& #39;s "preregistered" badge. Here are the criteria for awarding that badge, from the journal& #39;s web site. /2
Let& #39;s look for the results of any preregistered analyses in the article. The strings "prereg" and "pre-reg" find only a few hits. Here& #39;s the first: a keyword! Strong start there. This article is clearly going to be a feast of open science practices. /3
(By the way, that image also contains the date on whiuch the manuscript was received: January 25, 2019. Keep that in your mind.) /4
The next three mentions of "prereg" are in the Methods section, where for some reasons the authors seem to have become somewhat lukewarm about preregistration. ¯\_(ツ)_/¯ /5
Still, there must be plenty of other examples, right? After all, "preregistered" is a keyword. Let& #39;s find the next occurrence of "prereg".
Oh. It& #39;s in the coda.
Still, we get a link to this doubtless extensive preregistration. I might start getting excited again. /6
Oh. It& #39;s in the coda.
Still, we get a link to this doubtless extensive preregistration. I might start getting excited again. /6
Here& #39;s the preregistration. It was "preregistered" on June 3, 2019. That& #39;s 130 days ***after the manuscript was submitted***. It describes a preregistered analysis of... the possible effect of adding an extra explanatory variable to the models that are already in the paper. /7
So the article got the "preregistered" badge (arguably the journal& #39;s problem), and the authors themselves used the keyword "preregistered", on the basis of exactly one exploratory analysis, suggested by the reviewers, and written up in the supplement as having unclear effects. /8
Aside: A friend who works for a large consumer products company once told me that when they make a 10,000-litre batch of their "herbal extract" shampoo, they add 10 millilitres of actual herbal extract to the vat. That& #39;s what this feels like. Semi-homeopathic open science. /9
I hope the reader will forgive me if I say that my reaction to this is more likely to be spontaneous combustion than spontaneous applause. /10 /end
