I take from this article that the decline effect is merely an illusion. What initially appears to be a scientific breakthrough is found out to be nothing more than (i) random noise or (ii) hype resulting from publication bias (publish only the positive results and ignore the negative).
I can't seem to find the link at the moment, but someone wrote up a parody patent on something like, Method and Apparatus for Converting Random Data to Publishably Significant Results. Basically involves taking an infinite stream of random data, sampling N points at a time, checking for statistical significance at the p < whatever level, and repeat until you get it. Obviously an unsound statistical procedure, but many corners of science follow something not that dissimilar from it: collect data, analyze it to see if it's interesting, and if not, collect different data and repeat.
Comments
I take from this article that the decline effect is merely an illusion. What initially appears to be a scientific breakthrough is found out to be nothing more than (i) random noise or (ii) hype resulting from publication bias (publish only the positive results and ignore the negative).
I can't seem to find the link at the moment, but someone wrote up a parody patent on something like, Method and Apparatus for Converting Random Data to Publishably Significant Results. Basically involves taking an infinite stream of random data, sampling N points at a time, checking for statistical significance at the p < whatever level, and repeat until you get it. Obviously an unsound statistical procedure, but many corners of science follow something not that dissimilar from it: collect data, analyze it to see if it's interesting, and if not, collect different data and repeat.