The problem is we are publishing the wrong answer too many times. The bias in journals to publish positive responses (not to mention the drive to create company profits!) means people aren't rewarded for finding out they are wrong.
Fix the incentives, and I think we'll have better science.
The incentives are research companies offering money for successful researchers. Since having failed attempts makes a researcher appear less successful to businesses, they hide it.
Not sure how you would 'fix' those research companies.
One possible way is that, if you are in the position to make decisions based on, or make use of, the research results of others (e.g. you're the FDA, or the journal Nature) you require that companies publish/register the methodology of any research they intend to conduct before it begins. If someone submits/relies on the results of a study which was not pre-registered, the data is ignored.
Any by pre-registering, you can follow up on research that has been silently "forgotten about". If too much research is forgotten about, you stop trusting the results you do hear about.
But: What if they instead do research secretly. Then on research success, they publicly notify the journal of what they intend to do and that it will take 5 months to research. Then 6 months later (an extra month to look plausible), they publish their successful data.
Comments
I think scientists see this as a success. You get data and you revise your hypothesis. You get more data and you revise it again.
A lot of people want science to be like politics. They want you to pick a side and stick to it regardless of the data.
IMO, when conventional wisdom isn't at least occassionally overturneed -- that's when I'll begin to think science is failing us.
The problem is we are publishing the wrong answer too many times. The bias in journals to publish positive responses (not to mention the drive to create company profits!) means people aren't rewarded for finding out they are wrong.
Fix the incentives, and I think we'll have better science.
The incentives are research companies offering money for successful researchers. Since having failed attempts makes a researcher appear less successful to businesses, they hide it.
Not sure how you would 'fix' those research companies.
One possible way is that, if you are in the position to make decisions based on, or make use of, the research results of others (e.g. you're the FDA, or the journal Nature) you require that companies publish/register the methodology of any research they intend to conduct before it begins. If someone submits/relies on the results of a study which was not pre-registered, the data is ignored.
Any by pre-registering, you can follow up on research that has been silently "forgotten about". If too much research is forgotten about, you stop trusting the results you do hear about.
It sounds like a good idea!
But: What if they instead do research secretly. Then on research success, they publicly notify the journal of what they intend to do and that it will take 5 months to research. Then 6 months later (an extra month to look plausible), they publish their successful data.