Yes, that’s a huge issue. Another issue is that the reward for doing peer reviews is far too low, and publishing negative peer reviews comes with the risk of making an enemy in the same field, who might do your next peer review. So you only call out egregiously bad science or just rubber stamp every peer review, because there’s nothing in it for you to publish a negative peer review.
I’ve read meta studies that said that huge amounts of published scientific studies cannot be reproduced. I can’t remember the exact number, but it was >30%.
So if the published science itself is already full of garbage, how is a journalist (who is themselves not a scientist or at least not a scientist in the specific field) know what study is good and what is garbage? And even then, how many people read science journalism compared to boulevard media?
John Bohannon comes to mind, with his purposeful bogus study that claimed that eating chocolate can help with weight loss. He used overfitting and p-hacking to create a study that was purposely garbage and got it published. His goal was to show how easy it is to publish a sensationalist-but-garbage paper. This went so well that every trashy boulevard paper but also many major newspapers ran it, often as a title page news story.
In an interview he said that he got hundreds of calls, all on the level of “Which brand of chocolate helps best?”, and only a single serious inquiry doubting his methods.
He published his own debunk shortly after publishing the original story, it it got pretty much no media attention at all.
He basically couldn’t even recall his own bogus study, and to this day many people worldwide still believe that chocolate can help with weight loss.
Yes, that’s a huge issue. Another issue is that the reward for doing peer reviews is far too low, and publishing negative peer reviews comes with the risk of making an enemy in the same field, who might do your next peer review. So you only call out egregiously bad science or just rubber stamp every peer review, because there’s nothing in it for you to publish a negative peer review.
I’ve read meta studies that said that huge amounts of published scientific studies cannot be reproduced. I can’t remember the exact number, but it was >30%.
So if the published science itself is already full of garbage, how is a journalist (who is themselves not a scientist or at least not a scientist in the specific field) know what study is good and what is garbage? And even then, how many people read science journalism compared to boulevard media?
John Bohannon comes to mind, with his purposeful bogus study that claimed that eating chocolate can help with weight loss. He used overfitting and p-hacking to create a study that was purposely garbage and got it published. His goal was to show how easy it is to publish a sensationalist-but-garbage paper. This went so well that every trashy boulevard paper but also many major newspapers ran it, often as a title page news story.
In an interview he said that he got hundreds of calls, all on the level of “Which brand of chocolate helps best?”, and only a single serious inquiry doubting his methods.
He published his own debunk shortly after publishing the original story, it it got pretty much no media attention at all.
He basically couldn’t even recall his own bogus study, and to this day many people worldwide still believe that chocolate can help with weight loss.