health

Fake research can be harmful to your health; a new study offers a tool for rooting it out

10 Comments
By Lisa Bero

The requested article has expired, and is no longer available. Any related articles, and user comments are shown below.

© The Conversation

©2024 GPlusMedia Inc.

10 Comments
Login to comment

Unfortunately, reproducibility problems plague many scientific studies, especially in the medical field. The somewhat extreme essay by Ioannidis (https://en.wikipedia.org/wiki/Why_Most_Published_Research_Findings_Are_False) did create a lot of debate, and several statistical analysis methods have been established to detect suspicious data sets. I believe AI reviewing will become a big tool in the coming years.

The good thing is that science is quite "self-healing", and poor science is gradually discarded and updated with genuine and reproducible results.

6 ( +6 / -0 )

For example, ivermectin, an antiparasitic drug that is typically used in veterinary medicine and that was promoted by some without evidence as a treatment for COVID-19, was widely embraced in some parts of the world. However, after ruling out fake or flawed studies, a systematic review of research on ivermectin found that it had “no beneficial effects for people with COVID-19.”

On the other hand, a systematic review of corticosteroid drugs like dexamethasone found that the drugs help prevent death when used as a treatment for COVID-19.

A very clear piece of information that demonstrate without room for doubts how fake, unreliable or low quality research ends up being discarded from the scientific literature. It also proves the usual excuse (that this is done to prevent cheap drugs to be used instead of more expensive ones, since dexamethasone is dirt cheap and it never had this problem).

It is not a coincidence that at least 10 different papers about ivermectin use for covid have been retracted after findings of unreliable data, fabrication or falsification.

0 ( +5 / -5 )

Peer review is not designed to catch fake or misleading data.

Umm…then what exactly is peer review designed to catch? The point of peer review is to weed out shoddy research. Fake and misleading data is one of the major reasons for bad research. If peer review isn’t catching that, then what use is peer review?

In the current state of medical research, between half and two-thirds of all published studies in prestigious journals (never mind the truly awful publication mills) prove false. For the general public, this means we are better off doing literally the opposite of what studies recommend.

Beyond this, most people are not equipped to sniff out the statistical shenanigans, data manipulation, and other games played in sham research. Relying on studies not funded by pharmaceutical companies or on the clinical trial being registered is not enough. Researchers are under intense pressure to publish positive results. Careers and grant funding can go down the tube if research fails to produce results.

The entire academic industry is plagued with this pressure to publish, and thus there is extremely high motivation to fake results and low motivation by peers to point out the fakery.

5 ( +7 / -2 )

then what exactly is peer review designed to catch?

It can vary by publication, but generally it is intended to check that proper scientific methods were followed and that any conclusions from a study are logical. What it generally can't do is check that the data from experiments is valid. That would require conducting the entire study again which is generally not practical.

The systematic review approach described in the article is probably more appropriate for medical research where the consequences of bad research can be potentially dangerous.

2 ( +3 / -1 )

Systematic reviews require tenacious efforts but are a prerequisite process to prevent P-hacking.

Thanks for all the researchers' perseverance. Hopefully, they are decently paid.

0 ( +1 / -1 )

Vicky, peer review is done on a voluntary basis, and it is not financially compensated. In most of the cases it is also anonymous, even if recently journals are thinking about ways to give more recognition to the reviewers. Science in academia will not get you rich...

3 ( +3 / -0 )

Umm…then what exactly is peer review designed to catch? 

Invalid methods, conclusions not supported by the evidence presented, etc.

The "peers" are scientists, not detectives, unless the article is about how to falsify data or images it is unlikely that the peers have any capacity to detect these kind of problems.

 If peer review isn’t catching that, then what use is peer review?

Discard manuscripts where the methodology is not adequate, the evidence is no enough to support a conclusion made, the discussion do not include pertinent information that contradicts the report, etc. etc.

The entire academic industry is plagued with this pressure to publish, and thus there is extremely high motivation to fake results and low motivation by peers to point out the fakery.

There is plenty of motivation by peers to point out anything suspicious that they can find, a bad actor losing a prized postion means someone else now have a chance to fill that spot, same with a grant, project, collaboration, etc. What not many people have is the capacity to do it.

The problem is expecting pre-publication peer review to be what detects fake research, the best tool for that is post-publication peer review, many people can fool 3 reviewers for a few months, nobody can fool the entire scientific community forever.

Pubpeer is showing how powerful this tool is, with hundreds of correction and retractions done thanks to anonymous people that see a gel being spliced, exactly the same cells in several panels or a table with impossible numbers.

-1 ( +1 / -2 )

then what exactly is peer review designed to catch?

It can vary by publication, but generally it is intended to check that proper scientific methods were followed and that any conclusions from a study are logical.

Yeah, that is the intention.

One problem however is that authors can recommend reviewers, as well as requesting that certain reviewers be avoided. I understand the reasons for this, but I suspect it is frequently taken advantage of to ensure friendly reviewers.

-1 ( +1 / -2 )

For example, ivermectin, an antiparasitic drug that is typically used in veterinary medicine and that was promoted by some without evidence as a treatment for COVID-19, was widely embraced in some parts of the world. However, after ruling out fake or flawed studies, a systematic review of research on ivermectin found that it had “no beneficial effects for people with COVID-19.”

Interesting that they would choose ivermectin as an example, and the way they misrepresent it considering that many studies (including several RCTs) show positive results. If they really wanted an example of fake research they should have used the "Lancet gate" paper, a completely made up study, yet highly publicized because it wrongly showed HCQ to be ineffective and dangerous. But considering this is a The Conversation report, I am not surprised...

-1 ( +1 / -2 )

One problem however is that authors can recommend reviewers, as well as requesting that certain reviewers be avoided. I understand the reasons for this, but I suspect it is frequently taken advantage of to ensure friendly reviewers.

Which is precisely why this is no longer the norm and many journal specifically call for editors to avoid it.

Interesting that they would choose ivermectin as an example, and the way they misrepresent it considering that many studies (including several RCTs) show positive results.

Low quality studies, which mean they are a much less reliable source of information than the best possibly made studies that clearly show it has no use.

The studies that showed the strongest effect have instead been retracted when it was found that the data was fabricated or falsified, which is exactly what the article is talking about. A systematic review is not only collecting studies and giving them all the same weight, it requires specific rules to include and give value to each study being included so a valid metanalysis can be done, this has proved beyond any rational doubt that ivermectin is worthless against covid, and so it became the scientific consensus.

When at least 10 different papers about ivermectin had to be retracted (something that usually is a very rare occurrence) it becomes clear the evidence that many tried to use to push for its use was just fake.

-1 ( +1 / -2 )

Login to leave a comment

Facebook users

Use your Facebook account to login or register with JapanToday. By doing so, you will also receive an email inviting you to receive our news alerts.

Facebook Connect

Login with your JapanToday account

User registration

Articles, Offers & Useful Resources

A mix of what's trending on our other sites