The Longest Now

The Kostoff knowledge: Elsevier fakes peer review of COVID click-bait

The Kostoff knowledge v.8

Earlier this month, Elsevier‘s Toxicology Reports (CiteScore 6.2, top quartile) published a special issue on the COVID-19 pandemic.  Its includes a remarkable article by Kostoff, et al., claiming that getting a COVID-19 vaccine is, “extremely conservatively“, 5x as likely to kill people over 65 as it is to save them, and even more harmful to younger people. (Kostoff, et al., Tox. Rep. (2020), 7, 1448-1458)

This echoes the fraudulent claims of German homeopath Harald Walach, who briefly published a similar article in MDPI Vaccines in June, before it was promptly retracted.  A few of the most outrageous claims are listed below. None of this is subtle – unbelievable assertions start in the second paragraph of the abstract; the lead author has no past experience in the field; and the article puts “pandemic” and “vaccine” in scare quotes, and makes regular use of bold italics to emphasize points that are exaggerated.

This is why we have peer review, and editors, to distinguish research from polemic. Access to a reliable + competent body of reviewers is, in theory, a primary service that giant publishers like Elsevier offer to editors. Another is their name: being an Elsevier journal means you will be taken seriously out of the gate, and added to the major indices.

We should all be concerned that our publishing model allowed such a deceptive essay to be given the veneer of legitimacy – for weeks now, without correction.  And we must hold both journals and publishers accountable for fraud that they support or legitimize – through deceptive practice, lack of claimed review, or inaction.

I want to come back to this, and discuss ways to remedy this, and some current steps in the right direction.  But first let’s look at this instance in detail – as the errors were the most obvious that I’ve seen, related papers have been retracted in recent months, and it is impossible to imagine even casual peer review missing them.  And because, as we will see, this particular Elsevier journal has been gaming the system for some time.

Article-level fraud (by the authors)

1. Extensive misuse of VAERS data: VAERS is an open public registry of unvetted self-reports of health events occurring after vaccination. Most events are not caused by vaccines, but this is a starting point for further analysis. Doctors are supposed to report any deaths or hospitalizations occurring within a week of vaccination, regardless of potential causal link.

The very openness of this data has led to it being widely cited in anti-vax propaganda, misinterpreting VAERS as a catalog of known harms and side-effects. (“Don’t Fall for VAERS scares“)

  • 1a: The article mentions that VAERS data is not causal; but then after a brief hand-waving assumes it is causal in all calculations.  (inflating their result by a factor of ~1000)
  • 1b: A 2010 working paper suggested that 1% of all adverse events, and perhaps 3-40% of serious adverse events, temporally following a vaccination, are reported to VAERS. The article cites this to suggest that 1% of deaths are reported, although detailed studies show that most potentially-causal deaths are reported. (adding a factor of 100)
  • 1c: VAERS reports of all kinds fall off quickly in the days after a vaccine is administered. However the article assumes reporting rates are constant over time, leading to a large undercounting of the base rate of non-causal events, and an overcounting of events in the first week as causal. (used to justify the inflations above)

2. Misstating risks of COVID:

  • 2a: The article suggests disingenuously that when COVID-19 is listed alongside other comorbidities on a death certificate, COVID is not the cause of the death; and ignores all such deaths in his calculations. This is a ridiculous claim copied directly from anti-vax propaganda. (adding a factor of ~20)
  • 2b: Throughout the article + appendices the authors drop in vague and counterfactual claims to claim these numbers are conservative, and actual death rates are higher.  These claims only start to make sense if you read anti-vax conspiracy theories (misrepresenting how PCR tests are run, claiming no long-term effects from the disease but long-term effects from vaccines, ignoring the communicability of the disease). (adding a factor of 2-20)

Overall, basic statistics is abused; sources misquoted, and standard knowledge and practice misrepresented, extensively, to confirm a desired result. The topline numbers claimed in the article differ by a factor of 5 million from the best serious estimates of risk/benefit analysis for the vaccines. (Walach, in his previous attempt at the world record for vaccine risk inflation, only managed a fudge factor of 500,000.)

The authors also did not do basic diligence in searching for existing studies of the subject — the potential adverse effects of COVID-19 vaccines have been studied extensively, and good analyses exist.  From the top page of a quick scholarly search:

  • Klein, et al (with a group of statisticians and epidemiologists) published a statistical analysis of 12 million shots: “Surveillance for Adverse Events After COVID-19 mRNA Vaccination“. They found exactly two significant effects – a higher risk of temporary myocarditis the week after the shot, and a lower risk of appendicitis in the 3 weeks after.
  • Canada published a comprehensive overview of the adverse effects from their first 20M vaccination shots, attributing 1 death, and contributions to 3 other deaths to those 20M shots.

Journal-level fraud (by the editors)

How did an article that was problematic by the end of the abstract, and fraudulent just beneath the surface, get published in a modestly successful Elsevier journal?

Background radiation:

  • The editor-in-chief of Toxicology Reports (Aris Tsatsakis) who took over in its fourth year, regularly appears as co-author on articles in his own journal.  The current special issue on COVID, in which Kostoff published two articles, lists Tsatsakis as co-author of 4 of the 8 articles published; three of these listed him as the ‘handling editor’ responsible for recommending acceptance.  (The journal has since issued two three updates, removing his name and instead listing one of his frequent co-authors, though that means little post-acceptance.)
  • Other authors who are published regularly by this journal are unaffiliated researchers who focus exclusively on discrediting the efficacy of vaccines. For instance Neil Z Miller, author of the infamous anti-vax bestseller Miller’s Review of Critical Vaccine Studies, is the sole author of the most-downloaded article in the journal’s recent history.

Editorial misconduct here:

  • The editor-in-chief (Tsatsakis) is a co-author of the paper, and the handling editor who accepted it for publication.  Weeks after publication, he was replaced as handling editor by his frequent co-author Poulas.  Poulas has been an outspoken champion of Walach’s earlier fraudulent analysis.
  • Tsatsakis and another co-author (Calina) are also co-authors of another article in the same volume, evaluating the risks associated with vaccination in the EU, whose assumptions and conclusions directly contradict those in this article. Each article has a title and thesis crafted to appeal to a different audience.
  • Peer review does not seem to be happening.  Most articles in this special issue are by frequent co-authors of the current editors, or by the editors themselves.  Multiple articles in this volume (a low-N ivermectin study whose conclusions misstate its results, a data-mining paper) suffer from methodological, statistical, or textual oversight that even a brief peer review would catch.
  • The direct implication of this paper is that noone, particularly no children or elderly people, should get a COVID-19 vaccine. By all but the most fringe scholarship on the matter, this is a deadly recommendation that will lead to widespread death and chronic illness, and an ongoing degradation in public health.  It was immediately picked up by explicit anti-vax campaigns. There has been no recognition of this potential impact, through an accompanying letter from the editor or some other response to the public outcry against this work.

Publisher-level fraud (by the managers)

In the last three months, we have seen three very high-profile examples of COVID conspiracists publishing dangerously flawed research as fact, under the banners of MDPI, JAMA, and Elsevier – some of the giants of the industry.  Moreover, while extensive complaints by hundreds of readers can sometimes lead to retractions, many flawed papers are never retracted, and the visibility of retractions is always a fraction of that of the initial flawed work.

Not only is there no accountability for the publishers, they may benefit from the attention of anti-vax and anti-science networks. And they directly benefit from the high publication volume, and volume-driven reputation metrics, that lets such fraud thrive.  This is a crisis at a grander scale, defrauding us all of an effective knowledge commons.

How did we get here?

Elsevier runs a popular and profitable journal factory. They encourage people to start new journals. A legitimate and respected scholar founded this one, led it for a few years, then passed on the reigns to someone else. The second lead editor began using it to publish himself and his circle of colleagues. A journal’s peer review is only as good as its reviewers and process, and there is no way to tell if opaque reviews are happening at all. Current models of scholarly recognition and tenure privilege raw publication and citation counts over most other things, so running your own journals (either publishing yourself or publishing people who can publish you) and randomly adding the names of others to your papers, directly advances your career.

Peer review, when done properly, is a scalable way to improve quality, annotation, and context of new research.  But it can also be faked, as a way to grant undue reputability to poor or downright harmful work.

How do we remedy this?

  • We need transparent descriptions by journals of their peer-review process (Transpose!), what responsibility they take for retractions and following up on misinformation they publish.
  • We need accountability for authors, editors, and journals, when major mistakes make it past review – and a step up in accountability when there is intentional fraud or systematic manipulation.
  • We need effective responses to fraud and retraction that invest energy in propagating the retraction proportional to the harm and misinformation propagated by the initial publication. (see NISO’s CORREC recommendation)

Kostoff and co-authors should not be welcome to post other less visibly-deceitful papers without passing a higher bar for diligence and review.  Toxicology Reports should not be included in mainstream metrics until it has changed its editiorial process.  The responsible editors should be held accountable by their co-editors and fellows in their discipline.  Elsevier, as the umbrella publisher giving their work legitimacy, should be accountable for allowing conflict-of-interest editing, poor science, and poor review to continue for years in this journal, until it reached this extreme state.  And metrics maintainers like Scopus and other maintainers of journal metrics should take responsibility for capturing this sort of fraud and manipulation in their metrics, not only after the fact for the small percentage that are caught by the public, but actively (through systemic review) and retrospectively (identifying past patterns that match recently-caught fraud).

There have been a few tiny steps towards this sort of review lately, but for the most part the authors, journals, and publishers have downplayed responsibility, blamed ‘hackers’, and evaded any accountability.

  • This month, Springer Nature added “expressions of concern” to 400 articles in Special Issues of “Arabian Journal of Geosciences” and “Personal and Ubiquitous Computing
  • In July, Elsevier issued “expressions of concern” for 400 articles in Special Issues of Microprocessors & Microsystems
  • In 2019, Scopus stopped indexing two of the four Impact Journals, Oncotarget and Oncoscience, both high-volume publications. However they declined to share a reason, so it is unclear whether the same reasoning might apply to the other two IJ journals (which share an Editor-in-Chief).

3 Comments so far
Leave a comment

Dear Sam
Well done. Far too many crap articles are being published. Didn’t the tone of the article–even its Abstract–raise any suspicions from the reviewers?


Comment by Patrick Moriarty 10.04.21 @ 4:01 pm

[…] Center for Internet and Society at Harvard, began tweeting criticism of the paper, which he has now gathered at his blog. One […]

Pingback by Author Claims COVID-19 Vaccines Kill Five Times More People 65+ - Health News 24 - Health News 10.05.21 @ 3:43 pm

Patrick – judging from the replies of Kostoff and Editor-in-chief Tsatsakis reported by Retraction Watch, they both renewed their insistence that the article is legitimate.

If the reviewers that the editors choose are already anti-vaccine, and interested in publishing work that confirms their beliefs, the tone might have seemed a positive signal. That’s not surprising in itself — there are journals devoted to all sorts of non-science; Infinite Energy Magazine is going to continue to publish reports of perpetual motion machines.

The question is why a top publisher such as Elsevier has health-related journals with such editors.

Comment by metasj 10.07.21 @ 7:34 pm

Leave a comment
Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Bad Behavior has blocked 336 access attempts in the last 7 days.