The Longest Now

The Kostoff knowledge ^_v.14

^{Updates: Elsevier retraction (5/9), concern (12/17). EIC Tsatsakis removed. (~3/25).
Analyses by Schneider (10/6) & Morris (10/14). Kostoff’s article is top 1% by Altmetric.
K. publishes 3× more extreme version (10/13). Tox.Rep’s CiteScore grows 5% in Oct.
15 of Kostoff’s last 18 papers written w. Tsatsakis, the other 3 in Tsatsakis journals.}

Earlier this month, Elsevier‘s Toxicology Reports ^{(CiteScore 6.4, top quintile)} published a special issue on the COVID-19 pandemic.  Its includes a remarkable article by Kostoff, et al., claiming that getting a COVID-19 vaccine is, “extremely conservatively“, 5x as likely to kill people over 65 as it is to save them, and even more harmful to younger people. ^{(Kostoff, et al., Tox. Rep. (2020), 7, 1448-1458)}

This echoes the fraudulent claims of German homeopath Harald Walach, who briefly published a similar article in MDPI Vaccines in June, before it was promptly retracted.  A few of the most outrageous claims are listed below. None of this is subtle – unbelievable assertions start in the second paragraph of the abstract; the lead author has no past experience in the field; and the article puts “pandemic” and “vaccine” in scare quotes, and makes regular use of bold italics to emphasize points that are exaggerated.

This is why we have peer review, and editors, to distinguish research from polemic. Access to a reliable + competent body of reviewers is, in theory, a primary service that giant publishers like Elsevier offer to editors. Another is their name: being an Elsevier journal means you will be taken seriously out of the gate, and added to the major indices.

We should all be concerned that our publishing model allowed such a deceptive essay to be given the veneer of legitimacy – for weeks now, without correction.  And we must hold both journals and publishers accountable for fraud that they support or legitimize – through deceptive practice, lack of claimed review, or inaction.

I want to come back to this, and discuss ways to remedy this, and some current steps in the right direction.  But first let’s look at this instance in detail – as the errors were the most obvious that I’ve seen, related papers have been retracted in recent months, and it is impossible to imagine even casual peer review missing them.  And because, as we will see, this particular Elsevier journal has been gaming the system for some time.

Article-level fraud (by the authors)

1. Extensive misuse of VAERS data: VAERS is an open public registry of unvetted self-reports of health events occurring after vaccination. Most events are not caused by vaccines, but this is a starting point for further analysis. Doctors are supposed to report any deaths or hospitalizations occurring within a week of vaccination, regardless of potential causal link.

The very openness of this data has led to it being widely cited in anti-vax propaganda, misinterpreting VAERS as a catalog of known harms and side-effects. ^{(“Don’t Fall for VAERS scares“)}

• 1a: The article mentions that VAERS data is not causal; but then after a brief hand-waving assumes it is causal in the calculations. VAERS reports are not causal: most reported events have no connection to getting vaccinated (this mistake inflates their risk calculation by a factor of ~1000)
• 1b: A 2010 working paper suggested that 1% of all adverse events, and perhaps 3-40% of serious adverse events, temporally following a vaccination, are reported to VAERS. The article cites this to suggest that 1% of causal deaths are reported, although detailed studies show that most potentially-causal deaths are reported. (this mistake multiplies their calculated risk by a factor of 100)
• 1c: VAERS reports of all kinds fall off quickly in the days after a vaccine is administered. However the article assumes reporting rates are constant over time, leading to a large undercounting of the base rate of non-causal events, and an overcounting of events in the first week as causal. (this mistake is used to justify the inflations above)

2. Misstating risks of COVID:

• 2a: The article suggests that when COVID-19 is listed alongside other comorbidities on a death certificate, COVID is not the cause of the death; and ignores all such deaths in calculations. This claim is false on its surface, and seems copied directly from anti-vax propaganda. (this mistake multiplies their calculated risk by a factor of ~20)
• 2b: Throughout the article + appendices the authors drop in vague and counterfactual claims to claim these numbers are conservative, and actual death rates are higher.  These claims only start to make sense if you read anti-vax conspiracy theories. (They misstate how PCR tests are run, they claim no long-term effects from COVID (which has crippling and well-documented long term effects), they claim terrible and unknown long-term effects from vaccines (which are designed precisely to avoid those effects), they ignore the communicability of the disease & related public health impact). (these mistakes add a factor of 2-20)

Overall, this is an exercise in motivated reasoning. Basic statistics is misused, sources misquoted, and standard knowledge and practice misrepresented, to confirm a desired result. The topline numbers claimed in the article differ by a factor of 5 million from the best serious estimates of risk/benefit analysis for the vaccines. (Walach, in his previous retracted paper, only managed a fudge factor of 500,000.)

The authors, in compiling this pure thought experiment, also do not review or reference existing studies of the subject. The potential adverse effects of COVID-19 vaccines is an important topic, which has been studied extensively by epidemiologists and virologists (which none of these authors are). From the top page of a quick scholarly search:

• Klein, et al (with a group of statisticians and epidemiologists) published a statistical analysis of 12 million shots: “Surveillance for Adverse Events After COVID-19 mRNA Vaccination“. They found exactly two significant effects – a higher risk of temporary myocarditis the week after the shot, and a lower risk of appendicitis in the 3 weeks after.
• Canada published a comprehensive overview of the adverse effects from their first 20M vaccination shots, attributing 1 death, and contributions to 3 other deaths to those 20M shots.

Journal-level fraud (by the editors)

How did an article that was problematic by the end of the abstract, and fraudulent just beneath the surface, get published in a modestly successful Elsevier journal?

Background radiation:

• The editor-in-chief of Toxicology Reports (Aris Tsatsakis) who took over in its fourth year, regularly appears as co-author on articles in his own journal.  The current special issue on COVID, in which Kostoff published two articles, lists Tsatsakis as co-author of 4 of the 8 articles published; three of these listed him as the ‘handling editor’ responsible for recommending acceptance.  (The journal has since issued two three updates, removing his name and instead listing one of his frequent co-authors, though that means little post-acceptance.)
• This has happened before. In May, 2020, Kostoff and Tsatsakis published a similarly conspiracy-tinged thought experiment about the harms of 5G mobile networks, titled “Adverse health effects of 5G mobile networking technology under real-life conditions”.  It was published in a special issue of Toxicology Letters, a sister journal to Tox.Rep., also edited by Tsatsakis.
• This has happened before… a lot. Kostoff wrote 14 of his last 17 papers with Tsatsakis, all published in journals Tsatsakis is involved with.
• Other authors published regularly by Tsatsakis are independent researchers who focus exclusively on discrediting the efficacy of vaccines. For instance Neil Z. Miller, author of anti-vax bestseller Miller’s Review of Critical Vaccine Studies, is the sole author of the most-downloaded article in the journal’s history.

Editorial misconduct here:

• The editor-in-chief (Tsatsakis) is a co-author of the paper, and the handling editor who accepted it for publication.  Weeks after publication, he was replaced as handling editor by his frequent co-author Poulas.  Poulas has been an outspoken champion of Walach’s earlier fraudulent analysis.
• Tsatsakis and another co-author (Calina) are also co-authors of another article in the same volume, evaluating the risks associated with vaccination in the EU, whose assumptions and conclusions directly contradict those in this article. Each article has a title and thesis crafted to appeal to a different audience.
• Peer review does not seem to be happening.  Most articles in this special issue are by frequent co-authors of the current editors, or by the editors themselves.  Multiple articles in this volume (a low-N ivermectin study whose conclusions misstate its results, a data-mining paper) suffer from methodological, statistical, or textual oversight that even a brief peer review would catch.
• The direct implication of this paper is that noone, particularly no children or elderly people, should get a COVID-19 vaccine. By all but the most fringe scholarship on the matter, this is a deadly recommendation that will lead to widespread death and chronic illness, and an ongoing degradation in public health.  It was immediately picked up by explicit anti-vax campaigns. There has been no recognition of this potential impact, through an accompanying letter from the editor or some other response to the public outcry against this work.

Publisher-level fraud (by the managers)

In the last three months, we have seen three very high-profile examples of COVID conspiracists publishing dangerously flawed research as fact, under the banners of MDPI, JAMA, and Elsevier – some of the giants of the industry.  Moreover, while extensive complaints by hundreds of readers can sometimes lead to retractions, many flawed papers are never retracted, and the visibility of retractions is always a fraction of that of the initial flawed work.

Not only is there no accountability for the publishers, they may benefit from the attention of anti-vax and anti-science networks. And they directly benefit from the high publication volume, and volume-driven reputation metrics, that lets such fraud thrive.  This is a crisis at a grander scale, defrauding us all of an effective knowledge commons.

How did we get here?

Elsevier runs a popular and profitable journal factory. They encourage people to start new journals. A legitimate and respected scholar founded this one, led it for a few years, then passed on the reigns to someone else. The second lead editor began using it to publish himself and his circle of colleagues. A journal’s peer review is only as good as its reviewers and process, and there is no way to tell if opaque reviews are happening at all. Current models of scholarly recognition and tenure privilege raw publication and citation counts over most other things, so running your own journals (either publishing yourself or publishing people who can publish you) and randomly adding the names of others to your papers, directly advances your career.

Peer review, when done properly, is a scalable way to improve quality, annotation, and context of new research.  But it can also be faked, as a way to grant undue reputability to poor or downright harmful work.

How do we remedy this?

• We need transparent descriptions by journals of their peer-review process (Transpose!), what responsibility they take for retractions and following up on misinformation they publish.
• We need accountability for authors, editors, and journals, when major mistakes make it past review – and a step up in accountability when there is intentional fraud or systematic manipulation.
• We need effective responses to fraud and retraction that invest energy in propagating the retraction proportional to the harm and misinformation propagated by the initial publication. (see NISO’s CORREC recommendation)

Kostoff and co-authors should not be welcome to post other less visibly-deceitful papers without passing a higher bar for diligence and review.  Toxicology Reports should not be included in mainstream metrics until it has changed its editiorial process.  The responsible editors should be held accountable by their co-editors and fellows in their discipline.  Elsevier, as the umbrella publisher giving their work legitimacy, should be accountable for allowing conflict-of-interest editing, poor science, and poor review to continue for years in this journal, until it reached this extreme state.  And metrics maintainers like Scopus and other maintainers of journal metrics should take responsibility for capturing this sort of fraud and manipulation in their metrics, not only after the fact for the small percentage that are caught by the public, but actively (through systemic review) and retrospectively (identifying past patterns that match recently-caught fraud).

There have been a few tiny steps towards this sort of review lately, but for the most part the authors, journals, and publishers have downplayed responsibility, blamed ‘hackers’, and evaded any accountability.

• This month, Springer Nature added “expressions of concern” to 400 articles in Special Issues of “Arabian Journal of Geosciences” and “Personal and Ubiquitous Computing“
• In July, Elsevier issued “expressions of concern” for 400 articles in Special Issues of Microprocessors & Microsystems
• In 2019, Scopus stopped indexing two of the four Impact Journals, Oncotarget and Oncoscience, both high-volume publications. However they declined to share a reason, so it is unclear whether the same reasoning might apply to the other two IJ journals (which share an Editor-in-Chief).