The Problem With(out) Peer Review

Both the mass media and scientific journals have relaxed their rules about access to information regarding SARS-coV2. Paywalls are down on sites like NY Times and the Washington Post, as well as on scientific journals. Peer review and common sense seems to be waived too.

On March 16, 2020, a pre-print of an article found on MedRxiv tried to make the claim that "people with blood group A have a significantly higher risk for acquiring COVID-19 compared with non-A blood groups, whereas blood group O has a significantly lower risk for the infection compared with non-O blood groups."

This was subsequently reported on MSN, Newsweek, Fox News and more. No doubt people who are type A are starting to freak out even more than they already were. But they really shouldn't be. Here's why...

MedRxiv is a server that allows authors to submit manuscripts that have not been peer reviewed or published for the research community to view. By submitting to the server, the manuscripts will be evaluated by dozens of actual scientific journals which actually have a peer review process. They have a bolded disclaimer on their home page and About Us page that states:

"Preprints are preliminary reports of work that have not been peer-reviewed. They should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information."

So other scientists haven't verified the claims made by the authors. And despite clear guidance from the organizers of MedRxiv not to extrapolate clinical significance from reports submitted to their server, some journalists (likely without any science background at all) began to spread these unverified claims.

Source: BioNinja

Before we dive into an analysis of the paper, it would be good to review ABO blood types. The ABO blood group was discovered by Karl Landsteiner in the early 1900s. The 4 types - A, B, AB, and O - refer to the configuration of specific carbohydrates on the surface of red blood cells. These blood types are genetically determined and do not change over the course of your life.

Now let's look at the actual claims.

"The proportion of blood group A and O in COVID-19 patients were significantly higher and lower, respectively, than that in normal people (both p < 0.001)."

A previous study of 3.8 million Chinese people found the following distribution of the ABO types: A 30.5%, B 29.4%, AB 9.7%, and O 30.4%. Keep that in mind when we look at their data. Here's what they reported:

The authors claim that they did a Chi square test to check for statistically significant differences between the 4 blood groups. The p-value represents the probability of something occurring. We can interpret small p-values as certainty that our hypothesis is correct - that there is a real difference between categories not due to random chance. A p-value < 0.05 is like saying we are 95% certain it's not random chance at play.

Right away, I smelled a rat. The sample size of those infected with the virus was quite small compared to the control group. Ideally, you want as many data points as possible (bigger sample size is better) and you want your experimental and control groups to be similar in size. And eyeballing the data (I've done enough Chi square tests to have a sense for when something isn't going to be significant), it didn't appear to have enough of a difference to warrant a p-value of <0.001 (equivalent to 99.99% certainty). 

So I decided to run a Chi square test of my own (see the chart below). Whether I looked at Wuhan and Shenzhen separately or as aggregated data, there was no significant difference between folks with the virus and the controls for any of the blood groups. Not at a p-value of 0.01 (99% certainty) or a p-value of 0.05 (95% certainty) - and definitely not for the strict p-value they reported.

Wuhan Data from Paper

What are the takeaways here? Should you be freaked out if you have type A blood? No. There's nothing to suggest from this report that you are anymore likely to be infected than anyone else. Should you be angry that people are reporting on preliminary data that hasn't been reviewed by other experts? Yes. I'm all for more transparency in research, for making raw data available to more people more quickly. BUT it's hugely irresponsible for researchers to rush to publish something without actually having data to support the claim they are making AND for journalists without a solid grasp of the content to be raising alarm bells. My final analysis of this research and the lay press reporting on it?