Filed under: metrics
Below is a letter that Encyclopedia Britannica sent out today to some of its customers, in response to the December Nature article comparing the accuracy of articles in Wikipedia and Britannica. A more detailed review of the Nature study, including responses to each alleged error and omission, is linked from the front page of www.eb.com; you can also see an HTML version of the review here (thanks to Ben Yates).
In one of its recent issues, the science journal Nature published an article
that claimed to compare the accuracy of the online Encyclopædia Britannica
with Wikipedia, the Internet database that allows anyone, regardless of
knowledge or qualifications, to write and edit articles on any subject.
Wikipedia had recently received attention for its alleged inaccuracies, but
Nature’s article claimed that Britannica’s science coverage was only slightly
more accurate than Wikipedia’s.
Arriving amid the revelations of vandalism and errors in Wikipedia, such a
finding was, not surprisingly, big news. Perhaps you even saw the story
yourself. It’s been reported around the world.
Those reports were wrong, however, because Nature’s research was invalid.
As our editors and scholarly advisers have discovered by reviewing the
research in depth, almost everything about the Nature’s investigation was
wrong and misleading. Dozens of inaccuracies attributed to the Britannica
were not inaccuracies at all, and a number of the articles Nature examined
were not even in the Encyclopædia Britannica. The study was so poorly
carried out and its findings so error-laden that it was completely without merit.
Since educators and librarians have been among Britannica’s closest
colleagues for many years, I would like to address you personally with an
explanation of our findings and tell you the truth about the Nature study.
Almost everything Nature did showed carelessness and indifference to basic
research standards. Their numerous errors and spurious procedures included
the following:
* Rearranging, reediting, and excerpting Britannica articles. Several
of the “articles” Nature sent its outside reviewers were only sections of,
or excerpts from Britannica entries. Some were cut and pasted together
from more than one Britannica article. As a result, Britannica’s coverage
of certain subjects was represented in the study by texts that our editors
never created, approved or even saw.
* Mistakenly identifying inaccuracies. The journal claimed to have
found dozens of inaccuracies in Britannica that didn’t exist.
* Reviewing the wrong texts. They reviewed a number of texts that
were not even in the encyclopedia.
* Failing to check facts. Nature falsely attributed inaccuracies to
Britannica based on statements from its reviewers that were
themselves inaccurate and which Nature’s editors failed to verify.
* Misrepresenting its findings. Even according to Nature’s own
figures, (which grossly exaggerated the number of inaccuracies in
Britannica) Wikipedia had a third more inaccuracies than Britannica.
Yet the headline of the journal’s report concealed this fact and
implied something very different.
Britannica also made repeated attempts to obtain from Nature the original
data on which the study’s conclusions were based. We invited Nature’s
editors and management to meet with us to discuss our analysis, but they
declined.
The Nature study was thoroughly wrong and represented an unfair affront
to Britannica’s reputation.
Britannica practices the kind of sound scholarship and rigorous editorial
work that few organizations even attempt. This is vital in the age of the
Internet, when there is so much inappropriate material available. Today,
having sources like Britannica is more important than ever, with content
that is reliable, tailored to the age of the user, correlated to curriculum,
and safe for everyone.
Whatever may have prompted Nature to do such careless and sloppy
research, it’s now time for them to uphold their commitment to good
science and retract the study immediately. We have urged them strongly
to do so.
Nature responded with a polite declination.
Here’s a response from Nature:
Comment by sj 03.24.06 @ 1:33 am