Jerry Lawson responded today to my Saturday posting about misleading weblog traffic statistics with a Comment here and a similar post at eLawyer Blog. Jerry points out that “The difficulties in measuring web site traffic that you cite are nothing new. I explained some of them in my book.” Based on his experience, Jerry concludes:
“I have no trouble believing that a decent blog can easily attract three times as many readers as most conventional web sites. And yes, I do consider this significant“
garbage in?I readily acknowledge Jerry’s experience and I know the measuring difficulties “are nothing new” (which is why I quoted from an article written in 2000). It’s because the traffic numbers have so many ambiguities that I think using them to suggest the marketing power of weblogs is deceptive. “Three times more readers as most conventional web sites” is simply not a useful claim, when there is no way to tell how many human beings have been going to the comparison static web sites (all you have is the same uncertain traffic numbers) and no way to tell the “quality” of the additional traffic generated by a weblog..
If an ad said “Our medicine starts to work three times faster than our competitor’s product,” the wise consumer would want to know (1) is it three times 1 minute or three times one hour? and (2) is the quicker relief as effective and long-lasting as with the competitor’s product? If the advertiser isn’t willing to tell you the answer to both questions, paying attention to the claim is foolhardy and, although the claim may be “true,” it may be misleading or meaningless.
When this weblog has a 1000 page-hit day, I literally have no idea whether the number represents 20 actual human beings looking in or 200, but I know that it is unlikely to be more than 200. Telling me the number of “unique visitors” does very little to solve this problem, although letting me know how many “visitors” stay more than two seconds might help. The overall traffic numbers alone are, as I stated yesterday, so fraught with uncertainties to be meaningless. The multiplicand may indeed be 3 but just what it is multiplying is a mystery, as is the resulting figure.
Update: Jerry and I have continued our dialogue via the e&hEsq Comment box, which can be reached by following this thread, and his Netlawblog Comments, too. Dave! also chimed in and I hope you will, too.
update (Sept. 10, 2005): For several months now, I’ve been using two (free) services that count “unique visitors.” It appears that actual visitors are about 50% of my “page loads,” and tend to be 6 to 10 percent of “hits” measured by my webserver.
Not everything useful can be quantified with precision.
If an experienced doctor notices that his patients tend to recover significantly faster after being treated with medicine X instead of medicine Y, most won’t care much if the rate of recovery is twice as fast or three times as fast. Similarly, most doctors won’t care if each dose of medicine X kills 1.5 million germs or 2.7 million germs.
Doctors can tell that certain medicines have better effects, even though the precise numbers behind the beneficial phenomenon are uncertain. When someone as experienced as Dennis Kennedy tells me he believes he is seeing three times as much web site traffic since he added a blog to his conventional web site, that means just as much to me as an observation from a skilled doctor that a particular medicine is very effective.
Something significant is happening with blogs, and the difficulty of measuring it precisely is no excuse to ignore it.
Comment by Jerry Lawson — February 16, 2004 @ 11:11 am
Not everything useful can be quantified with precision.
If an experienced doctor notices that his patients tend to recover significantly faster after being treated with medicine X instead of medicine Y, most won’t care much if the rate of recovery is twice as fast or three times as fast. Similarly, most doctors won’t care if each dose of medicine X kills 1.5 million germs or 2.7 million germs.
Doctors can tell that certain medicines have better effects, even though the precise numbers behind the beneficial phenomenon are uncertain. When someone as experienced as Dennis Kennedy tells me he believes he is seeing three times as much web site traffic since he added a blog to his conventional web site, that means just as much to me as an observation from a skilled doctor that a particular medicine is very effective.
Something significant is happening with blogs, and the difficulty of measuring it precisely is no excuse to ignore it.
Comment by Jerry Lawson — February 16, 2004 @ 11:11 am
My point remains: The inability to measure what is happening precisely is no excuse to assume that what is happening is significant for marketing purposes.
Your hypothetical doctor is seeing a result that is valuable for the patient. More traffic is “a result”, but even the most experienced observer cannot say if it has any meaningful impact on a business’ bottom line.
Saying “significant,” “significant”, “significant” is meaningless without saying what the increased traffic is likely to mean for the person who is being sold the marketing pitch.
Comment by David Giacalone — February 16, 2004 @ 12:58 pm
My point remains: The inability to measure what is happening precisely is no excuse to assume that what is happening is significant for marketing purposes.
Your hypothetical doctor is seeing a result that is valuable for the patient. More traffic is “a result”, but even the most experienced observer cannot say if it has any meaningful impact on a business’ bottom line.
Saying “significant,” “significant”, “significant” is meaningless without saying what the increased traffic is likely to mean for the person who is being sold the marketing pitch.
Comment by David Giacalone — February 16, 2004 @ 12:58 pm
You are precisely right. For instance, I read both your blog and Jerry’s, and I do it from my laptop. When I’m at work, it uses one IP, school is another, and home yet another. I’m *three* unique visitors a day. And as people who read blogs also tend to (in my opinion) be somewhat tech savvy, they also may be reading from mulitple locations. In which case, Jerry’s “three times” statistics may be completely and utterly useless, and in fact, misleading. Until there are reliable ways to audit such information, the stats really don’t mean a thing.
-Dave!
Comment by Dave! — February 16, 2004 @ 1:16 pm
You are precisely right. For instance, I read both your blog and Jerry’s, and I do it from my laptop. When I’m at work, it uses one IP, school is another, and home yet another. I’m *three* unique visitors a day. And as people who read blogs also tend to (in my opinion) be somewhat tech savvy, they also may be reading from mulitple locations. In which case, Jerry’s “three times” statistics may be completely and utterly useless, and in fact, misleading. Until there are reliable ways to audit such information, the stats really don’t mean a thing.
-Dave!
Comment by Dave! — February 16, 2004 @ 1:16 pm
Seems to me that ever since man decided that ‘stats’ were a good measure of anything, there has always been a ‘+/-‘ factor built in as a failsafe.
Thus it should be with web traffic too. Of course, no-one should realistically expect 100% accuracy, nor are they getting it with an assertion of something being ‘3 times’ more or greater.
That is about as broad as it could possibly be, it would appear to me!
Steve
Comment by Steve — November 9, 2007 @ 8:12 am