Friday, May 29, 2009

Online Metrics

I have always wondered about numbers and such, say, from NPD where the number of console sales, quarter after quarter (or is it month after month?) end in nice little zeros. Every time. I had no idea that millions of people around the earth would somehow organize their purchasing in order to precisely buy items only in thousands of units. People are so amazing! Unless, of course, it's a... no, it couldn't be! Could it? An... an... estimation?

I have been thinking about this one for a while, and I was away at ICA, but the numerical disparity between Hulu's numbers for Hulu and Nielsen's numbers for Hulu is insane. The Hulu people have their own web server logs, Nielsen does not. I have seen a marketing exec trash Nielsen for lame digital know-how. I'll go with Hulu on the numbers. 
While Nielsen reported 8.9 million visitors to Hulu in March, another measurement firm, comScore, counted 42 million. 
Clearly, one or both of those are hugely wrong. Hugely. (A.k.a. to you purists, "orders of magnitude," there feel better?)
Web publishers are never entirely happy with the online ratings they receive from measurement companies. Their internal numbers, collected via clicks to their servers, are almost always higher than the third-party estimates.
So, do you want actual hard figures from the web server, or do you want an estimate
Nielsen counts video streams by using “beacons,” which inform the company whenever a video starts playing.
So, if I watch Hulu, I somewhere agreed to allow Nielsen to spy on my clickstream? When did I do that? I don't think so! Oh what's that you say, in the EULA that no one ever reads? Maybe? Not sure?

Hulu could just release their server logs. That would solve most issues. Why not? IP addresses, cookie data. Honestly, this stuff is fairly straightforward. Amazing that people get paid to make a mess of it.