Monday, April 7, 2008

Recent NPD Data - The danger of PR - When figures just don't look right (Thanks Regan and Dino for contributing the cited articles)

NPD recently reported that 72% of Americans are gamers:

http://www.gamasutra.com/php-bin/news_index.php?story=18107

This data was widely reported on the web before people began asking the obvious question - is this really believable? Soon the PR was being corrected:

http://www.gamedaily.com/articles/news/npd-72-of-us-plays-games-online-gaming-still-relatively-small/?biz=1


Indeed the original story did not square at all with what we've seen in 4-5 quarters of Gameasure and people began to ask questions about NPD's findings. Gameasure typically finds that just under half of Americans aged 12-65 have played video games. The question in Gameasure is very broad and asks people if they have ever played video games on every conceivable platform. They only need to have played a video game once to qualify. They do not need to play regularly. Adding in a requirement that they play for at least an hour per week, the percentage drops to 46% in Wave 4. This is a far more realistic figure than the 72% originally reported by NPD, and closer to the revised figure of 59% (still too high if you ask me). A figure closer to 2 rather than 3 out of 4 people seems more realistic, and is backed by numerous waves of data in Gameasure.

The only data point that reaches near the higher NPD figures in Gameasure was the casual gaming data gathered in Wave 4. When asked if they had ever played a casual video game, 68% of people said they had at some point. We only have one data point right now, but will soon be able to see if this was an anomaly. If it is not an anomaly, a large proportion of people are identifying with PC based casual games, but not with video games. Nonetheless it also points to how misleading the data can be. It is likely that the 22% of Americans who are in-between active gamers (46%) and total casual gamers (68%) don't have much idea about video games - at least not yet.

Some of the figures are close to ours. NPD finds that 3% of Americans own 2 out of 3 current generation consoles (still called 'next-gen' in article). Gameasure finds this is as high as 3.6%. On the other hand Gameasure only finds that 1% own all three consoles, not 2%. The 1% was also an unstable base size, 84 respondents out of 9,089.

The NPD press release shows how misleading improperly defined market research can be, especially once it is filtered through a press release. The report was successful at gaining press, but likely resulted in many people questioning the credibility of NPD's data. There does not appear to have been an objective look at the numbers before press release. It's important to step back and see oneself as the client - is what you have to report really believable?

No comments: