Nielsen Ratings: Fact, Fiction or Wishful Thinking? at DVinfo.net

James Emory · June 29th, 2004, 06:25 PM

The question I have is how could supposedly educated, successful corporate leaders continue to buy into this method of viewership measurement? The link below is detailed about how this system works, but the fact that the words estimated and theoretically are used so often when the decision to spend hundreds or millions on an airtime slot is at stake is troubling. Almost only counts in horseshoes and hand grenades. According to the info, around 5000 peoples' viewing habits are monitored to determine what the rest of us like. It is just aggravating and insulting to hear reports that 20 million people watched the Super Bowl when they truly don't know just how many did. They should say ALOT of people watched. But these marketing people just keep buying into it so I guess they deserve the 2.5 million dollar bill for their :30. How could you take someone seriously that even tried to give you a viewership number like that without literally visiting millions of homes for verification? Radio is the same situation. At least the print industry can say, we have this many subscribers, but that doesn't mean that all of them see your ad. Why don't they try to put a black box in as many homes as possible to get the most accurate data? Are they afraid of the truth? Is this not a racket? Give me a break!!!

www.nielsenmedia.com/whatratingsmean

www.nielsenmedia.com

Keith Loh · June 29th, 2004, 08:07 PM

James, 5000 is a statistically large amount. Federal election polls are frequently around 1000 people.

//don't they try to put a black box in as many homes as possible to get the most accurate data? //

Cost and the probability that the accuracy won't go up. It's been awhile since I've taken a statistics course but I was convinced then that pools really don't have to be all that large.

That said, pollsters were totally taken aback when the results of last night's Canadian Federal election came in and took a swing at their credibility. But in that case, the pollsters are recording people's opinions which isn't the same as actually recording activity in the case of the Nielsen families.

Back to the Nielsen's. Broadcasters have long recognized that the Nielsen Family Ratings aren't the be all and end all. That is why they do a hell of a lot of testing including audience testing, focus groups and gathering other data like letters from fans, analyzing DVD and other merchandising sales, monitoring boards and taking polls of their own.

You should know that the Nielsen's and other similar viewer habit services are influential in more than just recording the mass numbers, they also record when audiences drop off. I attended a seminar where a TV show runner told us she knew exactly when her show was in trouble because the numbers began dropping off at a certain time during the show. So then she knew she had to rejig the format of it. So even if you don't trust the audience totals, the reporting at least can tell you patterns during the show.

Rob Lohman · June 30th, 2004, 03:22 AM

Keith: 5000 may be a statistically large amount but that ofcourse
totally depends on which 5000 people they are using. In other
words, is the spread of types of people in real life the same in
the 5000 people group?

Anyways I'm outside the states so what do I know on this <g>

Rob Belics · June 30th, 2004, 08:09 AM

Keith is correct. Interviews are done, I believe, with those selected for the ratings so they fit a profile but it is known that the accuracy of the system does not improve much with the number of people used for the survey.

Barry Gribble · June 30th, 2004, 09:50 AM

James,

The math involved is rather complex, but the fact of the matter is that if you are trying to establish the average opions of one million people and you select 10 at random, you are going to get inaccurate results. However, if you select 1000 truly at random and get answers from them then you get pretty darned close to the average opinions of the million. And if you go to 10,000 instead, it doesn't actually get much closer.

You get problems, of course, because the subset of people willing to allow Nielson in to their house is not a perfectly representative set of the general viewing audience, but there are ways to normalize that out to reduce the error.

Fundamentally there is soooo much money at stake, that I have to believe that the networks trust Nielson's methodology and are not just giving them control of billions of dollars out of tradition. And the networks certainly know more than you or I about the accuracy.

I admit that it does seem strange tho....

I am guessing that the Nielson numbers are more useful in determining relative numbers than absolute ones.

There is a great entry on Gallup's website about sample size and polling methods that has some applicability here... you may want to check it out - it is a great overview of general polling:

http://www.gallup.com/help/FAQs/poll1.asp

also, a Wired article on Nielson:
http://www.wired.com/news/print/0,1294,63080,00.html

and an article about the recent uproar over a Nielson methodology change that some say is going to undercount minorities:
http://www.plastic.com/article.html;...06/05/14031275

Rob Belics · June 30th, 2004, 12:02 PM

And what is the alternative? You can't poll everyone so that leaves guessing. Neither of which would be acceptable.

James Emory · June 30th, 2004, 03:01 PM

Guessing is just what this ratings thing is. These folks are trusting that the statistical info is correct. They deserve to get financially screwed if they are that naive. I think it's safe to say that there are experts in every industry. They are often used in court to decide someone's guilt or innocence. The average person doesn't understand how DNA is processed for evidence but we trust these experts which they MAY very well be. This is just for argument and for no particular procedure, but what if they were wrong about a procedure and the guilty were released and innocent ones were held. Experts have been found to be wrong you know. I guess the question is this. If you were a potential advertiser, and a sales person said that there was a space available for $100,000 for :30 would you write them a check on, "well 10 million people saw last week's episode." I would send them right out the front door. Why don't they just go ahead and say 20 million people saw it and get $500,000 out of these suckers because the truth is NOONE knows the true number of viewers! It should be done accurately or not at all.

Keith Loh · June 30th, 2004, 04:07 PM

James you wrote:
//It should be done accurately or not at all.//

And it has been already said in this thread that it would be a question of resources and statistically it doesn't improve the accuracy much. The alternative, guessing, as has already been said, is guessing.

You're babbling about 'experts' and 'DNA'. Stay on the topic.

In any case, we may be getting to toward the 'box in every home' idea when more and more people install set-top boxes, satellites, (pay-per-view obviously gives the exact data) which *can* track viewing patterns. And then, as now, you will have people wanting to opt out because they are not crazy about having some corporation know what they are viewing.

James Emory · June 30th, 2004, 04:45 PM

I am not babbling, I am rambling! I only used DNA as an example of a complex test that not everyone understands but we trust as we're told by the experts, just like this viewer statistics racket.

As far as a black box recording viewing habits, didn't Tivo get in trouble a couple of years ago for recording its subscribers' viewing habits without telling them. They said something like they never assigned the stats to individual accounts or identities so noone's privacy was violated.

Keith Loh · June 30th, 2004, 05:02 PM

DNA test methodology are open to scientific review. In the past year there have been more than a handful of articles publiched in New Scientist about DNA testing so you don't have to be an expert to be able to look back and see what the issues are.

We are both recalling the same TiVo story. That certainly is an issue, especially as not everyone views the same 'safe' programmes that others do. I certainly wouldn't want John Ashcroft knowing that my viewing patterns and it has been shown that the DoJ and other agencies has been able to get that information from corporations without much of a fight (I'm speaking of the airlines specifically).

Anyway, I don't see how your criticisms can be satisfied. If you don't like the methodology and there are privacy concerns with tracking all viewership that leaves you with notions and guesses.

Where exactly were you going with all of this? Were you pitching a show and some executive threw back negative data?

James Emory · June 30th, 2004, 07:36 PM

Keith said "Where exactly were you going with all of this? Were you pitching a show and some executive threw back negative data?"

No, I just think it's a joke. Especially how millions of dollars are spent on what ifs. If I had proposed a plan that said I am going to tell you how many people watched your show by only monitoring 5000 out of a potential 15 million + viewers and then dare to put a million dollar price tag on it, I would have been laughed out of the room. I think it's probably the best of the worst estimated, maybe, guess method available.

Like I said earlier, if you were a potential advertiser, and a sales person said that there was a space available for $100,000 for :30 would you write them a check on, "well 10 million people saw last week's episode" and hope that they were right. I'd have to say, go visit those 10 million homes and then show me the reports with the signatures of each viewer next week and then I'll pay you. Ha, if they actually had to do that, then it just wouldn't get done! Why should they make any extra effort when it's too easy to estimate and/or guess and take these people's money.

They sure didn't ask me what I was watching!

Rob Belics · June 30th, 2004, 08:36 PM

James,

These audience numbers are based on statistics, one of the mathematical sciences. This is nothing new and a proven method.

There are actual formulas created to determine these results and prove their accuracy. You can go to the library and find books on the topic.

Search the web and you'll find sites that explain how it all works. It's not a hidden topic and the method is easy to learn, at least if you're into that sort of thing.

Michael Wisniewski · June 30th, 2004, 08:42 PM

Also, I'm sure the Nielsen ratings are just a part of the data that the advertisers use to figure out how much different time slots are worth.

A sample of 5000 homes could get you a pretty good guess as to what's going on if you cross reference it against all kinds of other viewer statistics.

For example, start with U.S. Census data, cross reference against historical sales and inventory data from companies, and then throw in the Nielsen data. That would give you a pretty good estimate.

I used to work with a statistical data company that would buy all kinds of databases from hundreds of public, private, and governmental institutions. They could probably figure out any trend within an acceptable range of error.

Jeff Donald · June 30th, 2004, 11:23 PM

the last time I checked the Law of Large Numbers was still in effect.

James Emory · July 1st, 2004, 12:14 AM

I don't think there's much more I can say without repeating myself again. One thing is for certain though. I'm not putting any of my millions on that ratings method anytime soon!!

June 29th, 2004, 06:25 PM	#1
James Emory Regular Crew Join Date: Aug 2002 Location: Atlanta, Georgia Posts: 1,892	Nielsen Ratings: Fact, Fiction or Wishful Thinking? The question I have is how could supposedly educated, successful corporate leaders continue to buy into this method of viewership measurement? The link below is detailed about how this system works, but the fact that the words estimated and theoretically are used so often when the decision to spend hundreds or millions on an airtime slot is at stake is troubling. Almost only counts in horseshoes and hand grenades. According to the info, around 5000 peoples' viewing habits are monitored to determine what the rest of us like. It is just aggravating and insulting to hear reports that 20 million people watched the Super Bowl when they truly don't know just how many did. They should say ALOT of people watched. But these marketing people just keep buying into it so I guess they deserve the 2.5 million dollar bill for their :30. How could you take someone seriously that even tried to give you a viewership number like that without literally visiting millions of homes for verification? Radio is the same situation. At least the print industry can say, we have this many subscribers, but that doesn't mean that all of them see your ad. Why don't they try to put a black box in as many homes as possible to get the most accurate data? Are they afraid of the truth? Is this not a racket? Give me a break!!! www.nielsenmedia.com/whatratingsmean www.nielsenmedia.com

June 29th, 2004, 08:07 PM	#2
Keith Loh Air China Pilot Join Date: May 2002 Location: Vancouver, B.C. Posts: 2,389	James, 5000 is a statistically large amount. Federal election polls are frequently around 1000 people. //don't they try to put a black box in as many homes as possible to get the most accurate data? // Cost and the probability that the accuracy won't go up. It's been awhile since I've taken a statistics course but I was convinced then that pools really don't have to be all that large. That said, pollsters were totally taken aback when the results of last night's Canadian Federal election came in and took a swing at their credibility. But in that case, the pollsters are recording people's opinions which isn't the same as actually recording activity in the case of the Nielsen families. Back to the Nielsen's. Broadcasters have long recognized that the Nielsen Family Ratings aren't the be all and end all. That is why they do a hell of a lot of testing including audience testing, focus groups and gathering other data like letters from fans, analyzing DVD and other merchandising sales, monitoring boards and taking polls of their own. You should know that the Nielsen's and other similar viewer habit services are influential in more than just recording the mass numbers, they also record when audiences drop off. I attended a seminar where a TV show runner told us she knew exactly when her show was in trouble because the numbers began dropping off at a certain time during the show. So then she knew she had to rejig the format of it. So even if you don't trust the audience totals, the reporting at least can tell you patterns during the show. __________________ -- Visit http://www.KeithLoh.com \| stuff about living in Vancouver \| My Flickr photo gallery

June 30th, 2004, 03:22 AM	#3
Rob Lohman RED Code Chef Join Date: Oct 2001 Location: Holland Posts: 12,514	Keith: 5000 may be a statistically large amount but that ofcourse totally depends on which 5000 people they are using. In other words, is the spread of types of people in real life the same in the 5000 people group? Anyways I'm outside the states so what do I know on this <g> __________________ Rob Lohman, visuar@iname.com DV Info Wrangler & RED Code Chef Join the DV Challenge \| Lady X Search DVinfo.net for quick answers \| Buy from the best: DVinfo.net sponsors

June 30th, 2004, 09:50 AM	#5
Barry Gribble Major Player Join Date: Aug 2003 Location: Los Angeles, CA Posts: 817	James, The math involved is rather complex, but the fact of the matter is that if you are trying to establish the average opions of one million people and you select 10 at random, you are going to get inaccurate results. However, if you select 1000 truly at random and get answers from them then you get pretty darned close to the average opinions of the million. And if you go to 10,000 instead, it doesn't actually get much closer. You get problems, of course, because the subset of people willing to allow Nielson in to their house is not a perfectly representative set of the general viewing audience, but there are ways to normalize that out to reduce the error. Fundamentally there is soooo much money at stake, that I have to believe that the networks trust Nielson's methodology and are not just giving them control of billions of dollars out of tradition. And the networks certainly know more than you or I about the accuracy. I admit that it does seem strange tho.... I am guessing that the Nielson numbers are more useful in determining relative numbers than absolute ones. There is a great entry on Gallup's website about sample size and polling methods that has some applicability here... you may want to check it out - it is a great overview of general polling: http://www.gallup.com/help/FAQs/poll1.asp also, a Wired article on Nielson: http://www.wired.com/news/print/0,1294,63080,00.html and an article about the recent uproar over a Nielson methodology change that some say is going to undercount minorities: http://www.plastic.com/article.html;...06/05/14031275 __________________ Barry Gribble Integral Arts, IMDB

June 30th, 2004, 03:01 PM	#7
James Emory Regular Crew Join Date: Aug 2002 Location: Atlanta, Georgia Posts: 1,892	Guessing Guessing is just what this ratings thing is. These folks are trusting that the statistical info is correct. They deserve to get financially screwed if they are that naive. I think it's safe to say that there are experts in every industry. They are often used in court to decide someone's guilt or innocence. The average person doesn't understand how DNA is processed for evidence but we trust these experts which they MAY very well be. This is just for argument and for no particular procedure, but what if they were wrong about a procedure and the guilty were released and innocent ones were held. Experts have been found to be wrong you know. I guess the question is this. If you were a potential advertiser, and a sales person said that there was a space available for $100,000 for :30 would you write them a check on, "well 10 million people saw last week's episode." I would send them right out the front door. Why don't they just go ahead and say 20 million people saw it and get $500,000 out of these suckers because the truth is NOONE knows the true number of viewers! It should be done accurately or not at all.

June 30th, 2004, 08:09 AM	#4
Rob Belics Major Player Join Date: Apr 2003 Location: St. Louis, MO Posts: 581	Keith is correct. Interviews are done, I believe, with those selected for the ratings so they fit a profile but it is known that the accuracy of the system does not improve much with the number of people used for the survey.

June 30th, 2004, 12:02 PM	#6
Rob Belics Major Player Join Date: Apr 2003 Location: St. Louis, MO Posts: 581	And what is the alternative? You can't poll everyone so that leaves guessing. Neither of which would be acceptable.

June 30th, 2004, 04:07 PM	#8
Keith Loh Air China Pilot Join Date: May 2002 Location: Vancouver, B.C. Posts: 2,389	James you wrote: //It should be done accurately or not at all.// And it has been already said in this thread that it would be a question of resources and statistically it doesn't improve the accuracy much. The alternative, guessing, as has already been said, is guessing. You're babbling about 'experts' and 'DNA'. Stay on the topic. In any case, we may be getting to toward the 'box in every home' idea when more and more people install set-top boxes, satellites, (pay-per-view obviously gives the exact data) which can track viewing patterns. And then, as now, you will have people wanting to opt out because they are not crazy about having some corporation know what they are viewing. __________________ -- Visit http://www.KeithLoh.com \| stuff about living in Vancouver \| My Flickr photo gallery

June 30th, 2004, 04:45 PM	#9
James Emory Regular Crew Join Date: Aug 2002 Location: Atlanta, Georgia Posts: 1,892	I am not babbling, I am rambling! I only used DNA as an example of a complex test that not everyone understands but we trust as we're told by the experts, just like this viewer statistics racket. As far as a black box recording viewing habits, didn't Tivo get in trouble a couple of years ago for recording its subscribers' viewing habits without telling them. They said something like they never assigned the stats to individual accounts or identities so noone's privacy was violated.

June 30th, 2004, 05:02 PM	#10
Keith Loh Air China Pilot Join Date: May 2002 Location: Vancouver, B.C. Posts: 2,389	DNA test methodology are open to scientific review. In the past year there have been more than a handful of articles publiched in New Scientist about DNA testing so you don't have to be an expert to be able to look back and see what the issues are. We are both recalling the same TiVo story. That certainly is an issue, especially as not everyone views the same 'safe' programmes that others do. I certainly wouldn't want John Ashcroft knowing that my viewing patterns and it has been shown that the DoJ and other agencies has been able to get that information from corporations without much of a fight (I'm speaking of the airlines specifically). Anyway, I don't see how your criticisms can be satisfied. If you don't like the methodology and there are privacy concerns with tracking all viewership that leaves you with notions and guesses. Where exactly were you going with all of this? Were you pitching a show and some executive threw back negative data? __________________ -- Visit http://www.KeithLoh.com \| stuff about living in Vancouver \| My Flickr photo gallery

June 30th, 2004, 07:36 PM	#11
James Emory Regular Crew Join Date: Aug 2002 Location: Atlanta, Georgia Posts: 1,892	Keith said "Where exactly were you going with all of this? Were you pitching a show and some executive threw back negative data?" No, I just think it's a joke. Especially how millions of dollars are spent on what ifs. If I had proposed a plan that said I am going to tell you how many people watched your show by only monitoring 5000 out of a potential 15 million + viewers and then dare to put a million dollar price tag on it, I would have been laughed out of the room. I think it's probably the best of the worst estimated, maybe, guess method available. Like I said earlier, if you were a potential advertiser, and a sales person said that there was a space available for $100,000 for :30 would you write them a check on, "well 10 million people saw last week's episode" and hope that they were right. I'd have to say, go visit those 10 million homes and then show me the reports with the signatures of each viewer next week and then I'll pay you. Ha, if they actually had to do that, then it just wouldn't get done! Why should they make any extra effort when it's too easy to estimate and/or guess and take these people's money. They sure didn't ask me what I was watching!

June 30th, 2004, 08:36 PM	#12
Rob Belics Major Player Join Date: Apr 2003 Location: St. Louis, MO Posts: 581	James, These audience numbers are based on statistics, one of the mathematical sciences. This is nothing new and a proven method. There are actual formulas created to determine these results and prove their accuracy. You can go to the library and find books on the topic. Search the web and you'll find sites that explain how it all works. It's not a hidden topic and the method is easy to learn, at least if you're into that sort of thing.

June 30th, 2004, 08:42 PM	#13
Michael Wisniewski Wrangler Join Date: Jul 2002 Location: Makati, Metro Manila Posts: 2,706 Images: 32	Also, I'm sure the Nielsen ratings are just a part of the data that the advertisers use to figure out how much different time slots are worth. A sample of 5000 homes could get you a pretty good guess as to what's going on if you cross reference it against all kinds of other viewer statistics. For example, start with U.S. Census data, cross reference against historical sales and inventory data from companies, and then throw in the Nielsen data. That would give you a pretty good estimate. I used to work with a statistical data company that would buy all kinds of databases from hundreds of public, private, and governmental institutions. They could probably figure out any trend within an acceptable range of error.

June 30th, 2004, 11:23 PM	#14
Jeff Donald Warden Join Date: Mar 2002 Location: Clearwater, FL Posts: 8,287	the last time I checked the Law of Large Numbers was still in effect. __________________ Jeff Donald Carpe Diem Search DVinfo.net for quick answers \| Where to Buy? From the best in the business: DVinfo.net sponsors

July 1st, 2004, 12:14 AM	#15
James Emory Regular Crew Join Date: Aug 2002 Location: Atlanta, Georgia Posts: 1,892	No Chance I don't think there's much more I can say without repeating myself again. One thing is for certain though. I'm not putting any of my millions on that ratings method anytime soon!!