You Can Never Reliably
Count On Survey Results
By Kel Fabie
Surveys often serve a purpose, and it is rarely “to find out the truth and what the people are really thinking.” Even with the noblest of intentions, though, surveys simply can’t be relied on for anything significant. Here are 8 reasons why.
8. Who wanted the survey commissioned will always taint the results.
Did a certain candidate commission a survey about who people want to vote as president? Did a new polling firm materialize from out of nowhere just to give results favorable to a “surprised” candidate (What’s up, D’Strafford?)? Politics aside, how about a survey about how safe tobacco is, conducted by the tobacco industry itself? Knowing who’s behind the survey will often tell you what the results will be. After all, if someone paid good money for a survey to be commissioned, do you really think they’d release the results if it didn’t favor them in any way?
7. The wording of the question itself would easily affect results.
If someone asked you right now: do you support the killing of known or suspected drug personalities without due process? Regardless of whom you voted for president, it is highly unlikely that you would just go and say “yes” to it, for whatever reason. It isn’t hard to make people not want “killing” on principle.
But what if you worded the question as: do you support the war on drugs and the drastic measures that are sometimes necessitated by the situation?
You will find that the support for what is essentially the same thing drastically changes, which is why you are presented with ridiculously contradicting results where SWS would say something like “8 out of 10 people are afraid of getting killed in this drug war (the other 2 are riding in tandem),” yet, in the same survey, it turns out, “85% of the respondents were ‘satisfied’ with the government’s war on drugs.”
6. The sample size is never 100%.
They say that there is a certain margin for error with these surveys, but what people don’t understand is that the accuracy of a survey is always affected by how far it is from 100% of the population that is being represented.
Let’s take for example a survey that is taken by a thousand respondents. Not only do those 1,000 respondents need to be representative of the demographics of the entire population (e.g., if we have only 1% of the population in socioeconomic class A, you shouldn’t be asking more than 10 people from class A in this survey.), it also has to consider how big the population itself actually is.
In our case, with a population of well over a hundred million people, a survey of 1,000 people accounts for a mere .001% of the total population of the Philippines. That’s about as reliable as asking your housemate what the best movie in the world is, then proclaiming “One More Chance is the best film of all time, according to 100% of our respondents.”
5. Surveys shouldn’t dictate how you think.
Surveys could be accurate for all we care, but if we subscribe to it, it only becomes a self-fulfilling prophecy, instead of an exercise in choice.
The ability of surveys, no matter how inaccurate, to set our expectations is called “trending,” and we don’t mean when people tweet about something a lot. It’s when people believe that the survey is so accurate, that deviations from it become suspicious. The problem with that, especially in voting, is that it makes us try to vote for the most “winnable,” instead of voting for the candidate we genuinely want for ourselves.