I read a comment in the Millstone that said: “In this day and age you can create an online poll in minutes”. That got me thinking about the value of non-professional on-line polling.
I am assuming that the reason one would have a poll is to get an accurate gauge of public opinion. This is not as simple as just churning out on-line polls.
Accurate polling is a non-trivial exercise. The phrasing of the question can influence the results and therefore provide inaccurate data. Consider the following two versions of the same question:
- Version 1 – Do you support the resolution?
- Version 2 – Do you oppose the resolution?
Essentially the same question but depending on which version of the question you use, you will get different results.
Another important consideration is the order of questions and even the order of the choices for a given question. Consider the following two versions of the same question:
Which of the following most represents your views?
- I think it is a good resolution
- I think the resolution should be deferred
- I have no opinion on the resolution
- I think the resolution should be accepted
Which of the following most represents your views?
- I think the resolution should be deferred
- I think it is a good resolution
- I think the resolution should be accepted
- I have no opinion on the resolution
Both of the above questions are exactly the same but the order of the answers is different. This variation will result in different results.
There are even more subtle things that can influence polling results. The words used for the preamble to the poll can influence results. Even the colour and font can skew the results.
The above points are just a few factors on the content of the poll. Then there is the statistical side to be considered.
How do you know that you are getting a representative sample? From what I can remember of my two years of Stats courses, selecting the right sample size is very significant. I can no longer recall the formula for determining sample size but if you want to have a meaningful poll, you should figure it out.
You also have to know what type of distribution you are dealing with and things like the standard deviation. It is a struggle for me to recall how to determine these variables but if you are serious about polling, you should familiarize yourself with the math.
How to ensure that you sample is truly random? This is essential for getting reliable data. A self-selecting sample, like people who read the Millstone or people who follow a Facebook page, is almost useless – it tells you nothing about the population as a whole.
I recall attending a talk at Ottawa U where Nik Nanos was a speaker. If memory serves me correctly, he said something like: “a poll can be constructed to give you any answer you want”.
My final point on polling is probably obvious: People with vested interests should not be conducting polling – it should be done by independent pollsters.
I hope that I have convinced you that ad hoc polling using things like Facebook or SurveyMonkey is of dubious value. I certainly hope that our councillors will not be swayed by unscientific polls done by non-professionals.
Robin Sukhu
