We don’t need more SLOPs
Campaign for Vermont took a wrong turn this week when it released the results of its first website visitor survey. “Citizens Want Property Tax Accountability” the press release asserted. “88% think $27 million should be restored to Education Fund.” To try to give the findings legitimacy, the press release compared the “poll” to similar surveys done on other websites and by state Sen. Bill Doyle each spring.
People who do real public opinion polling have an unflattering term for these kinds of surveys. They predate that Internet and are known as SLOPs, which stands for self-selecting listener opinion polls. The problem lies in the “self-selecting” part. Polls that are not based on carefully designed random samples do not represent anything but the opinions of the people who chose to respond to the survey. And because the respondents selected themselves, neither we nor the people who compiled the results have any idea whether the answers were representative of the entire population of the state.
Statistically valid surveys don’t require a lot of responses as long as the people questioned are picked at random. You only need a sample of about 400-500 people to get good results about the population as a whole. Such results are defined by the margin of error and the confidence level. For example, from a sample of about 400 randomly chosen Vermont residents, you would be able to say you were 95 percent confident that the results were plus or minus 5 percentage points of the results you would get if you surveyed everyone in Vermont.
The fact that thousands or even tens of thousands of people might answer an online survey—or the Doyle poll—doesn’t make it more accurate than a random survey of 400-500 people.
In its mission statement, Campaign for Vermont talks about positive consensus building and greater government transparency and accountability. And in the interest of promoting transparency and accountability, the organization’s principle backer, Bruce Lisman, has supported efforts to make information available to help shape public policy. Campaign for Vermont is a sponsor of the website developed by Public Assets and Ethan Allen Institute, Vermont Transparency, to make state fiscal data more easily accessible.
If Campaign for Vermont wants to fund some real polling, that would be a useful contribution to the debate about public policy. It would be helpful to know how Vermonters really feel about some of the tough issues being discussed.
These online surveys don’t build consensus; they are inherently divisive because they don’t reflect the opinions of all Vermonters. Instead, they represent the opinions of a small minority that has answered the survey, and Campaign for Vermont becomes the megaphone for that minority. We happen to agree that the $27 million should be restored to the Education Fund, and it may even be true that a majority of Vermonters support that idea, too. But this kind of survey does not contribute to good public policy debate.
Campaign for Vermont should reconsider.
