There’s a Town Council work session tonight. On the agenda is an idea from Councilwoman Patel. The idea is that the Town might routinely include a brief survey in the Vienna Voice monthly newsletter.
This would be a way for the Town to gauge where the average citizen stood on the issues of the day.
The unsigned, un-attributed Town staff response is quite a piece of work.
Maybe my take on this is just my own bias, as I have done surveys as part of my job. But I think not. So I’ll ask you just to read the rest of this, and see if you can get a sense for whether this is a fair and even-handed discussion of this proposal. Assess how much it strives to present an accurate assessment of the pros and cons of this approach. (Hint: See if you can find any pros.)
I’m not going to comment other than to make a single technical point. Mailing a survey to 100% of residents is, by definition, a random-sample survey. It’s just that the sample rate is 100%.
Here it is, in its entirety. Literally cut-and-paste. Here’s the Town staff response, to the idea of asking you what you think, on a routine basis, as part of the Vienna Voice mailing. You can find it in its original format on this Town of Vienna page.
Councilmembers Want to Know… survey initiative
Staff questions/concerns
• Unlike the National Citizen Survey, which uses random sampling, this type of survey is not statistically valid. (However, over time, data from survey may be quoted/used as if it is statistically valid – people tend to forget that part.)
• Crafting survey questions is a science. It’s very easy to accidentally inject bias into the question or to shape the question to generate the response desired.
• Is Council setting an expectation that all decisions will be made by referendum?
• Issues and decisions that must be made are often complex. Despite educational efforts, some residents will not be aware of all of the intertwined considerations that factor into decision-making.
• The timeline of utilizing the newsletter for monthly surveys is awkward, and data may not be available in timely enough matter to impact some Council decisions. Newsletter deadline is the 10th of the month preceding publication.
Example of how process might work:
o Councilmember questions due to editor December 10.
o Survey published in January newsletter around January 1.
o Deadline for responses? If January 10, could publish results in February issue; if later than that, would have to be in March issue.
• Who will “vet” questions posed? Editor, individual Councilmember, all of Council?
• Who will be responsible for analyzing and reporting data?
• Sets up unrealistic expectations? What if Council chooses to go in a direction that is different from survey results?
• Limited response, especially as time goes on.
• Will survey responses be anonymous or identified? No way to know that hearing from a representative set of voices.
• How will surveys be returned to the Town? Will people make the effort to drop off or mail to Town Hall?
Engagement best practices – “Meeting People Where They Are,” Sept. 1, 2019 ICMA article
• Conducting outreach and surveying using only one communications channel almost guarantees biased results.
• Need to use all outreach methods at our disposal, traditional and virtual, to provide more residents an opportunity to engage.
• Need to meet residents where they are. E.g., pop-up opportunities at events where people can respond to a survey on their phones or provided iPads; outreach at community events.
• Keeping the barrier to participation low means being able to instantly engage: no usernames or passwords, no creating an account.
• Being mobile-minded is one of the best ways to increase engagement.
• Messaging is best when it shows the value that public input will have on the decision-making process. For example: “Your input will help set priorities for our 2040 Transportation Plan.
• While a boots on the ground approach does build relationships, it’s difficult to scale, especially given time and staff constraints.
• The best way to optimize engagement and increase equity is to combine traditional and online outreach into a cohesive process and build a public participation database so you can analyze input, report findings, and make strategic decisions.
• Selectively leverage technology, budget, and staff time.
