How would you rate the services Mecklenburg County provides compared to the taxes you pay? A new survey by the county would have you believe that people in the county overwhelmingly approve of it, but the survey has some flaws.
Mecklenburg County uses the annual Community Survey to track how aware and satisfied the people of the county are with the services it provides.
“We would say that our overall performance ranges from successful to exemplary,” Monica Allen, Mecklenburg County’s Corporate Performance and Evaluation Manager, told the Board of Commissioners on Tuesday.
The survey found 86 percent of residents agree that county services provide value for tax dollars (with a +/-3 margin of error), a far higher number than similar questions on city surveys in Asheville or Durham, for instance, where about 40 percent agreed to some degree.
So, why are Mecklenburg’s numbers so much higher?
“There’s a number of problems here,” says Paul Biemer, a UNC-Chapel Hill professor in research methodology. “Think this was a poorly designed questionnaire. I think it’s very biasing.”
The survey pushes respondents for a positive answer, Biemer says. For instance, before Question 2 it reads: “Mecklenburg County’s Board of County Commissioners has a vision for Mecklenburg County to be a community of pride and choice for people to live, work, and recreate.”
“People go through it and they hear about all of these marvelous things, for example, ‘Mecklenburg county spends my local tax dollars on services that help residents improve their local community,’ says Biemer. “To me that sounds like, ‘Aren’t we great?’
“It’s not just asking, ‘Do you agree with that?’ It’s stating that’s what’s happening.”
Biemer says another phenomenon called “acquiescence bias” may have influenced the results—people agree to an answer, because they think it is what the surveyor wants to hear. So, the survey conductor would tell the respondent about a county service and then ask the respondent if he or she knew it exists.
That may be why the survey found 88 percent of the public knew they could use the county website to “check to see if someone has been arrested or has been jailed in Mecklenburg County” and 74 percent knew they could make a public records request online.
Neither the county nor MarketWise, the company that performed the survey for it, responded to request for comment by press time.
Read the Public Awareness Findings.
Update: Mecklenburg County researchers are responsible for crafting the questions and conclusions of the survey, according to Nancy Bunlap, president of market research company MarketWise.
Bunlap says her company was hired in 2006 or 2007 to conduct the questioning and process the raw data, while the county supplies the questions and interprets the findings. The survey itself has been conducted for more than a decade.
Bunlap also says that reframing questions in the survey, even if it eliminated potential bias, would prevent researchers from tracking longer-term trends, because the old questions would not match up with new ones.
"You could look at a question or two and say ‘could that question be improved?’" Bunlap says. "Maybe it could be improved, but do you want to lose all of your tracking data has been the issue, because they’ve tracked this data for a very long time.”