Surveying the Landscape
WNEU Polling Institute Is Making a Name for ItselfThe high-profile 2012 U.S. Senate race between incumbent Scott Brown and challenger Elizabeth Warren was memorable for a number of reasons.
Start with the amount of money spent — $68 million, making it one of the most expensive Senate contests of all time. There was also the heated rhetoric, epic debates, and, perhaps most importantly, the stakes — most analysts said this race was about nothing less than control of the Senate.
But Tim Vercellotti will also remember it for something else.
He considers that race the moment when the Western New England University Polling Institute, which he has directed since 2008, came into the national spotlight — and essentially came of age.
Launched in 2005, the institute included questions concerning that Senate contest in more than a half-dozen polls between the spring of 2011 and the days just before the election in November 2012. The headlines on the press releases announcing the polls’ results essentially mirrored what was happening in that pitched battle, as Warren, well behind when the contest began, gathered steam and, with the support of those also backing President Barack Obama, triumphed on election day:
• “Brown Holds 8-point Lead in Massachusetts Senate Race” (March 4, 2012);
• “Senate Race a Toss-up as Warren Closes Gap on Brown” (June 2);
• “New Poll Shows Warren Leading Brown in Senate Race” (Sept. 16);
• “Warren Leads Brown by Five Points in Latest Senate Survey” (Oct.7);
• “Poll: Warren Maintains Four-point Lead in Senate Race” (Nov. 4)
She would eventually win by eight points, said Vercellotti, noting that 4% of those polled near the end were still undecided, and the poll had a margin of sampling error of plus or minus four percentage points.
“So we were right there; our polls were correctly indicating what was happening,” said Vercellotti, noting that, beyond the level of accuracy and its impact on overall credibility, the institute’s work during the closely watched race gained considerable national exposure, with mentions in the New York Times, the Wall Street Journal, the Washington Post, MSNBC, CNN, and other news outlets. “That was our high-water mark … I’m not sure when that kind of clash of the titans will happen again.”
In the meantime, the institute has been garnering public opinion on everything from economic confidence and expectations for holiday shopping (two subjects in the most recent poll, undertaken in early November); from casino gambling to the recent government shutdown and which party was more responsible for it; from the ‘death with dignity’ poll question on last year’s ballot to healthcare reform.
And, while doing so, it is making strides in the all-important work to establish a reputation for accuracy and transparency, a process that can take years and perhaps decades, he said, but one in which he believes the WNEU facility is making solid progress.
“The longer you’re in the field and the more successful you are at building a record of accuracy, the better off you are,” he said, adding that the Brown-Warren race certainly enhanced the institute’s scorecard. “But it only takes a couple of bad polls to undo all of that, and that’s why I take this very seriously and think long and hard about the surveys and how they’re written.
“One of the challenges is that, in politics today, people want answers, they want absolutes, and surveys are merely exercises in probability — that’s why there’s a margin of error,” he went on. “What you’re saying is that, ‘19 times out of 20, we think the answer in the population is within this margin of error. But one time out of 20, it’s not, and that’s life; that’s just how it works.’ But if that one time in 20 is your final pre-election poll in a major, high-profile race, you can talk about probability all you want, but the audience can be very unforgiving.”
For this issue, BusinessWest takes an in-depth look at the work being conducted at the institute and the intricacies involved with the often-misunderstood world of polling.
Questions and Answers
While effective polling is both an art and a science, Vercellotti told BusinessWest, it is mostly the latter.
Elaborating, he said that strict attention must be paid to everything from how the questions are phrased to the order in which they are asked; from how political candidates are identified to how their names are pronounced, in order to ensure that the results are reliable and accurately reflect the thoughts of those being asked the questions.
As one example, he pointed to the most recent statewide survey, which polled respondents on the economy, holiday spending, casino gambling, medical marijuana, and other topics.
There were several questions about the health of the national economy, respondents’ personal financial position, and when and if improvement was forthcoming, he said, adding that he was careful to ask them after the queries on holiday shopping so as not to influence replies to that specific line of questioning.
“There was concern that if the shopping question came after the questions about the economy and people were gloomy about the economy, that would shape how they would answer those holiday questions,” he explained, adding that such nuances are shaped by experience and large amounts of pre-testing with surveys.
As another example, he cited polls on specific political races. Those asking the questions don’t have to go in alphabetical order with the candidates, said Vercellotti, but they should change the order of the names on a systematic basis, because a percentage of respondents to such queries will simply favor the first name they hear. Likewise, party affiliation must be mentioned with each candidate (when applicable) because some respondents are inclined to favor candidates by party affiliation.
“You have to rotate the order of those names so that each name appears first half the time,” he explained. “And it’s important to put party label with the name, because we know from political-science research that the less-engaged voters will often default to that label; they won’t know much about the candidates, so they’re just going to go with the party.”
These are just some of the things Vercellotti has learned during a career that has blended graduate-school training in survey research and questionnaire design, work at another college-affiliated polling institute (this one at Elon University in North Carolina), teaching, and his current post. He came to WNEU in the 1990s and became director of the institute in 2008.
He said there was a lot of learning while doing at Elon, and the education process essentially continues with each polling assignment.
“Survey researchers write some clunkers of questions,” he noted. “And with almost every survey, I get to the end and say, ‘I should have asked this.’ You file that away and try to do that the next time.”
Tracing the history of the WNEU program, polling institutes at colleges and universities, and polling in general, Vercellotti said the practice of gauging public opinion dates back to the 1930s, when firms such as the one started by George Gallup would go door to door seeking answers to specific questions.
The telephone became the preferred polling method in the ’60s — when the percentage of households with one reached a critical level — and it remains the best option today, he went on, although Internet polling, considered less reliable by many, is gaining some traction.
In recent decades, several colleges and universities have created polling institutes, he told BusinessWest, with the twin goals of raising the profile of the institution and generating data for a society that has developed an appetite for ever-increasing amounts of it.
The Eagleton Center for Public Interest Polling at Rutgers is one of the best-known of these facilities, but there are many others, including some in the Bay State at Suffolk University, UMass Lowell, and UMass Boston, said Vercelloti, adding that the WNEU model is based loosely on programs at other, smaller colleges, and especially Quinnipiac in New Haven, Conn.
WNEU started small, with 12 calling stations in a computer lab in the College of Business, said Vercellotti, adding that it has since grown to 23 stations, all staffed by thoroughly coached students.
Over the past several years, the institute has conducted two or three polls each semester funded by the university itself, he went on, adding that there have also been several projects — including many of those aforementioned Warren-Brown polls — undertaken in conjunction with the Republican and masslive.com.
There have also been a few contract assignments, including one commissioned by the West of the River Chamber of Commerce, which aimed to gauge the thoughts of Agawam residents on potential development of a large parcel on Tennis Road.
There is the potential for more of that kind of work, he explained, but the price tag — roughly $10,000 for an eight-minute, 24-question survey with 500 respondents — is higher than most inquirers expect and often beyond their means.
As he talked about the polling institute and its work, Vercellotti made early and frequent use of the word transparency.
Such a facility simply must have it in order for the results it generates and then publicizes to have credibility, he said, adding that objectivity is another trait that a successful polling institute must possess, and this explains why he’s turned down a number of potential contract assignments.
“We have been approached for contract work on the casino issue, and from people on both sides — opponents of gaming and advocates for casinos — and I’ve turned down the work,” he explained. “It’s critical that we’re objective; we’re not going to take sides with an election issue. Candidates will sometimes approach us, including some who are alumni of this university, and I make the same point: we’re not here to be engaged in partisan politics or take one side of an issue.
“We’re extremely transparent about who hires us and the methodology we use,” he went on, adding that he’s a member of AAPOR (the American Assoc. for Public Opinion Research), has signed its code of ethics, and is part of an endeavor known as the Transparency Initiative. “The only way the audience can make an informed judgment about the credibility of polling material is to know how it was gathered.”
Overall, the process of reputation building, which affects both credibility and accuracy, takes years and is certainly ongoing, he said, adding that it takes multiple election cycles to establish that a program is reliable.
To date, the institute has amassed a fairly solid track record, or scorecard, said Vercellotti, adding that he does maintain a record of how well the surveys project what is to come. It has not projected the wrong winner in any political race or referendum question, at least when one takes into account the margin for error that accompanies each polling assignment.
That caveat is necessary with the ‘death with dignity’ question on the 2012 state ballot, he explained, relating the story of how public opinion on that measure changed dramatically between the institute’s first polling exercise on the subject and the last one just before the election.
“When we polled on it in the spring of 2012, it was way ahead, a slam dunk,” he recalled. “But I think that, in some ways, opinion on that question was very wide, but not very deep. And by the fall, opponents of that proposal had gotten organized, they started running some advertising, the Kennedys came out against it, the Catholic bishops came out against it.
“Our final pre-election poll, just five days before the election, had it passing by a two-point margin,” he recalled. “Therefore, I said it was too close to call, and it failed by two percentage points, but that’s within the margin of error.”
While the institute is still in its relative youth at only eight years old, it is, Vercellotti believes, gaining the respect from the many constituencies that are seeing and judging its work — from the general public to the press; from campaign operatives to special-interest groups, such as those on both sides of the casino issue.
And then, there are the growing numbers of bloggers, he said, who take polling data and analyze it — and usually aren’t shy about voicing opinions on the sources of that data.
“They’re the toughest audience of all, because they’ll take the time and go through the methodology and raise questions,” he said, adding that the emergence of this audience is a relatively recent phenomenon. “When I got into the business in 2001, there wasn’t that kind of audience, but now there is, and they’ve developed a level of expertise.”
Therefore, he pays attention to what the bloggers are writing. Summing up the reviews to date, he noted that many are positive, but some are what he called “dismissive,” a tone he attributes to the small size of the school and the youthfulness of the polling program.
Looking ahead, Vercelotti said that 2014 could be an intriguing and busy year for the institute. Indeed, the Deval Patrick era is ending — WNEU’s facility conducted its first poll on the coming contest earlier in the fall, headlining the release “Democrats Out in Front Early in Governor’s Race” — and there will be a race for the Senate seat captured in a special election earlier this year by Ed Markey after John Kerry became secretary of State. Meanwhile, there may also be some interesting ballot questions, including the possibility of another referendum on casino gambling.
But after that, things will likely get quieter, he said, noting that, while there is a presidential race in 2016, there won’t be another governor’s race or Senate race (unless someone else leaves office before their term expires) until 2018.
There may be some local, state, or regional issues to fill the void, he went on, adding quickly that, in an age when the public’s thirst for information only grows, there is unlikely to be a shortage of issues on which to conduct polls.
Making the Call
Referring back to all the science involved in polling, Vercellotti said it extends even to what time individuals are called — or should be.
He said the recent World Series, won by the Red Sox, posed some challenges; many were not happy about their viewing being interrupted by a pollster. Likewise, Patriots games add a layer of intrigue to Sunday afternoons, one of the times when those staffing the phones at the institute are most busy.
“I try to make the most of halftime — I make sure the callers are active then,” said Vercellotti, adding that mastering such nuances is all part of the process of making the institute successful — and respected.
Nearly nine years after it started soliciting opinions, this facility is well on its way to achieving those goals.
George O’Brien can be reached at [email protected]