It isn’t exactly cool to speak up for the opinion polls these days. The pollsters’ failure to predict the Labour surge in Britain in 2017 has now joined the failure to predict the Tory majority in 2015 and the failure to predict the Brexit vote in 2016 to produce a hat-trick of polling incompetence. In some quarters, opinion polls are dismissed as not just unreliable but as a malevolent distraction, a form of ideological intervention to be spurned. So to admit, however quietly, to taking opinion polls seriously means at the very least having one’s tin hat at the ready.
Two years ago, the polling industry was forced to eat crow after former British prime minister David Cameron sailed back into Downing Street in an election in which the pollsters almost uniformly anticipated a hung parliament. An inquiry found that they were not sampling the voters accurately enough. This year, however, the pollsters have been less apologetic — although they are hardly in triumphalist mood. That’s because, as professor John Curtice, Britain’s most prominent poll analyst, told me last Thursday: “In 2015, all the polls were wrong. In 2017 we had a spread. The failure wasn’t collective this time.”
That’s true, as far as it goes. Even before all the votes had been counted, the British Polling Council (BPC), of which Curtice is president, issued a combative statement. All the final polls had got the Tory share broadly right, and had called the Scottish Nationalist Party (SNP), Liberal Democrats, United Kingdom Independence Party (Ukip) and the Greens correctly, said the BPC. But it admitted that it did worse with Labour. In the past, polls have tended to overstate Labour support. This time, though, the companies’ final polls understated Labour’s 41 per cent share by an average 5.2 per cent.
There is no way that can be spun as a success, because it is a dismal failure. Certainly that is the view of the Labour peer David Lipsey, who last Thursday was appointed to chair a new House of Lords committee on political polling and digital media. Its brief is to examine the effects of polling and digital media on British politics and to report in March. It will look at poll performance, methodology and accuracy, as well as whether the polling companies’ self-regulation through the BPC is fit for purpose.
When I first knew Lipsey long ago, this former adviser to two 20th century Labour giants, Anthony Crosland and James Callaghan, followed the opinion polls as attentively as he followed the greyhounds and the horses. These days, however, he has become a forceful poll sceptic. For the second time in a row, the polls had got an election wrong, Lipsey said. In a Guardian article in April, he wrote that the pollsters’ inability to know whether they were interviewing a representative sample of voters was their “dirty little secret”.
In Westminster terms, Lipsey’s select committee is the most prominent proof yet of a more widespread mistrust of and crisis for the opinion polls. It follows an election campaign in which, chastened by 2015 and 2016, press and broadcasting media used polling much more sparingly and cautiously than in the past. Although papers such as this one, for instance, continued to carry the findings of the long-established Guardian-ICM series , they were not prominently displayed or promoted. That was the right instinct, but it doesn’t mean polls can be ignored.
The reality, in any case, is that pollsters will go on polling, in private if necessary, and political parties will continue to feast on them. Even if polls were regulated or banned at certain times — as some members of Lipsey’s committee may want — polling will continue because it can supply genuinely useful information to which intelligent politicians should respond.
Without polls, the parties will simply promote more spurious data, like their own canvass returns, always a favoured ploy in byelections. Banks and hedge funds will stay in the private polling game too, if necessary working from offshore to avoid regulation. So the questions that ought to be asked about opinion polls are, first, how they can be made more reliable and, second, whether journalists and politicians can learn to interpret them better.
Patrick Sturgis, who chaired the BPC’s report on the 2015 debacle, admits part of the problem is that polling is getting harder. Knocking on doors no longer works. Nor does stopping people in the street. Land lines are less common. Subscribers dislike cold calls. There is no register of mobile phone numbers. And although online polling is cheaper, faster and more direct than old methods, it necessitates large corrections to weight the sample accurately. In addition, not everyone tells the truth.
Nevertheless, Sturgis thinks that, judged over time, polls are in fact getting more accurate than in the past. But he admits there is no methodological gold standard that will make everything instantly reliable. As the BBC’s former polling guru David Cowling also points out, the polling companies have a strong reputational self-interest in accuracy — their political polling accounts for a huge element of their publicity but a small element of their business.
In 2017 they all tried very hard, too hard, to correct for their earlier pro-Labour bias. Their remodelling made assumptions about young voters and uncertain voters that proved false this time (and may or may not next time). But don’t forget that they did pick up the swing away from the Tories and towards Labour, and that the swing seems to have accelerated at the very end, making it harder to pinpoint.
Though the pollsters need to up their game, the parties and the media must learn the larger lessons about political polling too. This, after all, was an election which only took place because Theresa May ended up having too much faith in the public mood on the basis of polls. The right-wing press followed her down the same road, failing to see that polls are better reflectors than they are predictors and ignoring the possibility that campaigns can change the mood dramatically.
Polling failures are intensely frustrating. But it is as foolish to ignore the polls at all times as it is to be led by them at all times. Polls matter. Accurate polls matter even more. They are part of the story that politics must tell. In the end, judgment matters more than anything. No one can predict the future, not even the pollsters. But good politics needs good polling.
— Guardian News & Media Ltd
Martin Kettle is an associate editor of the Guardian and writes on British, European and American politics, as well as the media, law and music.
