Did the polls help?

Ahead of the general election HOT attempted to put the polls on voting intentions into perspective – what can we realistically expect to learn from them? With the election now decided, how did the polls do in indicating what the outcome would be? Did they contribute positively to the democratic process? Text by Nick Terdre, research and graphics by Russell Hall.

People respond to polls, they influence voting behaviour, so it’s important not only that they are reasonably accurate, but also that people understand what they tell us, and what they don’t tell us. For the general election 2024 we can say they generally speaking got it right – they foresaw a big Labour victory and that’s what we got.

The polls also attempt to tell us how seats would be divided between the parties, and the result of individual constituency contests, which are more difficult aims to achieve. So how did they do? And which ones got closest to the actual outcome?

In the event the votes were shared out in Great Britain (ie not including Northern Ireland) thus: Labour 34.7%, Conservatives 24.3%, Reform UK 14.7%, Liberal Democrats 12.5%, Green Party 6.9%, Scottish Nationalist Party 2.6% and others 4.3%.

(In passing it is worth noting that the combined Tory/Reform vote share was 39%, well above Labour’s 34.7%).

The average projection of 15 polls analysed by HOT was 38.7% for Labour and 21.1% for the Conservatives. So they tended to overestimate the Labour vote and underestimate the Tories’, by 4% and 3.2% respectively.

It can be clearly seen from this interactive graph plotting 133 polls of voting intentions after the date for the general election was announced that they all over estimated Labour's vote and most of them underestimated the Conservative vote.

As pointed out in our pre-election piece, what the polls give us is a snapshot of the voting intention of those polled at the time they are polled. And that can change. It seems probable that, looking at a likely large Labour victory, some of those who were planning to vote Labour primarily to get the Tories out, felt confident enough that this aim would be achieved without their vote to switch it instead to an opposition party they preferred, such as the Lib Dems or Greens.

The Lib Dems did better than the polls on average expected - 12.5% against 11.1% - but the average projection for the Greens was almost spot on, 7% against 6.9%.

The sizeable defeat apparently looming for the Conservatives is also likely to have persuaded some Tory voters to switch to Reform - had the contest been closer, such voters might not have switched.

Evidene of late decision-making is provided by Lord Ashcroft Polls, whose analysis shows that across all parties some 36% of voters made up their minds either on polling day or in the few days before.

With the exception of Labour and the SNP, more than 20% of voters for all other parties only made up their minds on polling day. Late decision-making can catch the polls out.

In particular 51% of Green voters only decided on polling day itself or in the few days before, and 44% of Lib Dem voters in the last week, 25% of them on polling day or just before. As many as 39% of Tories only resolved how to cast their vote in the last few days, including polling day, and 31% of Labour voters.

For such reasons the polls can be led astray. In particular Labour did not do as well as the polls suggested. It also seems that the defection of Tory voters to Reform was over-estimated, as Reform achieved 14.7% on the day against an expected average of 16.5%, while the Tories, as explained above, did better than expected.

Overall the pollster which came closest to capturing the Labour/Tory shares was Verian, which projected 36% for Labour and 21% for the Conservatives. Even they, however, over-estimated the spread between the two parties by some way, putting it at 15% against the actual 10.4%.

A closer result on the spread was achieved by Norstat, at 13% (37% against 24%) while JL Partners, like Verian, projected 15% (38%-23%). Furthest out, at 20%, were People Polling (36%-16%) and Opinium (41%-21%).

A widely accepted measure of error in polling is the root mean squared error. Applying this to the projections of vote shares for all parties by the various pollsters, it is Verian again which emerges with the smallest error, 1.5%, followed by Norstat at 1.6%. At the other end of the scale the estimated error for People Polling was 4% and for Opinium 3.2%.

Going from an estimate of national vote share per party to deciding how this will translate into the number of seats which each party is likely to win is a more uncertain exercise, as we can see if we look at the root mean squared error for these calculations. Our benchmark is the actual division of seats: Labour 412 (including the Speaker’s seat which traditionally is not contested), Conservatives 121, Lib Dems 72, the SNP nine, Reform five, the Greens and Plaid Cymru four each and independents five.

The smallest error was 9.8 seats recorded by More in Common, who correctly called the winner in three quarters or 472 of the 632 British constituencies, followed by YouGov with 9.9 seats. More in Common had Labour at 430, the Tories 126 and the Lib Dems spot-on at 72. But it was well out with the SNP for which it projected 18 seats.

The most wayward projection was Savanta with a root mean squared error of 42.4 seats (Labour 516 seats, Tories 53).

Estimating the result for each constituency involves a further degree of complication.

Here, as explained in our previous article, we need to look at polls using the sophisticated statistical technique of multi-level regression and post stratification (MRP). Our analysis includes some dozen companies applying this technique, most of which produced more than one poll.

In the case of Hastings & Rye, looking at the root mean squared error, the top poll was YouGov’s second MRP at 1.6%. It  underestimated Labour’s vote by 0.6% and overestimated the Tory vote by 3.2%. For all the parties it was out by only 1.4% on average.

The interactive chart above shows that YouGov's second MRP poll, based on fieldwork between 11 and 18 June, was more accurate than its third and final MRP one, with fieldwork 19 June and 3 July. So we cannot assume that a later poll is necessarily more accurate than an earlier one! None of the 22 MRP polls analysed here however erred by suggesting that Labour would not win.

In Bexhill & Battle it was a closer contest, with polls differing over whether the Tories would hold the seat or Labour win it. As the above chart of the three YouGov MRP polls shows, on 18 June Labour was in the lead, according to the pollster's second MRP poll, but by the time of its third, published on 2 July, the Tories were back in front by 2.2%.

The Tories' eventual margin of victory, at 5.6%, was somewhat greater, but YouGov’s third MRP poll was overall the closest, with a root mean squared error of just 1.3%. It was alone in its degree of accuracy, however, the next closest error being returned by its second poll, at 3%.

Also in: Elections

Putting Labour’s landslide into perspective »