Lies, damned lies and...
Transforming the UK's evidence base
Like a schoolboy eager to tell the whole class on the first day how he spent his summer holiday, Jeremy Hunt, Chancellor of the Exchequer, stood before the House of Commons on Tuesday morning and gleefully declared:
“On Friday, the Office for National Statistics published an update to the UK’s GDP growth figures, which shows that the UK economy was 0.6% larger than pre-pandemic levels by the fourth quarter of 2021. It means that our economy had the fastest recovery from the pandemic of any large European economy, thanks to decisions such as furlough that protected millions of jobs. For that growth to continue, we need to halve inflation, which I am pleased to report is now nearly 40% below its 11% peak.”
The revision from the ONS came due to changes to the methodology and economic data it uses to calculate key economic statistics.
The ONS said:
“Upward revisions to annual volume GDP growth in 2020 and 2021 mean that GDP is now estimated to be 0.6% above pre-coronavirus (COVID-19) pandemic levels in Quarter 4 (Oct to Dec) 2021; previously this was estimated as 1.2% below.”
It’s a considerable shift in outcome.
The Government are now waving this new information in the faces of all the naysayers — or, as Liz Truss called them, “gloomsters”.
When so much in politics and the economy rides on these official statistics, this raises more fundamental questions than optimism and pessimism. It puts the whole operation of official statistics in doubt.
As Gareth Roberts asked in The Spectator:
“if they can get this one wrong, just what else are they cooking up or nodding through?”
It’s not the first time this sort of thing has happened.
In the past year, I’ve written about the great uncertainty around the economic data and forecasting of the Office for Budget Responsibility and the massive changes to the ONS’s R&D statistics back in December.
And in May the ONS announced changes to how it measures migration had resulted in a revision of net migration upwards by over 100,000.
Right on cue, this week the House of Commons Public Administration and Constitutional Affairs Committee took evidence on “Transforming the UK’s Evidence Base” from Professor Sir Ian Diamond, National Statistician at the Office for National statistics.
While Governments and central banks demand quick figures from the ONS, they publish revised figures each year in their “Blue Book”.
Professor Diamond explained that of the 8 quarters across 2020 and 2021, only two were subject to revision. Those were Q2 2020 and Q2 2021, covering the start of the pandemic and the start of the recovery respectively.
He said there were four factors that led to the earlier underestimation of growth:
Firms were stockpiling more than the ONS thought.
Costs facing firms went up overall.
Retail and wholesale margins held up better than expected.
The NHS was able to continue functioning better than expected.
He also said that the UK measured health and education outputs differently from other countries, which led to further under-estimation of growth.
Such significant revisions must be embarrassing for the ONS, suggested Conservative MP David Jones, especially given the political consequences.
Professor Diamond insisted that the ONS was following “international best practice”, which meant publishing uncertain data to meet the demands of Governments and central banks, and then revising the results once more data comes in.
Policy decisions, Professor Diamond explained, can be based on a series of “what ifs” that might be shared only with civil servants and Ministers trying to come up with policy.
Anything that will be released to the public has to meet the standards of the Office for Statistics Regulation.
Some analysis, therefore, is never published or only published after a significant delay.
The ONS is trying to speed up its data collection operation by shifting to what they call “born-digital” and administrative forms of data — i.e. data that is submitted automatically or scraped, from HMRC payroll to phone data, rather than being submitted through a survey of some kind.
Incredibly, during the pandemic, the ONS was still using “a person with a clipboard in the supermarket” to measure inflation, but is now shifting to scraping scanner data from seven major supermarkets, though it’s not quite there yet.
They are also scraping telephone data to track our movement in real time. As an example, Professor Diamond cited the Queen’s lying in state, during which:
“telephony data were used to monitor the speed of the queue and also to identify when to close that queue.”
Tom Randall MP asked whether all this data scraping raised ethical concerns and how the ONS would deal with them.
Professor Diamond mentioned the National Statistician’s Data Ethics Advisory Committee as a safeguard against unethical data use, but was generally vague in his answer:
“The use of radical new datasets and the possibility to link some of those datasets offer immense opportunities to answer questions that we have only been able to dream about answering and to answer questions that have the potential to impact positively on the lives of our fellow citizens.
We should only do so if those data are analysed and put together ethically, if privacy is maintained, if we are transparent in the way we use them, if it is absolutely in the public good and if there has been a programme of public engagement around it.”
However, when discussing the use of HMRC data to measure income at a local level, he admitted:
“It absolutely needs to be privacy-enhanced”.
Why that might be so, and how it could be done, were questions that no MP bothered to ask.
It seems that as Governments seek to use more and more data that we provide automatically online, statisticians are going to be more and more important in shaping how that data is used.
Professor Diamond suggested that we might be “moving away from a census”, at least as we have traditionally understood it, and towards using more “administrative data”. That is, rather than the Government asking you every few years about your life, they will just pull the data from elsewhere.
He later added, regarding moving away from a traditional census:
“A number of countries have moved away from a census but into different models. The French moved to a system whereby they have a series of large biannual surveys that they link together. The journey that we are on is similar to that which other countries are on, driven in some cases not only by the availability of administrative data and not only by the need for more frequent data, but also by, in some countries, the challenges of doing a census where overall enumeration rates have been somewhat lower than ideally we would like.”
Using administrative data or more regular surveys will mean more statisticians shaping the questions that get put to you day-to-day, perhaps without even knowing it.
Professor Diamond concluded, unsurprisingly, with a call for greater importance for his own profession:
“That may require making some changes to administrative systems. In my view, that requires the involvement of statisticians right at the beginning because they are the professionals who know how to answer questions.”
