TThe new administration faces many challenges as it seeks to boost economic growth, improve public services, and enhance family well-being. But it also has opportunities, some of which are right in front of us.
One of the most important is that the potential of the data that public authorities collect on individuals and businesses for their own purposes has yet to be realized. By more effectively sharing, linking, analysing and communicating this data, governments can improve policy design and implementation. This can be done directly, and also by helping external researchers and analysts scrutinise policy choices and generate new ideas.
The Office for Statistical Regulation (OSR), the UK’s statistics watchdog and part of the UK Statistics Agency, highlights this issue in its report on the state of the statistical system to 2024. It is encouraging to find that statisticians across the public system are already making extensive and creative linking and using administrative data to respond to new and complex questions posed by government and society. But we found several barriers to using this vital national asset as effectively as possible.
First, official statistical systems are under greater strain than before. They face the same financial constraints as other public sector sectors, and this is exacerbated by the fact that falling response rates and rising collection costs are making it increasingly difficult and expensive to maintain the quality of the household surveys they rely on to understand key parts of the economy and society – a trend seen in other statistical agencies around the world.
The statistical system is also more decentralised than in many other countries. The Office for National Statistics (ONS) is the largest single producer of official statistics, but most comes from other government departments and public bodies. This has the advantage that statisticians can work closely with policy makers and implementers in each field. But it has the disadvantage that data and analysis are often siloed. Each organisation holds a piece of the puzzle that, when combined, could provide richer insights.
During the pandemic, the nation’s sense of emergency provided an urgent impetus for data sharing, which could have enabled valuable insights, such as identifying targets for communication campaigns to boost vaccination rates. But the system has not fully capitalized on similar opportunities since then, leaving many benefits untapped.
Why is this the case? The OSR found that while there are instances of innovative and ambitious data sharing and collaboration within government, it is not widespread or the norm. The debate continues to focus less on potential benefits and more on the risks to individual privacy and the reputational risks data owners may face if weaknesses in their data are revealed or if the data is used to draw conclusions or narratives that are unfavourable to institutions and their leaders.
While these risks should not be taken lightly, real-world examples show that they are manageable and should be accepted for the benefit of society as a whole. For example, surveys show that citizens are in favor of sharing their data for research and statistics, as long as they are informed about how the data will be used and how it will be kept private. And even if sharing data exposes weaknesses in a nation’s data infrastructure or policy thinking, this should not be seen as a risk but rather a welcome challenge.
After newsletter promotion
Recent independent inquiries, including the inquiry conducted by Professor Dennis Leavesley on behalf of the Cabinet Office and the Parliamentary inquiry into transforming the UK’s evidence base, have all quickly reached the same conclusion: now is the time to rise to this challenge. Data sharing across government should be the rule, not the exception. But given the perceived risks, strong political leadership is essential to make this happen. The decision to centralise responsibility for data and digital policy in the Department for Science, Technology and Innovation could help bring a new focus, but will need the enthusiastic support of Downing Street, Cabinet Office and the Treasury.
The new government has a new opportunity to set expectations and drive a data-sharing revolution across Whitehall and the wider public sector. If the government is serious about being “mission-led” – defining and pursuing goals that cross departmental boundaries – it will be even more important that it seizes this incredible opportunity.
Sir Robert Choate is chairman of the UK Office for Statistics. Statistics Regulatory Agencybody regulation function, Office for National Statistics, National StatisticsAl Research Institute
