Governments can learn from Silicon Valley when it comes to AI and data science

A new study from the Oxford Internet Institute (part of the University of Oxford) and the London School of Economics explores how governments have gradually adopted Silicon Valley ways of working.

The changes offer valuable lessons for the UK’s new Labour government as it seeks to embrace AI and data science in public services. But the study also highlights ongoing challenges, including outdated IT systems, reliance on conservative technology providers, and the dominance of big tech companies.

The study, titled “The Political Economy of Digital Government: How Silicon Valley Companies Drive the Adoption of Data Science and Artificial Intelligence in Public Management,” was published in the journal Nature. Public funding and management.

The study shows that US, UK and Australian governments’ reliance on large traditional computer companies, known as “system integrators,” has slowed the modernization of public services. Civil servants have also been hesitant to adopt Silicon Valley methods, which has hindered progress for decades. To make better use of AI, the study suggests the public sector must avoid repeating the mistakes of the past.

Professor Helen Margetts from the Oxford Internet Institute explains: “Our research looks at how government contracting for information systems has evolved over time. For many years, civil servants working with very traditional global technology companies mistakenly believed that Silicon Valley approaches did not apply to the public sector.”

“But this thinking has changed with the growth of large platform companies, the adoption of Silicon Valley technologies such as cloud computing and AI, and the realization that these companies are often better at cybersecurity and managing sensitive information.”

The rise of Silicon Valley technology and cloud services has weakened the influence of traditional systems integrators and enabled new tools to be embedded across government departments.

Patrick Dunleavy, professor emeritus of political science and public policy at the London School of Economics, added: “Over the next decade, technologies from Silicon Valley and platform companies will shape how government handles data-intensive tasks. Digital managers within government will have a key role to play in accelerating or slowing cultural change, as well as developing the expertise needed to manage diverse technology contracts.”

While these technologies offer significant benefits for improving public services and citizens’ online experiences, the researchers urge government authorities to proceed with caution and ensure these companies are held to account.

Professor Margetts concludes: “A major challenge for the new government will be to build the expertise and capacity needed to take full advantage of AI and address the challenges it poses. It will also need to effectively manage relationships with both traditional technology providers and Silicon Valley giants, while regulating the technology market. Our research provides valuable insights to help it learn from past mistakes.”