The AI Procurement Paradox: Why the Safest Playbook is Now the Riskiest
Government's traditional playbook was designed to buy known quantities . . . but it's going to fail for innovative AI procurement.
Governments and public sector organisations worldwide are confronting the same systemic challenge: how to strategically procure Artificial Intelligence. The traditional playbooks are failing, and the search is on for a new model.
Nowhere has this challenge been brought into sharper focus than in the United States, where a recent White House mandate provides a powerful case study in the shift from diagnosis to direct action.
In August 2025, the United States’ White House Office of Management and Budget (OMB) issued a landmark memorandum,
‘M-25-15: Advancing Governance, Innovation, and Risk Management for Federal Agency Use of Artificial Intelligence.’
In line with the general speed of initiative roll-outs in all categories across the Trump Administration, ‘M-25-15’ isn't intended to be the quintessential dust-gathering report; it's a direct order. It mandates that by December 1, 2025, every Federal agency must implement specific safety measures and governance structures for their AI systems.
An even more “hot off the press” report – the September 2025 U.S. Government Accountability Office’s (GAO) "Artificial Intelligence: Agencies Report Progress, but Need to Fully Implement Key Accountability Practices" amps up the pressure further.
In short, the OMB memo has set the policy and issued the deadline, but the GAO document has highlighted the arguably chasmic gap between what agencies and departments are being ordered to do, and what they actually have the capability and the smarts to do.
Central to bridging that chasm is genuinely strategic procurement: acquiring the right AI capabilities in the first place. Not just some fuzzy use of the word “strategic.” Actually strategic. And strategic, in the context of Artificial Intelligence – with its characteristically uncertain outcomes, continuously evolving capabilities, and deep dependence on an agency's own unique data – does not necessarily carry the same definition, thinking, and set of processes as the procurement of well-defined goods and services with fixed specifications.
Deterministic vs Probabilistic . . . Procurement Professionals Have Been Struggling
Procurement professionals – and, ironically, even seasoned IT procurement pros – have been struggling . . . struggling to determine what they need, to articulate what they need, resultantly to actually procure what they need, and then ultimately to ensure what they bought, keeps meeting the required outcomes going forward. Their (certainly, hard-won) expertise is in de-risking the purchase of complex but ultimately predictable systems, defining known a quantity and required outcome specifics down to the last micro-detail. But AI is not a known quantity. It is not a static piece of software with a finite feature list; it is an evolving capability that learns from, and creates its outputs from, data.
To re-frame the same problem: Traditional software, no matter how complicated, is deterministic. It operates on a fixed set of pre-programmed rules. In the most general sense, given the same inputs, it will always produce the same output. The very essence of AI is probabilistic. It doesn't follow rigid rules; it identifies patterns in data to generate its outputs. Thus, the expertise required to buy a predictable, rule-based system is fundamentally different from the expertise needed to acquire an evolving, pattern-based capability.
In short, this new domain comes with highly unfamiliar core technical realities – and navigating the strategic implications of those realities will be the root and the starting point of successful AI procurement outcomes.
The Architecture of A New Playbook
Diving deeply into the problem and spending enough time evaluating it begins to suggest exactly what the strategic navigation of the component challenges would look like.
The influential think tank, the Brookings Institution, in its 2022 report, 'Why government struggles to buy AI: And what to do about it', provided a deep-dive, clear diagnosis of the problem in no uncertain terms, essentially demonstrating that the government's traditional playbook for buying technology is fundamentally broken when it comes to AI.
These traditional methods are not prescriptive enough to guarantee safety, yet not flexible enough to foster true innovation. And certainly, overall, not agile enough. Not agile enough to manage uncertainty, to adapt to evolving technology, or to engage with the innovative start-ups that are building the future.
The Brookings report made clear that this is a systemic challenge that’s been quietly building for years. The evidence for this long-standing, structural mismatch lies in their core finding: "Traditional procurement methods, which are designed to acquire well-defined goods and services, are ill-suited for AI, which is an evolving capability that requires experimentation and iteration."
The New Playbook: From Procuring 'Things' to Procuring Outcomes
Speaking as a strategist, once the problem is defined this clearly, the architecture of the solution starts to reveal itself.
And it's clear that consistent, sustained success is going to require a fundamental paradigm shift.
The traditional RFP asks for answers. An AI-ready procurement process must learn to ask brilliant questions. It moves from, "Here is the exact thing we want; give us a price" to "Here is our core operational challenge or strategic opportunity; propose how your AI capability can solve it."
Procurement professionals can, though, take comfort in the fact that this isn't about abandoning certainty. It’s about building a new kind of certainty – a certainty based on process and performance, rather than on fixed specifications.
Building 'New Certainty'
So what would that process look like in a procurement, in practical, in-action terms?
A search for, and investigation of, the ideal solution would, arguably but very logically, start here: Instead of a single, massive contract based on promises, modern procurement builds confidence through three key phases:
Certainty of Viability:
Before committing to a multi-million dollar program, agencies can award multiple smaller, fixed-price contracts for 90-day pilot projects. The goal isn't a finished product; it's to answer one question: "Can this vendor's approach actually work with our messy, real-world data?" The certainty being purchased is proof. This "try before you buy" approach, championed by government innovation hubs like 18F, a technology and design consultancy within the U.S. General Services Administration (GSA), de-risks investment by making "small bets" to find out exactly what works – and what doesn’t – before scaling.
Certainty of Performance:
With a viable partner identified, the contract would logically shift to focus on outcomes – but the what, or the satisfactory nature of the results, rather than the how. As opposed to the conventional "build a system with X features" approach, it's "deliver a system that achieves Y result" contract. For example: "The final system must identify fraudulent transactions with 95 percent accuracy on live data." The certainty being bought is a performance stipulation. Thus, it’s still a non-negotiable measurable.
Certainty of Cost:
No procurement professional wants to sign a blank cheque. Structuring the overall contract in delivery-and-projectable evolution phases, each with a clear budget control, would provide the guardrails against doing so. For example: "Phase 1: Deploy to the first department for $X. If performance metrics are met, we will then authorise funding for Phase 2." The certainty being purchased is budgetary control, ensuring the next step is funded only if the previous one was a proven success.
Managing An Evolving System: Guardrail Specifics
The continually, and rapidly real-time, evolving nature of Artificial Intelligence makes it clear no contract award will end up being a one-shot, “implement-and-call-us-if-you-need-us” one-time transaction (unless your procurement and your provider were indeed a lemon); it's going to be, quite essentially, the beginning of a structured partnership.
The procured “solution” (to resort to standard IT terms that don’t quite fit the new AI dynamic) is a "living" capability that will continue to evolve. To ensure it evolves in the right direction, a framework of human-centric oversight is essential. Logically, this would include at least the following policies and practices:
Constant Performance Monitoring:
The contract must specify that the vendor provides a real-time dashboard tracking the AI's vital signs (e.g., accuracy, false positive rates). If a key metric dips below the agreed-upon threshold, it triggers an alert, obligating the vendor to investigate and fix it before a major failure occurs.
Human Governance:
The agency or department must establish a cross-functional governance team – an 'AI Board', if you like – to provide strategic and ethical oversight. This board would meet regularly to ask the “big questions”: Is the AI's advice still aligned with our agency's policies? Has it developed any unforeseen biases?
Safe Testing:
The golden rule of AI procurement appears to be never to let a more advanced or updated version of the AI be deployed into the live operational environment without testing it first. For example, when a vendor proposes a new, "smarter" version of the AI, that new model must first prove it is better than the current one. This is typically done in a safe, offline "sandbox" environment, where the new version is run against recent historical data and its results are compared directly to the old version's. The update is only deployed if it is demonstrably better. It's the equivalent of a flight simulator for your AI.
In a forthcoming article, I’ll write about the powerful real-world example of this new approach as provided by the United States Defense Innovation Unit (DIU) and its "Other Transaction (OT) authority", which has compressed the procurement cycle to see it award prototype contracts in a stunning 60 to 90 days – proving that speed and governance can co-exist, an essential in light of the new U.S. governmental mandates.
References:
- Office of Management and Budget (OMB). (August 2025). Memorandum M-25-15: Advancing Governance, Innovation, and Risk Management for Federal Agency Use of Artificial Intelligence. The White House.
- U.S. Government Accountability Office (GAO). (September 2025). GAO-25-288SP, Artificial Intelligence: Agencies Report Progress, but Need to Fully Implement Key Accountability Practices.
- Engler, A., & Desouza, K. (February 1, 2022). Why government struggles to buy AI: And what to do about it. The Brookings Institution. Retrieved from: https://www.brookings.edu/articles/why-government-struggles-to-buy-ai-and-what-to-do-about-it/
- 18F, U.S. General Services Administration. (September 24, 2020). De-risking government technology projects. Retrieved from: https://18f.gsa.gov/2020/09/24/de-risking-government-technology-projects/
