Building an AI Centre of Excellence That Actually Ships

I've reviewed the AI Centre of Excellence structure at about thirty enterprises in the past 18 months. Twenty-five of them produce strategy decks; about five of them actually ship production AI. The five aren't smarter, better-funded, or in better industries. They follow a different operating model. The pattern across them is consistent enough that I now use it as a diagnostic — if a customer's CoE structure matches the strategy-deck pattern, I can predict within an hour of conversation that they haven't shipped production AI in the last quarter, regardless of what their internal narrative says. If it matches the shipping pattern, they almost certainly have. Here is the pattern we've seen work, drawn from customers in London, Riyadh, Mumbai, Hyderabad and Lagos.

The strategy-deck CoE: what doesn't work

The pattern: a small central team (typically 3–6 people) reporting to the CDO or CTO, with the mandate to develop AI strategy, identify use cases, partner with business units, and govern AI deployment. The team is staffed primarily with strategy and product profiles. They produce a portfolio view of AI initiatives, a roadmap deck, and a governance framework. They don't ship anything themselves; they convene business units and external partners to ship. Three years later, the strategy deck has been iterated through versions 1–4, the roadmap has slipped twice, and the production AI footprint is similar to where it was at year zero. Most enterprise AI CoEs we've reviewed are this pattern. The diagnostic is whether the CoE has shipping engineering capacity in-house; if not, it's a strategy function with an AI label, not a CoE in the operational sense.

The shipping CoE: what does work

The pattern: a CoE that actually contains shipping engineering — typically 8–15 people including ML engineers, platform engineers, MLOps specialists, and a small product/strategy capacity. The CoE owns the AI platform that business units deploy on. The CoE ships the first version of each use case in partnership with the business unit, then transitions operations to a delivery team while keeping platform responsibility. The CoE measures itself by production AI deployments and operational AI metrics, not by strategy documents shipped. This is what the five enterprises we've seen actually shipping look like. The headcount in the shipping CoE is larger than the strategy-deck version because the work is different — shipping engineering needs engineers.

The reporting line decision

Where the CoE reports matters more than its name. CoEs reporting to the CDO are typically strategy functions with limited delivery authority. CoEs reporting to the CTO are typically engineering teams with delivery authority but limited business engagement. The shipping CoEs we've seen most commonly report to a CDO who has explicit delivery responsibility — sometimes structured as a Chief AI Officer role — or to a COO who has operational delivery responsibility for the AI portfolio. The reporting line determines whether the CoE has the political weight to actually deploy in production, and whether it has the engineering accountability to deploy at quality.

The relationship with the business units

Strategy-deck CoEs treat business units as customers: identify their use cases, recommend solutions, hand off delivery. Shipping CoEs treat business units as joint deployment partners: pair an engineer with a business analyst, scope the first deployment together, deploy together, transition together. The hand-off model produces a steady stream of strategy documents and a thin stream of deployed systems. The joint-deployment model produces a steady stream of deployed systems and a thin stream of strategy documents. The choice between these models is a strategic decision that mostly gets made implicitly by how the CoE is staffed and incentivised.

The metrics that distinguish the two patterns

Strategy-deck CoEs measure themselves by deck iterations, governance documents shipped, framework adoptions, and use cases inventoried. Shipping CoEs measure themselves by AI workloads in production, monthly active users on AI-enabled features, business outcomes delivered (revenue lift, cost reduction, cycle-time compression), and platform reliability (uptime, latency, eval scores). The metrics drive the behaviour; an enterprise that wants a shipping CoE has to choose shipping metrics from the start. An enterprise that's accidentally built a strategy-deck CoE can move it to shipping by changing the metrics — but it requires accepting that several quarters of strategy work doesn't translate to several quarters of deployed work.

What about external delivery partners

Shipping CoEs often work with external partners (firms like ours) for accelerator-based first deployments. The pattern that works: external partner brings the platform and the first deployment; the CoE pairs an internal engineering team with the partner; the second and third deployments are joint; by the fifth deployment, the CoE is shipping independently with the partner consulting. The pattern that doesn't work: external partner ships everything; the CoE governs but doesn't build. In the second pattern, the CoE never develops the muscle to ship independently and the dependency on the partner becomes permanent. Most shipping CoEs we've seen prefer the first model. /capability covers MindMap's AI Capability CoE engagement model.