Sovereign LLM Platform at a West African Tier-1 Bank — A Single On-Prem Foundation for 40+ AI Use Cases

We led with Sovereign LLM Platform (Sl) as the model-serving foundation, LLM Gateway (Lg) as the access layer (model routing, cost controls, rate limiting), Guardrail System (Gs) for safety, RAG Builder (Rg) for the use cases that needed retrieval, and Model Benchmarker (Mb) for ongoing model evaluation. Together these formed a platform that the bank's data-science teams, the business-line product teams and the third-party vendors building bank-specific solutions could all build against, with a uniform API and a uniform governance model.

Phase one was the platform build — twelve weeks to a production-ready cluster with three foundation models live (Llama 3.1 70B and 8B, plus Mistral 7B), the gateway and guardrails operational, SSO integration with the bank's existing identity platform, and the audit-and-cost-tracking dashboards live for the CDO's team.

Phase two — running in parallel from week eight — was the migration of the existing pilots. Each pilot's integration team was given a four-week window to refactor onto the new platform, with MindMap engineers embedded in each migration to handle the integration mechanics. By week twenty-eight, twelve of the legacy pilots had been migrated and the remaining four had been retired (in favour of a use-case rebuild on the new platform).

Phase three was the new-build acceleration. With the platform live, new use cases could be stood up in days rather than months — the plumbing was already there, the security review was a delta-only review rather than a full one, and the cost-tracking governance meant the CFO knew what each use case actually cost to run. Within the first year post-platform-go-live, twenty-eight new use cases were added beyond the migrated twelve.

The platform runs entirely inside the bank's primary data centre in the home market, on a dedicated GPU cluster. There are no outbound calls to any external LLM provider — the platform is fully air-gapped from the public internet. The bank's regulator audited the deployment and certified it as compliant with the draft data-residency directive in advance of the directive coming into force.

The GPU fleet is sized to handle peak concurrent load across all use cases. The largest models — Llama 3.1 70B for the heavier reasoning use cases (credit-memo drafting, complex case narrative generation) — run on a cluster of thirty-two H100 GPUs in tensor-parallel groups of four, served via vLLM with continuous batching. The smaller models — Llama 3.1 8B and Mistral 7B for the high-volume, lower-complexity use cases (intent classification, simple summarisation, document classification) — run on a separate fleet of L40S GPUs.

The LLM Gateway is the access layer every use case interacts with. It handles model routing (the same logical request can be routed to different physical models based on cost and capability), rate limiting per use case and per user, cost tracking with per-use-case showback to the business line, request and response logging with the bank's required retention, and the integration with the bank's SSO for user-level authentication and authorisation.

The Guardrail System sits in line on every model request, applying PII detection (with the bank's own configurable redaction rules), prompt-injection defence, jailbreak detection, and output content moderation. Different use cases can configure different guardrail profiles — the contact-centre chatbot has a stricter profile than the internal credit-memo drafting tool — but no use case can bypass the guardrails entirely.

The RAG infrastructure uses Qdrant as the vector database, with separate logical indexes per use case to enforce data-access boundaries. Embedding generation uses BGE-M3 (multilingual) and a fine-tuned domain-specific embedding model for the more technical use cases. The platform team operates the RAG infrastructure; individual use case teams own their data ingestion and index management.