A recurring problem in client discussions about AI is that organisations are still comparing models using headline specifications that sound important but often tell them very little about whether the model is actually right for the job. More parameters. Bigger context windows. More tokens. More compute. All great ‘headology’ if your selling your latest Frontier model you want organisations to get suckered in. However the snake oil hides a very real flaw in the sales pitch.
A useful analogy is a person at work. Think of ‘Parameters’ as your brains capacity. They represent long term learned capability such as the patterns, relationships and behaviours embedded during model training. Training data then can be seen as the education and experience. It shapes what that capability was exposed to and what the model learned before you ever interacted with it. The ‘Context window’ then can be visualised as your desk size. It determines how much information can be laid out in front of the model at once and this is where procurement conversations often go wrong. A person with a huge desk is not necessarily more intelligent. Equally, a model with a gigantic context window is not necessarily a better reasoner.
Then there is ‘RAG’, which stands for Retrieval Augmented Generation. ThisIt is a way of giving an AI model access to information it was not trained on, such as your internal policies, client records, technical standards or previous decisions, at the moment it answers a question. Stored in a structured architecture it becomes your institutional memory, you enterprise documents are the library your AI activities can consult.
Finally there is ‘Inference’. Inference compute is like a thinking budget, it is how much processing resource you allow the model to expend reaching an answer.
You may also come across ‘Fine-tuning’ is specialist training, often oversold, organisations should be realistic about what that means. Most are not going to retrain frontier models or somehow insert their corporate knowledge permanently into them. However what they can do is fine tune smaller or open models, use techniques such as LoRA (Low-Rank Adaptation) or adapters, train models on approved examples, specialise behaviour, terminology or classifications and tune models for narrow tasks. Increasingly, I would be surprised if much of what organisations actually want from ‘fine-tuning’ cannot often be better achieved through prompting, retrieval, tools, workflows and policy controls. It’s also a dam site cheaper than the compute needs when training models.
That distinction leads to a much more important architectural question. Why should an organisation send every problem to the same model? The truth is they should not.
This is where that other funky term you will hear comes into play, the ‘harness’. An enterprise AI harness is a piece of orchestration technology, architected to sit between users, corporate information and multiple AI services. Its role is to determine whether a task needs a frontier reasoning model, a cheaper commodity model, a specialist internal model, several models working together or (hush) no AI at all because the answer sits in a boring old relational database that some thoughtful developer interfaced so it was available through your RAG.
The model therefore becomes interchangeable intelligence rather than the place where the organisation accumulates its memory. That second point may prove even more important.
Institutional knowledge should ideally remain under organisational control. Policies, decisions, previous interactions, client context, risk positions, corporate terminology and business history can reside in an enterprise controlled memory and retrieval layer. The harness exposes only the information necessary for a particular task to whichever model is selected. That produces a fundamentally different architecture:
- Models provide intelligence.
- The enterprise retains memory.
- The harness decides how the two meet.
This is why the current obsession with AI model leaderboards and context window sizes risks missing the strategic issue entirely.
Finally, we cannot move today without compliance adding its friction into almost everything. This is where the governance dividend in keeping institutional memory inside the enterprise pays off. If your policies, decisions, client context, risk positions and historical knowledge remain in an organisation controlled layer, then retention, residency, access, deletion, legal hold and audit-ability can be governed once, at source. The harness can also be audited to demonstrate compliance as the decision making layer as to what each model is allowed to see, stripping out what it does not need, restrict providers or jurisdictions, require human approval and log every material interaction. That is a far more defensible position under GDPR, the EU AI Act, DORA, NIS2 and sector regulation than allowing institutional knowledge to seep into a patchwork of external AI services, no matter how compelling the sales language of the AI vendors.
Think in simple terms, govern the memory once, govern access by policy and keep the models disposable (sorry Anthropic, OpenAI etc, but that is you place). When the regulator, insurers or other accountable body inevitably asks, ‘What did this model know, where did it get it from and where did that data go?’, you should not need a forensic expedition across half a dozen vendors to find out you cannot realistically answer the question.
So the message to procurment and boards infatuated with sporting the latest ‘frontier’ model, the winning enterprise AI architecture may not be the one with access solely to the biggest model, it will be the those that can continuously choose the right brain, give it the right desk, open the right books, allow the right amount of thinking and most critically never hand over ownership of the library.
Posted on August 29, 2026
0