AI doesn't make you special: your data and infrastructure do

It took only a couple of years, but LLMs have officially hit commodity status.

Previously, leveraging a large language model (LLM) might have given a business a competitive advantage. Today, it is merely the price of entry. When everyone is using models from open-source libraries, tapping into a hyperscaler’s APIs or leveraging frontier models, the competitive differentiation evaporates.

In fact, I’ve recently noticed a maturation in the market. IT leaders today care less about which model they’re using than they do about extracting value from whatever model they decide on. Driving this shift is the understanding that the model alone is no longer the source of competitive advantage: value comes from the organisation’s sovereign data and the infrastructure that brings the two together.

Two-tier intelligence

Exactly how an organisation measures the value AI delivers might vary from one entity to the next. At the end of the day, however, the simplest question to answer when measuring value is: did you get more out of it than what you spent? It is for this reason that the idea of two-tier intelligence is gaining traction.

As AI evolves from comparatively quaint chat interfaces and into the era of agentic AI, token economics is forcing a rethink of how models are chosen. AI agents run 24x7x365. Autonomous agents run tirelessly as they retrieve context, reason across multiple steps and interact with tools to solve complex business problems. As these agents consume and generate tokens by the millions to complete these tasks, the financial reality of rented infrastructure no longer stacks up — particularly when using the latest frontier models.

Rather than face the bill shock inherent in such a strategy, two-tier intelligence takes a more nuanced approach. With an agent gateway that sits securely between applications and models, requests can be intelligently routed with policies enforced across multiple LLM providers. This allows organisations to reserve expensive frontier models for the small fraction of tasks requiring that level of complexity, while the majority of high-volume tasks (data retrieval, standard code generation, formatting and continuous agent validation) are routed to open-weights models. Not only can these models be optimised for your own business and data, they can also run securely on your own private infrastructure.

Bridging the gap

Unlocking this level of control requires what is known as an AI factory. Despite the name, this does not require a development application from the local council. At its core, an AI factory is an infrastructure model that allows an organisation to manage both virtual machines (VMs) and Kubernetes.

The vast majority of your organisation’s data lives on VMs. AI applications and LLMs run best in cloud-native Kubernetes deployments. An AI factory allows you to bring the two together while simplifying and streamlining management of both from a single interface. This is crucial for AI success. Bridging the gap between the traditional world of VMs and AI’s cloud-native needs is pivotal to delivering value with AI.

Flexibility is the future state

Just as with the previous revolution in enterprise IT (cloud computing) the initial reaction for many was to go all-in on a single public cloud provider. Once the market had matured, the ‘cloud-first’ strategies that initially dominated evolved into ‘hybrid multicloud strategies’ that understood every workload has different requirements. We’re seeing the same evolution with AI: rather than commit to a single frontier model, different tasks will require varying degrees of sophistication, scale and cost.

With that in mind, the organisations that win in the AI era won’t be those who threw the most money at it. It will be those who built for control and made flexibility the foundation of their future state.

Image credit: iStock.com/da-kuk