AI in Australia must be built as a data system, not just a compute system

For the past few years, the AI infrastructure conversation has been dominated by one metric: compute. For GPUs, CPUs, memory, interconnect speeds and power density these performance benchmarks have multiplied. When the immediate priority is to train large models and move AI from experimentation into real-world use, this focus on compute made sense.

However, as AI adoption matures across Australia, the structural gap between compute and data becomes critical. Training remains important, yet the next phase of AI will not only be defined by how much compute organisations can deploy, but by how much data AI systems will consume, generate, retain and reuse over time.

The difference becomes even clearer as AI moves into production and inference to drive business value. AI does not just use data: it creates new data continuously, from context and metadata to outputs, histories and operational exhaust that many organisations will want to retain for governance, model improvement or future use.

Different AI workloads also require different storage tiers — from data ingestion and training to inference and long-term retention — because each stage carries fundamentally different requirements for performance, capacity and cost. Once inference begins, the divergence becomes clearer: compute may scale in waves, but data keeps growing without pause.

Over time, AI production environments begin to behave more like data systems than pure compute systems, because the accumulation of data starts to define how those systems scale, operate and deliver value. This is especially relevant in Australia, where AI adoption is accelerating under different combinations of scale, cost pressure, energy constraints and regulatory complexity.

Scaling AI data in Australia

A report from Deloitte showed that APAC is set to become the world’s next data centre hub, with approximately US$800 billion in data centre investment expected by 2030. AI infrastructure planning can be complex: the APAC region encompasses a mix of hyper-growth digital economies, markets with established infrastructures and emerging AI native environments with different priorities.

Deloitte also highlighted that while Australia is better placed than many regional competitors to develop digital infrastructure due to factors like land, capital and supportive regulation, other countries are moving fast to develop their pipelines.

As such, the real bottleneck in AI is increasingly less about bursts of processing power but more about managing data at scale. As AI environments grow, organisations must support different tiers of data across the lifecycle: hot data requiring fast access, warm data used intermittently, and cold data stored for the long term. Collapsing everything into one single high-performance tier may work on a small scale; however, it becomes inefficient and economically unsustainable as data grows.

Australia’s AI growth will place pressure not only on compute deployment, but also on the broader data architecture that supports it: architecture now matters as much as raw speed. When it comes to scaling, it is about availability, durability, resilience and the economics of retaining and managing data over time. What matters now is whether the underlying architecture can keep pace as data volumes rise, workloads change and cost pressures intensify.

The long-term cost of AI

At scale, the total cost of ownership (TCO) of data storage is shaped by the cumulative impact of drives, power consumption, cooling units, rack space, and the operational burden of managing surging volumes of data.

This is why sustainability becomes inseparable from infrastructure design. The issue is not only how to power compute, but how to build AI as data systems that use capacity, energy and space efficiently. Not all data needs to live in the same performance tier. Matching storage resources to workload needs allows that.

For infrastructure leaders, this means treating sustainability and TCO as design priorities from the start. Assumptions made about retention, tiering, durability and availability have long-term consequences once systems are in production and become costly to revisit at scale. Organisations that build with the full data lifecycle in mind will be better positioned to scale AI in a way that is both economically sustainable and operationally resilient.

The next phase of AI will be defined by architecture

The industry is moving beyond a phase where AI infrastructure was framed mainly around chips, benchmarks and peak model performance. The next phase will be shaped by architecture choices that determine whether systems remain affordable, adaptable and sustainable as usage expands.

That means asking harder questions, such as:

• How much data should be retained, and for how long?
• Which workloads require premium performance, and which do not?
• How should organisations balance access, resilience, governance and cost?
   

These issues are central to whether AI can scale in a commercially viable and operationally durable way.

The next winners in AI will not simply be the organisations that deploy the most compute. They will be the ones that understand how AI systems behave over time and that build around the reality that AI creates intelligence, but also creates data. At scale, that data becomes the system.

Image credit: iStock.com/imaginima