AI is exposing the real problem in enterprise IT: infrastructure

Artificial intelligence is changing the IT landscape in radical, unprecedented ways. It’s rewriting the rules of code generation, automating customer service and providing data insights that used to be impossible to extract.

For IT leaders across Australian organisations, AI represents a massive shift in infrastructure requirements. It’s a tool that needs to be used strategically, and that strategy starts at the hardware and virtualisation layer.

As organisations develop their AI strategies, infrastructure must be considered from the outset. You cannot build a skyscraper on unstable ground, and you can’t build a reliable, secure AI strategy on legacy infrastructure that wasn’t built for high-density, GPU-accelerated workloads. This is where the conversation shifts from software to the bedrock of your data centre.

Broadcom’s VMware Cloud Foundation 9 (VCF 9) is designed to support this shift, bringing AI into the mainstream of enterprise private cloud infrastructure and enabling it to be managed at scale.

For IT leaders in Australia, the challenge is how to adopt these capabilities without significant capital investment or adding unnecessary complexity to already diverse IT environments. This is where infrastructure strategy becomes critical. Many organisations are looking to Infrastructure as a Service (IaaS) models built on proven VMware-based platforms to provide a more flexible and scalable foundation for AI initiatives, helping reduce the need for major infrastructure changes as AI workloads grow.

The infrastructure gap in AI adoption

Most AI discussions focus on the models: large language models (LLMs), generative pre-trained Transformers (GPTs), and neural networks. However, successful AI adoption increasingly depends on having the right infrastructure in place to support these workloads at scale.

AI workloads require significant compute, fast storage, and low-latency networking. Traditionally, supporting these workloads meant building dedicated silos of hardware, such as racks of expensive GPUs that sat idle when not training models, or shadow IT environments created by data teams to bypass slow provisioning processes and corporate security protocols.

This creates an infrastructure gap. On one side, you have data scientists who need speed and agility. On the other, you have IT operations demanding security, governance and cost control. Bridging this gap requires a platform that treats AI resources, especially GPUs, much more like any other virtualised resource (CPU and RAM). This includes pooled capacity, policy controls, and clear governance, instead of one-off hardware silos.

VCF 9 addresses this by unifying AI and traditional workloads within a single private cloud platform.

A new standard for private AI

VCF 9 supports the next phase of enterprise AI adoption through private AI, which balances the power of AI with the privacy, sovereignty and compliance requirements that are especially important for Australian organisations.

This is particularly relevant in Australia, where data sovereignty, Australian Prudential Regulation Authority (APRA) requirements, and sector-specific regulations across financial services, healthcare and government strongly influence technology decisions.

Rather than sending data to public AI models, organisations can bring AI to their data, maintaining greater control and security.

Key capabilities of VCF 9

• Centralised model management: AI models can be stored, versioned and governed from a central repository, improving security and control.
• GPU as a service: GPU resources can be shared and allocated across teams and workloads, improving utilisation and reducing waste.
• Integrated Kubernetes: Kubernetes and virtual machines can be managed within the same platform, helping simplify operations in hybrid environments.

The role of IaaS in your AI strategy

Understanding the capabilities of VCF 9 is one thing. Implementing and maintaining the infrastructure required to support it is another.

Building AI-ready private infrastructure requires significant investment, specialised power and cooling requirements and ongoing hardware management.

As a result, many organisations are exploring Infrastructure as a Service (IaaS) models to access secure and flexible infrastructure without needing to build and manage everything internally. This approach can help organisations scale AI workloads and containerised applications more efficiently as their AI strategies evolve.

Security and governance

AI introduces new attack vectors and compliance challenges. For organisations running AI workloads in private cloud or on-premises environments, the underlying infrastructure must be designed and implemented with security and governance in mind.

As AI adoption grows, organisations are placing greater focus on secure infrastructure, compliance requirements, and operational resilience, particularly in sectors such as healthcare, financial services and government.

Maintaining control over data, AI models and workloads while operating within secure and resilient environments is becoming an increasingly important part of enterprise AI strategy.

Preparing for tomorrow

VCF 9 reflects the growing connection between AI and infrastructure. Capabilities such as GPU sharing, integrated Kubernetes, and centralised model governance are becoming key requirements for enterprise IT.

As organisations continue their AI journey, infrastructure will play a central role in enabling performance, security and scalability.

Next steps for IT leaders

If your organisation is exploring AI, now is the time to assess whether your infrastructure is ready.

Audit current capacity: Review compute, storage and networking capacity to understand current limitations. Assess VCF 9 capabilities: Consider how features such as GPU support and Kubernetes integration align with your technology roadmap. Review infrastructure options: Explore how Infrastructure as a Service (IaaS) models can support AI adoption without large upfront investment.   A strong AI strategy depends on having infrastructure that can support performance, security and future growth.

Top image credit: iStock.com/MikeMareen