Bendigo and Adelaide Bank reduces compute costs by 60%

Financial service provider Bendigo and Adelaide Bank (Bendigo) started its digital transformation journey on Amazon Web Services (AWS) so it could comply with Australia’s open-banking mandates. In March 2020, Bendigo quickly migrated its infrastructure to the cloud using Amazon Elastic Compute Cloud (Amazon EC2), which provides secure and resizable compute capacity for virtually any workload.

Since completing its cloud migration, Bendigo has focused on improving its architecture so that it can increase cost savings and improve the performance and availability of its digital offerings. To balance performance with cost savings, Bendigo started running its nonproduction workloads on Amazon EC2 Spot Instances, which let customers take advantage of unused Amazon EC2 capacity in the cloud. The bank also achieved greater resiliency and scalability by implementing other AWS features for fault-tolerant workloads.

With this new infrastructure, Bendigo has reduced its compute costs by approximately 60%, increased the resiliency of its workloads by nearly 30% and improved the performance of its banking system by roughly 20%.

Meeting open-banking regulations by migrating to the cloud on AWS

Bendigo’s vision is to become Australia’s bank of choice by providing its 2.1 million customers with data transparency and simple banking solutions. The bank’s purpose is to feed into the prosperity of its customers and communities — not off them. Bendigo holds over $83.4 billion in deposits, and it serves over 110,000 shareholders while employing over 7000 people across 317 branches.

In May 2018, the Australian Government passed the Consumer Data Right legislation, which required all major financial institutions to comply with open-banking practices by July 2020. This legislation also meant that financial institutions must provide their customers with access to and control over their personal data. Facing these industry changes, Bendigo expected an increase in demand on its on-premises infrastructure. To prepare, the bank decided to undergo a digital transformation on AWS.

Bendigo chose AWS as its cloud service provider because of the maturity of its services, and in March 2020, it kicked off its cloud migration using Amazon EC2. The bank also began running its workloads using Amazon EMR, a cloud big data solution for running large-scale distributed data processing jobs, interactive SQL queries and machine learning applications using open-source analytics frameworks. To meet open-banking mandates, the company configured clusters that would help it maintain a high availability across its banking systems.

After completing this project, Bendigo wanted to improve its cloud architecture and reduce compute costs by running its clusters using Amazon EC2 instance fleets.

“Cost savings wasn’t something that we were thinking about initially,” said Adam Hobbs, development infrastructure engineer for Bendigo.

“But we realised that we wanted to improve the performance of our architecture.”

In 2021, the Bendigo team entered the second phase of its digital transformation journey, and the team began experimenting with new solutions and features to meet the bank’s computing needs.

Using Amazon EC2 Spot Instances to reduce compute costs

Bendigo engaged with the AWS team to test Amazon EC2 instances that would maintain a high availability while optimising compute costs. During this time, the Bendigo team trialled Amazon EC2 Spot Instances, which deliver Amazon EC2 instances at a lower price compared to Amazon EC2 On-Demand Instances. Because Bendigo’s developers use its nonproduction environment for iterating features for its digital offerings, the bank identified this as an area where it could reduce compute costs. Now, the bank runs its nonproduction workloads entirely on Spot Instances.

“Amazon EC2 Spot Instances have been an effective way for us to save on compute costs while maintaining an elastic workload,” said Leandro Silva, data lead for the open banking project at Bendigo. By making these changes, Bendigo has reduced its compute costs by approximately 60%.

The bank also tested different Amazon EC2 instances for running its Amazon EMR clusters, and it found the best performance using diversified Amazon EC2 instance types, which include varying combinations of CPU, memory, storage and networking capacities. By diversifying its instance types, the bank has reduced the likelihood of service interruptions and downtime and made its workloads more fault-tolerant.

“We went through stages of configuring different instance types,” Hobbs said.

“We spent some time with the AWS team identifying which instance types have the highest availability, which has delivered a much better experience.”

Under this new model, the bank has improved the resiliency of its near-real-time workloads by roughly 30%.

Bendigo’s Amazon EMR clusters consist of different node types, which use Amazon EC2 instances for cluster capacity planning.

“We have been setting up core nodes,” Hobbs said. “We went from a 5-node cluster to a 30-node cluster.”

To facilitate effective cluster scaling, the bank implemented task nodes, which it can spin up or spin down to increase capacity as needed and save on compute costs. Since then, the company has grown to use 60 core nodes for its production environment. With multiple task nodes, Bendigo's Amazon EMR clusters can use other available task nodes if one fails, minimising service interruptions and downtime. By increasing the elasticity of its infrastructure, the bank has increased the performance of its system by approximately 20% while driving down compute costs.

The bank also started using Managed Scaling for Amazon EMR, which automatically resizes its clusters for best performance at the lowest possible cost. Using Managed Scaling, the company can manage its compute costs more effectively by setting up minimum and maximum limits for its compute capacity.

“We gain deeper insights about our workloads, which helps us monitor our usage and true costs,” said Ash Austin, service owner of the cloud platforms team at Bendigo.

“Working on the cloud helps us enhance our risk and vulnerability management in a highly secure, robust and controlled environment.”

Building a data lake to enhance machine learning capabilities

Bendigo is currently modernising its core banking system on AWS so that all of its branches can use a unified banking system that provides transactional updates in near real time. The bank is also building a data lake using Amazon Simple Storage Service (Amazon S3), an object storage service offering industry-leading scalability, data availability, security and performance. By securely storing its unstructured data in the cloud, the bank is laying the foundation for integrating machine learning and artificial intelligence alongside its digital offerings.

“AWS has been great at helping our organisation find the right tools for the job and navigate the regulatory landscape,” Austin said.

“It brings us the global expertise that we need to accelerate our digital transformation journey.”

Image credit: ©stock.adobe.com/au/Jakub Jirsák