Business intelligence in the middle

Much of the trouble that organisations are having with business intelligence comes down simply to a lack of skills, analysts say.

While some mid-sized organisations in Australia are successfully implementing business intelligence (BI) programs, most are failing to capitalise on its potential, with analysts blaming a lack of data-related skills for this failure.

Mid-sized organisations in Australia don’t seem to be doing too well with BI, according to local analysts. IBRS analysts Joe Sweeney and Guy Cranswick said that in general, “most Australian mid-sized [organisations] are not using BI in any sophisticated way”.

To get an idea of how medium-sized organisations are using BI tools, we must take a closer look at what BI actually comprises. According to Sweeney, BI should be viewed as four separate but interlinked services, each of which addresses a different business need. Specifically, these needs are: reporting, self-direct data exploration, operational decision support and data science.

With this definition in mind, Sweeney and Cranswick say that some individuals within SMEs are using standalone data visualisation tools — such as Microsoft’s PowerBI — to analyse data for niche use cases, “but these are few and far between”.

The majority of mid-market organisations using BI tend to fall into two categories, the IBRS analysts said. The first of these involves extracting and massaging sales data in Excel. “This aligns to the ad hoc/data visualisation use case,” they said. “Basically, mid-sized organisations tend to be using such data in a manner which is similar to reporting, but easier to consume: they wish to understand how well their business is performing.”

The second category involves the use of built-in analytics dashboards/reporting within the organisation’s CRM, financial or productivity solutions. “The use of the ever-growing BI tools (customisable dashboards, KPIs meters, custom visualisations) embedded in mainstream business systems is often overlooked. However, a quick glance at the user interface of today’s modern accounting or CRM suites (eg, JCurve NetSuite, Sage 300, Salesforce, Saasu, etc) clearly shows how BI is now baked into day-to-day operations,” Sweeney and Cranswick added. “Certainly, such sophistication of capabilities were not readily available to the mid-market 10 years ago.”

Many mid-sized organisations are migrating to cloud-based software solutions for various business functions (such as CRM, sales, customer channel management and so on), and these tools already include BI functionality like the ability to create performance dashboards and KPI meters, the IBRS analysts said.

“Over the next few years these solutions will move beyond visualisation and reporting services. Some solutions (eg, Microsoft Dynamics, Sitecore, etc) will call upon cloud-based Machine Learning services (eg, Azure Machine Learning) to provide predictive analytics, especially in the areas of customer churn, product recommendations, forecasting, preventative maintenance, etc,” they said.

As a result of the mid-market latching onto these cloud-based, BI-loaded tools, BI will increasingly become “just a part of doing business”.

“However, use of specialised BI to identify then answer the questions that drive deep innovation will remain rare among mid-sized organisations. This is due less to technology and more to a lack of skills in data science, lack of awareness of the business potential,” Sweeney and Cranswick added.

Ian Bertram, managing VP at Gartner, said that the state of BI in the Australian mid-market “all depends on what you call a BI tool”.

He noted that some people consider Microsoft Excel a BI tool, adding that “the older versions of Excel, I would argue, are a bit of a duct tape solution”.

“However, Microsoft themselves have been investing in more data-governed, data-discovery environments. So even with the Microsoft products — which many of the medium-sized organisations have available to them — I would suggest many have a lot of the BI capabilities available to them,” Bertram said.

Have tools, won’t necessarily travel

But the fact that an organisation has some BI-capable tools in its arsenal doesn’t mean they’re being used properly.

“Whilst they might have … Power BI and PowerPivot, are they using them to their full potential? I suspect not. Just because someone has downloaded something doesn’t mean they’re actually going to use it,” Gartner’s Bertram said.

IBRS’s Sweeney and Cranswick reckon that organisations of all sizes — not just the mid-market — are failing to capitalise on BI’s promise.

“Alas, most organisations — large and small — are not doing well with BI.” The problem lies not with the technology, the IBRS analysts said, but rather the “underlying intention of BI itself”.

“In large organisations, we consistently see the organisation looking to BI to ‘solve business issues’, where few people can articulate the actual business issue to be solved. In mid-sized organisations, we see growing use of ad hoc visualisation tools, but again without a clear business outcome in mind.”

Organisations that are looking to BI tools should start with a clear and concise objective, such as “Identify how to reduce subscriber churn by 30% over the next 24 months”, Sweeney and Cranswick said.

“Only by asking the right questions, and then finding the appropriate data sources, and applying the required tools and techniques, can value be generated from BI investments. So the secret is to ask not what value can be generated from BI, but rather, what value do you need?” they said.

The role of skills

Much of the trouble that Australian organisations are having with business intelligence seems to come down to a lack of relevant skills in the workforce. To illustrate this, Sweeney and Cranswick point again to a current trend they’re seeing in Australia (and the US) — the uptake of ad hoc visualisation tools.

“Tableau, Click, Yellow Fin and more recently PowerBI are being used by people who would have traditionally used Excel to delve into data. While these visualisation tools certainly make BI more attractive and easier for non-specialists to work with data, they generally do not address a very important issue: the skills needed to assess data quality. Such skills are lacking in mid-sized organisations (and indeed in larger organisations where there is an ad hoc approach to data analytics in the mid-management),” they said.

This use of BI tools without the proper theoretical understanding of data-related issues can lead to some very misleading analyses. “The result can be an explosion of very interesting looking analysis, based on data that may be of questionable quality (not complete, selection biased, or simply wrong),” the analysts said.

Bertram reckons that when it comes to business intelligence, people often consider an investment in a particular tool or technology as “the silver bullet that’ll fix all of our problems”. However, he said, “quite often it’s more the investment in the capabilities of the people that is the silver bullet” that fixes many of an organisation’s problems. “Investment in the skills and capabilities is far more important than just the tools themselves.”

Note that Bertram isn’t just referring to a person’s ability to use BI tools — to click the buttons in the required order and produce some kind of output. He’s speaking of a more abstract understanding of issues surrounding the analysis of data: “the storytelling skills, the interpretation skills, the understanding the data skills; it’s not just understanding the tools themselves”.

The impetus for bringing in these skillsets has to come from the upper tiers of an organisation, Bertram said.

“Like in any organisation, if the CEO values the investment in the data, and wants to have a data-driven culture, then those types of things tend to happen. Even in a medium-sized organisation, if the person at the top doesn’t want to invest in these types of environments, and their decisions [to] be driven based on facts and understanding of data and understanding of markets, then these things aren’t going to happen. So it still needs to come from the top of the organisation,” he said.

However, Bertram noted that it may be quite difficult for organisations to hire someone with all the necessary education, training and skills — the “elusive beast that is a data scientist”.

“Don’t think you have to hire one — look within your organisation,” he said.

Bertram raises the idea of the ‘citizen data scientist’ — an employee who, although not trained as a data scientist, has some statistical background or a sense of curiosity, who actually uncovers some great things that are going on in your organisation.

“I’ve been advising many organisations, particularly in the last 12 months, you’ve really got to hunt out the people that are those citizen data scientists, because they’re the ones that have some of those skills. They might not have all of the skills of a data scientist, but they’re the ones you’ve got to invest and develop into what you might consider to be a data scientist type practice within an organisation,” Bertram said.

Other considerations

While a skills deficit is one problem facing business intelligence programs, there’s a variety of other questions for interested organisations to consider, also. The first of these is that old IT chestnut — who controls it, and who’s responsible for it?

There are several potential options for BI ownership. Briefly, these are: owned and operated by (1) the IT department; (2) the line of business (LOB); (3) a separate business unit devoted to BI; (4) some combination of those three.

Melanie Disse, ANZ market analyst, software and analytics at IDC New Zealand, said the decision of organisational ownership “depends on the use case”.

“BI capabilities are nowadays often embedded in ERP or CRM solutions, making it easy and affordable for SMBs to have access to BI capabilities. If all they want is an overall snapshot of the organisation it might sit with LOB (increasingly popular with CMO departments). There are a lot of affordable and easy-to-use (self-service) solutions that are quickly deployed and often cloud based,” she said.

IBRS’s Sweeney and Cranswick advise that: “Increasingly, medium-sized organisations will leverage BI built into their cloud-based ERP/CRM/Sales solutions and supplement these services with ad hoc data visualisation tools, also largely based in the cloud.”

There’s also the selection of vendor, which can seem complicated given the variety of vendor types, including traditional larger BI vendors, more recent self-discovery vendors, targeted industry vertical vendors, open source options and so on.

Disse said it again depends on your organisation’s specific-use case. “It should also be considered whether BI (often backwards looking) is what the organisation is actually after or whether a more sophisticated advanced and predictive analytics solution is more suitable to fulfil the needs.”

Vendor selection is also related to the question of who, in the organisation, has responsibility for the tools.

“[A] lot of traditional BI/analytics players have changed their portfolio or at least expanded their portfolios to include less expensive and less sophisticated solutions, often targeted towards non-technical end users with a self-service data discovery purpose. With the increasing competition from newer/smaller BI players, demand has shifted from big BI implementations under IT towards easier-to-use, less-technical solutions for LOB. With the lower price attached to those solutions, the entry point for smaller and medium-sized organisations has been lowered,” Disse said.

First steps

For those mid-sized organisations that are keen to pursue business intelligence, IBRS’s Sweeney and Cranswick offer a series of first steps. “The senior business executive in mid-sized organisations should start their BI journey by first identifying the questions that will make a difference to their business. What do they need to know in order to make decisions that will improve the organisations?” the analysts said.

Next, the executives should work with IT and LOB management to identify the location of the data needed to answer these questions. “Where this information is stored within existing solutions (especially cloud-based solutions), examine the extent to which these solutions can provide dashboards/reports — that is, use the built-in BI that already exists,” they said.

“Where ad hoc BI is needed, look to cloud-based data visualisation tools and assign non-technical teams to analyse the data. These teams may call upon IT for help as required. Also, where skills are lacking, consider hiring in specialist BI firms to assist and provide skills transfer.”

IDC’s Disse points out that there are a lot of supposedly ‘free’ trials available for BI tools. “Most vendors offer the option of a trial period these days.”

But beware: “[T]hey might not always be as ‘free’ as claimed,” she warned.

Image courtesy FutUndBeidl under CC