Every business needs data to drive decisions, but how do leaders create an information system able to keep up with their changing objectives?
They used to call it business intelligence
Businesses have always needed data to make decisions. As far back as World War II, it could be argued that the power of data driven insights delivered unquestionable advantages. Prior to the Battle of Midway, the plans of the Japanese Fleet were signposted by US Intelligence teams. And Churchill was convinced that the code breakers of Bletchley Park were his biggest secret weapon.
Jump forward seventy years or so, and the world has shifted from thinking about data as something that gets reported at the end of a long analytical process, towards real-time data insights interpreted as they are created—Business Intelligence.
From reporting past events to living with data
The fundamental change we’ve seen in digital businesses is the opportunity to capture data from all points of the compass as part of work tasks, largely because more and more software robots and machine-to-machine interfaces exist. Instead of waiting for documents to be scanned, a human to write things down, or a data entry screen to be keyfilled, today business systems can capture data through robots and present insights instantly.
I don’t mean to say that business people don’t report on anything today—that certainly isn’t the case. There are still millions of organizations around the world that continue to use manpower and spreadsheets as a workhorse to find out things. For some at least, the promised digital summer never happened after the analogue report writing winter.
Where spreadsheets and self-service business intelligence tools fall down as reporting tools
Spreadsheets (like Microsoft Excel) create huge productivity wastage owing to the demands they place on humans to perform highly repetitive tasks that machines tend to be better at. But, push the issue of productivity to one side for a moment, and there remains the issue that spreadsheets are a poor solution for making sense of business data for a number of reasons:
1. They are prone to formulaic inaccuracies (creators making fundamental mistakes)
2. They suffer from version control (people referencing old reports by mistake)
3. They shroud original data as the purest evidence of data integrity (when you create data, or republish it in a spreadsheet, it’s impossible to drill down to where the original sourced data came from in the first place)
4. They don’t cope well with large data files (the data processing capacity of spreadsheets is often limited to your desktop processing power)
5. They are easy to manipulate
BI is popular but not pervasive
With all the hyperbole about digital business, and the need to be data driven, one might imagine that every business has an active strategy for embracing decisioning, but that isn’t the case. While Gartner predicted the Business Intelligence would hit $16.9 Billion in 2016, even now, the figure is only just hovering around $17.7 billion mark according to Select Hub. So, why hasn’t the growth appeared as industry pundits expected?
Speak to people in the ‘enterprise data industry’ and they think it’s more down to people than technology. Nick Lawrie is Managing Director of the data science consultancy, NDMC Ltd; a company with a 20 year track record in tackling some of the world’s most challenging data applications. Lawrie believes it’s ‘people issues’ that are holding innovation back. “Our experience working with management teams is that almost without exception, departmental managers are hungry to harvest and use data more readily, but they face resistance from business units and individuals who seek to retain control over their spreadsheeting empires. While executives can see arguments for cloud-driven data science solutions, the path to achieve this panacea is spanned by political and emotional barricades in many organizations.”
The shift from ‘known-decisions reporting’ to the powering of curious minds
It’s not all bad news for the business intelligence industry. This year, Information Week published research suggesting a third of large-scale organizations plan to adopt decision intelligence by 2023. They argue, one of the main reasons for this significant growth comes from businesses that see a role for business intelligence beyond reporting of outcomes.
Since the dawn of the business intelligence hype-curve, the primary driver for investing in tooling has been to analyze data in response to known questions. Organizations today increasingly want to equip their people with tools to start asking new questions of what they do, and how they do it; to equip the curious with the answers they seek by harnessing the value of known and new data.
Lawrie argues that, while the tools of the BI trade are similar to those that came before, their application is very different.
“Management teams want their people to be proactive in improving customer experience and business outcomes, but they can’t do that without benchmarking performance today, and asking new questions about processes, customers, use of resources and assets, etc. The bigger challenge for technologists and data scientists, becomes less about presentation of data, more about how to harvest data from multiple system silos, data quality, and integrity.”
The way forward
Finding solutions to powering the curious begins not with business intelligence, but in data abstraction, organization, aggregation, and quality enrichment—what enterprise IT architects would label the extract, transform and load (ETL) layer. Cloud technology is coming to market in the form of PaaS solutions like Mendix, Encanvas, Boomi, and Talend to put all of your data to work by providing the ability for decision makers to harvest and re-use their data for other purposes, such as asking new questions.
These new digital platforms bring the power of artificial intelligence and cloud infrastructure to bear in taking your data, helping to overcome the ETL and data integration challenges that have hampered business intelligence projects in the past. Only time will tell, whether PaaS technology solutions ultimately create a data driven business world.
