Decoding buzzwords: big data, predictive analytics, business intelligence

I must admit, I can get easily confused in discussing predictive analytics, or business intelligence, then adding the term big data in the mix, the sense-making becomes more challenging. With this market being so enormous and promising claims of unparalleled competitive edge, a few simple insights to decipher what’s in the market playbook – can only help us all learn more rapidly.

What is business intelligence (BI)?

First caution, please do not confuse traditional business intelligence (BI) with predictive analytics. Forrester Research (2013) defines business intelligence in one of two ways:

• Business intelligence is a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making.
• When using this definition, business intelligence also includes technologies such as: data integration, data quality, data warehousing, master data management, text and content analytics, and many others that the market sometimes lumps into the information management segment. Therefore, analyst firm Forrester Research refers to data preparation and data usage as two separate, but closely linked segments of the business intelligence architectural stack.
• Forrester defines the latter, narrower business intelligence market as just the top layers of the BI architectural stack such as: reporting, analytics and dashboards. BI is about reports, dashboards, and advanced visualizations.

What is predictive analytics?

Predictive analytics is more complex and requires advanced skills in engineering, coming from diverse backgrounds in artificial intelligence (AI), machine learning, advanced mathematics, and advanced statistics and one could argue, this is a sub-segment of the business intelligence market.

However, one would find the traditional approaches BI to be very model centric, with high professional service costs, versus being in the cloud, plug it in, cost-effective, and securing rapid predictive insights in real-time. These are fundamental differences in what is happening – so customers need to be far more informed to make wise investment decisions. A few pointers:

• Predictive analytics uses artificial intelligence and machine learning algorithms on large and small data sets alike to predict outcomes. But predictive is not about absolutes; it does not guarantee an outcome. Rather, it’s about statistical probabilities.
• Predictive analytics is the practice of extracting information from existing data sets in order to determine patterns and predict future outcomes and trends. Predictive models and analysis are typically used to forecast future probabilities with an acceptable level of reliability.
• Applied to business, predictive models are used to analyze current data and historical facts in order to better understand customers, products and partners and to identify potential risks and opportunities for a company.
• Predictive analytics is business intelligence technology that produces a predictive score for each customer or other organizational element. Signaling these predictive scores is the job of a predictive model, which has, in turn, been trained over your data, learning from the collective experience of your organization, both from past and current data patterns– the accumulation of all directions at once – finding the hidden math for a an more informed future possibility.

What is big data?

Big data is the term for a collection of data sets that are so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include: capture, curation, storage, search, sharing, transfer, analysis and visualization (Wikipedia, 2014). With increasingly pervasive big data environments, companies must not only sense the present, but also see the future and proactively shape it to their advantage.

The market for data is exploding, just a few factoids:

• 2.7 zetabytes of data exist in the digital universe today.
• The Obama administration is investing $200 million in big data research projects.
• IDC estimates that by 2020, business transactions on the Internet, both for business-to-business and business-to-consumer, will reach 450 billion per day.
• Facebook stores, accesses, and analyzes 30+ petabytes of user-generated data.
• More than 5 billion people are calling, texting, tweeting and browsing on mobile phones worldwide.
• Decoding the human genome originally took 10 years to process; now it can be achieved in one week.

To put this in perspective, the production of data is expanding now at an astonishing pace. Experts point to a 4300 per cent increase in annual data generation by 2020.

How big is the predictive analytics market?

This market is big business. Business advisors McKinsey & Company is already making claims that predictive analytics in big data can mean 10 per cent continued annual growth rate (CAGR) to companies. Bottom line – you cannot afford to have a business plan that doesn’t leverage big data and predictive analytics. A few market factoids:

• The market for predictive analytics software is estimated to be worth US$2 billion today, and is expected to exceed US$3 billion in 2017 (Forrester Group, 2013).
• Other market research reports forecast the market for predictive analytics software to reach USD 6,546.4 million globally by 2019. The market growth is driven by increased demand for ‘customer intelligence’ and ‘fraud and security intelligence’ software.
• Cloud hosted predictive analytics software solution is seen as an emerging market and is expected to drive growth in the near future. Globally, the predictive analytics market was valued at USD 2,087.3 million in 2012 and is forecast to grow at 17.8% CAGR from 2013 – 2019 (Transparency Market Research, 2013).