Now one of the hottest careers, data science has come a long way since its early days. To understand how analytics has transitioned to what we use today, let’s first briefly review the history of business intelligence.
As companies began moving their business’s online in the late 1990’s and early 2000’s, this inevitably led to massive increases in the volume and velocity of data creation. The first generation of business intelligence software emerged alongside this first wave in order to manage and analyze all this newfound data, although it certainly wasn’t widely adopted right away.
As social media became much more widely adopted in the mid-2000’s and data creation skyrocketed, the second wave of business intelligence emerged and became much more a necessity for many companies.
Even though more and more companies adopted BI and data analytics, these first and second generation tools certainly weren’t what you see today. Instead, they were entirely reactive—meaning you would feed in KPIs and metrics into a dashboard, set static thresholds for monitoring purposes, manually adjust these thresholds as new patterns and behavior arise , and then use traditional statistical analysis to try and derive insights from it. Often these static thresholds were only able to compare changes on a week to week basis, missing much of the granularity in the data.
Now that big data, AI, and machine learning are becoming ubiquitous in the corporate world, we’ve entered a third wave of business intelligence. In particular, the volume of data has gotten simply too large to deal with manually and with traditional statistical techniques. As a result, we’ve moved to an era of AI-based analytics that has shifted the paradigm from a reactive to a proactive approach.
In order to understand this shift in analytics, let’s first review how businesses handle data in order to make business decisions. To do so, we’ll review the difference between push vs. pull analytics.
The Difference Between Push vs. Pull Analytics
At a high-level, there are two main ways to take raw data, transform it, and get it into the hands of decision makers in a useful format: push and pull analytics.
Let’s review each of these approaches in a bit more detail:
- Push Analytics: A push approach to data analytics involves sending certain insights to end users, which are users then used to make informed business decisions with. Insights can be pushed to end-users for a number of reasons, although typically they will be pushed based on certain thresholds being hit, or they will be simply be pushed on a periodic basis.
- Pull Analytics: A pull approach to analytics, on the other hand, pulls certain insights in order to answer specific business questions. This means that, since thresholds won’t have been hit, the end-user will often need to perform further data analytics such as running queries or performing root cause analysis in order to be able to use the insights for better decision making.
As we’ll discuss, in the traditional, reactive approach to analytics, both push and pull analytics have severe limitations that can cause anomalies and insights to go undetected.
Many companies have realized that only an AI-based analytics solution can simultaneously push anomalies and pull key insights to decision makers in a real-time, proactive approach.
Moving from Reactive to Proactive Analytics
Now that we’ve reviewed how companies deal with data from a push and pull perspective, let’s look at the shift from reactive to proactive analytics.
As mentioned, before the age of big data and AI, analytics was entirely reactive. It involved taking a raw data set, applying some form of analytical technique (typically statistical analysis), and attempting to find insights or patterns in the data.
The reality is, though, that even if the insights were valuable in making better business decisions, there is always a lag between the event and action with a reactive approach.
In today’s data-driven world, analytics often involves monitoring and analyzing thousands of metrics and billions of events each day. If companies still have this reactive approach to data analytics, revenue-impactful events can easily get lost in the sea of data. If these incidents are found, the slower time-to-detection still causes harm to the bottom line.
Proactive analytics refers to monitoring data in real time and analyzing events in order to either prevent incidents or to adjust conditions before a critical moment.
What many companies have realized, however, is that traditional statistics and static thresholds simply can’t monitor all these events in real-time. In addition, these tools can’t adjust to normal changes in data that occur every day—for example, normal shifts in consumer behavior due to seasonality.
Instead, companies have realized that they need to turn to AI and machine learning based approaches to monitoring and analytics. Specifically, a branch of machine learning called unsupervised learning is able to learn each individual metrics behavior on its own. This not only allows the system to monitor each data point and find correlations between others, but it can also detect anomalies in real-time and adjust to new normal behaviors of data.
As the shift to AI-based solutions has occurred, companies are now finally able to take a proactive approach to analytics.
An Example of Proactive Analytics: Retail Demand Forecasting with AI
To give you a real-world example of proactive analytics, let’s look at the incredibly challenging task of retail demand forecasting.
Any type of forecasting is challenging since we’re dealing with future predictions, but the fact that retail demand is a human-generated metric means that it is particularly complex, dynamic, and difficult to forecast. In particular, inventory planning requires the demand forecasting solution to take into account many factors such as seasonality, geographic location, and countless other factors.
Machine learning is changing the field of demand forecasting as it’s able to factor in the numerous data points available and create more accurate forecasts than traditional methods. As you can imagine, when it comes to inventory planning, producing more accurate demand forecasts ultimately leads to higher customer satisfaction and increased sales for any retailer.
Summary: From Reactive to Proactive Analytics
The early approaches to data analytics involved feeding in several KPIs and metrics to BI dashboards, setting stat thresholds for critical events, and applying traditional statistical techniques to extract insights from the data.
The issue that many companies realized is that, as the volume of data creation increased at an exponential pace, these reactive approaches led to incidents going unnoticed until it was too late. As a result, revenue was impacted and often the customer experience was damaged.
Now that machine learning and AI have become more and more prevalent in the business world, companies today can finally take a truly proactive approach to analytics. In particular, unsupervised learning techniques can be used to monitor 100% of the data and make adjustments to changes in real time.
As discussed in the example of demand forecasting with AI, this proactive approach to analytics is a significant competitive advantage for early adopters.
Amit Levi is VP of product and marketing at Anodot. He is passionate about turning data into insights. Over the past 15 years, he’s been proud to accompany the development of the analytics market. Having held managerial positions in several leading startups, Amit brings vast experience in planning, developing, and shipping large scale data and analytics products to top mobile and web companies. An expert in product and data, his mantra is “Good judgment comes from experience and experience comes from bad judgment.”