And, risking on making this piece another one in that series, they got it right … Well, part of it! Surely there are a lot of hidden value opportunities in looking through your data and extracting knowledge from it, but upon research, you will find that rather than a gentle wave, data mining is more of a relentless tide of concepts seemingly unwelcoming for beginners.
Let us navigate said tide, breaking down some prominent data mining concepts from a business perspective, while discussing how they can translate into a more efficient and profitable operation. The entry-level concepts of associate rule mining, clustering and anomaly detection all have an intuitive philosophy and they have been broadly adopted by analytics-savvy organizations, as you will see from the business cases that we cherry picked from various activity sectors.
Associate rule mining is all about crunching transactional data – let us say every in-store transaction for a given retailer – searching for rules that map buying patterns and that are meaningful for the business.
Billions of transactional records can be reduced into a manageable set of rules of type occurrence A leads to B happening (A=>B). No records are skipped, no data is set aside due to time restrictions.
Companies can then identify precursor products to which they apply aggressive pricing strategies, create product or service bundles, promote targeted offers for customers who already acquired A but not yet B, in short, they have the power to reorganize their value propositions based on the tacit rules that buyers use while shopping. This sort of principle helped massive loyalty programs get going and it is still a driving force that is pushing them along (if you want to expand on customer-driven promotional portfolios, read this insight).
Humans have a limited cognitive window that makes us effective while dealing with single-digit numbers of items, but rather useless when addressing thousands or even millions of individual cases. Additionality, human cognition is limited by the hypothesis we are creative enough to formulate.
An analyst will not test the data in areas where he/she never thought to look. A machine, however, does not fall into such pitfalls. It leaves no stone unturned, going far beyond intuitive relations.
Clustering is a dimensionality reduction strategy that aims at finding naturally emerging patterns in existing data, congregating millions of records into a more concise set of groups that share a somewhat identical behavior. Planning the product assortment of thousands of retail outlets might sound like a dreadful task while doing the same for only nine types (clusters) of store locations seems painless (check this success case).
Setting up challenging but fair performance goals for a sales team comprised of hundreds of people is a steep hill to climb until you cluster the individuals together and end up facing less than a dozen profiles.
Unexpected events are intrinsically sporadic, but they can heavily disturb the operational flow of any organization. A supply shortage may halve the sales volume of an article, a defective bearing may slow down an entire production line and a global pandemic (as we are painfully aware) can bring the world to a halt (if you have the time, check out our piece on market modelling in the pandemic theater).
Anomaly detection is a collection of statistical and machine learning approaches that aim at identifying (and eventually correcting) outlying data points generated by events whose effect is too prominent to be neglected. As a rule of thumb, anomaly detection should be the entryway to any planning or forecasting exercise.
It is far too common to find budgeting exercises that skip it, ultimately underestimating the sales potential of a product hit in the past by a supply shortage highly unlikely to be repeated, while casting a spotlight on references that spiked in sales due to concentrated purchases from a single customer that is heading out the door.
In short, data mining is a complex field that is expanding its numerous arms even further but be rested assured that is not just academic gibberish, in there are tangible efficiency gains you can draw on today.
This article might not have sent you straight to the ocean, but it just may have gotten you interested in the smell of the sea breeze. Get going and head out to sea, it not nearly as daunting as it initially appears. After all, in analytics, you can dial up the intensity of the waves as you progress.
