Retail Data: Signal vs Noise
Our love for data
In the 21st century, we LOVE having data around to aid in our operations. We use it to celebrate successes and lament over failures. We use it to make hiring and firing decisions. We use it for planning new campaigns and killing old ones.
But what if we were using it wrong? The horror...
What do you mean wrong?
I mean you might be listening to the wrong indicators.
Don't fall for this rookie mistake. Learn to recognize the root cause of issues. Failing to recognize the difference between good signals and white noise can be fatal in the long term.
It's akin to assuming that your vehicle is working fine just because you haven't had an accident.
Signal vs Noise
Signal: Root indicators of the health of a business/ initiative and or campaign.
Noise: Distractions that will lead you astray in the long-term.
In today's data-rich environment, taking the easy path and only tracking the daily sales numbers may give teams the false impression that the business is doing well. This myopic form of analysis has been exacerbated by the overly simplistic operational dashboards that we see everywhere today.
I'm willing to wager that more than 60% of e-commerce practitioners still think of revenue growth as an indicator of long-term success.
That is entirely false. (or at least only half true.)
Sales is a lagging indicator of success.
To get to the sale, there are at least 4 other major stages that we should note...
- Discovery of Need
- Refer/ Repeat purchase
It would be embarrassingly simplistic to brag about a 10% month over month sales growth when you had to double your marketing budget every single week just to make the numbers work.
Two Examples of Noise
In order to get to the truth, we have to first learn what the false positives look like...
Whenever an online promotion doesn't work, we usually assume that it was due to the lack of appeal of said promotion.
We tend to look to tweaking the conditions (e.g. shopper segmentation, minimum spending amount) or the type (e.g. dollar off, percentage off) of promotions but fail to consider other factors.
Noise: Promotional Conditions
Possible Signal: Impressions
Whenever we run a new online promotion, it's not enough to simply switch up the design of the homepage. It is usually accompanied by lots of outreach campaigns to ensure that the message gets pushed out.
Start at the impression level! Did the social media post for this campaign get the same amount of social shares? Was the search ad budget cut for this particular period? Was there a content strategy to drive organic traffic for this promotion?
Bonus: In 2-3 years, even understanding impressions will not be enough. We more advanced tools such as Facebook Audience Insights, bleeding-edge marketers will be trading notes about audience segmentation.
For those of you who have already implemented a multi-channel/ attribution funnel on your website. Good on you! You are way ahead of the curve.
For those of you who haven't, this is a short description:
Attribution: The practice of scoring channels, initiative or campaigns based on their ability to drive sales.
Noise: Last-Click Attribution
Possible Signal: Multi-Funnel Scoring
Imagine you are a brand marketer.
1. You send me an email detailing the latest offers...
2. What if I clicked through, entered the site and then exited...
3. Then went back to Facebook and saw a video ad...
4. Then went to Google to look for alternatives and was shown another search ad...
5. Then I clicked on that ad and finally made my purchase.
Is it really be fair to assume that the search ad at the end was really what drove my purchase? Of course not.
We have to be careful here because if that were the case, we simply cut off email marketing because it would seem to have to impact whatsoever on the final purchase decision.
Pro Tip: Don't just score channels by the revenues they generate. Spend the extra enough to quantify the cost of those channels (E.g. A Facebook ad is not just the $20 per day bidding budget, it is also the time spent on collateral any software used to automate the posting)
The Atomic Unit: Cohorts
For those of you who got lazy and simply scrolled through. STOP.
This section is important.
If you take anything away from this post is that if you have to track the long-term sustainability of your business, don't track the month-over-month revenue growth. Track your month-over-month customer retention & referral.
Cohorts are the definitive indicator of the longevity of your business.
Some venture capitalist who even go so far as to term it as the “atomic unit of measurement”.
A cohort, in Jan 2020, will consist of
1. New Shoppers
2. Old Shoppers
In this segment of “Old Shoppers”, it can be broken down into:
– From Dec 2019
– From Nov 2019
– From Oct 2019
– From the first month
A cohort analysis will reveal the long-term health of your business and this historical analysis (adjusted for optimizations done over time) should reveal how well your stores will perform in the long-term.
Depending on when you started,
You will have to start worrying if every month more than 90% of sales came from new customers.
It means that you have to work MUCH harder next month just to see revenue growth.
Building a long-term data strategy
In essence, this strategy comes down to establishing the infrastructure required for analytics teams to move fast and execute. These teams need three things-
Depending on the contents of the raw data, some departments might be uncomfortable disclosing the numbers to the analysis.
Say...a marketing department wildly overspends on an unsuccessful promotional campaign. It might be embarrassing for them to share these numbers with others in the organisation.
Your duty as someone part of this organisation (even if you don’t have a leadership title) is to encourage this sharing of data.
Like it or not, the success of this strategy hinges on the involvement of the senior management.
It doesn’t matter how much you invest in your analytics team if their work is not going to be organization by another department.
They are going to get jaded and just work on improving some other part of the organization while that portion remains as it is.
Individuals who possess strong analytical skills are in hot demand.
They are generally hard to recruit and retain. Don’t make them work with old tools that take forever to load.
Try letting them choose the tools that they prefer working with because they will be more aware of the limitations of the various tools and the data set available.