Move Up the (Data) Property-Ladder

Retailers are data-driven businesses. But today, analytics are often limited to basic ‘pre-canned’ reports shared with small internal populations. Usually they provide a snapshot of a specific function or business process. And they are usually rearward looking. Our own research shows that best-in-class retailers are running a couple of hundred thousand queries a day and driving perhaps £3bn of annual value as a result. More traditional retailers are performing far fewer than that and seeing proportionately less value.

In the near future, retailers will need to run millions of queries everyday just to compete. We expect that those that can do so will see an expected ROI of £15bn per year. But scaling to these levels needs an enterprise-wide vision and a strategy to overcome conceptual and structural barriers embedded in current practices. CEOs need to buy rather than rent the data estate to support growth.

How to run 50 million queries per day

To really capture the economic advantage of being a digital business, retailers must be able to predict and organise around models of every product in every store, every delivery truck on every route, and every customer in every channel. Data points as varied as stock availability and location, weather reports, traffic conditions, and local events, not to mention the preferences, previous purchases and social media comments of individual customers must all be considered. That means scaling to 1,000s of concurrent ongoing model queries and millions of hyper-personalised interactions. That’s approaching 50 million queries per day!

As noted previously, retailers already have plenty of data. They need scale to analyze all of these data points in real-time. Plus, insight is no use without action. So, the outputs of analysis must be available across the organisation and in every channel at the same time to ensure everyone has the information needed to get the right products to customers in the right place at the right time.

Scale horizontally as well as vertically

These numbers can be mind-boggling – so what does analytics at this scale look like in the real world? Let me share two examples.

A retailer wants to understand more about propensity to purchase specific products and promotions across its customer base. Using basket transaction data, it creates a model based on millions of observations of actual customer purchase behaviour. It applies that model to its 60 million loyalty customers. The whole process takes little more than an hour and gives the business insight into near-real-time customer behaviour that is immediately actionable. We call this approach Vertical Scalability.

Others benefit from what we call Horizonal Scalability. Take, for example, a typical grocer – with 50,000 products in 3,000 stores. Modelling the performance of every product in every store against a range of factors such as weather, time of day, day of week, etc. could create 150 million predictive models running every single day. Horizonal scaling allows them to do this. Repeatable and comparable analysis conducted at speed and scale provides actionable predictions and insights shared across the business. Suddenly data analysis becomes part of day-to-day operations with measurable impact on business outcomes.

Getting beyond storage and compute concerns

Data is an asset, and decisions around its strategic value must get beyond the cost of storing or analyzing it. A cost per query metric might look fine on a departmental project dashboard but becomes meaningless if you are running hundreds of million per day. If data is really an asset, then retailers need to move beyond measuring it as a cost. The scale, frequency, pervasiveness and strategic value of the data the organisation owns goes far beyond the costs of storing and processing it.

Yet many organisations opt for per-use, or per query metrics to try to define the value they get from their data. Cloud-based solutions offer ‘rental’ models which may deliver short-term gains and flexibility, but as anyone can tell you, owning real estate is far more cost effective in the medium- and long-term. What’s more, just like real estate, careful management, investment and development of your data asset will ensure it gains value.

To stretch the analogy, many retailers find themselves renting low-cost apartments and lock-ups scattered all over town, and then paying to move data between them every time its needed. Or worse, paying to furnish duplicate facilities. The up-front cost of buying and equipping a house is more, but it ensures all your data is in one place and can be combined, analyzed and used to create value. What’s more, it is safe and beyond the prying eyes of nosey landlords.

With all your data in one ‘residence’ or data platform, you can optimise the furnishings and layout to best meet your needs. Finding the right combinations of hardware and software to deliver the right mix of speed, scalability, and cost for your business, with the ability to extend or redecorate as your usage changes. Our calculations suggest that at scale you could be spending eight times what you need through poorly optimised rented infrastructure.

More importantly, a well-architected data platform will create the firm foundations for the radical simplification needed to drive effective, repeatable and automated data-driven processes necessary for hyperscale growth.