Challenges in retail – Search for data

Management information is the life blood of any business, but some businesses have more complex needs than others. Retail in particular presents complex issues and challenges to providers and consumers of business intelligence software. In this article I want to take a look at what it is about retail that makes it unique, and to identify some of the challenges faced by retailers in the successful adoption of business intelligence techniques and systems.

First and foremost retail presents a complex multi-dimensional problem. Most business users will identify with analysing data according to Time, Cost Centre and Product or Service. Retail is special though in having to cope with thousands of SKUs across hundreds of outlets. In addition to this retailers constantly measure themselves against not just budget but against last year and last week, and are also addicted (often due to pressure from the City) to “like for like” analysis. As technology allows it, managers are also increasingly starting to demand near real time analysis. The volumes of data that these requirements need to support meaningful analysis can be quite staggering.

This multiplicity of data points brings problems of scale to retail business intelligence as the potential volumes of data explode into billions of data cells. For example just the annual daily sales data for 1,000 SKUs in 250 stores creates nearly 100 million potential base level data cells over the course of a year. If we have to calculate and store all of the potential summaries we encounter the phenomenon known as data explosion. It is not unusual to find that data explosion causes storage requirements to increase by a factor of 10, giving us nearly 1 billion potential data nodes for our sales data. Of course, not every store sells every item every day so we also have the problem of how to deal efficiently with the resulting data sparsity from a technical viewpoint. Continued rapid advances in processing power and sophisticated data management techniques developed by leading vendors mean that storing and analysing these vast amounts of data is now a practical proposition. This in turn has led to a new problem – Instead of asking “what data can we store”, we are now faced with the rather more subjective and challenging questions “what data should we store, at what levels and for how long?” The challenge of data relevance is fast becoming one of the most significant issues in retail business intelligence.

The sheer volumes of data required by retail merchants and finance executives used to result in the situation where whole rain forests-worth of reports were laid on people’s desks on a Monday morning (or, more often than not, Monday afternoon). However, this gave way long ago to interactive screen based reporting applications. Today, retail executives can expect to arrive in the office and instantly have access to reports that show them which areas are performing best and worst and where the key actions need to be taken. Yesterday’s leading edge of dashboards, traffic lights and balanced scorecards is now being superseded by a new generation of interactive and dynamic analytics where the software trawls the data looking for exceptions and reduces the incomprehensible clutter of data into succinct suggestions for actions to gain competitive advantage.

Another important characteristic of retail is that it is an incredibly fast moving environment. Today’s “Must Have” product is soon consigned to tomorrow’s bargain bin, and the pace is increasing rather than slowing down. For example, “fast fashion” retailers now carry stock levels as low as 4 weeks of supply whereas 10 years ago it would have been unusual to have carried less than 16 – 20 weeks stock cover. The concept of “newness” has become fundamental to successful merchandise assortments, and effective and targeted decision support for managing old stock out of the business can be the difference between making a profit or a loss.

At the same time supply chains are extending as purchasing becomes ever more globalised. For example the Asian region now accounts for nearly 50% of global exports in the world’s clothing industry. This means that finely tuned decisions about short life inventory investment are being made sometimes months in advance. Riding this knife edge calls for ever more timely and accurate data that has to be presented in meaningful ways. You might be forgiven for thinking that this problem is limited to clothing, but think again! With UK supermarkets selling sweet corn from Thailand, prawns from Ecuador and apples from New Zealand it is plain that grocery retailers also have supply lines that stretch across the globe creating complex cost structures that have to be monitored and managed.

As well as changes in the retail business model we are also seeing significant developments in the supporting IT structures that feed business intelligence. RFID, for example, promises to transform the accuracy of inventory movement data, but as we have seen implementations have not always been as rapid or as far-reaching as initially seemed likely. Having said that, improved data accuracy for inventory resulting from the adoption of RFID promises to be one of the highest value developments in retail business intelligence over the next few years.

Like all other industry sectors retailers are becoming ever more conscious of controllable costs. In a world of selling price deflation and constant pressure on margins the only route to increasing or even just maintaining profitability can often be effective analysis and control of these costs. The sheer volume and geographical spread of stores and administrative cost centres involved in retail means that whilst collating and presenting relevant data may be time consuming and complex, the potential for bottom line saving is enormous. This can only be realised through effective measurement using up to date business intelligence techniques. Recent developments in web enablement of business intelligence software mean that it is now a practical proposition to allow branch managers to access centrally held management information. More importantly they can also now collaborate in target setting, meaning that store budgets get real buy in and can leverage local knowledge in a way that was imply not practical when planning systems were tied to the desks of a few individuals in head office.

As we all know though, such measurement means having benchmarks and this means having effective planning systems. As Business Intelligence evolves, the artificial distinction that has sometimes been created between reporting, planning and tactical action is rapidly disappearing. Finance departments have been embracing the concept of Business Performance Management (BPM), and this in turn has spawned the new discipline of Operational Performance Management where planning and evaluation processes are streamlined and integrated across the entire business, allowing the concept of truly joined up planning to be realised.

Such planning exercises have typically been carried out at a macro, strategic planning, level, but business intelligence is also winning its spurs at the micro level as well. Predictive analytics takes the body of data collected and manipulates it using sophisticated algorithms to provide dynamic suggestions for tactical action. An example of this kind of application would be profit optimisation software, By integrating merchandising, optimisation and supply chain systems vendors claim to have increased sales volumes, margin percentage, and stock turn all at the same time. The bottom line impact of combining even small improvements in these KPIs is potentially immense.

We can see then that retail faces a variety of challenges and opportunities in the effective deployment of business intelligence. In such a swiftly changing and dynamic environment, flexible and responsive business intelligence is one of the key differentiators between the leaders and those who fall by the wayside. What we have to remember is that all the software and technology in the world won’t help a retail business on its own. Business Intelligence is just a tool, and the real benefits that can be derived from it come from supporting relevant and innovative management activity.

    Send Me a Message

    Looks good!
    Please enter in your name
    Looks good!
    Please enter in a valid email address
    Looks good!
    Please fill in what you are contacting us about

    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.