IT strategy - Get real about warehousing

How many consultants does it take to create a data warehouse?

There’s no amusing answer to this riddle, but there is a rather sad one – almost always more than are justified by the customer’s real-life needs. And this is just as likely to be the fault of the customer’s over-enthusiasm as the consultancies’ over-selling. Indeed, it is up to management and IT consultancies to give clients advice on how to avoid getting carried away on the warehousing tide. This advice may not yield as many pounds in short-term projects as pushing warehousing projects for all they’re worth, but it is likely to guarantee long-term customer satisfaction and loyalty: companies are already starting to realise that, as in so many IT fads before, they may have been steamrollered into projects that are far more ambitious than their business needs require.

This is not to say that warehousing is unnecessary – indeed, it is one of the most genuinely useful IT trends of the past decade, and one that responds to a real corporate need rather than one hastily cooked up by the marketing departments of the IT suppliers or, it must be said, the consultancies. It goes some way to solving the problem that faces all companies which need to make rapid, informed decisions in a fast-changing competitive market – how to access and make sense of the deluge of information flooding the company network every day. As such, it has been driven not by techies or suppliers, but by managers, particularly those in departments that depend heavily on analysing large quantities of data, such as marketing or financial planning.

This is also not to say that consultancies should stay out of warehousing projects. If a company is really implementing a data warehouse, rather than just a glorified database with some query tools attached, then it will almost certainly need consultancy involvement at the strategic and the IT planning stages. The warehouse is too large and expensive an undertaking to do lightly and without being very sure that it fits into the business structure and the needs of users. An objective view of what these needs are, and what sort of IT solution would best suit them, can often be more effective than a view from the board, the IT department or the users.

The commercial benefits that will accrue from the warehouse must be clearly defined and significant, or it will not be worth the effort.

And that is where the consultancies have an interest in stemming the over-excitement of many corporations. There is a predominant feeling that, unless you have a warehouse in the making, you are not a serious “information company”. But, as with a physical warehouse, it’s what you put into it, and even more importantly, what you need to get out, that really counts. Most companies have closed down the warehouses they used to rent for storing internal records or old stock because they were a waste of money. Now they are building electronic warehouses for masses of data that they will rarely need to use and that isn’t much use when they do.

Similarly, some companies are building warehouses that are well structured and populated with very relevant and useful information, but they are greatly over-estimating their actual need to use this data. How many people really need minute-by-minute updates on corporate information?

So that when they go off for a cup of tea they have to access it all over again? Sounds like a recipe for endlessly postponing making any executive decisions at all.

Robert Legg, who, as director of information resources at pharmaceuticals giant Smithkline Beecham serves managers and users with more pressing information needs than most, sums up the danger of getting carried away with the potential of data warehousing while ignoring the real needs of the business. “We need time-stamped data, updated daily at the most.

We all get carried away with technology but we don’t need data updated every minute. Nor can we allow entirely free access to the data all the time because of the strain on the network. Fourteen years ago we would have had to upgrade our minicomputer just to support two marketing people who wanted access to corporate data. It’s not so different now.”

In other words, who really needs unlimited access to data, how up-to-date does this really need to be, will the business suffer by having a slightly less state-of-the-art warehouse, but one which will be quicker to implement and create fewer nightmares for the IT department that has to support it? Users are starting to ask these questions more realistically and to realise that they do not have the time or resources to spend two years converting and cleaning data for the perfect warehouse, hence the rise of “datamarts” or local warehouses that provide a relatively rapid solution to improving the information situation for specific departments.

If they have embarked on a massive warehousing project, companies are realising, in many cases, that it is probably well beyond their real or future requirements. This may be down to lack of awareness of how warehouses work, to pressure from end users to deliver the most comprehensive solution, or from the type of corporate machismo that seems to influence IT investment to a ridiculous extent. But whatever the reason, the most obvious people to blame when customers decide they have wasted their money will be the consultancies. So it will be in everyone’s interests to put the mammoth projects to the back of our minds and get real about warehousing.