How can a company justify the cost of eliminating data marts and shifting to a centralized warehouse?
From a cost perspective, it is a lot cheaper. It costs between $1 million and $2 million a year to maintain a data warehouse. After careful analysis, we select some of the data marts that are good candidates for elimination. Usually, they are at the end of their depreciation cycle for hardware. They are redundant and are there only for a small number of users. How has active data warehousing evolved, and where is it headed?
In the early days, people viewed their data warehouse as static data that didn't change often. They used it primarily to write reports and do queries about what happened. Then it evolved to marketing departments using the data to predict which customers had a high propensity to buy. Companies today are spending a lot of money on getting a comprehensive integrated set of accurate data from their data warehouse, and now they're just leveraging it more and more to get the information where it needs to be. What role will the DBA play in the data warehouse of the future?
With our previous data warehouses, DBAs spent 80% of the time moving the data around and 20% of their time with users. Now we've completely reversed that. DBAs are working with users to try to implement the analytic applications that help users do better work. In addition, there is an interesting career direction for DBAs. Data mining is coming into its own. Data mining was something DBAs were never involved in, but now we're turning it into something that is a natural extension of what they already know. There have been differences of opinion on whether to shift to a centralized database. What's your take?
The vendors who don't agree with centralizing just can't handle it. Data marts were a pretty good idea, but you get to your 20th application and you're still extracting the data out of the same sources. You end up with redundant storage in all these data marts and extraction, transformation and loading (ETL) becomes a pretty expensive process. This is where 'active warehousing' comes into play. You do it once, and then you've got this pretty comprehensive set of data that you can use for nearly all your applications. What impact will the Sarbanes-Oxley Act have on the use of the data warehouse?
I see companies doing different things with a warehouse that they probably didn't do before. The focus will be on customer analysis and customer profiling and, of course, a company's financials. Companies will be looking for trends and consistency from one year to the next, and that is hard to do using just a company's financial systems. Once you pull the data from different sources, clean it, and add whatever demographic data and external data you want, you really get the single version of the truth.
FOR MORE INFORMATION
Check out a Featured Topic on Analytics.
Check out a Featured Topic on data warehousing.
To provide your feedback on this article, contact Robert Westervelt.What industries are starting to use active data warehousing, and where do you see it emerging?
Banks are learning to predict the long-term value of customers, and they want to predict the default rate of their loans. Our customers have been very innovative and pushing the envelope, including our airline partners. In that industry, you have to use demand forecasting and yield management, and you have to integrate extensive data. They can't keep it in separate pockets like they used to.