Data warehousing technology has been filtering down from large companies to small lately, as businesses across the corporate landscape scramble to take advantage of business intelligence (BI) reporting and analytics. With a strong product and lower costs, Microsoft SQL Server has positioned itself as a serious contender for the midmarket data warehousing business and -- increasingly -- the data warehousing market as a whole, analysts...
“Ever since SQL Server 2005 came out, Microsoft has been capable of supporting data warehouses,” said Donald Feinberg, an analyst at research firm Gartner Inc. in Stamford, Conn. “Because SQL Server is generally less expensive than most other databases, its rise to prominence should be no surprise.” SQL Server 2008 R2 brought more features to the party, particularly ones that supported both data warehousing and business intelligence.
As Microsoft’s flagship relational database management system, SQL Server has long suffered from a market perception that it’s best for handling smaller databases and lower-level transactions, Feinberg said. Higher-end data warehousing was thought to be the playground of bigger kids like Oracle, IBM and Teradata.
“That is why you are often seeing recommendations of SQL Server for department use and Oracle for the enterprise. That is a difficult perception to break,” Feinberg said.
Even so, SQL Server is grabbing its share of the market. A recent SearchDataManagement.com survey found that 80% of respondents now have or are considering acquiring data warehouse technology. Of that group, nearly half currently run or are considering SQL Server to power their data warehouses.
Feinberg said vendors may not be taking Microsoft’s growing presence in the market seriously. Most are not worried about SQL Server at the low end because it has long been considered “OK for a few terabytes but no more.” In fact, there are quite a few SQL Server data warehouses with more than 10 TB of storage, Feinberg said. “The other vendors aren’t scared, but they should be, especially if you analyze the cost of licenses for SQL Server compared to a product like [IBM’s] DB2.”
And cost goes a long way in the data warehouse market, Feinberg said. “There is another truth in this industry that people have lost track of, namely, that if a product is good enough and costs less, it will win,” he said. “The fact is SQL Server is better than ‘good enough,’ and it is less money than the competitors,” he said.
Microsoft is a very different vendor compared with most others in the space, said Mark Beyer, another Gartner analyst. He said Microsoft data warehouses are mostly -- probably more than 95% -- running data volumes of 5 TB or less, he said.
“We have seen a big uptick in midsize companies, in the $500-million-to-$1-billion revenue range, inquiring about their ability to use Microsoft SQL Server in the data warehouse role.”
Those companies are no strangers to data warehousing, Beyer noted. Most are now storing around 5 TB of data and are likely to see that grow to as much as 10 TB over the next three to five years.
“Generally, they already have a SQL Server data warehouse and they want to know if they can plan to expand,” he said.
Beyer said around 90 percent of the Global 5000 companies have some kind of data warehouse or data mart analysis that is critical to them and another few percent “probably just don’t realize they have one,” he says. That means “greenfield” opportunities aren’t there either for retailers or vendors.
As a result, noted Beyer, when most companies look at data warehouse products, it isn’t to set up their first data warehouses, it is to improve on what they already have. Therefore, he noted, companies are looking for cost-control mechanisms and ways to simplify what they have, including opportunities to consolidate vendors, he said.
When all the issues are addressed, overall cost will re-emerge as a driver for company decisions, Beyer said, and that will lead people to think about cheaper options.
“That is where Microsoft has the potential to move in,” Beyer said. So if a midsize company has had good results after experimenting with a small SQL Server installation, he said, it’s almost certain that someone will suggest that they keep going with that.
And then there are newer concerns, like handling “big data” -- massive sets of structured and unstructured information companies are just now coming to terms with. It’s often seen as the purview of specialized vendors, but Microsoft has jumped into the big data pool. The company recently announced it would hook its upcoming SQL Server 2012 into distributed computing framework Hadoop, and it has big-data plans for Windows Server and its cloud platform Azure as well.
Beyer said SQL Server may not be able to support the number of data types as, say, DB2 or Oracle Database, but Microsoft will still rack up customers, especially from among cost-conscious Microsoft shops who are happy with what they have.
“If my company is not doing big data now, my data warehouse is growing and I’m taking a cost-control approach, all of a sudden, Microsoft is on the list.”
Microsoft’s deeper foray into data warehousing
In addition to offering SQL Server as an engine for data warehousing, Microsoft has also taken steps to beef up its offering and make it more capable, easier to implement and more appealing to a wider audience. There are two main elements of this expansion:
- Parallel Data Warehouse (PDW) appliance, offering scalability to up to 600 terabytes, components such as SQL Server Reporting and Analysis Services and integration with Apache Hadoop, an open source distributed computing framework that processes massive, variable data sets known as “big data.” Microsoft has teamed up with hardware vendors to provide what it calls a turnkey data warehouse. For instance, Hewlett-Packard offers its Enterprise Data Warehouse Appliance built on SQL Server 2008 R2. Dell offers the similar Dell Parallel Data Warehouse Appliance.
- Fast Track Data Warehousing. Microsoft has also put together a reference architecture -- a grouping of components from a number of vendors, pre-tested for interoperability with Microsoft SQL Server 2008 R2 -- that aims to reduce costs, reduce risk and save time, with configurations it claims can cost as little as $11,000 per TB.
According to Microsoft, interest in data warehousing is increasing as companies find they have more data to store, manage and analyze, citing a study that shows information volume growing worldwide at a minimum annual rate of 59%.
Gavin Payne, an independent SQL Server consultant in the U.K., sees the expanded offerings and the move deeper into data warehousing as a natural progression. He said Microsoft is responding to the changes over the last few years in the software-buying process.
“The buyers of SQL Server are now increasingly business rather than technical managers who are looking to meet a business rather than a technical need,” he said. “By packaging SQL Server’s marketing-leading business intelligence functionality into easy-to-consume appliances, Microsoft has taken away the technical barriers to adoption.”
Alan R. Earls is a Boston-area freelance writer focused on business and technology.