As more customer data has come to reside in the cloud, and as on-premises data warehouses bulge with ever-increasing amounts of data, cloud-based relational data warehouses are giving user organizations new technology options.
Amazon Web Services forged the category with Amazon Redshift, and has been countered by Google BigQuery, Snowflake and other offerings. With Azure SQL Data Warehouse, Microsoft is in the thick of the fray -- and the company moved its cloud data warehouse forward this week, disclosing a preview-mode update that boosts the system's data processing scale by threefold.
In a blog post coinciding with Microsoft's Inspire 2017 channel partner event, Rohan Kumar, general manager of its database systems group, said the Azure SQL Data Warehouse scaling threshold has increased from 6,000 to 18,000 Data Warehousing Units, or DWUs. The DWU is a Microsoft in-house measure of CPU, memory and I/O resources required for parallelized data warehousing work.
The boost in DWUs is done without putting limits on the data warehouse's associated columnstore indexing and storage. Such column stores, intended to speed concurrent data warehouse user queries, have become a hallmark of advanced analytics implementations.
Column stores on the march
"Column stores, from a business intelligence perspective, make a lot of sense. It's great for BI because everything is indexed, and you can efficiently query against it," said Rick Sherman, managing partner of consulting firm Athena IT Solutions.
Rick Shermanmanaging partner, Athena IT Solutions
Sherman said most cloud data warehousing activity he has encountered is still experimental, but that could change. That is because users foresee the need to collect and store more and more data, requiring more and more processing, and cloud vendors' great banks of elastically scalable computers look like an apt place for such processing.
"Elasticity is the pitch for things like Azure SQL Data Warehouse. You pay for what you need, while leveraging the economy of scale that vendors gain with vast data farms across the planet," he said. "Microsoft is definitely helping to push things to the tipping point."
Cloud data warehouse implementers face limits in the number of concurrent users they can efficiently support for BI queries. That, too, makes Microsoft's cloud data warehouse advances important. But the situation is fluid, according to Sherman. "Microsoft may offer more substantial scale-out than Amazon right now," he said. "Redshift has to grow up a little bit more, but I wouldn't count Amazon out."
Beyond data storage limits
Removing storage limits on Azure SQL Data Warehouse is a useful step for Microsoft in the cloud, according to Warner Chaves, a SQL Server principal consultant at technology services company Pythian.
"This move is directed at supporting Azure SQL Data Warehouse's largest existing clients -- ones that were hitting the limits of storage, or were willing to pay for more compute to get results faster," he said.
Without these new options, people would be forced to move to a Hadoop-style big data platform, or to competitive cloud data warehouses such as those from Amazon and Google, Chaves indicated. "It's important that Microsoft has given them a path to continue to scale," he said.
Capacity limits can also slow cloud adoption, Chaves said. "Having something that is at a fixed capacity or that is cumbersome or too slow to scale up or down takes away part of the attractiveness of the cloud."
Relational data warehouse as workhorse
While much attention has focused on analytics done via innovative Hadoop and Spark applications, both on premises and in the cloud, updates to cloud versions of more traditional SQL-oriented data warehouses, such as Azure SQL Data Warehouse, are garnering attention, too. "The relational data warehouse is a workhorse," Sherman said.
Updates to Azure SQL Data Warehouse show the traditional data warehouse may also be a workhorse prepared to flourish on the cloud server farms of Microsoft and others.
One particular Azure SQL Data Warehouse elasticity trait that finds favor with customers, Chaves said, is the system's ability to pause processing completely, yet turn back on quickly as needed.
Kumar said the public preview of Azure SQL Data Warehouse with 18,000 DWUs will be available this month, along with another version that supports 9,000 DWUs. Pricing for the 18,000-DWU implementation varies based on the region the system runs in, with a low cost of $217.75 in the East U.S. 2 region. For now, the preview will only be available there and in these regions: South Central U.S., Canada East, North Europe and Southeast Asia.
Listen to a podcast about Azure SQL Data Warehouse
Find out how MPP changes data warehousing
Gain a DBA's perspective on SQL Server innovations