Microsoft StreamInsight charts course in unfamiliar waters

Microsoft StreamInsight, a complex event processing product, is a stab at a whole new space for the company, and not a bad one at that, experts say.

This article can also be found in the Premium Editorial Download: SQL Server Insider: Microsoft Master Data Services boosts SQL master data management:

It’s the same old story of Microsoft making aggressive and strategic moves into new areas.

This time around, it’s StreamInsight, a complex event processing (CEP) platform introduced last spring for SQL Server 2008 R2 that the company says can process, analyze and take action on a constant stream of events from multiple sources in near real time. Until StreamInsight came along, CEP was largely the domain of players with top-end, enterprise-focused products. But according to experts and analysts in the field, Microsoft has ventured into the right exclusive corner.

“In some sense, a CEP product like StreamInsight might be considered sort of ‘out there’ for Microsoft, but in other ways it’s a typical business model for them,” said Kevin Kline, technical strategy manager at Quest Software, a vendor of monitoring tools.

For example, he noted, high-end reporting had previously been the domain of products like Business Objects Crystal Reports. “You would spend thousands of dollars to get a good reporting system and you would have to train people to use it,” he said.  Then, Microsoft plugged similar capability into SQL Server and offered it for free. Now there are people who buy SQL Server primarily to get Reporting Services, Kline said. The company also infiltrated the ultra-high-end extract, transform and load market—where entry-level capabilities could start at $120,000 -- with a far cheaper, packaged SQL Server Integration Services.

“So, at first blush, offering StreamInsight, which is also bundled with SQL Server, might seem to be peripheral to what Microsoft does, but when you look at the bigger picture, this is a very standard kind of play for them,” he said.

What’s more, according to Kline, StreamInsight is a good product. Often, Microsoft’s first version of a product follows the 80/20 rule -- 80% of the people will be happy -- and that is typical. As with most products, “The smaller your company, the happier you will be with what you are getting at that price point,” he added.

W. Roy Schulte, a distinguished analyst specializing in application architecture and middleware at research firm Gartner Inc., said Microsoft StreamInsight has had a fairly good start in the market.

“So far it seems like the adoption of StreamInsight is a little low for the time it has been available, but of course low for Microsoft is high compared to its competitors,” such as IBM WebSphere Business Events, Progress Apama, Sybase Aleri and Tibco Business Events, he said.

At first blush, offering StreamInsight might seem to be peripheral to what Microsoft does, but when you look at the bigger picture, this is a very standard kind of play for them.

-- Kevin Kline, technical strategy manager at Quest Software

Schulte said when he talks to clients he asks almost immediately what kind of environments their applications are running in. “If it is a very heavy Windows environment, I tell them StreamInsight should be on their short list of applications to acquire,” he said. But if it’s a mixed environment with AIX and Linux, for example, Windows-focused StreamInsight would not be as a strong competitor as the native offerings for AIX and Linux, he added.

That said, Schulte said CEP -- and StreamInsight in particular -- can have broad applications. It is used in what Microsoft calls OT, or operational tech, which encompasses activities such as pipeline monitoring. It is also a natural for serving complicated capital markets (think Wall Street trading systems), where Microsoft has done some work.

However, although about 30% of CEP deployments are in the high-powered financial services arena, “Microsoft’s penetration there is almost zero,” Schulte said. For its part, Microsoft foresees StreamInsight applications in vertical markets such as manufacturing, oil and gas, utilities, financial services, health care, Web analytics and IT and data center monitoring.

Schulte said that the 2010 CEP market was “around $200 million.” Microsoft’s StreamInsight is not counted in that market because its revenue is intermingled with SQL Server, so its revenue market share is 0%, according to Gartner’s methodology. “If you did a unit count rather than a revenue market count, Microsoft probably has 5% of the CEP market,” Schulte noted.

“If you ask which industry and which functions are most suited for CEP, the answer is pretty much the same as it was for RDBMS [relational database management system] -- there is nothing industry-specific about it at all, and almost all companies need that. The same is true of CEP,” said Schulte, adding that the reason some industries, such as financial markets, got to CEP early is simple. “They will adopt anything early.”

Conversely, noted Schulte, unlike the RDBMS, which is used by practically all businesses, CEP is unlikely to achieve that same level of penetration, since near-real-time analytics is only needed when you are quickly looking for certain patterns in data. “That does not describe 90% of business requirements the way it might for RDBMS,” he said.

But that’s not likely to be the end of the story. Microsoft has saved a spot for StreamInsight in its ambitious cloud strategy.

The company has been aggressive in getting to the cloud, in particular with the Windows Azure cloud platform, Schulte noted. In fact, the company has reversed course on its original approach, which involved making application features available on Azure after introducing them in traditional software. “Now, the new features come out first on cloud Azure, and then they go back into the on-premises software,” he said.

Schulte said the cloud version of StreamInsight, dubbed Project Austin, works the same as the on-premises version except it is what Microsoft calls a hybrid configuration. Either the sender or the receiver is in the cloud as opposed to the traditional on-premises model, in which neither is in the cloud.

Kline said the cloud option, combined with SQL Azure, Microsoft’s cloud database service, could appeal to many smaller companies once they do the math. Even with their relatively modest licensing fees, when combined with hardware and operational costs, the cost of SQL Server and StreamInsight can add up quickly to six figures, so this Software as a Service (SaaS) option can be attractive, he said.

Schulte was less optimistic. He said a SaaS provider might be interested in that arrangement, but the typical customer would not. “It isn’t as end-user-friendly as some other CEP offerings; you must be a serious developer to make it work.”
 


StreamInsight in action
Although analysts said StreamInsight is still in the early stages of adoption, some users have taken the plunge and started to plumb the product’s potential. Among them is Johan Åhlén, an IT pro in Sweden who, until recently, was a consultant working for organizations in the high-frequency trading field.

Åhlén said he began to work with StreamInsight almost as soon as it became available last year, initially on customer systems reserved for testing and development. “In this industry, the cost of failure is very high, so we always perform extensive testing,” he explained. “No one is about to let you interface with any other system until they know it is safe.”

With those factors in mind, Åhlén said his team spent about six months working with StreamInsight before connecting to trading systems a few months ago. “So far we have only tried it with small amounts of money,” he said.

Because it was a new product, Åhlén said he was initially concerned about whether StreamInsight would scale; so far, all indications are that it will. “Our load tests show that StreamInsight has the capacity to handle over 100,000 events per second on a single server,” he said.

“Usually with Microsoft products, they think about extensibility from scratch and also make it so you can integrate successfully,” he added.

“In my opinion, they have done a great job with the product and they have continued to develop it further,” Åhlén said. For example, he noted, in the last release, Microsoft added support for user-defined stream operators that “are very suitable for efficient signal processing.”

Åhlén said he is excited about the potential of cloud-based StreamInsight, now in development, “although it wouldn’t work well for high-frequency trading.

“If you are looking for some kind of near-real-time processing or business activity monitoring, you should look into StreamInsight,” he added.

Alan R. Earls is a Boston-area freelance writer focused on business and technology.

Dig deeper on Microsoft SQL Server 2008 R2

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

SearchBusinessAnalytics

SearchDataCenter

SearchDataManagement

SearchAWS

SearchOracle

SearchContentManagement

SearchWindowsServer

Close