Microsoft is not only putting its weight behind Hadoop, the open source platform for crunching large amounts of data across thousands of servers. It’s abandoning the proprietary platform it built to do much the same thing.
Last last week, a blog post from Redmond announced that the company would stop development on LINQ to HPC, aka Dryad, a distributed number-crunching platform developed in Microsoft’s Research Lab. Instead, the company will focus on its effort to port Hadoop to its Windows Server operating system and Windows Azure, its online service for building and deploying applications.
“Hadoop has emerged as a great platform for analyzing unstructured data or large volumes of data at low cost, which aligns well with Microsoft’s vision for its information platform,” read the blog post from Don Pattee, of Microsoft’s Windows High Performance Computing group.
Pattee said that the company had updated the preview version of Dryad, but that it would not release an official “production” version. Microsoft declined to comment on the blog post.
According to Matt Aslett, an analyst with research outfit 451 Group, the move isn’t surprising, but it shows how much Microsoft has changed. “It is still early days for enterprise adoption of Hadoop, but it’s already gained enough attention to make it difficult for competing projects to gain traction,” Aslett told Wired. “A few years ago it would have been almost unthinkable that Microsoft would support an open source project written in Java, let alone contribute to its development.”
In early October, Microsoft announced that it would integrate Hadoop with future versions of its relational database, SQL Server, as well as Windows Azure. Hadoop was built for Linux, so the platform must first be ported to Windows. Doug Leland, general manager of product management for SQL Server, told Wired that the company plans to eventually release its work back to the open source community.
Hadoop — named after a yellow stuffed elephant that belonged to the son of the project’s founder — was originally bootstrapped at Yahoo!, and it helps run several other big web names, including Facebook, Twitter, and eBay. But the traditional IT giants and various startups are also pushing it into everyday businesses. IBM, Dell, SAP, EMC, and even Oracle offer software or appliances that use the platform.