Forget about the business intelligence suites from IBM, Oracle, and SAP Business Objects, the analytics from SAS, and even the hot data visualization tools like Tableau Software. New platforms like Hadoop and NoSQL databases demand new tools that are purpose built for these environments.
This is a core theme that Jay Parikh, VP of infrastructure engineering at Facebook, and Ping Li, a partner at venture capital firm Accel Partners, discussed on stage on Thursday at the DataWeek 2012 Conference in San Francisco. Their talk was about the challenges and opportunities facing startups and young companies in the big data arena, and Parikh and Li shared their message with InformationWeek by phone just hours before they took to the stage.
There's little doubt that Hadoop, NoSQL databases, and other emerging big data platforms are quickly evolving, says Li, "but we're hoping to see more new applications on top of these platforms." Parikh and Li are encouraging more innovation because there's not enough speed and breadth of development to truly feed a rich big data community, they say.
Citing a "huge gap" in connecting big data business users to the new underlying platforms, Li says there's also ample room for new business applications, like CRM, and new vertical industry applications for data-intensive fields, such as oil and gas.
Li manages Accel Partners' Big Data Fund, which clearly stands to benefit if there's a crop of new startups to invest in that ultimately succeed. But why is Facebook taking a stand?
"We've had a long history of innovating on infrastructure very openly and contributing back into various open source projects," Parikh explained. "There's a lot more work to be done on these platforms, but we're not going to hire every smart engineer on the planet. We want to be able to collaborate with the people that we can't hire in the open through various communities."
In its earliest days, Facebook helped push the envelope with open source projects like Memcached and MySQL. The social network giant has since made significant contributions to Hadoop, including foundational work on Hive and many contributions to HBase, HDFS, and MapReduce. The company has been forced to innovate because it runs the largest Hadoop deployment in the world, with more than 100 petabytes of information.
"We built Hive as a way for business users to get what they needed out of our [Hadoop] big data infrastructure," said Parikh. "Writing MapReduce jobs is fine for engineers, but if you're an analyst or a product manager and you want to extract reports or do analysis, you need an easier interface for that data. Hive gave our users a SQL-like interface to Hadoop."
The "we need new tools" thesis seems to write off products that have made huge strides in connecting to new platforms. Parikh grants that there is a bias in the big data community toward "shiny new things," and doesn't believe there's "one magical piece of technology that's going to wipe out everything done in the past."
Li, too, grants that the relational database and the applications built for it will survive, "but we're seeing enough new green field applications that will require a new set of tooling." Most relational databases and BI platforms have sprouted connections to Hadoop, with one of the latest wrinkles is HCatalog-based access to Hadoop data without data movement. But over time, Li foresees new tools built natively for the new platforms.
没有评论:
发表评论