This week, Apache Kylin graduated from the Apache Incubator to the status of a top-level project, which means that Kylin's community and products are "well-governed" according to the Apache Software Foundation's requirements and principles.
Kylin is an open-source distributed analytics engine that provides a SQL interface and multi-dimensional analysis (OLAP) for Hadoop, which provides support for extremely large (petabyte-scale) datasets.
Bridging a Gap in Big Data
Kylin is designed to bridge the gap between Big Data exploration and human use by enabling interactive analysis with sub-second latency on massive datasets. The result is that Kylin brings business intelligence back to Hadoop.
The project has its beginning as eBay, which initially developed Kylin before it was submitted to the Apache Incubator about a year ago.
Kylin has relationships with several other Apache projects. Luke Han, Vice President of Apache Kylin explained that "we have tightly integrated Apache Calcite as our SQL Engine, and we provided a Kylin Interpreter to Apache Zeppelin." In addition, Kylin is a consumer of Spark, Kafka, HBase, and Zookeeper.
A Good Omen
The rapidly growing Big Data market in China has been quickly adopting Kylin as an analytic platform. The name "Kylin" itself seems to come from the Chinese qilin or 麒麟, "a mythical hooved chimerical creature known in Chinese and other East Asian cultures, said to appear with the imminent arrival or passing of a sage or illustrious ruler. It is a good omen thought to occasion prosperity or serenity. It is often depicted with what looks like fire all over its body." [Source: Wikipedia]
Its logo is reminiscent of the mythological creature.
High-level Thoughts from eBay
"Apache Kylin is the best OLAP engine on Big Data so far," said Wilson Pang, Senior Director of Data Services and Solutions at eBay. "At eBay, we collect every user behavior on every eBay screen. While other OLAP engines struggle with the data volume, Kylin enables query responses in the milliseconds. Moreover, we are also starting to leverage Kylin for near real-time data streaming storage and analytics engine. Altogether, Kylin serves as a critical backend component for eBay’s product analytics platform."