Apache Kylin

Definition

Apache Kylin is an open-source distributed analytics engine that provides OLAP (online analytical processing) over very large datasets by precomputing multidimensional cubes so that dashboard-style queries return in sub-second time. Instead of scanning raw data at query time, Kylin builds and stores aggregated cube results in advance, trading storage space and build time for fast, predictable query latency at massive scale.

Overview

Apache Kylin was built to solve a specific pain point in big-data analytics: dashboards and BI tools often ask the same shapes of aggregation query — sums, counts, and averages sliced across a handful of dimensions — repeatedly against datasets with billions of rows. Scanning that much raw data on every query, even with a fast SQL engine, produces latency that feels sluggish for interactive dashboards. Kylin's answer is the OLAP cube: a precomputed structure that stores aggregated values for combinations of dimensions ahead of time, so a query becomes a lookup rather than a full scan. Mechanically, Kylin defines a cube over a star or snowflake schema, specifying which dimensions and measures matter, then runs a build job — historically on Hadoop MapReduce or Spark — that computes and materializes the aggregations across the relevant dimension combinations, storing the results in a fast key-value store such as Apache HBase or in Parquet files for newer versions. When a query arrives through Kylin's SQL interface, its query engine determines which precomputed cuboid can answer it and returns the result directly from storage rather than recomputing from source data. Against its neighbors, Kylin differs from engines like Impala, Presto, or Spark SQL in that those tools compute results on demand from raw or lightly indexed data, giving flexibility for arbitrary queries at the cost of scan-time latency, whereas Kylin only answers queries that fit within its precomputed cube definitions but answers them far faster. This makes Kylin closer in spirit to traditional OLAP cube technology from data warehouse vendors, adapted to run on big-data infrastructure. In practice, Kylin is deployed behind BI tools and reporting dashboards where the set of common aggregation queries is known in advance, letting teams define cubes that cover those patterns and get consistent sub-second response times regardless of how much source data has accumulated. It is typically paired with a Hadoop or Spark-based data pipeline that feeds the cube build process on a schedule. The central trade-off is flexibility: any query outside the dimensions and measures a cube was built for either falls back to slower processing or cannot be answered at all, and cube definitions must be maintained as reporting needs evolve. Build jobs also consume compute and storage proportional to the number of dimension combinations, which can grow quickly with high-cardinality dimensions. Teams needing fully ad hoc, unpredictable query patterns are usually better served by an on-demand SQL engine rather than a cube-based system like Kylin.

Key Features

• Precomputed OLAP cubes for sub-second aggregation query response
• Standard SQL interface for connecting existing BI tools
• Cube build pipelines on Hadoop MapReduce or Apache Spark
• Storage of materialized cuboids in HBase or columnar Parquet files
• Incremental cube builds to add new data without full recomputation
• Star and snowflake schema modeling for dimensions and measures
• Query engine that selects the best-fitting precomputed cuboid
• Support for very high concurrency dashboard workloads

Use Cases

Alternatives

Frequently Asked Questions

Related in AI Tools