Apache ORC

Definition

Apache ORC (Optimized Row Columnar) is an open-source columnar storage file format originally developed to speed up Apache Hive queries on Hadoop, storing data in column-oriented stripes with built-in indexes and statistics that let query engines skip irrelevant data. It competes closely with Apache Parquet as a standard format for analytical data, offering similar column-pruning benefits with different internal layout and compression defaults.

Overview

Early Hive deployments running on plain text or row-oriented sequence files suffered from slow query performance because engines had to read entire rows even when a query only needed a few columns, and there was little built-in metadata to help skip irrelevant data. Apache ORC was created within the Hive project specifically to fix this, introducing a columnar on-disk format with lightweight indexes so Hive queries could prune both columns and row ranges before reading them. Mechanically, an ORC file is divided into stripes, each containing column data stored together, an index for that stripe, and row-level statistics such as min, max, and count for each column. A file footer holds the overall schema and stripe locations, and a postscript at the very end of the file holds compression parameters, so a reader can jump directly to the footer, decide which stripes are relevant to a query's filters using the embedded statistics, and skip the rest — similar in spirit to Parquet's row-group pruning but with ORC's own indexing structure and encoding choices. Among columnar formats, ORC and Parquet are close peers solving the same core problem; ORC grew up tightly coupled to Hive and often defaults to slightly different compression and encoding choices, while Parquet became the more common default in the broader Spark, Trino, and cloud-warehouse ecosystem. The practical difference between them today is smaller than their separate origins might suggest, since most major engines read and write both, and the choice often comes down to which ecosystem a team's existing tooling defaults to. In practice, ORC remains the default or a well-supported option in Hive-centric data platforms and is also used in some Spark and Presto/Trino deployments, particularly where legacy Hive tables already exist in that format. Choosing between ORC and Parquet for a new project is often more about consistency with an organization's existing stack and tooling defaults than a decisive technical advantage of one over the other. ORC's limitations are largely shared with Parquet: it is not designed for frequent row-level updates or deletes, since changing data typically means rewriting stripes or files, and it is a poorer fit for transactional workloads than analytical, write-once-read-many datasets. Table formats such as Apache Hudi and Apache Iceberg can layer transactional semantics on top of ORC-based storage, similar to how they do for Parquet, for teams that need update and delete support. In newer projects without a Hive legacy to carry forward, the choice between ORC and Parquet is often driven less by technical necessity and more by which surrounding tools default to which format, since query engines across the ecosystem support both roughly equally well today. Teams standardizing a new data lake are just as likely to pick based on their primary query engine's defaults and their team's existing familiarity as on any measured performance difference between the two.

Key Concepts

• Columnar stripes containing per-column data, indexes, and statistics
• File footer and postscript enabling quick schema and stripe lookup
• Row-range pruning using embedded min/max and count statistics
• Origins tightly coupled to Apache Hive's query engine
• Comparable column-pruning performance to Apache Parquet
• Support from Spark, Trino, Presto, and Hive query engines

Use Cases

Frequently Asked Questions

Related in Database