Apache Hive

Definition

Apache Hive is a data warehouse system built on top of Hadoop that lets users query and manage large distributed datasets using HiveQL, a SQL-like query language, without writing low-level MapReduce code.

Overview

Hive originated at Facebook in 2007, when engineers needed a way to let analysts familiar with SQL query the huge volumes of data being stored in Hadoop's HDFS, without requiring everyone to write raw MapReduce jobs. It was donated to the Apache Software Foundation and became a top-level project in 2010. Hive translates HiveQL queries into execution plans that can run on MapReduce, Tez, or Apache Spark, and applies a 'schema-on-read' approach, meaning data can be loaded into Hadoop first and structured only when queried, which suits large, semi-structured datasets. A central component, the Hive Metastore, stores table and schema definitions and is frequently shared with other query engines like Spark and Presto/Trino, effectively acting as a catalog for the broader data platform. Hive queries are commonly scheduled as part of larger pipelines orchestrated by tools like Apache Airflow, and Hive itself is often compared to fully managed cloud warehouses such as Snowflake or BigQuery — SQL skills built with Hive transfer well to those platforms, a connection explored further in SkillVeris's SQL Mastery course.

Key Features

• SQL-like HiveQL query language for querying big data
• Schema-on-read approach suited to semi-structured, large-scale data
• Pluggable execution engines: MapReduce, Tez, or Spark
• Hive Metastore acting as a shared schema/catalog across tools
• Partitioning and bucketing for improved query performance
• Support for custom user-defined functions (UDFs)
• ACID transaction support in modern Hive versions
• JDBC/ODBC connectivity for BI and reporting tools

Use Cases

Frequently Asked Questions

Related in DevOps