Accumulo

Definition

Apache Accumulo is a distributed key-value store built on the Hadoop ecosystem, modeled on Google's BigTable design, that adds cell-level security labels as a core feature, letting individual cells within a table carry access-control attributes rather than only table- or column-level permissions. It stores sorted, sparse data across a cluster using HDFS for underlying storage and was originally developed within a U.S. government context that required fine-grained data access control.

Overview

Some organizations, particularly government and defense agencies, need to store data where different users are authorized to see different individual pieces of information within the same table, rather than being granted or denied access to entire tables or columns. Accumulo was created to meet that specific requirement by extending the BigTable model with a security label attached to every cell, so authorization can be enforced precisely at query time based on the labels a user presents. Mechanically, Accumulo organizes data into sorted key-value pairs much like BigTable and HBase, using row, column family, and column qualifier components, but each cell also carries a visibility label expressed as a boolean expression over security tokens. When a client queries the table, Accumulo evaluates whether the client's presented authorizations satisfy each cell's visibility expression and filters out cells the client is not permitted to see, all enforced server-side. It runs on top of HDFS for storage and uses a tablet-server architecture similar to HBase's region servers for distributing data across a cluster. Among BigTable-inspired systems, Accumulo's defining differentiator from HBase and Hypertable is this native cell-level security model; HBase and Hypertable both offer coarser-grained access control mechanisms layered on top rather than built into the core data model, making Accumulo the more direct choice when fine-grained, per-cell authorization is a hard requirement rather than a nice-to-have. In practice, Accumulo has been used in government, defense, and intelligence contexts where handling classified or multi-tenant sensitive data at different clearance levels within a single dataset is a core requirement, and it has also seen adoption in some large enterprise analytics settings that value its fine-grained authorization model. The main trade-off is that Accumulo's security model and operational profile add complexity that is unnecessary for workloads without a genuine per-cell access control requirement; for straightforward wide-column storage without that need, HBase's larger ecosystem and simpler operational model are usually the more practical choice. Operators also need to design visibility label schemes carefully up front, since retrofitting a cell-security model onto data that was written without consistent labeling conventions is far more disruptive than establishing that convention before ingestion begins, and getting it wrong undermines the very guarantee that makes Accumulo worth choosing in the first place. Because of that, teams evaluating Accumulo should treat the labeling scheme as a core piece of upfront data modeling work, on par with schema design in a relational system, rather than as an operational detail to be handled later.

Key Features

• Cell-level security labels enforced server-side on every read
• BigTable-style sorted, sparse key-value data model
• Runs on HDFS with a tablet-server cluster architecture
• Boolean visibility expressions evaluated against client authorizations
• Originated in a U.S. government context requiring fine-grained access control
• Distributes data across a cluster similar to HBase's region servers
• Part of the broader Apache Hadoop ecosystem

Use Cases

Alternatives

Frequently Asked Questions

Related in Database