Databricks Details Two New Apache Iceberg Specs for Cross-Catalog Governance
Databricksは、エンジンやカタログ間でのポリシー適用を標準化することを目的とした、Apache Icebergの新たな2つの仕様「読み取り制限(read restrictions)」と「カタログラベル(catalog labels)」を発表した。
ひとことで言うと
What did Databricks announce about governance in the Open Lakehouse?
Databricksによると、Apache Icebergコミュニティは、Iceberg REST Catalogに2つの新たな仕様、「読み取り制限(read restrictions)」と「カタログラベル(catalog labels)」を採用したという。読み取り制限により、カタログは信頼できるエンジンにポリシーの実施を委任できるようになる。一方カタログラベルは、Unity CatalogやSnowflakeといった連携システム間で、PIIタグなどのガバナンス関連メタデータをやり取りできるようにするものだ。
要点
- Databricksによると、Apache Icebergコミュニティは、Iceberg REST Catalogに2つの新機能を追加する作業を進めている。読み取り制限とカタログラベルだ。
- Read restrictions let a catalog evaluate access policy for a user and return row-filter and column-projection instructions that a trusted engine, such as a securely configured Trino deployment, must enforce; Databricks says untrusted engines like Spark or DuckDB require centralized enforcement instead.
- Databricksによれば、読み取り制限仕様では9種類の定義済み列射影アクションと標準化された行フィルター式が規定されているが、カタログがエンジンとの間でどのように信頼関係を確立するかについては定義されていないという。
- Catalog labels, according to Databricks, allow catalogs such as Unity Catalog, Snowflake, AWS Lake Formation and Google Cloud Knowledge Catalog to exchange key-value metadata at the table and column level, which the receiving catalog maps into its own tagging or policy model.
- Databricksは、ラベルはあくまで共通の意味を持たない不透明なキーと値のペアにすぎないと指摘しており、企業側はカタログ間で一貫した解釈を行うため、独自の運用ルールを整備する必要があるとしている。
Databricks published details on two new specifications adopted by the Apache Iceberg community for the Iceberg REST Catalog: read restrictions and catalog labels. The company says both are meant to standardize how governance policies are enforced when data is accessed across different compute engines and federated catalogs.
Read restrictions
Databricks describes read restrictions as addressing a common scenario in which an organization governs data in one catalog but wants to query it using different engines or tools. Under this model, the catalog evaluates a requesting user's identity and context, decides what row filters or column masks apply, and returns those restrictions to an engine it trusts to enforce them.
Databricks distinguishes this from centralized enforcement, where all steps happen inside the catalog's own environment. The company gives its own implementation as an example, saying Databricks applies fine-grained access control on dedicated compute by routing queries through what it calls a secure filtering fleet, and that its Unity Catalog Cross-engine ABAC feature extends this to other engines through the Iceberg REST catalog's scan and plan APIs.
For delegated enforcement, Databricks says the engine does not receive the policy as an administrator wrote it, but rather the resulting filtering or masking instructions for that specific user. The company states the initial spec defines nine predefined column-projection actions and standardized row-filter expressions, such as comparisons or set membership, and notes that policies relying on subqueries, lookup tables, or custom UDFs cannot currently be expressed this way.
Databricks also notes the specification does not define how a catalog establishes that an engine is trustworthy, saying a claim from the client alone is not sufficient and that system administrators must apply their own security mechanisms. The company points to Iceberg community discussions that have considered options including mTLS and OAuth, while stating that trust remains outside the protocol itself. Databricks says a securely configured Trino deployment is an example of a trusted engine, while Spark and DuckDB are considered untrusted when users control the runtime and can execute arbitrary code.
Catalog labels
The second specification, catalog labels, addresses governance across federated catalogs, according to Databricks. The company notes that many enterprises now connect multiple catalogs, including Unity Catalog, Snowflake, AWS Lake Formation, and Google Cloud Knowledge Catalog, each with its own identity models and policy languages.
Databricks explains that labels let a producing catalog attach lightweight key-value metadata to tables and columns, which a consuming catalog then maps into its own classifications or tagging system before applying its native policies. As an example, the company describes a producing catalog labeling a column "ssn" with "pii=ssn," which a consuming catalog could use to trigger a masking policy.
Databricks states that because enforcement stays local to the consuming catalog, each system preserves its own policy expressiveness and maintains independent audit trails, without needing to call another catalog for every access decision. The company cautions that labels are opaque key-value pairs with no shared semantics or stable identifiers defined by the standard, meaning a consuming catalog sees only the resolved label, not the purpose it was intended for. Databricks says enterprises will still need their own conventions or explicit mappings to interpret labels consistently.
Applying the two models
Databricks summarizes the choice between approaches as depending on the destination: centralized enforcement for untrusted engines, read restrictions for trusted engines, and catalog labels for catalog-to-catalog federation.
Source: Databricks Blog, "Unifying governance across engines and catalogs in the Open Lakehouse," published September 10, 2026.
よくある質問
- What are read restrictions?
- Databricksは読み取り制限について、新しいIceberg REST Catalog契約として説明している。この契約では、カタログがリクエストしたユーザーに対するアクセスポリシーを評価し、行フィルターと列投影の指示を返す。信頼されたエンジンは、テーブルを読み取る際にこれらの指示を適用しなければならない。
- What are catalog labels?
- Databricksによると、カタログラベルを使うことで、データを提供する側のカタログはテーブルや列にキーバリュー形式のメタデータを付与できる。例えば、ある列に個人を特定できる情報(PII)が含まれていることを示すといった具合だ。フェデレーション時には、データを利用する側のカタログがこの情報を自らのガバナンスモデルに反映させることができる。
- Which engines count as trusted under read restrictions?
- Databricksは、行フィルターや列マスクをネイティブに強制できる安全に構成されたTrinoの導入例を、信頼できるエンジンの典型として挙げる一方、SparkやDuckDBについては、ユーザーがランタイムを制御できる場合には信頼できないものと位置づけている。
- Does the read restrictions spec define how trust is established?
- いや、違う。Databricksによれば、この仕様が定めているのは信頼されたエンジンが何を強制すべきかであり、カタログがその信頼をどう検証するかではないという。Icebergコミュニティではこれまで、mTLSやOAuthといった仕組みが議論の対象になってきたとも同社は指摘している。