Skip to content
DigitalNeuron
Herramientas y productos

Databricks Details Two New Apache Iceberg Specs for Cross-Catalog Governance

Databricks describe dos nuevas especificaciones de Apache Iceberg, restricciones de lectura y etiquetas de catálogo, orientadas a estandarizar la aplicación de políticas entre motores y catálogos.

Por DigitalNeuron Desk3 min de lectura

Respuesta rápida

What did Databricks announce about governance in the Open Lakehouse?

Databricks afirma que la comunidad de Apache Iceberg ha adoptado dos nuevas especificaciones, restricciones de lectura y etiquetas de catálogo, para el Iceberg REST Catalog. Las restricciones de lectura permiten a los catálogos delegar la aplicación de políticas en motores de confianza, mientras que las etiquetas de catálogo permiten a los catálogos intercambiar metadatos de gobernanza, como etiquetas de información personal identificable (PII), entre sistemas federados como Unity Catalog y Snowflake.

Claves

  • Databricks informa de que la comunidad de Apache Iceberg ha avanzado en dos incorporaciones al catálogo REST de Iceberg: las restricciones de lectura y las etiquetas de catálogo.
  • Read restrictions let a catalog evaluate access policy for a user and return row-filter and column-projection instructions that a trusted engine, such as a securely configured Trino deployment, must enforce; Databricks says untrusted engines like Spark or DuckDB require centralized enforcement instead.
  • Databricks afirma que la especificación de restricciones de lectura define nueve acciones predefinidas de proyección de columnas y expresiones estandarizadas de filtrado de filas, pero no establece cómo un catálogo genera confianza con un motor.
  • Catalog labels, according to Databricks, allow catalogs such as Unity Catalog, Snowflake, AWS Lake Formation and Google Cloud Knowledge Catalog to exchange key-value metadata at the table and column level, which the receiving catalog maps into its own tagging or policy model.
  • Databricks señala que las etiquetas son pares clave-valor opacos y carecen de una semántica compartida, por lo que las empresas siguen necesitando sus propias convenciones para interpretarlas de manera coherente entre catálogos.

Databricks published details on two new specifications adopted by the Apache Iceberg community for the Iceberg REST Catalog: read restrictions and catalog labels. The company says both are meant to standardize how governance policies are enforced when data is accessed across different compute engines and federated catalogs.

Read restrictions

Databricks describes read restrictions as addressing a common scenario in which an organization governs data in one catalog but wants to query it using different engines or tools. Under this model, the catalog evaluates a requesting user's identity and context, decides what row filters or column masks apply, and returns those restrictions to an engine it trusts to enforce them.

Databricks distinguishes this from centralized enforcement, where all steps happen inside the catalog's own environment. The company gives its own implementation as an example, saying Databricks applies fine-grained access control on dedicated compute by routing queries through what it calls a secure filtering fleet, and that its Unity Catalog Cross-engine ABAC feature extends this to other engines through the Iceberg REST catalog's scan and plan APIs.

For delegated enforcement, Databricks says the engine does not receive the policy as an administrator wrote it, but rather the resulting filtering or masking instructions for that specific user. The company states the initial spec defines nine predefined column-projection actions and standardized row-filter expressions, such as comparisons or set membership, and notes that policies relying on subqueries, lookup tables, or custom UDFs cannot currently be expressed this way.

Databricks also notes the specification does not define how a catalog establishes that an engine is trustworthy, saying a claim from the client alone is not sufficient and that system administrators must apply their own security mechanisms. The company points to Iceberg community discussions that have considered options including mTLS and OAuth, while stating that trust remains outside the protocol itself. Databricks says a securely configured Trino deployment is an example of a trusted engine, while Spark and DuckDB are considered untrusted when users control the runtime and can execute arbitrary code.

Catalog labels

The second specification, catalog labels, addresses governance across federated catalogs, according to Databricks. The company notes that many enterprises now connect multiple catalogs, including Unity Catalog, Snowflake, AWS Lake Formation, and Google Cloud Knowledge Catalog, each with its own identity models and policy languages.

Databricks explains that labels let a producing catalog attach lightweight key-value metadata to tables and columns, which a consuming catalog then maps into its own classifications or tagging system before applying its native policies. As an example, the company describes a producing catalog labeling a column "ssn" with "pii=ssn," which a consuming catalog could use to trigger a masking policy.

Databricks states that because enforcement stays local to the consuming catalog, each system preserves its own policy expressiveness and maintains independent audit trails, without needing to call another catalog for every access decision. The company cautions that labels are opaque key-value pairs with no shared semantics or stable identifiers defined by the standard, meaning a consuming catalog sees only the resolved label, not the purpose it was intended for. Databricks says enterprises will still need their own conventions or explicit mappings to interpret labels consistently.

Applying the two models

Databricks summarizes the choice between approaches as depending on the destination: centralized enforcement for untrusted engines, read restrictions for trusted engines, and catalog labels for catalog-to-catalog federation.

Source: Databricks Blog, "Unifying governance across engines and catalogs in the Open Lakehouse," published September 10, 2026.

Preguntas frecuentes

What are read restrictions?
Databricks describe las restricciones de lectura como un nuevo contrato del Iceberg REST Catalog, en el que un catálogo evalúa la política de acceso para un usuario solicitante y devuelve instrucciones de filtrado de filas y proyección de columnas que un motor de confianza debe aplicar al leer una tabla.
What are catalog labels?
Según Databricks, las etiquetas de catálogo permiten que un catálogo productor adjunte metadatos de clave-valor —por ejemplo, marcar una columna como contenedora de información personal identificable (PII)— a tablas y columnas, que un catálogo consumidor puede mapear a su propio modelo de gobernanza durante la federación.
Which engines count as trusted under read restrictions?
Databricks presenta una implementación de Trino configurada de forma segura como ejemplo de motor confiable, ya que aplica de manera nativa los filtros de filas y las máscaras de columnas, mientras que considera a Spark y DuckDB como motores no confiables cuando los usuarios controlan el entorno de ejecución.
Does the read restrictions spec define how trust is established?
No. Databricks sostiene que la especificación define lo que un motor de confianza debe cumplir, pero no cómo un catálogo verifica esa confianza, y señala que la comunidad de Iceberg ha debatido mecanismos como mTLS y OAuth.

Fuentes

  1. Unifying governance across engines and catalogs in the Open Lakehouse | Databricks Blog — Databricks
Etiquetasdatabricksapache-icebergdata-governancelakehouseunity-catalogopen-source

Lecturas relacionadas

{"duration_api_ms":2911,"stop_reason":"end_turn","session_id":"40a692f4-4211-480b-a48e-ae8b8d16ad58","total_cost_usd":0.006905400000000001,"usage":{"input_tokens":2,"cache_creation_input_tokens":912,"cache_read_input_tokens":8132,"output_tokens":48,"output_tokens_details":{"thinking_tokens":0},"server_tool_use":{"web_search_requests":0,"web_fetch_requests":0},"service_tier":"standard","cache_creation":{"ephemeral_1h_input_tokens":912,"ephemeral_5m_input_tokens":0},"inference_geo":"not_available","iterations":[{"input_tokens":2,"output_tokens":48,"cache_read_input_tokens":8132,"cache_creation_input_tokens":912,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":912},"type":"message"}],"speed":"standard"},"modelUsage":{"claude-haiku-4-5-20251001":{"inputTokens":1077,"outputTokens":14,"cacheReadInputTokens":0,"cacheCreationInputTokens":0,"webSearchRequests":0,"costUSD":0.001147,"contextWindow":200000,"maxOutputTokens":32000,"thinkingTokens":0,"canonicalModel":"claude-haiku-4-5","provider":"firstParty","costBasis":"list"},"claude-sonnet-5":{"inputTokens":2,"outputTokens":48,"cacheReadInputTokens":8132,"cacheCreationInputTokens":912,"webSearchRequests":0,"costUSD":0.0057584,"contextWindow":1000000,"maxOutputTokens":64000,"thinkingTokens":0,"canonicalModel":"claude-sonnet-5","provider":"firstParty","costBasis":"list"}},"permission_denials":[],"terminal_reason":"completed","fast_mode_state":"off","fast_mode_disabled_reason":"sdk_opt_in_required","subagent_stats":{"spawned":0,"requested":{"background":0,"foreground":0,"unset":0},"started_in_background":0,"max_depth":0,"spawned_by_subagents":0,"completed":0,"failed":0,"killed":{"parent":0,"user":0,"system":0},"refused":{"depth_limit":0,"concurrency_limit":0,"budget":0},"by_type":{}},"is_error":false,"num_turns":1,"subtype":"success","api_error_status":null,"result":"Databricks afirma que su equipo de marketing interno usa los datos tres veces más gracias a Marge, un asistente basado en Genie","ttft_ms":2448,"type":"result","duration_ms":2518,"uuid":"0b9c18c9-93cd-4863-a5d4-afa66b7fd08a","ttft_stream_ms":1785,"time_to_request_ms":506,"queued_turn_count":0} Client.listTools() called but server does not advertise tools capability - returning empty list

Databricks afirma que su equipo interno de marketing utiliza datos tres veces más a menudo en sus decisiones desde que adoptó Marge, un asistente basado en Genie Agents construido sobre un Marketing Lakehouse gobernado. La empresa reporta una adopción superior al 85% entre los profesionales de marketing, más de 800 preguntas respondidas al mes y una caída del 25% en las respuestas incorrectas señaladas.

3 min de lectura

Databricks afirma que su asistente de IA interno triplicó el uso de datos entre los profesionales de marketing

Databricks afirma que su equipo de marketing desarrolló Marge, un asistente de análisis conversacional basado en Genie Agents, sobre un Marketing Lakehouse gobernado. La compañía informa que los profesionales de marketing ahora recurren a los datos tres veces más a menudo para tomar decisiones, la adopción supera el 85% de la organización de marketing y las respuestas incorrectas señaladas han disminuido un 25%.

3 min de lectura