Databricks Details Lakebase Postgres Compute Cache Improvements
Databricks says larger Postgres shared buffers and new huge-page memory support on large Lakebase compute nodes cut latency and CPU use, with production tests showing up to 2x throughput gains.
Quick answer
What did Databricks announce about improving the Lakebase Postgres compute cache?
Databricks says it increased Postgres shared buffers to 75% of DRAM on large fixed-size Lakebase compute nodes (CU 80+) and added dedicated huge-page memory support, replacing a local file cache setup that capped shared buffers near 1 GB. The company reports production throughput gains up to 2x, fewer storage reads, and lower CPU use and latency.
Key takeaways
- Databricks says it raised Postgres shared buffers on large fixed-size Lakebase compute nodes (CU 80 and above) to 75% of available DRAM, up from a roughly 1 GB cap, and disabled the local file cache on those nodes.
- The company says it introduced dedicated 2 MB huge-page memory support across its virtual machine infrastructure to reduce the memory-translation overhead created by larger shared buffers.
- Databricks reports benchmark tests showing huge pages cut tail read latency by up to about 40% and reduced CPU utilization by up to about 30%.
- In production examples cited by Databricks, one endpoint saw accessed-block throughput roughly double while storage-layer reads dropped about fivefold, and another saw CPU use fall from 20 cores to 4.
- Databricks says the rollout began region by region a few weeks before the September 10, 2026 post and that a planned second post will cover work on autoscaling shared buffers.
What Databricks announced
Databricks published a blog post on September 10, 2026, describing changes to how its Lakebase Postgres service caches data on compute nodes. The post, billed as part one of a series, is credited to David Wein, Sunil Kamath and Haoyu Huang. Databricks says the goal is to make more efficient use of the DRAM available on Lakebase compute by keeping more data in Postgres's fastest memory layer.
Background on the caching setup
According to Databricks, Lakebase Postgres uses a disaggregated storage model, with caching happening both in distributed storage and on the compute node itself. In a standard Postgres deployment, the database caches data in an in-memory area called shared buffers, while the operating system separately caches the same pages in its own page cache — a setup Databricks says causes double buffering, so 1 GB of cached data can end up consuming 2 GB of RAM. Databricks also notes that Postgres shared buffers are normally a static setting that requires a database restart to change, which the company says is a challenge for a serverless, autoscaling system.
To work around this, Databricks says it built a local file cache, or LFC, that has been running on all Lakebase compute since launch. The LFC sits on the compute node's local NVMe storage as a secondary tier behind shared buffers, which Databricks had tuned conservatively, capping them at about 1 GB regardless of a compute's total memory.
Larger shared buffers and huge pages
Databricks says it has now removed that roughly 1 GB cap for large fixed-size compute nodes. For computes with CU of 80 or higher, the company says it disables the LFC and sets Postgres shared buffers to 75% of available DRAM, keeping more frequently accessed pages in the fastest memory tier instead of falling through to the slower local cache. Databricks says customers can check the configuration by running "show shared_buffers" in a Postgres connection, with an 80 CU endpoint expected to return a value of 20971520.
Because larger shared buffers increase memory-translation overhead in Postgres's process-per-connection design, Databricks says it also introduced dedicated 2 MB huge-page memory support across its virtual machine infrastructure, opting for explicit HugeTLB pages rather than relying on best-effort transparent huge pages. The company says this required consistent support from the host level through the hypervisor to the guest kernel. Customers can verify the setting by running "show huge_pages," which Databricks says should return "on" for an 80 CU endpoint.
Reported results
Databricks says its benchmark tests found that huge pages reduced tail read latency by up to about 40% and cut CPU utilization by up to about 30%. The company says the rollout started region by region a few weeks before the post and points to three production examples measured after compute restarts. On one large endpoint, Databricks says accessed Postgres blocks per second doubled and reads from the storage layer fell from about 8,000 per second to about 1,500, while the customer reported lower p50 and p99 latency than the prior day, week and month. On a second endpoint, Databricks says throughput rose about 43% and the compute cache hit rate reached nearly 100%. On a third, Databricks says CPU use fell from 20 cores to 4 after the change, alongside a doubling of throughput.
Databricks says the current changes apply to fixed-size computes, since shared buffers cannot yet scale dynamically, and that a planned second post in the series will cover its work on autoscaling shared buffers.
Source: Databricks Blog, "Improving Lakebase Postgres compute cache," published September 10, 2026.
Frequently asked questions
- What is the Lakebase Postgres compute cache change Databricks announced?
- Databricks says it increased Postgres shared buffers to 75% of available DRAM on large fixed-size compute nodes (CU 80 and above), replacing a setup that had capped shared buffers near 1 GB and relied on a secondary local file cache.
- What performance improvements does Databricks report?
- Databricks cites production examples with throughput increases of roughly 1.3x to 2x, storage-layer reads dropping as much as fivefold on one endpoint, and CPU use falling from 20 cores to 4 on another.
- How did Databricks address the overhead from larger shared buffers?
- The company says it added dedicated 2 MB huge-page memory support across its virtual machine infrastructure, which in its benchmark tests cut tail read latency by up to about 40% and CPU utilization by up to about 30%.
- Does this change apply to all Lakebase Postgres computes?
- Databricks says the larger shared buffers configuration is currently live only for fixed-size computes with CU of 80 or greater, and that a planned second post will address autoscaling computes.