Databricks details Proteus system for specialized GPU kernel generation

Databricks has introduced Proteus, a system that uses agents to generate GPU kernels specialized for particular runtime shapes while applying controlled validation, repeated benchmarking and managed context.

The company says individual kernels produced for Qwen 3.5 122B were 1.8 to 5.2 times faster than the best implementations available in vLLM. Databricks presented the results alongside details of the harness it built to check proposed kernels and carry useful information between optimization attempts.

Validation before benchmarking

Proteus proposes kernels, verifies them against a controlled reference implementation, times candidates that pass verification and iteratively improves the strongest measured results. Databricks says validation became the central constraint because candidate kernels could appear fast without performing a fair version of the intended operation.

The company identified several sources of misleading results during development. A candidate could reuse compiled code from an earlier attempt, compare a replayed CUDA graph with a baseline that launched operations separately, or perform well only on input sizes visible during generation.

Databricks says its checker times both sides consistently and can use CUDA event timing, wall-clock time and CUPTI timing for cross-checks. It clears compiled state that should not persist, keeps setup and teardown ordering consistent, retests winning candidates and reserves some tests from the generating model. Automated consistency checks also flag claimed speedups above physical GPU bandwidth and compute limits, including results exceeding 100 times.

Limiting the context agents receive

Proteus also manages the information supplied to the kernel-generation model. Databricks says large prompts can include the current best kernel, failures, profiler guidance and notes from earlier runs, but may add cost and mix useful information with stale or conflicting advice. Prompts with too little context can cause repeated dead ends.

The company settled on a smaller knowledge layer containing actionable lessons tied to specific situations and concise failure notes from related parent runs. Proteus retrieves those lessons through hierarchical tag filtering and a hybrid of keyword and semantic search. Databricks says reorganizing and further distilling stored lessons should occur in background jobs rather than through repeated multi-step retrieval during every generation attempt.

Gated DeltaNet packed decode

Databricks tested the full loop on the Gated DeltaNet packed-decode path in Qwen 3.5 122B, using NVIDIA B200 GPUs and a Triton backend. The operation updates recurrent state and writes decode output from packed QKV inputs, gate parameters and state indices.

The reference implementation recorded latency of 0.025 milliseconds. An initial candidate passed validation but ran slower, after which Proteus divided the search into shape-specific paths. Databricks says Candidate 012 reached a 1.5-times result for the single-batch decode shape. On the serving-decode path, Candidate 030 recorded the lowest kernel latency at 0.018 milliseconds, while Candidate 036 delivered the highest shape speedup at 1.6 times.

The latter kernel specialized for a Batch=4, Key=128 and Value=128 layout and processed the value dimension in chunks of 64. Databricks described it as safe for that particular shape rather than a universal replacement.

Source: Databricks, “Achieving Extreme Efficiency through Specialized GPU Kernel Generation,” published September 4, 2026.